The concerns are highlighted in a Department for Education-commissioned report examining the trust's progress on improving the town's "inadequate"-rated children's services.
The report says that senior staff within the trust feel that "the extent of historic challenges" in the service mean that the speed of improvements expected by the DfE and Ofsted are unrealistic.
"Staff across the trust thought the trust needs time for the changes it has implemented to bed in across the organisation, particularly the change to the hub model and systemic practice and the shift in working culture," the report states.
It adds that the trust is also finding the ongoing scrutiny of the service by Ofsted and others is distracting it from focusing on delivering improvements. It also points out that the trust faces higher than usual levels of inspection and regulation because of its status as an independent fostering agency and voluntary adoption agency.
The report by consultancy firm Kantar Public suggests that the trust, the DfE, and Slough Council, work together to establish "realistic expectations" about how quickly services in Slough will improve.
The DfE ordered the formation of the trust to take over the operation of children's services in Slough in 2014 after Ofsted found the service was "inadequate".
The trust launched in October 2015 but in 2016 a fresh inspection by Ofsted found the service to still be inadequate.
In its most recent monitoring visit to Slough, Ofsted found there was still a lack of supervision and oversight of social workers and delays in responding when children are at risk.
Despite staff concerns about unrealistic expectations, Kantar Public's report said the trust has made "substantial changes" to children's services in Slough.
These changes included reduced bureaucracy by flattening the management structure and strengthening the front door teams with additional staff. The report also said the trust had adopted a "zero tolerance" approach to poor performance within the service that has led to staff feeling more accountable for the quality of their work.
Another success was that the trust and council were now working together after an initially "hostile" relationship that, the report says, "undermined the trust's progress and smooth transition from Slough Borough Council". Problems caused by this "dysfunctional" relationship included the trust having a lack of access to finance, performance and HR data.
"This report demonstrates the hard work of staff at Slough Children's Services Trust to raise standards for vulnerable children and families," said children's minister Nadhim Zahawi.
"I am especially pleased to see that children have been put at the heart of its culture change and that they have invested substantially in the workforce, reducing staff turnover.
"I hope the learnings in this report are helpful in setting up children's trusts in the future and that Slough continues its improvement journey so that the needs of children and families are met effectively."
Publication of the report follows the naming of Lisa Humphreys as the trust's new chief executive. Humphreys, who is currently the interim director of children's services at Wokingham Council, will replace the trust's interim chief executive Andrew Bunyan in September.