Munro, professor of social policy at the London School of Economics, found "widespread concern" among staff at the first 11 children's services departments to undergo the new Ofsted inspection over "the demands it puts on inspectors and local authorities and about the reliability and validity of the judgments reached”.
Despite this, Munro concluded the inspection framework – which combines inspections of care, adoption, fostering and child protection services – is robust and has widespread support across the sector, a feat she describes as a "major achievement".
The review, published today (Tuesday) alongside an Ofsted report, also backed the judgments reached in the first 11 inspections as fair.
Munro, author of the government's 2011 review of the child protection system, was asked by Ofsted to evaluate the effectiveness of the new single inspection framework following its launch in November last year.
Her findings, drawn from interviews with inspectors, senior Ofsted staff and those working at the 11 councils inspected between November 2013 and February 2014, reinforce previous concerns raised by children's services leaders about the amount of data needed to be collected and analysed.
She said: “It is not just that it required more data but also more types of data and the connections between data.”
This created a significant cost to councils, in terms of staff time and resources needed, the report says, and left council staff questioning “whether the demands were reasonable when the public sector is dealing with major cuts in funding”.
Some reported inspections had a negative effect on frontline support, with staff having to spend time with inspectors rather than families.
It also left inspectors struggling to make the best use of data to reach a judgment.
Other issues raised by councils included a lack of consistency in judgments, differences in verbal and written feedback from inspectors and a lack of transparency in how a final judgment was made.
Ofsted’s moderation process that reviews judgments also came under fire from three councils, who felt “it was inappropriate for people with no direct knowledge of their work to make such an influential judgment about them”, said Munro.
The moderation process was an issue raised in February by Sheffield Council after its initial rating was changed following moderation.
The overall “level of emotional impact” of an inspection on council staff was also a worry, Munro said. Many council staff told her they felt demoralised by the “stressful” inspection process and were left in fear of a negative judgment.
Munro suggests a factor in this heightened fear has been the single framework’s replacement of an "adequate" rating with "requires improvement". Staff told Munro that councillors and council chief executives are now viewing this as equally as damning as the lowest grade of "inadequate".
Munro added: “With so much riding on obtaining a good or outstanding judgment it is perhaps not surprising that inspections are daunting.”
Munro makes a number of recommendations for Ofsted, including:
- Qualitative data management software should be used to better analyse information
- Inspectors need training to improve their research and project management skills
- Set up a "consistency panel" to discuss, review and standardise judgments
- Ofsted moderation needs to be more transparent
- Overall judgments need a narrative indicating how close they were to the next category
- Improve advice to councils on how they can use feedback from children and families
- Use clearer language in inspection reports around risk management
- Inspectors need a better understanding of what constitutes good practice.
In its response to Munro’s recommendations, Ofsted has said it is trialling software improvements to help inspectors better understand data. It is also considering a training overhaul to improve inspectors’ research and project management skills.
A decision on whether a “consistency panel” is viable is to be made by the end of July, with a panel able to be up and running by September, says Ofsted. Revised guidance for inspectors on improving transparency is also due by the end of July.
Ofsted will consider including greater details in inspection reports on how a judgment was reached and how close the service was to being handed a different judgement. A decision on this will be made by November 2014.
In addition, Ofsted has pledged improvements to the way service-user assessments can be used.
The 11 councils evaluated were Bolton, Coventry, Derbyshire, East Sussex, Essex, Hartlepool, Hillingdon, Hounslow, Sheffield, Slough and Staffordshire.
