Troubled Families delivers gains despite damning study findings
Derren Hayes
Tuesday, October 25, 2016
In-depth research has found no link between the Troubled Families programme and improved outcomes for participants, but children's services leaders claim the government's flagship initiative still has merits.
The national evaluation into the effectiveness of the Troubled Families programme has concluded that the government's flagship initiative has had little impact on the lives of the people it worked with.
For critics of the programme, the long-awaited report from the National Institute of Economic and Social Research (NIESR) has confirmed their suspicions that the scheme was not worth the £440m the government invested in it, nor a further £900m spent on its expansion last year.
The NIESR findings, and subsequent media furore over them, has also called into question how ministers and civil servants interpret and use evidence to validate the success of policies.
In June 2015, then Prime Minister David Cameron claimed that the programme had successfully "turned around" 116,000 of the 120,000 families with the most entrenched social problems worked with in the first phase of the scheme, and at the same time saved £1.2bn in taxpayers' money.
The former Prime Minister's assertions were based on the number of payments made to local authorities for successfully achieving targets through a payment-by-results (PbR) system.
Under the first phase of the scheme, introduced in 2012, councils were paid £3,200 for working with every family they identified as meeting the needs criteria of committing antisocial behaviour, being long-term unemployed and on benefits, experiencing child welfare problems and having children who persistently truant from school.
A further £800 payment was made when a family was deemed to have been "turned around", which involved a number of targets being achieved, such as desisting from crime or attending school for an entire academic year.
Buoyed by the high number of families whose lives had been "turned around", 18 months ago, Cameron extended the scheme to a further 400,000 families with a wider range of problems including those affected by parental mental health.
Flawed evidence
But the NIESR report - based on information from 56 English local authorities covering 30,000 families - suggests the evidence base for this decision was fundamentally flawed.
It states: "Across a wide range of outcomes, covering the key objectives of employment, benefit receipt, school attendance, safeguarding and child welfare, we were unable to find consistent evidence that the Troubled Families programme had any significant or systematic impact.
"That is to say, our analysis found no impact on these outcomes attributable to the programme. The vast majority of impact estimates were statistically insignificant, with a very small number of positive or negative results."
The report authors make it clear that their findings are not at odds with councils meeting the targets under the PbR system, but point out that there is no way of proving whether families worked with through the scheme would have still achieved improvements if they were not involved.
Some experts say the findings confirm the government has based policy decisions not on evidence of positive impact for families, but a belief that the programme itself - and the mechanisms it uses - are the right ones to achieve change (see expert view).
Ministers and the Department for Communities and Local Government (DCLG) maintain the programme is achieving results.
Giving evidence to the public accounts committee inquiry into the Troubled Families programme last week, DCLG permanent secretary Melanie Dawes said the initiative had achieved "some really good results".
But she shied away from claiming any cost savings derived from the programme. "We are certainly not claiming any definitive cost savings," she said.
Former Troubled Families chief Louise Casey also accepted the research findings that the improvements in families' lives could not be directly attributed to the scheme.
But she added: "Did we oversell and under deliver? The answer to that, honestly, is no.
"Did we change the lives of 116,000 families? Yes, we did."
Casey's assessment tallies with that of the Local Government Association (LGA).
In its submission to the committee inquiry, the LGA said the programme had helped target resources and integrate services. Specific benefits it highlighted included the role of the key worker to co-ordinate services across different agencies, and better use and sharing of data, which had improved agencies' understanding of complex problems.
However, it adds: "It may be difficult to attribute these to the Troubled Families programme in isolation, particularly given the relatively short period of time it has been running."
Positive benefits
The Association of Directors of Children's Services (ADCS) also believes the study has not captured the benefits derived from the scheme.
"The positive impact of the programme is being evidenced locally in many areas," says Dave Hill, ADCS president.
"A lot of work is being done by committed and dedicated practitioners across the country to support families to recognise their strengths, empower them to make changes and provide them the tools to improve their situation.
"That's not to say it is without faults, but we need to look past the statistics and hear from families themselves, many of whom tell us they appreciate and benefit from the support provided through the programme."
For Keith Davies - associate professor at Kingston University and St George's, University of London, and author of a book on the Troubled Families programme - the national evaluation highlights the complex and sometimes difficult relationship between researchers and policymakers.
"What researchers expect is to get critical outcomes and lack of success, and then build up knowledge in that way," he explains.
"The relationship between policy rhetoric and research methods is more problematic than they [ministers] imagined."
Davies says the root of the problem could be based on the fact that it was "not so difficult" for councils to "turn around" families and trigger a payment due to the way the scheme was designed.
This gave an overly optimistic impression that families' problems had been fixed and was a "vulnerable" strategy for government to take when more in-depth research had been commissioned, he adds.
However, Davies says it would be "throwing the baby out with the bathwater" if the NIESR findings led to the Troubled Families programme being scrapped, as it has led to improvements.
"There's been a shift in the way services work together, which is reflected in the model of working with the whole family," he says.
"A success has been that some of the quantitative research has shown improvements in families' morale, sense of hope and functioning. That is at least one of the features that is likely to produce better results in the future."
The DCLG is due to produce a report on the progress of the second phase of the programme in early 2017 and then every six months thereafter.
Hill says he would like the government to replace the "crude monetary measures" with a more "sophisticated and nuanced approach" to assess the impact of the second phase of the programme.
Davies also says it would be sensible for ministers to scrap the term "turned around" to describe families' progress for this second phase as it "encourages overclaiming".
Such a move would also be supported by Casey. "It was right at the time, but I wouldn't use it in the future," she told the committee inquiry.
Expert view: Study discredits troubled families scheme
By Michael Lambert, tutor in early childhood studies at Liverpool Hope University and expert on the Troubled Families programme
The independent evaluation of the government's Troubled Families programme has finally been published. It does not make good reading for the government.
The evaluation report, by the National Institute of Economic and Social Research, has found that there has been "no significant and systematic impact across a range of key outcomes". At a time when the government has repeatedly cut funding to local authorities and continued its national commitment to financial austerity - despite claims otherwise - the princely sum of £448m for the first phase of the programme seems a lot of money spent, for not a lot in return.
Payment-by-results has brought payments, but no results.
Despite this clear failure, rather than beating a hasty retreat, the programme has been expanded.
In 2014, it was announced that the programme would "build on the success" of the first phase by radically expanding its remit. This was after an earlier 2013 announcement that the second phase would be piloted before it was rolled out. From 120,000 "troubled families", there would now be 400,000. From £448m, there would now be at least £900m. All of this, without the completion - let alone publication - of the first phase of the report.
At the time, academic and expert on the programme Stephen Crossley expressed his concerns of "fast policy" in action. Since then, his worst fears have been realised. The "troubled families" model - of payment-by-results on a targeted and defined group - has been exported to other policy areas. This, Crossley argued, was an instance of "market-tested policy solutions" and not, as the government claimed, evidence-based policy.
At the time, evidence of the superficial and counterproductive basis of these changes was noted.
In their 2013 evaluation of Lincolnshire's Families working together, Sue Bond-Taylor and Peter Somerville noted the increase in paperwork and data-gathering, rather than working with families, that happened when the programme was introduced to existing family intervention projects. The programme lost many of the benefits of close casework and intervention by expanding without proper training and caseload management. If there was any success in the Troubled Families programme, it was down to the tenacity of workers.
My own research on the historical antecedents of the programme in the "problem family" schemes run by local authorities in the 1940s, '50s and '60s provides a warning. In the post-war years, social work expanded enormously, particularly around schemes to "rehabilitate" the "problem family". It served to professionalise, modernise and standardise social services throughout Britain.
The Troubled Families programme is doing the exact opposite. It is rolling back the frontiers and responsibilities of the state. Concerns have been expressed about the privatisation of children's services. The Troubled Families programme is trying to construct an "evidence" base to support payment-by-results and reduce responsibilities of local authorities. The uproar caused by the intention of the Children and Social Work Bill to allow local authorities to opt out of statutory requirements is another worrying example.
The Troubled Families programme serves as a salutary warning about the claims of evidence-based policy. It is as much about "turning around" state services, as the families themselves. The "troubled families" model has been thoroughly discredited. Janus-faced austerity to local authority social services on the one hand, and spending money on a vanity flagship policy which had "no discernible impact" on the other should lead us to question the motives of the government in introducing reforms without evidence, consultation or considering the views of frontline staff.
- Michael Lambert submitted evidence to the public accounts committee inquiry into the Troubled Families programme