Controlled trials gain momentum

Randomised controlled trials (RCTs) are associated more with scientists testing new drugs on guinea pig patients than they are with children’s services.

But a Department for Education announcement this month of funding for two RCTs in education and child protection is further evidence of the method gaining popularity in the sector.

One trial announced by Education Secretary Michael Gove will assess the maths and science results of 480 schools so they can “compare and collaborate” on different methods to raise standards.

Alongside this, a new child protection assessment tool – the Safeguarding Assessment and Analysis Framework – will be tested to see how it compares with results gathered through existing alternatives.

RCTs are a prerequisite for drugs to be used in the NHS, and have become the preferred method in the health field for proving clinical and cost effectiveness.

Better evidence needed
Given current economic constraints and commissioners’ desire for evidence of successful outcomes, should children’s services providers be looking to use the method to test the effectiveness of their preferred interventions?

Leon Feinstein, head of evidence at the Early Intervention Foundation, says stronger evidence is required across the sector. “There is quite a strong view in government that the evidence underpinning a lot of social policy is a bit weak,” he says.

“It tends to be qualitative, or practitioner-led, or theoretical – but very little of what some would call hard evidence. We want to support service providers to develop more rigorous quantification of impact and benefits, and understanding of costs compared to benefit.

“If they are done well and are testing a clear hypothesis, RCTs can add a lot of knowledge. But we are also aware that social policy is not conducted in a laboratory, and there are all kinds of implications to that.”

While acknowledging the benefits of RCTs, Paula Lavis, policy and knowledge manager at mental health charity YoungMinds, says they can have drawbacks when used to assess social policy. In medical trials, for example, “blind” studies can take place whereby patients do not know whether they are receiving the trial medicine or a placebo – a method that can counter the effect of perceived benefits, or otherwise, of the subject.

“This wouldn’t be possible for all interventions,” Lavis says. “RCTs probably work better in a controlled environment, where you can control your variables effectively. This may be an issue for social care, where it would be difficult to control all environmental factors.”

The majority of RCT-tested programmes used in the UK – such as the Incredible Years parenting programme and the Family Nurse Partnership – were developed and tested in the US, where the method has become commonplace. But there are increasing examples of projects developed in the UK being tested in this way. The Social Research Unit at Dartington has carried out RCTs on a social and emotional learning programme in schools, and parenting interventions in children’s centres.

But Nick Axford, senior researcher at Dartington, says it is not the right method to use in every instance.

Choosing the right method
“It is a tough test and some interventions that are perhaps a bit early in their development would not stand up to scrutiny – you almost do them a disservice,”
he says.

“When an organisation decides it wants to put an intervention through an RCT, it is taking a bit of a risk – it is possible that the trial will show no effect or that the effect is harmful.

“But if you are concerned about improving children’s wellbeing, it is arguably a risk worth taking.”

The random nature of the trials has been criticised by some on ethical grounds, since some people do not receive the intervention. But Axford says measures can be incorporated to address these concerns. “RCTs are good or suitable where you have more people who need the intervention than you have spaces,” he says.

“You have to come up with a fair way of deciding who gets what, and random allocation could be a good way of doing that. When we conduct RCTs, we are always open.”

For example, one RCT Axford was involved in staggered the intervention so that all participants received it, but in two groups, one after the other. This overcame any ethical issues and was still able to measure effectively the impact of receiving and not receiving the intervention.

Cost is also an issue – it varies depending on the size and length of the study as well as the data collection method used. The cost of a single study can range from £50,000 to well over £1m.

“There are lots of evaluations that might not cost as much, but the results don’t tell us much about the impact, and arguably that is money not very well spent,” Axford says.

“RCTs done well might cost a bit more, but you should get a result that you can be confident in.”

Randomised controlled trials