Barriers to measuring outcomes

The government has announced it will establish a new National Stability Forum for Children's Social Care to advise ministers and civil servants on the best ways to promote stability, better life chances and outcomes for looked-after children.

In addition to providing national leadership, the government wants the forum to "galvanise" action by local commissioners and providers to join up and promote effective practice and decision making.

Although stopping short of creating a national "permanence board" to oversee all care placements and practice, some children's services expertssay the new forum could take the lead on developing a national outcomes framework (see below).

Such a move would be welcomed by independent care providers.

A report published last month by the Children's Services Development Group (CSDG) - a coalition of independent care providers - concludes that a national outcomes framework would allow commissioners to benchmark services from different providers based on value, quality, cost and outcomes.

"This would allow for outcomes-based commissioning, enabling commissioners and providers to make strategic use of data to ensure better placements for children, and afford them the time to make well thought through placement decisions," states the report. "Further consideration of how outcomes are identified and quantified would be required."

Here, Andrew Rome of Revolution Consulting examines the key challenges to delivering a national outcomes framework:

Calls for a national outcomes framework are not new. It has been tried in the past, for example alongside national contracts for special schools, fostering and residential care.

Those approaches largely fell by the wayside. The opting-out of national contracts by local authorities that disliked some of the more commercial terms meant the outcomes measures described by the contracts were never embedded into practice or monitoring.

This would indicate that robust leadership of a project to develop a national outcomes framework is needed. It is difficult to see this being effective without ministerial and DfE leadership, potentially backed by statutory implementation.

Could the National Stability Forum take on this responsibility? Here are some of the challenges.

Find the right measures

Many people have correctly reached the conclusion that the most meaningful way to examine outcomes is at the individual child level. For example, the aspired outcome for one child new into a placement might be to see a reduction in incidence and severity of self-harming, whereas for different children in different settings, the aspired outcomes might be to bring about engagement with education, reduce drug and alcohol use, or reduce absconding and risk-taking behaviour.

The ways in which the types of outcome might be described and evidenced are as diverse as life and human behaviour itself. Finding measures that apply to most or all children, and thus to services, is also challenging.

Convince social workers

Some in the sector argue that decision making based on a small sub-set of blunt measures made for all children and young people is flawed and therefore risky.

Social work professionals will need to be convinced that any framework is simply designed to help inform them better about options they may be considering, and that it is not to be used to deny them professional judgment and choice for the child.

Ensure context is considered

It is clear that some of our current outcome measures for looked-after children are a poor fit for the task of informing policy or to measure the impact of different services or providers. This should give motivation to an intelligent and balanced attempt to consult widely on potential framework indicators, and to consider if better indicators could be developed.

The purpose of such indicators also needs mature reflection. As a tool to help to describe different segments of need, and to inform commissioning and supplier development strategies, a framework may be worthwhile.

However, an understanding of causality is also essential.

Outcome measures for any service or provider need context - for example, an indicator that simply measures average length of stay will generate very different results for a short-term assessment unit compared to a long-stay children's home.

Outcome measures may also be affected by factors not under the control of the service or provider - for example, the unexpected closure of a state school could have an impact on fostered children.

Robust evidence is needed

Proposals to commission for outcomes have been around for 20 years, but little progress has been made, perhaps because when issues of context and causality begin to be considered, the challenge has become too great for those involved.

This reinforces the requirement that any attempt to develop a national framework needs to be properly funded, robustly supported and led from the centre; it needs to consult all parties widely and in depth, and with equal voice; it needs to have scientific and mathematical rigour applied so may need academic input; and framework measures may need to be verified and validated so the role of regulators in relation to such audit may need development; and it needs to be one system rather than a patchwork of regional variants.