Assessing character development

Paul Oginsky, chief executive of Personal Development Point (PDP), a specialist training provider that helped develop the government's flagship National Citizen Service initiative, has written a "white paper" advising the youth sector on how to attract investment into character development for young people.

In the paper, A Way Forward for Character Development: The Missing Piece of Education, Oginsky says that practitioners, commissioners and policymakers need to "differentiate effective character development from other forms of education, leisure and positive activities" if they are to make a "convincing case for funding even in times of austerity".

To do this effectively, youth leaders need to identify clear ways to show how services and interventions help young people. Oginsky says this poses a problem, because there is no "definitive and universally accepted" way of measuring the impact of their work.

"There is no such thing as a unit of confidence, loyalty, honesty, motivation or any of the characteristics which this kind of work seeks to impact," he says.

To overcome this, Oginsky argues that impact assessment should ask young people to describe if and how they have changed - with this being verified by those who know them. This can be conducted rigorously despite not being quantifiable or measured, he says.

"Some funders may still insist on measurable evidence for a wider impact - such as all young people having full and happy lives," he adds. "This is a waste of everyone's time.

"Far more effective is to ensure programmes include robust methods to assess the progress made by young people, as evidenced by them and witnessed by others."

Here, three youth work leaders give their views on Oginsky's recommendation.

Bethia McNeil, director, The Centre for Youth Impact

McNeil agrees with Oginsky's conclusion that there is no definitive and universally accepted way of measuring impact, and that asking young people about their experience of programmes, services, relationships, and what has changed, is "vital".

"This kind of systematic feedback should be part of every organisation's approach to developing its provision and evaluating it," she says. "It can be both qualitative and quantitative - both are equally important."

However, McNeil says that feedback is only one element of impact and should sit alongside a "clear understanding" of the type of provision, an assessment of its quality, and data or information demonstrating its wider benefit.

"The development of character brings additional challenges because young people will be influenced by all their interactions and relationships, and it will be very difficult to connect change to any one organisation or project," she adds.

"Debates about character education are still fairly contentious, and there is no agreement on outcomes so it is unsurprising that we cannot universally agree on how to measure it."

Joanne Hay, chief executive, Teens and Toddlers

Having outcomes verified by young people is a "powerful thing", says Hay, but warns that on its own it is not enough to support the case for investing in youth character education.

"If it's for a funder or commissioner and the programme is targeted then a mix of hard and soft outcomes could be used," she says. "If it's for the provider, they will need to know how well programmes are working, so a mix of measures can be useful along with performance information and of course feedback from young people."

In the same way that looking at attainment in isolation doesn't give the full picture, "nor does a statement on a character change", she says.

"We can measure what people are capable of but we also need to look at what they have actually done too.

"A mix of measures works best. The purpose and type of programme will determine the most appropriate mechanism and will likely include a mix of hard and soft measures that includes evidence of an actual change - what has actually happened as a result of the change."

Amanda Hill-Dixon, researchers lead, Project Oracle

Hill-Dixon says that it is never advisable to stick to just one tool or evaluation approach.

"We agree that the self-reflection and reports of young people and those around them can contribute to evidence of impact but there are many different ways of collecting this, including readily available robust quantitative impact tools, along with qualitative tools such as interviews, focus groups and observations, to name a few," she adds.

"The paper suggests youth practitioners ‘forget measurement'. While we recognise the challenges of measuring soft outcomes, we believe that a continued effort is vital to improving our understanding of what works, for whom and in what circumstances in children and youth provision.

"In essence, robust impact evaluation, including quantitative and/or qualitative approaches focused on a particular question, is essential for creating the learning culture [the paper] recommends."