Set against a backdrop of deep cuts to local authority youth work budgets – and the bulk of what remains being spent on targeted provision for the most at-risk young people – the £40m allocated to open access youth work projects through the Youth Investment Fund (YIF) has provided a much-needed shot in the arm for the sector.
The three-year programme is a joint investment between the Department for Digital, Culture, Media and Sport, and the National Lottery Community Fund.
Since being established in 2017, it has supported around 90 youth organisations to expand their provision in six regions in England – Liverpool, Tees Valley and Sunderland, east London, Bristol and Somerset, West Midlands and East Midlands.
As the YIF draws to a close, latest analysis shows that the projects funded through it have been well received by the more than 400,000 young people using them, and have achieved impressive satisfaction marks for safety, enjoyment and trust (see graphics).
A key measure was how young people used the experience to make positive changes in their lives.
The YIF Learning Project – led by New Philanthropy Capital (NPC) in partnership with the Centre for Youth Impact and a wider consortium of partners – has published the fourth in a series of evaluations on emerging findings from the programme.
Here, key early insights from the evaluation are explored, while experts analyse how youth service leaders can apply the findings to their own projects and practice (see below).
Valued services
Whether the activity is boxing, creative writing or health education, the way the scheme is delivered, experienced and the values that underpin it are what set open access provision apart from other types of youth work.
How young people experience the provision is the key “mechanism of change” that affects outcomes. This means that young people’s experiences can be used as a “criteria for quality delivery”, as their absence undermines positive outcomes.
Feedback for YIF-funded provision was generally positive across the board, suggesting that open access youth provision is highly valued by young people.
There were especially high ratings for young people’s feelings of safety, enjoyment, trust, value, quality and feeling that taking part was worth their time and effort.
The evaluation concludes that “these are the fundamental aspects of provision that underpin impact”.
Co-production
Youth organisations care about working in partnership with young people, yet data from the programme shows that delivering effective co-production is not easy.
“Young people’s experiences of co-production can sometimes fall short compared to other key mechanisms of change,” the evaluation finds. “This is perhaps unsurprising given that effective user-involvement is genuinely hard to get right.”
The proportion of young people believing it was “very likely” changes would be made to provision as a result of feedback rose from 49 to 72 per cent. Despite this, there was little evidence of organisations changing how they worked based on feedback. Those that were youth-led tended to be better at enacting changes.
Trust in staff
Young people show exceptionally high levels of trust in youth work practitioners – 82 per cent said they trust staff “a great deal”.
Yet they do not always feel this trust is reciprocated – just 64 per cent of young people felt that staff trusted them “a great deal”.
The evaluation team say this disparity may be linked to the limited success of co-production, as empowering young people to influence how provision is run demonstrates trust. The relative absence of authentic co-production may be seen by young people as a lack of trust in them.
Data collection
The YIF evaluation approach used five types of data to measure the impact of open access youth provision. This looked at who attended – age, ethnicity, gender – and how they engaged with provision – what activities they attended and how often – alongside understanding young people’s feedback and outcomes, and the quality of the setting.
The varied ways in which young people engage with provision, misalignment between evaluation approaches and youth work practice, and the practical challenges of collecting data from and about young people, mean that even the most basic data can be difficult to collect, the evaluation concluded. A particularly hard challenge is to develop a data collection system that works for different organisations and is compatible with existing systems.
“We don’t think this is a reason to reject evaluation in open access settings or to double-down on more traditional impact evaluation approaches,” the report authors state. “Rather, we believe the YIF learning project has advanced theory and practice in developing a more effective and practice-aligned approach to evaluation for open access youth provision beyond what would have been gained from traditional impact evaluations that focus narrowly on outcomes.
“Co-producing a shared theory of change alongside an evaluation framework and associated data collection tools is a significant step forward in understanding quality and impact in open access youth provision, a field that has typically been challenging to articulate at a sector level.”
Shared evaluation
The report concludes there is a strong interest in shared evaluation and learning across open access youth provision. However, it says more support is needed in organisations and across the sector to develop the capacity to conduct evaluation, reducing barriers to sharing knowledge and creating incentives to collaborate, and embedding a learning culture that values the sharing of good practice.
“This is not just an ‘ask’ of individual youth organisations; it requires ongoing infrastructure support,” it states.
- YIF learning and insights papers from: https://yiflearning.org/
EXPERT VIEW
What do the Youth investment fund emerging findings mean for the sector and its funders?
By Bethia McNeil, chief executive of the Centre for Youth Impact (left), and Karen Scanlon, programme manager for the YIF
Youth organisations do lots of different things for young people. This is neither new nor contentious, but it is an idea that traditional evaluation approaches have struggled to fully take on board. In response, we tend to be swift to say all youth organisations are unique (as are young people) and that “you can’t compare apples and pears”.
The challenge for the Youth Investment Fund (YIF) Learning Project was to design an evaluation approach that not only took account of the diversity and range of activities represented by the open access youth organisations that took part, but reflected it in a way that enabled us to understand more about its impact on the lives of young people.
Shared evaluation is a chance to build consensus across a field of practice on common goals, outcomes and value (beyond financial returns). By taking a common approach to measurement, organisations and the funder community can gain greater insight into individual and combined strengths, and identify areas for continuous improvements. Practitioners are empowered through these insights, and funders are able to support a more equitable approach to evaluation.
Shared insight
For the YIF evaluation, we actively looked for commonalities among the 90 YIF-funded organisations to develop measurement frameworks that would enable shared meaning and insight. In impact terms, most youth organisations are actually doing the same thing: creating safe and welcoming spaces for young people, where they build trusting relationships with skilled practitioners, learn new skills and grow in their awareness of themselves and others.
It’s this core, fundamental framing of what constitutes “quality practice” that is key to understanding its influence in the lives of young people and communities.
Understanding the influence – and impact – of open access youth provision is only possible with a holistic evaluation perspective. In practice, this means looking much further than outcomes and assessing data on who accesses provision and how often, young people’s experiences of it, the impact it has on their outlook and wellbeing, and what measures of quality there are.
Our approach looked at the relationships between these types of data, so we could explore whether young people had more positive experiences in higher quality provision, for example, or whether different groups of young people attending different types of provision reported differences in their social and emotional skills.
The early findings from the YIF Learning Project suggest that what young people experience within youth provision – which relates to both what is offered and how it is offered – is really important. We’re seeing early signs that young people’s feedback on their experiences correlates with practitioners’ assessments of quality practice, and that the feedback is more positive in the settings that participated in the quality self-assessment.
Routine feedback
We also found that gathering routine and light-touch feedback is feasible for small youth organisations, and can be easily integrated into their ongoing engagement with young people.
Our final findings are due to be published in spring 2021, but our early findings highlight the relationship between quality provision, youth experience and – we hope – outcomes. It means we have some tools and resources that can be used by youth organisations to reflect on, improve and understand the impact of their work. It has developed an approach in which funders can collectively invest to both capture and improve the impact of the sector’s work. And it means we have a voice to talk more clearly about how open access provision influences change – rather than just whether it does.
