PEER EVALUATION: Part of the team
PJ White
Wednesday, February 11, 2004
Youth agencies are now expected to involve young people in evaluating their services. PJ White finds out how peer evaluation projects can help the young inspectors as well as the services they are assessing.
It is a startling claim. But there is logic to it. An effective service to young people should be shaped by young people. It should have young people observing, evaluating and recommending changes. That process is an integral part of delivering a service. Young evaluators or researchers are doing a crucial job. Therefore all are co-workers.
If it is meaningful, large numbers of young people will be engaged in that research and feedback process. And if young people are co-workers, they should be paid for their work.
A paid job
Increasingly, both are happening. The days of two token young people on a management committee are disappearing. Wider and deeper consultations are helping determine all kinds of youth services, often using young people as researchers. Young people are acquiring skills in research and carrying out robust inspections using standard evaluatory techniques. They are finding innovative and creative ways to get more accurate information from their peers, building on the advantages of their understanding and rapport. And yes, despite some obvious problems, young people are being paid for their time and effort in shaping the services they assist.
Quite right, says Harry Wade, active involvement team manager at The National Youth Agency and co-author of the Hear by Right standards for young people's participation. He has seen the problems in the past when young people, fully engaged on a task alongside youth service providers, were distinguished solely by being the only people in the room not paid for their work. "Paying them is the honest way," he says.
The result is that Connexions South Central now employs two young people working a 30-hour week as "peer connexors". They work supervised by personal advisers particularly on the social aspects of relating to young clients.
When the first two appointees came to the end of their 14-month fixed-term contract, they gave a comprehensive evaluation of how they had found the job to the local management committee. They also made recommendations every three months to the committee throughout their term.
Different perspective
Connexions Shropshire, Telford & Wrekin also pays young people. Since November, three 18-year-olds have been full-time salaried peer researchers, offering a professional young person-led evaluation service to Connexions branches in the region. "When we go to a branch, we look at things from young people's point of view," says Moya Matthews-George, who before her appointment as a peer researcher for the NewView team was doing casual pub work. "We look at three main areas - information, atmosphere and branch layout." Like traditional inspectors, they carry checkslists and clipboards and use their knowledge to assess what is working well and what isn't.
They also advise branches on ways to improve their feedback from young service users. They encourage the use of methods like graffiti walls and suggestion boards.
Peer researcher Stacie Brown, 18, says: "We got a lot of help from the branches, from other Connexions staff." The processes and techniques of evaluation were an eye-opener to the young researchers, and they felt they learned a lot. But it was not so easy at first. Their learning curve was steep. They think their early reports could have been more detailed.
But they now feel they have earned the respect of the Connexions staff.
Peer researcher Jez Brookes says: "They seemed intrigued at first, as if they were judging us." In time, and particularly after a key presentation to the staff - they won them round.
Unique insight
Paul Facer, peer evaluation team manager at Connexions Shropshire, Telford & Wrekin, works closely with the NewView team. He is under no illusion that the openness that comes from the inspections is bringing a lot of knowledge to the company. The team is funded until the end of April 2004, jointly by Connexions and the Children's Fund, whose work it also inspects.
The venture is a recognition that young people have a set of skills and insights that adults don't have, says Karen Bradshaw, chief executive of the Connexions partnership. The team has been given a lot of support, with one worker more or less permanently assigned to them. That has paid off. To begin with all three young people were accompanied when they went on visits. Now they are much more independent. "It demonstrates how much they have grown," says Bradshaw.
Dibs Patel is planning a similar team in the London Borough of Barnet, where he is assistant head of leisure and youth services. "Are there better advocates and promoters of positive youth work than young people?" he asks rhetorically. Although he sees them as "useful at weeding out poor youth work", he sees no reason why they should be perceived as a threat to the service. "In my experience, young people are always positive and constructive about change," he explains.
Barnet will begin the process of recruiting and training young inspectors over the coming months. Patel is passionate about the contribution young people can make by bringing their own perspective to inspections. They have a few methods uniquely their own.
For instance, a mystery shopper-type sampling of services is a form of consumer research not available to a traditional middle-aged Ofsted inspector.
But the main advantage young people have is that they can get so much more out of existing methods than adults can. Conversations and focus groups can be used by anyone. But if young people run them they are much more fruitful, says Patel.
On payment there is no question. When young people simply turn up to meetings, they should be paid expenses. If they are actually doing a job of work, they should be properly paid. "It is the future," says Patel.
He sees payment as about young people's empowerment and part of an equality agenda. There are limits though. The pay rate for young inspectors in Barnet is not determined yet. But Patel confesses it is unlikely to match the dizzy heights of an Ofsted inspector's salary.
CASE STUDY - EVALUATING SERVICES IN NEWCASTLE
Newcastle's Play and Youth Service commissioned a review of services for Black and minority ethnic (BME) young people in the city. The tender was awarded to a joint project between the Regional Youth Work Unit North East and Save the Children. The research was carried out by young people.
"We thought recruiting young people as researchers would be a problem," says Cate Blatherwick, youth and community worker at the Regional Youth Work Unit. In the end, it wasn't. On the first training and information weekend 22 young people from 11 different ethnic backgrounds turned up.
Blatherwick had been hoping for 12.
They decided to involve everyone. It was the right decision, says Blatherwick.
But it took a lot of effort to train and support 22 peer researchers contacting nearly 300 BME young people over six months. They were trained in community research methods and spoke to small groups of young people about how they spent their spare time, the projects they used and what changes they wanted to see. Researcher Fatema Rahman, 25, says the group learned a lot from each other. "When I started, I didn't know any Black or Chinese people or asylum seekers, she says. "Now we are like one big family."
The researchers' findings include a recommendation for increased mixed youth provision, bringing together BME and White young people. They noted a need for more specialist single gender provision for certain groups. Young people also wanted more BME youth workers and more anti-racist training for White workers.
"Young people need to see evidence of change but they are also realistic that things are not going to change overnight," says Blatherwick. "Their involvement will increase the impetus for change because young people have been at the heart of the process."
WHAT TO CONSIDER
- Young people evaluating youth services need well thought-out training and support. That is not because they are young, but because they are new in a job
- Don't reinvent the wheel. You are not the first to use peer researchers. Use the Hear by Right guidelines and standards. These can be downloaded from The National Youth Agency's web site, www.nya.org.uk
- Let young people know what happened as a result of their involvement. Specifically, what changes came about. If nothing did change, ask yourself why you did it. Perhaps something went wrong?
- Make sure that young people are properly accredited for their learning and work
- Young people doing a job like to be considered as colleagues first and young people second
- Having your work evaluated by young people is no more or less threatening than being inspected by adults. But it can be a sensitive business, so be aware of likely staff responses.