The research, commissioned by Education Secretary Damian Hinds and conducted by the Social Mobility Commission, found "huge disparities" in participation rates depending on social background.
The study, An Unequal Playing Field which looked at activities such as arts, music, sport, dance, voluntary work and youth clubs, found participation increased in line with a rise in household income.
It shows young people aged 10 to 15 from wealthier families are more likely to take part in every type of activity, with music and sport scoring the highest for participation rates.
The study reveals a participation gap of around 20 per cent for sporting activities, drawing attention to a lack of overall provision for extra-curricular activities in parts of the UK.
"Some classes are expensive, but there are other barriers for the less affluent," the report states. "In certain areas, there are access difficulties - schools don't provide these activities and youth provision has been cut back by local councils."
Other barriers preventing disadvantaged children from taking part in these activities are because they lack confidence or fear they will not fit in, it adds.
The findings reveal 64 per cent of young people from the highest income households take part in sport compared to 46 per cent of those from the lowest income families.
Sport is the most popular extra-curricular activity, with nearly half of young people taking part compared to only eight per cent doing art.
The research also highlights wide geographical differences in provision and participation rates. Young people in Northern Ireland joined in most in activities such as attending youth clubs, whereas children in the North East were the least likely to take music classes than anywhere else - nine per cent compared to 22 per cent in the South East.
There are also significant variations in terms of participation among different ethnic groups: while only four per cent of British Pakistani young people take music classes, 28 per cent of British Indian and 20 per cent of White British young people do.
The commission sets out key recommendations for the government, voluntary groups and schools which include the introduction of a national extra-curricular bursary scheme for disadvantaged families.
It also wants funding to develop and extend voluntary sector initiatives that allow access to activities.
The capacity of schools to provide extra-curricular activities and extra information should be increased as well as improved data collection and further research into soft skill development, it said.
The team of researchers from the University of Bath, which carried out the study, found young people who did participate in these activities gained confidence and built up social skills.
They were also more likely to aspire to go onto higher or further education.
Dame Martina Milburn, the chair of the commission, said: "It is shocking that so many people from poorer backgrounds never get the chance to join a football team, learn to dance or play music.
"The activity costs too much, it isn't available or people just feel they won't fit in. As a result, they miss out on important benefits: a sense of belonging, increased confidence and social skills which are invaluable to employers.
"It is high time to level the playing field."
John Herriman, chief executive of Greenhouse Sports, a charity helping disadvantaged children through sport, said: "Extra-curricular activity is so important for young people, and can be especially advantageous in helping to improve the life chances of those from disadvantaged backgrounds."
Charlotte Hill, chief executive of the #iwill campaign, which promotes social action for young people, said education was not just about academic achievement.
"Taking part in volunteering and other activities is a great way for young people to develop their character, confidence and vital skills for the future," she said.