The Early Intervention Foundation report wants to see a greater focus on boosting cognitive skills of disadvantaged children at an early age. This includes their language development and understanding of numbers, objects and people.
It says health visitors need to play a greater role in leading this work and be backed by extra investment to ensure they can provide intensive home visiting support to focus on supporting low-income families during their children's first two years.
The report points to evidence that shows there is already a language gap between young children from affluent homes and those living with low-income families.
By the age of three, middle- and upper-income children have a vocabulary at least twice the size of their low-income peers, according to previous EIF research.
The report also says there is an economic case for investing in health visitor support to help with early development, as income-related learning gaps can already be identified by the age of four.
"Unless we recognise that children's language and their understanding of objects, people and numbers are a basic necessity, we're in danger of the country's most disadvantaged children already being significantly behind their peers when they start school, creating a gap that is likely to widen even further as they get older," said EIF chief executive Jo Casebourne.
"Increasing investment in health visiting is one crucial way to support children's cognitive development and ensure they have the opportunity to live happy, healthy and productive lives, whatever their circumstances."
Other recommendations include ensuring early years curriculums focus on children's awareness of the world around them and counting activities.
The EIF report comes weeks after a survey of health visitors found that four out of 10 feel so stretched that they fear a tragedy at some point.
The Institute of Health Visiting survey of more than 1,200 health visitors found that 43 per cent believed they may not recognise whether a child is in need until it is too late, due to rising demands in recent years.
Over the last three years health visitor numbers have declined markedly, from 10,309 in October 2015 to 7,852 in July this year.