Norfolk County Council said the seven remaining children's centres buildings will become "early childhood and family bases", with the majority of support being provided on an outreach basis within local community venues, such as libraries, village halls, schools, and in families' homes.
A consultation document on the plans states that "wherever possible", the council wants families' needs to be met through peer-led support - services set up and run by the local community, with help and support provided by others who have been through similar experiences in the past.
The council said it plans to find ways that local communities could use the rest of buildings currently designated as children's centres to benefit families with children under the age of five.
It said it also intends to make it easier for families to access the information and advice they need, through providing better joined-up online and digital services. The proposed changes will come into effect from October 2019.
Norfolk County Council needs to make savings of £125m across all services by 2021. In February the council announced the children's centre budget for 2019/20 had been cut from £10m to £5m.
It said extending current contracts maintaining operation of current children's centres from April 2019 to March 2020 would be unaffordable under the revised budget.
Another option, to operate 50 per cent of the current number of children's centres by focusing only on areas with the highest level of need, taking account of 50 per cent of current annual budget being available, was deemed unviable as there would be "gaps left for families".
The council said that by bringing the services out of the buildings and into the community, it will be able to spend a greater proportion of the budget on providing services and frontline staff to the children and families who need them the most.
Penny Carpenter, chairman of the children's services committee at Norfolk County Council, said: "We want to get the right help to children and families as early as possible and create services that are fit for today's families.
"By spending our money on frontline services, rather than buildings, we'll be able to provide more focused one-to-one and group support, with a more consistent service across the county.
"About a quarter of those families who live in areas of greatest need are not accessing children's centre services at the moment and we want to develop a service that gives them the support and help they need for their children.
"We've agreed significant investment over four years to develop new ways of working. This includes a range of projects to help support families to keep their children safe at home."
A consultation on the proposals will be open for eight weeks.