The warning came from former Ofsted chief inspector Sir Michael Wilshaw, who told MPs his own experience of excluding young people as a headteacher left them in "great danger of being drawn into crime".
Wilshaw, part of a panel of witnesses to give evidence, said earlier intervention is needed if the number of exclusions are to be reduced.
"I realised I was often committing them to a miserable few years afterwards so it was a very painful decision to make and we sent negative messages to that youngster and their family about themselves," he said.
He told the committee many of the young people excluded from school came from "difficult, often dysfunctional, chaotic homes".
"They presented behavioural problems often at an early stage in primary school before they reach secondary school.
"If we are going to make an impact on their lives, if we are going to make sure that they don't get excluded, then early intervention is absolutely critical," he said.
Wilshaw said it was not just a case of schools maintaining "tight behaviour policies", but ensuring there are sufficient resources to deal with pupils' emotional and mental health issues.
"There are two sorts of kids that get into trouble. One is the odd bout of misbehaviour which can be dealt with internally, but the really serious stuff is manifested by youngsters who are deeply troubled," he added.
"Schools need to have the resource to deal with those troubles at an early stage," he told the committee.
The one-off evidence session took place on Wednesday to explore a possible link between the rise in exclusions and increase in knife crime.
Since the start of the year, there have been 41 fatal stabbings in England and Wales, 10 of whom were aged under 20.
Robert Halfon MP, the committee's chair, cited a statistic from the Ministry of Justice in 2018 that showed 21 per cent of students who committed a knife possession offence were excluded from school.
However, of this group there was a 50/50 split between those whose first exclusion was prior to the offence and those who were excluded at some point after the offence.
Wilshaw told the committee he believed some of the young people carrying a knife in those instances may not have been "that much of a problem in school".
"They may have been carrying a knife for all sorts of different reasons including self-defence but if they had behaved well in school and had a pretty good record, there's no reason why they should be excluded," he said.
- Feature: A public health approach to youth violence
- Experts look to public health model to tackle surge in youth knife crime
The committee's witness panel included Will Linden, deputy director of the Scottish Violence Reduction Unit who spoke about Scotland's approach to tackling knife crime over the past decade.
He said there had been a cultural shift away from exclusion being the "last resort" and the "unintended consequences" of taking a young person off-roll were taken into consideration.
"In 2007, Glasgow had 140 permanent exclusions in one year, last year they had one. This year it is zero so far.
"So Glasgow still excludes but it was this idea of how can we intervene, who intervenes and what do you need to do in terms of risk management. It's about putting young people at the heart of what we are doing in Scotland," he said.
He said Scotland still had high levels of violence, including youth violence, but the "trajectory is in the right direction".
Last year, the education committee's report on alternative provision called on the government to tackle rises in school exclusions.
It warned a lack of early intervention and support was contributing to the increases and that the quality of alternative provision should be improved.
Committee member Lucy Allan MP said schools faced a difficult choice between protecting pupils and excluding young people that might then face a "more detrimental situation".
"Children with knife crime convictions will not be allowed to go to a pupil referral unit so what happens to them," she said.
Wilshaw said local authorities had been "marginalised" in recent years in terms of overseeing schools and there had to be a "better balance" in the part they played in monitoring all schools.
"A lot of the exclusions involve children with special educational needs and difficulties and the local authority should be able to track them through the key stages and make sure they don't fall through the net," he said.