Appearing before the education select committee, Hinds said that he is concerned about the high number of children being excluded and believes better information sharing by schools with a low exclusion rate could help reduce the numbers.
Latest figures show that 35 children a day are being excluded from schools. Earlier this month the government announced a review into school exclusions.
Hinds confirmed to MPs that the review will consider ways in which successful behavioural management initiatives and other efforts by schools to reduce exclusions can be used England-wide.
"It is clearly possible to reduce the number of exclusions we have now because there are individual schools and wider areas where there is a lower level," said Hinds.
"Our first priority should be spreading the information and knowledge on how those that do it best do it."
He added: "In my view schools in general do not want to be excluding children so we need to try and help them. I would like there to be fewer children excluded. I want that number to come down."
Last month Ofsted wrote to more than 100 head teachers across eight exclusion "hot spots" to raise concerns about the high number of children being removed from schools in their area.
Also at the committee hearing, Hinds defended the government's record on childcare and early years in response to questioning about cuts to Sure Start children's centres, and parental support programmes.
"In terms of children's centres, different local authorities will work in different ways. There are still plenty of bricks and mortar establishments out there," he said.
"There are other ways also of working with families - various outreach programmes and so on. For children in the most difficult circumstances we have got the [DfE social care innovation fund], for example, working with children's services departments across the country looking at different ways of working.
"On the overall question of investment in childcare and early years, this government is spending more than any previous government in Britain on early years and childcare - an extra £1bn."
Hinds' appearance follows a speech he gave yesterday for World Social Work day when he announced two new appointments to work closely with the social work profession and share best practice across the social care system.
Lord Patel of Bradford, a former social worker, will chair the new dedicated social work regulator, Social Work England, which will set professional, education and training standards for social workers.
And former children's minister Edward Timpson will chair the independent Child Safeguarding Practice Review Panel to consider and share learning from the most serious child safeguarding cases.
The Child Safeguarding Practice Review Panel will be responsible for overseeing reviews of serious child safeguarding cases that the panel believe include complex or nationally-important issues.
