Appearing before the education select committee, Wilshaw, whose term as chief inspector of Ofsted comes to an end in December, said the way in which directors of children's services spend their time is an "issue that needs to be looked at", suggesting that they may spend too much of it away from their local authority.
Asked, in light of three quarters of local authorities being rated as "requires improvement" or "inadequate" under the single inspection framework, whether children's social care is in crisis, Wilshaw said: "It's not good enough. We're quite clear that it has got to improve."
"The issue for us is why is it so poor? Particularly now that local authorities are losing much of their responsibility for education. Academisation means that they can focus much more upon the quality of children's services than ever before. We have to ask the question 'why is it going so badly wrong in so many instances?'"
Wilshaw said that "it seems a very common sense thing" for a director of children's services and the chief executive to make sure that they have enough social workers, and that agency workers are properly professionally developed and trained.
"It seems common sense to make sure that the case files are up to date," he added.
"It's common sense to make sure that the thresholds are clear and transparent for everyone and assessment is appropriate and timely, yet they get it wrong again, and again, and again.
"Are they away from their local authorities too much? Are they on too many conferences? Or are they really getting down and finding out for themselves what is happening on the frontline? That's an issue I think needs to be looked at."
Wilshaw said that the quality of children's social care is "a leadership issue".
"Like schools, it is a challenge for the system to ensure there are enough good leaders coming through to lead one of the most difficult parts of public services," he said.
Asked by Labour MP for Gateshead Ian Mearns about whose responsibility improving and monitoring leadership in local authorities is, Wilshaw said the Association of Directors of Children's Services have a role to play.
"They have got a big responsibility, not only to represent directors of children's services and senior officers in social care, but also to lead training. It's the responsibility of the Department for Education and the chief social worker as well to make sure that training and professional development is of high quality."
Wilshaw also expressed concern about the number of "underperforming" directors of children's services who "leave after a poor judgment and then find their way into another local authority".
"That's got to be monitored by the department," he said.
Eleanor Schooling, Ofsted's national director for social care, said that around half of authorities judged to "require improvement" are "good in part".
She said one of the main weak points was the provision of help and protection for children.
"They are not always working really well with their partners, because work in protecting children always involves health, police, schools and others," she said.
She added that caseloads is another recurrent issue.
"If you have got social workers who constantly change, it's very difficult to assess what a child and family's needs are," she said.
"In some local authorities we have seen caseloads of over 40. A social worker's work should be about engaging with the family, getting the family to understand what the issues are, and helping them to make the right changes so that there doesn't need to be any further intervention by a local authority.
"But that's absolutely not possible when you have got 40 cases with which to deal.
Schooling said that Ofsted would like caseload levels to be published as a local indicator, and a requirement placed on local authorities to evaluate whether or not caseloads are safe and right for doing the work.