Inspectors from Ofsted and the Care Quality Commission said parents of children with SEN are "very critical" of the support they receive locally.
"Parents are not convinced that the reforms have improved services for them or their children," a letter detailing the findings states.
"Parents are very frustrated about the quality of information and guidance, the timeliness of responses to concerns, the fairness of access to specialist services and sufficient recognition of the breadth of their children's and family's needs."
The inspection, which took place from 4 to 8 July this year, is one of a handful of new local area SEN inspections that have been carried out so far.
The inspections look at how well areas are supporting children with SEN and implementing the reforms set out in the Children and Families Act 2014.
Inspectors also found that support to children with SEN was inconsistent across the nine regional areas delivering special provision.
Inspectors found a discrepancy between the information the Department for Education and the local area have on the number of children with moderate learning difficulties and autism. This means the area does not have the information it needs to properly commission support, inspectors said.
The inspection also found that some education, health and care plans, which replace statements of SEN, were completed by the Hertfordshire County Council without the contribution of all relevant professionals.
The SEN inspections also examine how well an area has implemented their local offer - a list of services available for children with SEN and their families.
The report states that the local offer is a "source of much frustration among parents" because it does not meet their needs.
Richard Roberts, cabinet member for children's services at Hertfordshire Council, said, said: "I'm pleased that Ofsted has recognised some of the excellent work we are doing in this area.
"We are always reviewing what we do with the aim of getting better, and I'm confident that we are in a great position to further improve the lives of families of children and young people with special educational needs and disabilities."
