The Save Downhills campaign group said they were "disappointed but not surprised" their application was refused and were considering whether to appeal.
The group said in a statement: "Today's ruling shows that consultation during academy conversion is irrelevant.
"Parents' voices have been completely ignored. The law appears constructed to ensure that only one man, the Secretary of State, can make decisions about any school's future. Downhills is a happy, inclusive and improving school as the latest test results and Ofsted monitoring will show.
"We believe we are the victims of his wider political agenda. He publicly abused us as 'enemies of promise' and said he was 'marching towards the sound of gunfire'. We continue to believe Mr Gove has abused his power, denying parents and local communities their legitimate voice."
The Department for Education has welcomed the High Court ruling over Downhills Academy.
"Downhills has been under-performing for several years. Ofsted found that the school had failed to give its pupils an acceptable standard of education and that those responsible for leading, managing and governing the school hadn't demonstrated the capacity to secure the improvement necessary.
"The new academy will open in September and will be sponsored by Harris, a not-for-profit educational charity, which has already turned around a number of previously failing schools in London, nine of which have now been judged by Ofsted as outstanding."
Parents had argued that Downhills Primary was now thriving - and that turning it into an academy made no sense.
But, as he refused permission for a judicial review, Mr Justice Kenneth Parker said that it was a rational decision to turn it into an academy, in the light of the school's "egregious" past performance.
He said that it was clear that the Education Secretary "simply had no confidence" that the school would substantially improve if it remained within the control of Haringey Council.
On the other hand, he "believed firmly" that conversion to an academy would substantially improve performance.