At a meeting at York Racecourse on Wednesday night (27 April), all parties supported a motion agreeing that it was no longer appropriate for Prince Andrew to represent York and its residents.
The Duke of York was given the ambassadorial title in February 1987 in a ceremony shortly after his marriage to Sarah Ferguson.
But it came under scrutiny following revelations about his friendship with disgraced American financier, Jeffrey Epstein, who was convicted of sexually assaulting young women.
The campaign to remove the prince's title gathered pace after he was stripped of a number of other titles and then made an out of court settlement with the woman who had accused him of sexual assault.
Prince Andrew always denied the allegations.
Cllr Darryl Smalley, City of York Council’s Executive Member for Culture, said the Freedom of York was "the highest honour" the city could bestow.
He said: "The honour is held by many notable and accomplished people who carry it with pride and responsibly.
"Having been stripped of his military roles and royal patronages by the Queen, we believe that it is right to remove all links that Prince Andrew still has with our great city.
"I was pleased to see councillors of all parties support this motion and make it clear that it is no longer appropriate for Prince Andrew to represent York and its residents.
"The removal of this honorary title sends the right message that we as a city stand with victims of abuse."
Prince Andrew remains the Duke of York as he keeps the dukedom bestowed on him by the Queen. But York councillors have now made calls for him to relinquish that title too.
Councillor Darryl Smalley added: "The next logical step is now for Prince Andrew to do the right thing and relinquish his Duke of York title.
"If he fails to do so, the Government and Buckingham Palace must step in to remove his title to finally end Prince Andrew’s connection to York."
Some people ITV News spoke to in York today agreed that relinquishing his title is the "appropriate thing to do"