Sussex Police said a 67-year-old man, believed to be retired Reverend Vickery House, was also released.
The 67-year-old man arrested at his home address near Haywards Heath on suspicion of two separate sexual offences against two boys, aged 17 and 18 at the time, in East Sussex between 1981 and 1983, was interviewed in Sussex and released on police bail late this afternoon while enquiries continue.
The arrests result from a review and subsequent investigation over the past six months.
Sussex Police said a man arrested this morning, believed to be retired Peter Ball, 80, a former Bishop of Lewes, was released this afternoon on medical advice:
The 80-year-old man arrested this morning at his home address near Langport, Somerset, on suspicion of eight sexual offences in East Sussex and in one case elsewhere, during the late 1980s and early 1990's, was released at his home this afternoon on medical advice.
It is our intention to interview him at a later date. The offences were allegedly committed against eight boys and young men, all of whom were at time in their late teens or early twenties, except one who was 12. We will continue pursue allegations of sexual offending robustly.
The Bishop of Chichester, Martin Warner, confirmed an 80-year-old retired bishop and a 67-year-old retired priest were arrested this morning relating to "allegations of sexual abuse in the 1980s and 1990s".
The bishop said:
We can confirm that the retired bishop has had no ministry in Sussex for many years and no longer lives in this area. The retired priest has had his Permission to Officiate suspended.
Our co-operation with Sussex Police in this investigation continues our ongoing commitment to do all that is necessary to bring any alleged criminal matters to the attention of the public authorities, and to ensure that the Diocese of Chichester is a safe place for all in our church communities, whilst being an unsafe place for any who may seek to abuse them.
"This is a very complex enquiry, in the course of which many people, all now adults, have had to be traced, together with other witnesses and records from a wide variety of sources", the force said in a statement.
Police said there were no allegations of recent or current offending, and no suggestion that any young people are at risk.
They also said the claims are being treated separately and do not involve allegations the two men acted together.
A retired Church of England bishop was among two clergymen arrested by Sussex Police investigating historic allegations of child sex abuse within the scandal-hit Diocese of Chichester, according to sources.
Peter Ball, 80, was arrested at his home near Landport, Somerset, on suspicion of eight sexual offences committed against boys and young men during the late 1980s and late 1990s, sources said.
A second man, aged 67, was also arrested this morning at his home near Haywards Heath, West Sussex, on suspicion of separate sexual offences against two teenage boys in East Sussex between 1981 and 1983.
Sussex Police confirmed arrests have been made but did not reveal the identity of the suspects.
They are currently detained - in Somerset and Sussex respectively - for interviews by Sussex Police officers, the force said.