Rejecting the application, Lady Justice Rafferty said the sentencing judge had not "fallen into error" in what was a "difficult sentencing exercise".
She said that "mercifully" cases of the kind involving Rooke came before the courts "relatively seldom".
Rooke, 45, was jailed along with his wife Donna, 40, and son Jamie, 19.
Jamie Rooke admitted affray and a number of counts of causing actual bodily harm and was sentenced to four years custody.
Donna Rooke admitted a specimen count of battery and was jailed for four months.
At the Crown Court, Judge Peter Kelson said the family used Kinsella like a "punchbag" and treated him "like a dog".
He said: "'Man's inhumanity to man never ceases to shock."
After the hearing, Mr Grieve said: "This was a shocking case of exploitation of a vulnerable man by a family who abused and assaulted him on a regular basis.
"Mr Kinsella was tricked and trapped by people who posed as his friends and employers. In reality they were manipulative and exploited him.
"He lived in David Rooke's garage in appalling conditions and the abuse only came to light.