Cheltenham Borough Council has said the authority had ordered the owner to carry out "urgent works" on the house.
They were designed to stop further deterioration of the listed building, make it weatherproof and stop the risk of falling render.
Mark Nelson, enforcement manager at the Borough Council, said removing any pieces would be classed as a criminal offence.
We have endeavoured to protect the Banksy as much as possible and to this end the notice required the owner to have due regard for the Banksy mural whilst works were being undertaken.
We were aware of loose render on that part of the building where this was situated but the extent and how far it would affect the mural was unknown until work progressed.
The owner has left pieces of the render with attached Banksy Spy Booth mural on it with the council whilst investigations are continuing.
We would advise anyone against removing any further pieces from the location as this may be classed as a criminal offence.
Until the investigation is complete we are unable to say what if any further action will be taken.
An MP has called for an urgent inquiry after a Banksy mural was torn down - as sources close to the owner claim it has been destroyed.
The popular artist's repeatedly vandalised work was in Cheltenham, just a few miles away from GCHQ.