FA chairman Greg Dyke said he was "annoyed" by claims made by Fifa President Sepp Blatter that the British media was racist following allegations of corruption.
He said: "I was deeply upset at the allegations yesterday from Mr Blatter about the British media was somehow racist. I got annoyed by it.
"It's nothing to do with racism at all, so I just pointed that out."
Speaking to ITV News Brazil Correspondent Nick Ravenscroft, Mr Dyke added: "I think a lot of Uefa members feel that Fifa has a bad reputation, and as a result something needs to be done about that. It's hard to do that while Mr Blatter is there."
More top news
Two private citizens have already paid a "significant deposit" to take the week-long trip.
Slippy start tomorrow with frost & ice
The court will determine how much the UK government and travel firms knew about the risk of an attack on tourists holidaying in the area.