He said simply relocating the Lords was "not going to make a big difference" because "the nations and the regions do not feel part of this United Kingdom".
He said if the relocation was "combined" with the creation of the "senate of the nations and regions" that unified the country by representing all corners of it, then that "would be a difference".
"But just to move it from London 200 miles north - one anachronistic institution does not cease to be anachronistic if it's in another place but unchanged, you've got to change it," he added.
Mr Brown says SNP calls for independence have gained prominence because Scots feel a "remoteness and a distance" from an "over-centralised" Westminster government.
He said that could be helped if more decisions were made by regions within the UK and suggested the formation of a "council of the north, the council of the midlands".
"We have got to fundamentally change if we are going to make people feel comfortable within the United Kingdom in the future," he said.
He said Brexit has helped the SNP case for independence because leaving the EU involves powers being transferred from Brussels to London and nowhere else.
"They're not going to be transferred up north, either to the Welsh Assembly or the Scottish Parliament or the Northern Ireland Assembly," he said.
Despite admitting the country is "divided", he claimed the SNP's terms for a referendum had "not actually been met".
He said the party had spoken of needing consistent wins for the SNP of over 60% to show a mandate for a referendum.
And he claimed constitutional changes in the UK will soon make things in Scotland "move on" from calls for independence.
"You're going to have proposals for change that will change the face of Britain and I think make Scotland feel that they are a much bigger part of it."
Watch Robert Peston's full interview with Gordon Brown here:
He also said Labour would not be supporting calls for a second independence referendum in Scotland "at this time".
The former chancellor refused to reveal who he was backing for the Labour leadership, saying he would only back a candidate once names were on the ballot paper.
Those hoping to succeed Jeremy Corbyn must first receive backing from unions or Constituency Labour Parties (CLPs) before going on the ballot for a poll of members.
Mr Brown said whoever the next leader is, they must "have an economic vision for the Britain of the future that includes environmental considerations" and dealing with rising poverty in the UK.