Northern cities could be offered "serious devolution of powers and budgets" in return for electing mayors, George Osborne has announced.
The elected mayors of the North would enjoy similar powers to those of London Mayor Boris Johnson.
A referendum on the subject was held but rejected in 2012.
But Mr Osborne made clear that he was ready to offer a bigger carrot in terms of local decision-making power to tempt the cities to reconsider their opposition.
He said the combined economic and creative energies of Manchester, Liverpool, Leeds, Sheffield and Hull could compete on a global scale, just as the stars of their Premier League football sides could "wipe the floor with any competition" if they came together in one team.
Chancellor George Osborne has tweeted his plans to create a 'Northern Powerhouse' to rival London as he gives keynote speech in Manchester.
We'll back the science & universities & creative clusters in the North and offer the local power & control the #NorthernPowerhouse needs
Chancellor George Osborne has said the fastest growing economic activity right now is in the North-East.
Giving a keynote speech in Manchester, he said people were also joining the jobs market there at the fastest rate.
Despite being a Londoner, the MP, who represents Tatton in Cheshire, spoke of his love for the North.
Being a Londoner I am proud to represent a northern constituency.
I feel the buzz and the energy every time I’m here. And I see it too in the Treasury data."
A new high-speed rail connection and better roads could create an economic "powerhouse" in the north of England to rival the success of London, George Osborne said today.
The Chancellor is proposing the new transport link between Manchester and Leeds to connect a collection of northern cities, which if combined could 'take on the world.'
He said he was prepared to back it up with money in the sum of '£6 to 7billion.'
He said: "It's a vision of how you create better road and rail links across the Pennines.
"We are about to commit, later this year, many billions of pounds to investing in our economy to try to bring together a vision of how the north of England can have this northern powerhouse."
George Osborne has said there is a 'strong case' for more elected mayors to drive growth and development in northern cities and rival London Mayor Boris Johnson.
Speaking on BBC radio 4's Today programme the Chancellor said: "I think there is a strong case for elected mayors in places like Greater Manchester to make sure they have the same powers, the same clout, as the mayor of London does in our capital.
"I think it helps to have a single individual who is, of course, democratically accountable, but also bringing together powers over planning, housing and transport."
He added he wanted to bring together cities like Manchester, Leeds, Liverpool and Hull with new transport links.
George Osborne will say a collection of northern cities linked closer together by new transport infrastructure could "take on the world" as he bids to end London's "unhealthy" economic dominance.
The Chancellor will say in a keynote speech in Manchester today:
We need an ambitious plan to make the cities and towns here in this northern belt radically more connected from east to west - to create the equivalent of travelling around a single global city.
As well as fixing the roads, that means considering a new high speed rail link.
Today I want us to start thinking about whether to build a new high-speed rail connection east-west from Manchester to Leeds.
New railway plans proposed by Chancellor George Osborne mean commuters could soon travel from London to Manchester in just one hour.
In a speech in Manchester today Mr Osborne will say 'we must do more to connect our northern cities.'
The proposed line would be based on the existing rail route between the two cities "but speeded up with new tunnels and infrastructure" to create "a third high speed railway for Britain".
He will also propose a high-speed rail connection east-west from Manchester to Leeds.
A new high-speed rail connection and improved roads between England's northern cities could help create an economic powerhouse to rival London, George Osborne will say today.
He will use a keynote speech to say an upgraded, high-speed link between Manchester and Leeds should be considered as part of the review of the second phase of the HS2 project.
The Chancellor will say he wants travelling through towns and cities in the northern belt to be as easy as moving around a major global metropolis as part of a plan to help them "take on the world".
Under current plans the £50 billion HS2 high-speed rail project will link London to Birmingham in its first stage, before creating a Y-shaped network with lines to Manchester and Leeds by 2032/33.
But concerns about connectivity between the northern cities were raised in the report by HS2 Ltd's chairman Sir David Higgins earlier this year.
Transport unions have hit out at Virgins Trains after securing an extension deal to its West Coast Main Line franchise.
The leader of the TSSA rail union said Virgin Trains is "being rewarded for failure on the West Coast line" and argued that publicly-owned franchises "provide better value to both taxpayers and passengers".
"Ministers continue to ignore the lesson that publicly-owned East Coast has returned far more cash to the Treasury in the past five years than Virgin," he said.
Mick Cash, from the RMT union, has said Virgin Trains have "bullied themselves into a monopoly provider position" by "ignoring the public sector option".
"Rail franchising is a one-way ticket to the bank for greedy shareholders as these companies promise the earth and deliver just the bare bones in the interest of profit," he said.