The Chancellor said the Government had embarked on the "biggest infrastructure programme for a generation".
Asked why projects could not be brought forward to aid growth in the economy now, George Osborne told ITV News: "I'm absolutely for doing everything we can, as a Government, to support construction through infrastructure, through our help-to-buy scheme to help families get their first home.
"But I'm also clear that as a country we have to do that in a way that convinces the world that we can pay our way. You can't have the debt fuelled problems, the debt fuelled growth, that got Britain into this mess in the first place."
George Osborne said today's GDP figures are "better than forecast". The Chancellor added: ""Britain is on the mend, but we've got to stick with the plan because there's still a long way to go."
Shadow chancellor Ed Balls has given a cautious welcome to the preliminary economic growth figures:
After 3 damaging years of flatlining, this growth is both welcome & long overdue. But still no recovery in family living standards.
Today's economic growth figures are encouraging. We are on the right track - building an economy for hardworking people.
GDP stats better than forecast.Britain's holding its nerve, we're sticking to our plan, the economy's on the mend.But still a long way to go
The 0.6% growth is broad-based; it is the first time since the third quarter of 2011 that all four sectors of the economy are growing.
It was the first time since 2011 that the UK has seen back-to-back quarterly increases, after a 0.3% rise at the beginning of this year.
The largest contribution to growth in the second quarter came from the services sector, the Office for National Statistics said.
Chief Secretary to the Treasury Danny Alexander said "there's a lot more to do" after figures showed the economy grew by 0.6% in the second quarter of 2013.
Good, balanced growth figures show we are on the right track - but not time to get carried away, there's a lot more to do