Liverpool's defence of the League Cup ended as Brendan Rodgers' selection gamble backfired, as former club Swansea progressed to the quarter-finals for the first time in their history.
The Northern Irishman left the south Wales club in the summer to take over from Kenny Dalglish but it was the visitors who had the last laugh in the fourth round of the Capital One Cup at Anfield.
Chico Flores and Nathan Dyer put the Swans 2-0 up with 17 minutes to go and although substitute Luis Suarez quickly pulled one back De Guzman made it 3-1 in added time to loosen the eight-time winners' grip on the trophy.
It was only their 10th defeat in 102 League Cup games as the home team.
The match pitted together two sides with similar playing styles having been crafted by the same manager.
However, it was Rodgers' version one rather than his current side which came out on top.
The Reds boss made nine changes from Sunday's Merseyside derby draw and the starting line-up contained only three players - Jamie Carragher, Stewart Downing and Joe Cole - aged over 24.
Cole, who still appears to remain a misfit at the club he joined in the summer of 2010, started his first match since the Europa League qualifier against Gomel on August 2.
He did little to suggest he is any closer to regaining the first-team spot he loosely held in Roy Hodgson's brief tenure and lasted only 45 minutes.
Big-names stars Steven Gerrard and Suarez were left on the bench but it would only be 45 minutes before they were dragged into the action.
It took Swansea, who kept two clean sheets against Liverpool in the league last season, a while to find their feet at Anfield but they created the best chances.
Dyer's control let him down after Ashley Williams' long ball out of defence sent him racing clear of the defence, Ki-sung Yeung smashed a half-volley wide and Pablo Hernandez had a shot blocked by Carragher after a quick one-two with Michu on the edge of the area.
The best move of the half saw Michu rob Joe Allen - who followed Rodgers from the Liberty Stadium - inside Swansea's half, offload to Dyer who charged forward before squaring to Jonathan de Guzman whose chip drifted just over.
Liverpool's best opportunities, half-chances at best, had fallen to Stewart Downing, who had been the match-winner on this ground last week in the Europa League against Anzhi.
But he could not repeat his goalscoring feat, drilling one shot wide and forcing Gerhard Tremmel, replacing the injured Michel Vorm, to push the ball around a post.
Swansea, who were not short of confidence, finally took their chance in the 34th minute.
Yeung's low shot was parried wide by Jones and Chico beat Sebastian Coates to De Guzman's corner to power home a header.
It was their first goal at a Premier League ground since winning 5-0 at QPR on the opening day of the season.
The visiting fans, filling three-quarters of the lower tier of the Anfield Road end, sang 'Brendan, what's the score?' and also suggested Liverpool were just a poor imitation of Swansea - albeit in much more colourful language.
Rodgers wasted no time in sending for the cavalry, bringing on Gerrard and Suarez for the disappointing Cole and the out-of-his-depth 18-year-old Samed Yesil at half-time.
Instantly the hosts were lifted and Jonjo Shelvey fired over just two minutes after the re-start.
Gerrard came closer on the hour, firing a low shot against the post which rebounded to Downing only for him to miss the target from 12 yards.
Rodgers' final throw of the dice was to send on Raheem Sterling but Swansea were in no mood just to defend their lead and, if anything, they re-doubled their efforts with Jones making three saves in quick succession from Hernandez, Dyer and Michu.
That trio applied the finishing blow in the 72nd minute when they caught Liverpool exposed on the counter-attack, the two Spaniards combining for Dyer to score from close range.
"We don't need you any more," sang the Swansea fans, although technically there was little they could have done about Rodgers' summer departure after the manager admitted before the game moving to Anfield was a "no brainer".
Liverpool needed a quick response and they got it when Suarez headed in Gerrard's free-kick three minutes later but they could not force another and De Guzman made the result safe deep into added time.
Swansea's last cup game at Anfield was an 8-0 FA Cup third round drubbing in 1990. They left a lot happier this time.