Video report by ITV News Political Editor Libby Wiener
Liberal Democrats leader Jo Swinson hailed the victory as candidate Jane Dodds won the mid-Wales constituency by a margin of 1,425, swinging the vote nearly 12% from the Tories.
Speaking to ITV News, Ms Swinson said: “The Liberal Democrats are on the up. We’re winning again.
“People don’t have to settle between this choice of Boris Johnson and Jeremy Corbyn.
“We’re in different political times.The old rules do not apply. The rules of politics are changing.”
Boris Johnson will be nervous about seeking a new mandate from the electorate, says ITV News Political Correspondent Libby Wiener
The victorious Ms Dodds told ITV News she was happy to "burst (the PM's) bubble in the first week of his premiership" and send him a message on Brexit.
She added: "We really, really hope… that that he will listen loud and clear to the people of Brecon and Radnorshire who’ve said: take no-deal off the table."
Mr Johnson’s first major challenge as Tory leader became his first major defeat with the announcement of Ms Dodds’ victory in the early hours of Friday.
The Tory loss will add to the PM’s challenges to steer Brexit through Parliament and may increase the chances of a snap general election.
With the vote coming eight days after Mr Johnson was appointed leader, it is the quickest by-election defeat for any PM of the post-War period.
Welsh Lib Dems leader Ms Dodds pushed into second place the Conservative candidate Chris Davies, who was the constituency’s MP until he was ousted in a recall petition after he admitted submitting false invoices for expenses.
Despite 19% of the electorate petitioning for his recall, the Tories selected Mr Davies to fight to reclaim the seat.
The Lib Dems secured 43% of the vote, the Tories 39% and the Brexit Party 10%.
Fourth-placed Labour narrowly avoided losing its deposit by securing just over 5% of the share.
The vote may also be seen as an early victory for the “Remain alliance” of anti-Brexit parties, with Plaid Cymru and the Greens agreeing not to field candidates in order not to split the pro-EU vote.
In her victory speech, Ms Dodds thanked those two parties for their “courageous decision”.
Plaid Cymru leader Adam Price said the “spirit of co-operation” between the pro-Remain parties had led to Ms Dodds’s election, as he called for a second referendum.
“But if the prime minister is intent on a general election, he should know that Plaid Cymru and the other pro-Remain parties are committed to cooperating so that we beat Brexit once and for all,” he added.
Mr Davies won the seat from the Lib Dems in 2015 and secured a majority of 8,038 in the 2017 general election.
But this was overhauled with a swing to the Lib Dems of 11.96%.
The loss leaves the PM with the support of 319 MPs, including the DUP which props up his majority, while opposition parties now have 318.
Mr Davies’ recall came after he pleaded guilty in March to submitting two false expenses invoices for nine photographs costing £700 to decorate his new office.
He was fined £1,500, ordered to pay £2,500 towards legal costs and told to carry out 50 hours of community service.
In his losing speech, Mr Davies thanked his team and family “who have had a difficult time over the last few months”.