The 37-year-old succeeds Nicola Sturgeon, who formally tendered her resignation to the King on Tuesday morning after announcing her intention to stand down last month after more than eight years in the post.
Mr Yousaf's election at Holyrood comes after he won the SNP leadership contest on Monday, beating closest rival Kate Forbes by picking up 52% of votes compared to her 48%, when second preferences were included.
Scottish Conservative leader Douglas Ross, Scottish Labour leader Anas Sarwar and Scottish Liberal Democrat leader Alex Cole-Hamilton all stood against Mr Yousaf for the position of First Minister in the Holyrood vote but failed to secure enough support.
Mr Yousaf won the votes of 71 fellow MSPs with members of the SNP and Greens backing his candidacy.
The 31 Tory MSPs voted for Douglas Ross, while 22 of the Labour contingent voted for Anas Sarwar. The four Lib Dem MSPs backed their leader Alex Cole-Hamilton.
Speaking after the vote, Mr Yousaf said he would lay out detailed policies to Holyrood after the April recess.
His government, he said, would prioritise the fight against the cost-of-living crisis and he will speak to anti-poverty groups in one of his first engagements as First Minister.
He added that he will seek to reform the NHS and social care, extend childcare, maintain the promise made to care experienced young people, reform criminal justice and tackle drug deaths.
Mr Yousaf went on to say he hoped to "build on the record" of Nicola Sturgeon and John Swinney, to whom he paid tribute in his first Holyrood speech as First Minister.
MP for Dumfries and Galloway and Scottish Secretary Alistair Jack responded to the result saying: "My congratulations go to Humza Yousaf on being elected as the new First Minister of Scotland.
"The UK Government wants to work constructively with him on the issues that matter to people in Scotland.
"I was pleased that during his acceptance speech Mr. Yousaf made clear he also wants to work together. That is what people in Scotland, rightly, expect.
"There is a huge amount to be done - continuing to tackle the cost of living, growing Scotland's economy, tackling NHS waiting lists, ensuring our energy security and improving transport links.
"I hope that Mr Yousaf will govern for the whole of Scotland. In particular, I hope he will put his obsession with independence aside and concentrate on working with the UK Government to make life better for people in Scotland."
The Glasgow Pollok MSP will now officially be sworn in as First Minister at the Court of Session in Edinburgh on Wednesday.
Want a quick and expert briefing on the biggest news stories? Listen to our latest podcasts to find out What You Need To Know...