US warplanes have struck pro-Syrian government forces that were said to pose a threat to American troops and allied rebels operating near the border with Jordan.
The coalition said "apparent" Russian attempts as well as warning shots and a show of force, had failed to deter the pro-Assad forces moving toward Tanf in southern Syria, where British special forces are also based.
The Syrian Observatory for Human Rights said the strike killed eight militiamen, and a Syrian opposition media group, the Palmyra News Network, also said there were casualties from the attack.
Syrian state television said a US coalition airstrike had hit "one of our military points".
Russia has criticised the strike, with Gennady Gatilov, Russia's deputy foreign minister, said the attack was unhelpful in resolving Syria's crisis and added that civilians had been hurt, Russia's Interfax new agency reported.
There were no immediate details on any civilian injuries.
The US characterised the strike as "defensive", with US Defense Secretary Jim Mattis telling reporters: "We are not increasing our role in the Syrian civil war, but we will defend our troops."
The region - where the borders of Jordan, Syria and Iraq meet - has been considered a de-conflicted zone, under an agreement between the US and Russia.
The airstrike is at least the third time that US forces have fired on forces loyal to Syrian President Bashar al-Assad in Syria's increasingly crowded war zone.