Britain acted after claims Rwanda is funding the rebel army which is causing bloodshed across the border in the Democratic Republic of Congo
A rebel army, called M23, seized the provincial capital a week ago, defeating the hapless Congolese army.
Thousands of civilians have been forced to flee as M23 rebels in the Congo consolidated their position in Goma.
The United Nations has confirmed that a camp for internally displaced residents in the Democratic Republic of Congo has been attacked by armed men.
During the attack, which took place overnight, six women were raped and the Mugunga camp, located outside the city of Goma, looted, the UN said in a statement.
Earlier today, the camp's residents for food aid from the World Food Programme.
The attack on the camp comes amid a security vacuum in the area after the M23 rebel group's departure from nearby Goma.
Only several hundred policemen are securing the city, the Associated Press reported.
The rebels said today they would take Goma back if the Government does not agree to negotiate with them by tomorrow.
Rebel fighters, singing and brandishing weapons, pulled out of Democratic Republic of Congo's eastern border city of Goma today, raising hopes regional peace efforts could advance negotiations to end the insurgency.
The rebel withdrawal from Goma on Lake Kivu, a strategic hub in the country's war-scarred eastern borderlands, was agreed in a deal brokered by presidents of the Great Lakes states under Uganda's leadership a week ago.
After almost twenty years of close diplomatic relations, the UK's decision to withhold £21 million of aid to Rwanda signals a new direction in the relationship between the two countries.
A Rwandan minister has called the decision a "mockery", and the country's poor will be the ones likely to suffer most, with many public services previously paid for by aid.
ITV News Africa Correspondent Rohit Kachroo reports:
The Rwanda minister of finance and economic planning has called the UK's decision to withhold £21 million of aid from the country "a mockery of aid principles."
John Rwangombwa tweeted earlier this evening:
Britain has cancelled £21 million of aid to Rwanda, due to President Paul Kagame's support for rebels in the Democratic Republic of Congo.
Traditionally Rwanda has had close ties with Downing Street, but the decision comes after the United Nations asserted that Rwanda were helping fund the "M23" insurgents who have captured the city of Goma.
ITV News Africa Correspondent Rohit Kachroo reports:
In a report published today, the International Development Select Committee said aid to Rwanda should go through non-government channels. The cross-party group of MPs also said they "did not understand" why Andrew Mitchell concluded that the state was no longer supporting the M23:
Mr Mitchell has assured us that he carried out extensive consultations within the UK Government and with the government of Rwanda before making his decision.
The new Secretary of State agreed that the decision-making process had been robust.
We are not privy to all the information and advice upon which he made this assessment, but, on the basis of the other evidence we received, we do not understand how he reached the conclusion that support for the M23 had ceased.
Richard Burden, a member of the committee, welcomed the halt on £21 million of general budget aid:
People in that part of Africa are desperately poor but, in the light of the evidence given to the United Nations as well as the UK Government, it is important that aid now gets to the people of Rwanda via means other than general support for Rwandan government spending.
The Prime Minister's spokesman defended the decision taken in September to reinstate aid to Rwanda. The UK suspended the last tranche of £16 million of aid in July after the UN first highlighted the links between the Rwandan government and M23 fighters in DRC.
On his last day as International Development Secretary Andrew Mitchell controversially reinstated the aid; authorising £8 million as direct budgetary support to the government, and another £8 million to development programmes. The Prime Minister's spokesman said:
We stand by the decision that we made to release the last tranche of funding. This is our approach to the aid budget. We keep decisions under review.
The United Nations has accused the Government of Rwanda of supporting the M23 rebels operating inside the Democratic Republic of Congo in a number of reports. The UN says the "de facto chain of command" of the group culminates in Rwanda's Minister of Defence, General James Kabarebe:
The Government of Rwanda continues to violate the arms embargo by providing direct military support to the M23 rebels, facilitating recruitment, encouraging and facilitating desertions from the armed forces of the Democratic Republic of the Congo, and providing arms, ammunition, intelligence and political advice.
The United Nations has also accused the Government of Uganda of supporting the M23 by providing "direct troop reinforcements" as well as weapons, training, and other assistance.
M23 rebels tell me they are not armed by Rwanda. They say most of their weapons come from the Democratic Republic of Congo's own army, via Zimbabwe or elsewhere.