It is the first ever meeting between the two sides.
Both teams are challenging for promotion from League One and League Two respectively, with Rotherham the higher team.
Rotherham currently sit top of League One, a point clear of second-placed Wigan, while Sutton - who were only promoted from the National League last term - are 10th in League Two.
Fans have travelled to London to watch the Millers' fourth appearance at Wembley in 12 years.
If Rotherham do win the trophy then they will be the first South Yorkshire side to win the same competition at both the old and the new Wembley.