Culture, Media and Sport Secretary Jeremy Hunt said he was "thrilled" about England's victory and hoped they will continue to improve in a "quiet, British way".
After a speech at London's Victoria and Albert Museum, Mr Hunt said: "I watched bits of it (England's 1-0 victory over Ukraine). Obviously I'm absolutely thrilled and keeping fingers firmly crossed.
"England seems to do best when we don't have the highest expectations so I think it's best not to ramp up expectations too much. Hopefully we'll continue to go from strength to strength in a sort of quiet, British way."