Defeat in head coach's Oscar Garcia's first league game in charge at the Amex Stadium means Brighton have lost all three of their opening Championship and cup games.
Derby, who drew their opening game at home to Blackburn, settled quickly but failed to make the most of a surging run and cross from former Watford defender Craig Forsyth in the second minute.
Ulloa, who spearheaded the Seagulls promotion challenge last season with 11 goals in 21 games, gave goalkeeper Lee Grant no chance with a stooping header following a pinpoint cross by Spanish defender Inigo Calderon.
Derby responded positively to going behind, however, and their adventurous style of play saw them pull level nine minutes later.
The Rams would take the lead with a superbly worked goal only two minutes after the break.
A storming run by Kieron Freeman down the right wing led to his excellent cross being sidefooted home by Martin at the far post.
Good work down the left wing by Forsyth set up Martin at the near post and, after his initial shot was blocked, the former Norwich striker coolly fired home into the bottom of the net from close range with the outside of his right boot.