Nathaniel Clyne's long-range stunner gave Southampton a deserved victory over an Arsenal side whose fringe players did not take their opportunity in a fast-paced game at the Emirates Stadium on Tuesday night.
Ronald Koeman's side came from behind to advance after Alexis Sanchez's early free-kick gave the hosts a lead, before Dusan Tadic levelled matters from the penalty spot.
With just five minutes left until half-time, a Southampton corner was cleared only as far as Clyne and the right-back took one touch before releasing a swerving effort that flew past David Ospina.
Action-packed first half
Both sides came into the game on the back of victories in their last Premier League matches, and the confidence showed as the football flowed throughout the opening exchanges. Arsene Wenger made ten changes to his side from the weekend, whilst Ronald Koeman made only five.
It was the much-changed home side who took the lead after 14 minutes however, when Alexis Sanchez curled a peach of a free-kick over the wall and into the top left-hand corner of Fraser Forster's goal, giving the England goalkeeper no chance in the process.
The lead did not last long though, as Dusan Tadic restored parity from the penalty spot after Southampton debutant Sadio Mane was brought down in the area by Tomas Rosicky.
Then came Clyne's goal, which deserved to win any game. A Steven Davis corner was headed only half-away by the Arsenal defence and Clyne unleashed a vicious strike which flew past Ospina and into the net to give the away side the lead at the break.
Arsenal controlled the ball in the second half, but it was still the visitors who looked more likely to nick what would have been a third and decisive goal.
After a neat move down the Southampton left, Davis' prodded effort just did not have the legs to beat the Arsenal stopper who got down well to his right to save.
Sanchez then drew a fine point-blank save from Forster after good work down the Arsenal right, but the Saints threatened to punish the home side on the counter.
Mane flashed an effort wide of the far post, before Graziano Pelle headed over from eight yards with a free header.
Shane Long could not quite get on the end of Dusan Tadic's fizzed cross at the death, but neither side added to the scoreline and Southampton advanced to the Capital One Cup fourth round with a memorable first win at the Emirates.
Southampton player ratings
Forster - 7
Clyne - 8
Fonte - 7
Gardos - 6
Targett - 6
Wanyama - 6
Schneiderlin - 8
Tadic - 7
Davis - 7
Mane - 7
Pelle - 7
Long - 6
Bertrand - NA