Martin Demichelis and substitute Martin Palermo scored the goals to break Greece's resilience and end their World Cup hopes as Argentina recorded a 2-0 win at the Peter Mokaba Stadium.
Bayern Munich centre-back Demichelis fired home from close range in the 77th minute after his header from Lionel Messi's corner bounced back off team-mate Diego Milito.
Boca Juniors striker Palermo tapped in the rebound from Messi's shot to complete the victory a minute from time.
It sent Argentina into the knockout phase to face Mexico full of confidence on the back of three successive wins as Greece's best defensive efforts ultimately proved to be in vain. South Korea's 2-2 draw with Nigeria put them through as Group B runners-up.
For long periods Diego Maradona's side were frustrated by Greece, for whom defending is what they do best - having used the tactic to become surprise European champions in 2004.
Alexandros Tzorvas almost undid all Greece's hard work when he spilled Milito's low cross into the path of Sergio Aguero but Loukas Vyntra charged down his shot at the far post, but he redeemed himself in first-half added time when he saved from Liverpool midfielder Maxi Rodriguez and Messi in quick succession.
Three minutes into the second half Greece almost snatched the lead when Georgios Samaras beat Demichelis to another long ball out of defence and raced into the penalty area but drove wide of far post with his left foot - Greece's first shot of the match.
But injuries were not kind to Rehhagel's side as they lost Georgios Karagounis, Konstantinos Katsouranis and Vasileios Torosidis all within 10 minutes of the restart.
And Argentina took the lead when Demichelis drilled into the roof of the net with 13 minutes to go.
Things opened up after that and Messi smashed a left-footed shot off the post having weaved his way into the penalty area. His next effort was more productive as Tzorvas parried his strike into the path of the 36-year-old Palermo who took full advantage.
Copyright (c) PA Sport 2009, All Rights Reserved.
Do YOU want to write for GiveMeSport? Get started today by signing-up and submitting an article HERE: https://www.givemesport.com/writeforgms