Terpstra made his move 6.3km from the finish and timed it perfectly with his other main rivals failing to keep up with him.
The Dutchman finished 20 seconds ahead of second placed John Degenkolb who out sprinted last years winner Fabian Cancellara for the last podium place.
Cancellara has now placed within the top three in his last 12 Monument events.
Terpstra was delighted with the win, claiming it to be the best result of his career.
According to Sky Sports News the 29 year old said: "Of my career, it's the best day. I had some nice results, especially this year, and then to take the win in the biggest Classic of all is a dream come true.
"It's a race that suits me well. My shape this year was really good. I attacked and it was a good attack."
Terpstra who rides for Omega Pharma-Quick Step has already won the Tour of Qatar this year and can now add Paris-Roubaix to his accomplishments this year.
Team Sky's Geraint Thomas finished seventh with fellow team-mate Sir Bradley Wiggins finishing in ninth overall.
This was Wiggins' first Paris-Roubaix in three years and the former Tour de France winner admits he was disappointed with the result.
Wiggins said: "There's a tinge of disappointment because I really had the legs. Even in the final I felt strong.
"There was a part of it where I was pinching myself a bit. I don't mind admitting that. It was a real honour to be there in the final."
Thomas was happy with his finishing position but is disappointed he couldn't achieve higher.
The Welshman said: "It's nice to get a top 10 again. The way I rode it wasn't the easy way. We were out front for a lot of the time. It's satisfying but at the same time we wanted to get a podium."
Three of the top 10 riders were from the Omega Pharma-Quickstep team, with Zdenek Stybar finishing fourth and Tom Boonen tenth.
Paris-Roubaix is supposed to be the hardest of the one day races due to its many cobbled sections. This years race had 51km of cobbles throughout the 257km course.
The cobbled sections are good practise for the Champs-Elysees at the end of the Tour de France but with so many sections of cobbles throughout the race it makes the course extremely challenging.
Write for GiveMeSport! Sign-up to the GMS Writing Academy here: http://gms.to/1a2u3KU