Liverpool came from behind twice to prevent ten-man Newcastle claiming all three points in an entertaining game at St. James Park.
Cabaye's first half strike was cancelled out by Steven Gerrard's penalty after Mapou Yanga-Mbiwa was sent off.
Newcastle moved back ahead in the second half through 22-year-old Paul Dummett only for Daniel Sturridge to claim a point for the visitors with a header in the 72nd minute.
Heading into the game Newcastle made two changes from the side that won 2-1 away at Cardiff City last time out with Hatem Ben Arfa and Yanga-Mbiwa back in the starting line up.
Liverpool also made several changes with Glen Johnson and Aly Cissokho both returning while Lucas and Jose Enrique failed to even make the bench. Luis Suarez made his 100th appearance for the club.
The opening ten minutes were fairly even, with a half chance for Liverpool the only action on goal, Cissokho was looking quite nervy on his first start for the club, while Ben Arfa looked like a threat down the right.
Daniel Sturridge was unable to challenge keeper Tom Krul after a run from Victor Moses put him through on goal while Newcastle continued to press with Loic Remy cutting in from the left.
The best chance came after 20 minutes when Luis Suarez had a header cleared off the line though it did look as if the shot was going wide, however it was Newcastle who took the lead through a brilliant long range effort from Yohan Cabaye, powering past Simon Mignolet.
Liverpool looked for a quick response but Mamadou Sakho couldn't keep his header down following a brilliant piece of control from Suarez.
Weathering Liverpool's response Newcastle then posed the biggest threat going forward as the Red's highly rated front pairing were unable to get on the ball.
Cabaye continued to threaten from around 30 yards forcing another good save from Mignolet around ten minutes before half time, though a mistake from Cheik Tiote let Suarez in forcing a classy save from Krul.
Then came what appeared to be the major turning point as a great ball from Sturridge put Suarez through, he was brought down in the box by Yanga-Mbiwa earning a straight red card from Andre Marriner and Steven Gerrard slotted home the penalty. This, his 100th Premier League goal, giving Liverpool full control until half time.
At the beginning of the second half Liverpool continued to control possession with Glen Johnson forcing a good save from Krul at the near post.
A brief period of forward momentum from Newcastle was quickly ended as Liverpool's passing game was on display probing the two lines of defence setup by the Magpies.
However against the run of play Newcastle took the lead again as young defender Paul Dummett slotted home as Liverpool again failed to cope with a set piece.
From then Liverpool were unable to put real pressure on Newcastle as Brendan Rodgers brought on Luis Alberto in place of Sakho.
The Reds continued to come forward but great defending and a lack of ideas from the front three kept the score at 2-1.
Suarez had a shot saved by Krul before the so-called SaS partnership combined for Liverpool's equaliser, Daniel Sturridge heading in Suarez's cross.
Back level, the confidence grew and Liverpool came forward with a greater purpose, Sturridge and Suarez unlucky to convert two good chances.
Newcastle continued to pose a threat on the counter-attack but Liverpool looked far more likely to score.
A final change from Rodgers saw Raheem Sterling replace the returning Glen Johnson to add more pace to the attack.
The attacks continued but it was Remy who had a great chance for a late goal for Newcastle only to be denied by a last ditch challenge from Cissokho.
In added time Liverpool were unable to put a meaningful attack together but a late, late free kick by Suarez was superbly saved by Tim Krul and Newcastle hung on for a well deserved point.
In what will be seen as a huge missed chance for Liverpool, the Reds at least moved back top of the table before Arsenal play Norwich while Newcastle move up to 9th with their point.
Write for GiveMeSport! Sign-up to the GMS Writing Academy here: https://gms.to/130seMa
DISCLAIMER: This article has been written by a member of the GiveMeSport Writing Academy and does not represent the views of GiveMeSport.com or SportsNewMedia. The views and opinions expressed are solely that of the author credited at the top of this article. GiveMeSport.com and SportsNewMedia do not take any responsibility for the content of its contributors.
Do YOU want to write for GiveMeSport? Get started today by signing-up and submitting an article HERE: https://www.givemesport.com/writeforgms