But neither of them have given the 32-year-old a harder time than Dr Steelhammer.
They came up against each other five years ago at Wembley Stadium in April 2017.
AJ made the shock admission when asked the question during a Q&A at the Oxford Union.
He said: “The hardest opponent I have come up against would be Wladimir Klitschko definitely.
“The passing of the guard. The young lion vs the old lion. At the time I fought him I thought it was definitely too early but it was his last fight.
“So if I didn’t fight him then it would have been too late, so it was risk vs reward and I thought boxing needed it. Sometimes due to a lack of experience we make it harder than it need to be. He had definitely more knockouts on his record than I have fights and knockouts combined.
“The guy is very experienced and very strong and he gave me a tough fight.
“Before that stage I was knocking guys out within six rounds and seven rounds. Wladimir took me eleven rounds somewhere I had never been before but he was my toughest for sure.”
Joshua was also questioned about his biggest inspiration starting out – and he revealed his huge respect for the Ukrainian, even if he admitted he wasn’t everyone’s cup of tea.
He added: “Wladimir Klitschko was one of the guys that I looked up to when I first started because it’s not only wanting to be a champion, it’s the responsibilities that come with being a champion, and I felt like he educated himself.
“He spoke different languages, he wore suits and stuff, so when I was coming through as a young man, I didn’t want to have big chains, not educating myself properly, talking in a certain way.
“So he inspired me. People used to say, ‘Oh, Joshua’s fake, he’s this, he’s that’. For wanting to better myself?
“But you stay strong through the criticism because I knew the guy that I looked up to is someone that inspired me.
“Maybe not inspiring for someone else but I looked up to him. My idol became my rival but we have a great relationship still. Anyone coming up I’d say be mindful about the position that you’re in for sure.
“I’ve made mistakes coming up, sometimes saying the wrong thing, being in the wrong places.
“But it’s always from a good place at the end of the day and it’s part of the responsibility for being a champion.”