In the absence of Robin Van Persie and the injured Thomas Vermaelen, the 29-year-old Spaniard was handed the armband for the Emirates Stadium encounter.
"It was really pleasing for me to be captain for a club like Arsenal, it's unbelievable," Arteta told Arsenal Player. "I just want to thank my teammates for the effort, my manager for the support and hopefully we can build on this and win against Swansea.
"We had many chances to finish it off much earlier but it seems to be like it a lot this year at the Emirates. At least we won, we are through and it was a good result after the disappointment of the Fulham game."
Arteta reserved praise for his teammates' 'never-say-die' attitude, which helped them progress to the Fourth Round with a 1-0 victory, courtesy of a Thierry Henry strike on his emotional return to the club.
However, the midfielder also warned that the players need to take a more incisive approach in future fixtures, particularly when Arsenal have the home advantage.
"It is [frustrating] because if you score two goals in the first half it makes it much easier," he added. "It becomes harder to control a game [when it remains 0-0].
"Everybody is getting a bit anxious to score and it makes it harder. But we were really patient and in the end made the right decision to win the game."