Up to 30,000 passport applications have been hit by delays, David Cameron has said.
The Prime Minister insisted that hundreds of extra staff have been drafted in and new offices are being opened in Liverpool next week in to help tackle the backlog.
Ed Miliband called on the premier to "get a grip" as he warned that thousands of people face cancelled holidays as a result of the problems.
But Mr Cameron accused the Labour leader of trying to "frighten people".
He said: "We have 300,000 extra applications than is normal at this time of year. We have increased massively the staff. The level of applications outside the normal three-week limit is less than 10% of that 300,000."
Unions have claimed more than 500,000 applications are waiting to be dealt with and say delays in the service are down to the loss of 300 jobs and the closure of 20 passport offices over the past five years.
Photographs taken by anonymous members of the passport agency's staff have emerged that are claimed to show boxes of passport applications stacked up and waiting to be dealt with in the Liverpool office.
Earlier today Home Secretary Theresa May said the Passport Office has extended its hours to open from 7am until midnight, while staff are working longer hours and more days of the week to tackle the surge in applications.
Mr Cameron said he understood that people were "anxious" about delays but insisted the Home Office has "been on this from the very start".
Some 250 staff have been redeployed to frontline passport services who were prioritising all outstanding applications, which would allow for an extra 25,000 application examinations every week, MPs were told.
An extra 650 workers have been drafted in to work on customer helplines.
Mr Miliband said applicants were being forced to pay extra to try to fast-track their applications.
"To add insult to injury people are being told if they want their applications processed in the three-week target they have to pay £55 extra."
The Labour leader accused Mrs May of being too busy battling against Education Secretary Michael Gove to focus on the passport problems.
He said: "The truth of the picture of this Government is we have the Home Secretary fighting with the Education Secretary but not paying attention to the business of Government."
"The truth is that is tens of thousands of people are finding that their holidays are being cancelled because they are not actually getting a passport," he added.
"He says they have increased the resources at the passport agency - that is not the case. There are greater responsibilities for the passport agency since 2010 and fewer resources."