The U.S. has begun evacuating diplomats and staff from its embassy in Kabul as Taliban stormed the Afghan capital early Sunday, according to Business Insider.

“We have a small batch of people leaving now as we speak. A majority of the staff are ready to leave,” a U.S. official told Reuters. “The embassy continues to function.”

Helicopters were photographed leaving the embassy compound. Two U.S. military officials told the Associated Press that smoke could be seen rising from the roof of the building as officials destroyed sensitive documents.

The Biden administration has deployed 5,000 extra troops to help with the evacuation operation, with embassy officials taken to Kabul airport, where they have been seen boarding military planes.

A U.S. official told CNN that the U.S. embassy would continue operating from Kabul airport with limited staff.

Only weeks ago, Biden dismissed the prospect of the Taliban taking back control of the country as highly unlikely. The collapse of Afghan security forces appears to have taken the White House by surprise. The original date for the full withdrawal of U.S. forces in Afghanistan was September 11, the 20th anniversary of the 9/11 attacks.

Taliban militants have begun entering the city “from all sides” Reuters reported early Sunday. A Taliban leader in Doha told the agency that fighters had been instructed not to perpetrate violence in the city, and allow all who wanted to flee to leave the city.

In a statement, the Taliban said it had instructed fighters to remain at the city’s gates until the transition to a new government takes place.