Medea Benjamin, co-founder of Code Pink, told The Associated Press that police entered the embassy early in the morning. A State Department spokesman said authorities arrested and removed four people from the embassy.
About noon, a group of police cars drove away from the building. A video posted on social media by Code Pink showed Adrienne Pine, one of the four protesters, in the back of the police car, saying "this is an illegal order that (the police) are following."
Mara Verheyden Hilliard, a lawyer for the activists, said in a statement that they were charged with interference with certain protective functions.
The protesters consider Nicolás Maduro to be the legitimate Venezuelan president. But the United States and more than 50 other countries say Maduro's recent reelection was fraudulent and are backing congressional leader Juan Guaidó's claim to the presidency.
Guaidó's newly named ambassador had requested the help of U.S. authorities in clearing the building. Shortly after the arrests, Guaidó tweeted, "The process of recovery of our embassies around the world has started.
"We recovered this building," Guaidó's envoy Carlos Vecchio said during a press conference held across the street from the embassy. "Next one will be Miraflores" - the presidential palace in Caracas.
Vecchio said he expected to gain access to the diplomatic compound as soon as law enforcement ends an ongoing security search. He said he plans to use part of the facility as storage for humanitarian aid to be shipped later.
Vecchio also said his team has secured control all six diplomatic buildings owned by the Republic of Venezuela in U.S. territory, five in the nation's capital and the consulate in New York City.
The protest started more than a month ago with at least 30 activists staying at the embassy, but their numbers gradually dwindled. The building has been without power since last week and a crowd of Guaidó supporters has frequently gathered to heckle the protesters from the street.
The Maduro government criticized the law enforcement action, with Foreign Minister Jorge Arreaza tweeting that the United States was violating its international obligations and the right of the activists who had "protected" the embassy with "our authorization."
Maduro appearing on state TV said that in contrast to authorities in Washington, he has ordered security to be beefed up at the U.S. embassy in Caracas to protect the property belonging to the United States government.
The building is empty, last diplomatic staff pulling out over a month ago citing a breakdown in relations between the two nations.
"We are going to protect it even more because Venezuela complies with international conventions and international law," Maduro said of the U.S. embassy, calling inhabitants of the White House "brutes" and "criminals."
Associated Press writers Colleen Long and Matthew Lee in Washington, and Scott Smith and Jorge Rueda in Caracas, Venezuela, contributed to this report.
Follow Luis Alonso Lugo on Twitter: http://www.twitter.com/luisalonsolugo
Copyright 2019 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.