Search and rescue efforts continued into the night for more victims of the collapse, which occurred around 10:45 a.m. when a four-story building under demolition fell onto a neighboring two-story Salvation Army Thrift Store.
"We still do not know how many people were inside the thrift store or possibly on the sidewalk" at the time of the collapse, Nutter said at a news conference.
Authorities say the cause of the collapse, which occurred at 22nd and Market streets in the heart of Philadelphia's Center City, was still under investigation.
A witness told Reuters the building collapse shook the ground and knocked a man off his feet on the sidewalk outside the thrift store.
"It was ground-shaking. The shaking of the ground made the man fall down," said Jordan McLaughlin, 18, of Philadelphia.
Police urged the public to stay away from the area while rescuers dug through the rubble.
Authorities said the building that was being demolished had housed an X-rated book and video store. They said it was owned by Richard Basciano, a well-known owner of adult entertainment properties including Philadelphia's last X-rated movie house, which closed in 2012, and a New York Times Square pornography emporium known as Show World, which closed in 2004.
Basciano did not immediately respond to a call seeking comment.
(Reporting by Dave Warner; Additional reporting by Barbara Goldberg in New York; Writing by Dan Whitcomb; Editing by David Gregorio, Steve Orlofsky and Eric Beech)