The fire erupted shortly before 6 p.m. Monday and was mostly burning in the Angeles National Forest, away from populated areas. It was 5 percent contained.
Winds were light, allowing smoke and ash to settle on communities across the San Gabriel Valley.
Pollution regulators issued a smoke advisory for the valley and the San Bernardino Mountains to the east, saying areas directly impacted may experience air quality in the "unhealthy" range or higher.
"We're urging older people and people with asthma to stay inside, not exert themselves," said Nathan Judy, a fire spokesman for the U.S. Forest Service.
Evacuation advisories were lifted early Tuesday for residents of three foothill homes closest to the blaze, said Shawn Rollman with the Forest Service.
Although the fire was burning away from that area and into the forest, those homes and others in Azusa could be threatened if the winds change direction and push it to the south, officials said.
Rollman said some 450 firefighters from the Forest Service and Los Angeles County battled the flames. The fire's cause wasn't immediately known.
Super Scooper airplanes and water-dropping helicopters bombarded the fire with water taken from the nearby San Gabriel Reservoir.
The fire was burning in terrain east of the 250-square-mile area of the Angeles National Forest devastated by the 2009 Station Fire.
Although temperatures in the 80s were expected, weather conditions, overall, were good for firefighting. Winds were blowing at only about 5 mph, helping slow the flames' progress.
Forecasters said an upper-level, low pressure area moving into the Pacific Northwest Tuesday night and Wednesday would cause an increasing flow of cool, moist air from the ocean onto the land.