How Google researchers turned millions of Flickr images into time-lapse videos

Researchers at Google and the University of Washington mined 86 million publicly available images to create time-lapse videos of famous landmarks. The team's code sorts the images by date, stabilizes them, and normalizes their perspectives and aspect ratios.

You’ve probably seen time-lapse videos in which filmmakers placed a camera in a particular spot and recorded footage for hours or days before compressing the passage of time, revealing a flower blooming, a city waking up and beginning to bustle with activity, or the stars appearing to move across the night sky. Now, researchers at Google and the University of Washington are producing crowdsourced time-lapse videos, assembled from more than 86 million public pictures posted online, on sites such as Flickr and Picasa.

The researchers started by grouping more than 86 million photos by location. Some monuments and landmarks are photographed thousands of times per year by visitors, so the researchers took those locations and sorted the photos from oldest to newest. Then they used computers to slightly distort the perspective of the photos, giving them the appearance of having all been taken from the same spot. Finally, the researchers smoothed out the images so that when they were played back in order they gave the illusion of having been taken at perfectly regular intervals.

The results, some of which are available on the YouTube channel of the project’s lead researcher, Ricardo Martin-Brualla, show glaciers advancing and retreating, foliage blooming and dying with the seasons, and waterfall streams emerging and drying up. They also show new construction projects, such as the Goldman Sachs Tower in Manhattan, being built; and other buildings, such as the Basilica of St. Maria of Salute in Venice, being renovated. One time lapse even shows the Swiss Guard posted outside a door at the Vatican, standing in precisely the same spot day after day, year after year.

The researchers said they were able to assemble more the 20,000 time lapses from the photos mined. Each time lapse is made up of more than 1,000 pictures. But not all of the videos were successful. Some were blurry in areas of significant activity, such as buildings that were constructed very quickly. Others showed a strange twilight effect when daytime and nighttime photos were mixed together. And in cases when almost the entire photographed landscape shifted over time, such as a moving glacier, the stabilizing algorithm had a tough time keeping a single vantage point.

Most of the photos came from landmarks in North America and Europe, since those areas are heavily photographed. Comparatively few came from Africa and South America. The team said each time lapse took about six hours to assemble on a single computer. They plan to release the project code to the public so others can create their own time lapses. As more and more photos are posted online, time lapses can reveal changes to natural and manmade landscapes in even greater detail, and over longer periods of time.