Ghostly 3-D scans of London aim to show 'inner life' of a self-driving car

London design studio ScanLAB produced a video that uses scanning technology to show what self-driving cars 'see' as they navigate city streets.

ScanLAB Projects/The New York Times Magazine
A screenshot from a video produced by a London design studio shows a 3D scan of a car's vision as it navigates the streets of London.

3-D scanning has been used to uncover ancient architectural ruins buried deep below the ground and to make animated models — often of human subjects — for countless movies and TV shows.

Now, one London design studio wants to merge the two — using 3-D scanning technology to look closely at what self-driving cars may “see” as they become more common on city streets.

One October afternoon, architectural designers from ScanLAB Projects drove a specially-equipped Honda CR-V through the streets of London. They wanted to use the scanning technology’s imperfections — its difficulties seeing through fog, its tendency to be fooled by reflective surfaces — to capture the “inner life” of an autonomous car.

Their purpose was mainly artistic, with the resulting video showing a ghostly city filled with both precisely geometric buildings — the Houses of Parliament stand out — and blurry renderings of more natural elements.

A city bus, scanned repeatedly as it waits on a street corner, appears to take up an entire block. A group of pedestrians appears nearly frozen in time as they cross the street.

The design studio’s goal was to capture what self-driving cars may see only accidentally while cruising city streets. It's “the peripheral vision of driverless vehicles,’’ as William Trossell, an architectural designer from ScanLAB, told the New York Times Magazine, which collaborated on the video with the designers.

The designers’ work also reveals the challenges companies such as Google, BMW, Ford, and others face while testing self-driving cars, the Times’ Geoff Manaugh notes.

Their self-driving cars use a laser-scanning technology called lidar (a portmanteau of “light” and “radar”), which sends out tiny bursts of light that are invisible to the human eye, nearly a million per second, that bounce off buildings, people, and objects in the area.

It produces extremely detailed renderings of the resulting environment, down to the millimeter, making it particularly useful for architects. (ScanLAB previously collaborated with the BBC, PBS, universities, and museums to produce detailed renderings looking inside environments such as military bunkers after D-Day during World War II and beneath ancient Roman ruins.)

But lidar scanning has some limitations, particularly around mirrored surfaces, the Times notes. Some situations may be strange and unlikely, but they are also difficult for a computer to understand.

Consider the example of someone wearing a T-shirt with a stop sign printed on it, Illah Nourbakhsh, a professor of robotics at Carnegie Mellon University, told the Times.

"If they’re outside walking, and the sun is at just the right glare level, and there’s a mirrored truck stopped next to you, and the sun bounces off that truck and hits the guy so that you can’t see his face anymore — well, now your car just sees a stop sign.”

The chances of that actually occurring are highly unlikely, Professor Nourbakhsh told the paper, “but the problem is we will have millions of these cars. The very unlikely will happen all the time.’’

One cyclist in Austin, Texas, also discovered that Google’s cars can have difficulty distinguished common movements, which can lead to unusual “learning” situations.

The self-driving Lexus got to a four-way stop just before the cyclist did, he wrote on a biking forum. Because the car had the right of way, the cyclist stopped and waited, doing a track-stand to maintain his balance.

The car remained stopped, forcing him to keep balancing as he waited for it to react. Then it did something unusual — it began mirroring the biker’s movements, stopping and starting, as if seemingly paralyzed in fear of what he would do.

“The two guys inside were laughing and punching stuff into a laptop, I guess trying to modify some code to 'teach' the car something about how to deal with the situation,” he wrote.

Despite describing the situation as strange, he noted that he felt safer with the self-driving car — Google has often said they’ve taught the cars to be extra careful — than with a human driver.

Making use of those imperfections was the goal of the designers at ScanLAB, who told the Times they had turned off some of the scanning technology’s corrective algorithms — such as a tendency to truncate long periods where a car is stuck in traffic — for aesthetic reasons.

But while the videos now show what Mr. Trossell of ScanLAB calls “mad machine hallucinations,’’ it’s also possible these images can provide a glimpse into what goes on inside the minds of self-driving cars as they learn to navigate a new environment that may increasingly change to accommodate their existence.

You've read  of  free articles. Subscribe to continue.
Real news can be honest, hopeful, credible, constructive.
What is the Monitor difference? Tackling the tough headlines – with humanity. Listening to sources – with respect. Seeing the story that others are missing by reporting what so often gets overlooked: the values that connect us. That’s Monitor reporting – news that changes how you see the world.

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to

QR Code to Ghostly 3-D scans of London aim to show 'inner life' of a self-driving car
Read this article in
QR Code to Subscription page
Start your subscription today