Greetings from the back seat of the world’s smartest car.
Mercedes-Benz’s autonomous research vehicle—cleverly disguised as a large black sedan—has been taking this reporter on a tour of hallowed ground. In August 1888, Bertha Benz, the wife of Karl Benz, actually stole her husband’s prototype motorwagen, then the only automobile in the world, and drove it from Mannheim, Germany, to her mother’s house in Pforzheim, some 60 miles away. It’s a whole big thing with the company…
Look! Over there! A tree! See, you have a lot of downtime in this car, because it drives itself. To be clear, it drives itself under the wary eye of at least a couple of engineers at all times, who are watching the car’s autonomous-driving operation and carefully recording errors so that they can dial in software fixes back at the lab.
A fully autonomous vehicle is at least more than a decade away. That’s one way to look at it. The other is, Holy Cow! Robot cars in a decade? Where do I sign up? Dan takes a ride in Mercedes-Benz’s autonomous research vehicle, and looks at the robo-cars of today and tomorrow. Photo: Getty Images
Eberhard Kaus, sitting in the driver’s seat, is also there to grab the wheel in case the car becomes baffled by one of the route’s many intersections, roundabouts, parked cars or random pedestrians crossing the street in Bruchsal’s Imperial district.
At the frontier of making cars drive themselves, one big technical lift is the writing of pattern-matching algorithms so the machine brain understands what the machine vision sees. As the speed of traffic increases, these algorithms must become increasingly predictive, anticipating, as humans do, the probable intention of drivers in other vehicles.
To evolve, this research needs the car to experience randomness so that it will react appropriately in even the unlikeliest situations. “We have a washing-machine test,” said Dr. Ralf Herrtwich, a leader in the autonomous-research program. “Something suddenly falls off a truck ahead of the car. Can the system react faster and better than a human?”
As of this unseasonably warm day in September, the test car has 6,500 kilometers on the odometer. It will have a million before it retires. A fully autonomous vehicle, something you’d put your mom in, is “maybe more than a decade” away, said Dr. Herrtwich, with a touch of regret.
Mercedes-Benz’s autonomous research vehicleMercedes-Benz USA
That’s one way to look at it. The other is, Holy Cow! Robot cars in a decade? Where do I sign up?
Giving automobiles auto-piloting features—up to and including completely hands-free, eyes-closed operation with trusting souls aboard—is the Space Race of global auto makers, and you are the monkey in the capsule. Last month Nissan7201.TO +0.49% and Renault chief Carlos Ghosn promised that Nissan would bring affordable autonomous cars to the public by 2020. Mercedes-Benz already markets some of its driver-assist technologies as “semiautonomous”: automatic lane keeping (positioning the car between the lines during brief periods of hands-off operation); and Stop & Go Pilot, an optics-and-radar-based cruise control that can see traffic ahead and adjust speed in heavy traffic.
“To be honest the auto industry hasn’t had any wows in a while,” said Rachel Nguyen, director of global upstream planning for Nissan Motor Company. “This is something we can shout about.”
The first tremors of self-driving hit the market in 2007, when Lexus offered an auto-parking feature that automatically parallel parks the car, a skill few human drivers possess. The technology, which piggybacks on the ultrasonic sensors and low-cost cameras already on board to help the driver maneuver in tight quarters, has moved into mass-market cars such as the Toyota Prius, Ford Escape and VW Tiguan.
As other technologies have emerged—optically-based automatic lane-keeping assist and radar-based all-speed cruise control—cars have acquired fairly robust senses that allow them to briefly and tactfully fill in for the driver, in congested traffic or during long dull spells on the highway, or in the case of inattention.
The falling cost of computer horsepower is key. Moore’s Law can be applied to the rapid growth in the speed and performance of devices based on semiconductors, and it is playing out in the cost and sensitivity of the sensor technology, said Larry Burns, former GM research chief, professor at University of Michigan and a Google GOOG +12.26%adviser. “These systems’ computational demands are enormous,” said Mr. Burns.
Not if, but how
What is being considered isn’t whether, but how to give cars more automation in limited situations, including driverless operation, meaning nobody in the driver’s seat.
Brace yourself. In a few years, your car will be able to drop you off at the door of a shopping center or airport terminal, go park itself and return when summoned with a smartphone app. Audi demonstrated such a system at this year’s Consumer Electronics Show.
At your next dinner party, ask for a show of hands of the people who’d want that.
Anybody want a car that doesn’t crash? At this month’s Frankfurt auto show, mega-auto supplier Continental announced a partnership with IBM IBM -0.25% to help bring autonomous vehicles to market, with “zero accidents” as a possible result. Volvo has promised to injury-proof its cars by 2020. GM and Carnegie Mellon aim to develop autonomous technology to eliminate car accidents.
That would be nice. In 2011, Americans were involved in more than five million auto accidents, with more than 32,367 killed. The cost to the U.S. economy in mayhem, lost productivity and lives totals $300 billion per year, according to the AAA.
Google’s fleet of auto-piloting Priuses and Lexuses is perhaps the best-known autonomous-research program, and most recognizable, thanks to the cars’ roof-mounted laser emitters on the roofs. As a harvester and sorter of titanic amounts of information, Google is in a unique position to develop, and profit from, the data-enriched 3-D virtual maps upon which autonomous cars will depend. The Google project, was led by Sebastian Thrun, former director of Stanford’s Artificial Intelligence Lab, has logged hundreds of thousands of test miles without incident.
More on Cars
For the Bertha Benz autonomous route—a lovely tour through vale and village—Mercedes-Benz joined with Nokia, whose mapping service digitally reconnoitered the path, actively pinging the world to form a reliable data map.
At the end of the autonomy rainbow is a very different relationship with the automobile than we are accustomed to. Because if we can order vehicles hither and yon, and pay for them only as needed, why own them? The costs of transportation to young people are driving them away from automobiles in historic numbers. Now imagine a car-sharing service like Uber dispatching a car, or truck, or SUV, driverlessly. (Google Ventures invested $250 million in Uber in August.) It appears in your driveway when you want it, and disappears when you don’t. Mr. Burns noted that private vehicles spend 90% of their time parked and unoccupied.
Autonomy “changes everything,” said Mr. Burns. “Giving automobiles the ability to drive themselves is the biggest thing to happen to the automobile since the automobile.”
How will it work? No one is quite sure.
The idea of self-piloting cars has been around a long time, but the vision belongs to Norman Bel Geddes, the American utopianist who designed General Motors’ Futurama exhibit for the ’39 World’s Fair in New York. In the future city (as per Bel Geddes’ sprawling set piece of miniatures), cars would move in tight squadrons safely under the control of a central traffic authority. They would be in constant radio contact with each other and the road. They would be electronically crash-proofed in a way that rules out random collisions. Speeds could rise, proximities close and road carrying capacities increase.
“Can ‘Skynet’ serve the greater good for our automobiles?”
A beautiful order would be imposed on interurban traffic, and the cascading efficiencies would include a smaller infrastructural footprint, lower per-mile energy costs and higher system productivity (this vision being from the 1930s, the era of scientific management).
As of 2013, almost everything technically necessary to enact a real-world version of the Bel Geddes dream has been invented. But the details are shaping up differently. For this first generation of autonomy, for example, cars will rely on their own wits—their own cameras, sensors and map-keeping—rather than cede control to some master computer.
But, for now
In the near-term, autonomous cars will likely operate like “human beings on steroids,” said David Strickland, an administrator of the National Highway Traffic Safety Administration. Independent of the infrastructure or traffic information, these vehicles will rely on powerful sensors (including long-range radar and thermal imaging) and onboard brains to assist the driver and, if appropriate, take the wheel. Among the first functions likely to be highly automated is highway driving, when driver workloads are low and the safety risk is fatigue. And distraction. NHTSA reports that in 2011, 17% of injury crashes (387,000) and 10% of fatal crashes were caused by driver distraction (3,331 killed), such as eating, grooming or chatting on the phone.
A camera installed on the Mercedes-Benz autonomous vehicle. Mercedes-Benz USA
The Mercedes autonomous vehicle sees the world primarily through a wide-baseline stereoscopic camera installed ahead of the rearview mirror at the top of the windshield. This camera can provide range, closing rates and direction of other traffic. The car also uses radar sensors front and rear to paint a picture of the world at long range (up to 650 feet) and midrange. Additionally, two radar sensors are installed in the front fenders to look out for cross-traffic threats. If the car approaches a green light and detects another car running the light from another direction, the car should react, avoiding, if it can, the collision and otherwise bracing the car and occupants for impact.
The car uses two other video cameras. One forward wide-angle unit helps interpret lights. “The color of traffic lights has been a challenge,” said Dr. Herrtwich. “Which traffic light is your traffic light? Also, with a wide-angle camera, the light registers only as a few pixels. The computer has to recognize that pattern in real time, 100% of the time.”
One challenge for these systems is the weather sensitivity of onboard computation. Stuffed into the trunk of the Mercedes is what looks like a mini server farm, a rack of three desktop computers and assorted power electronics to crunch the numbers. These chipsets and motherboards have to tolerate bitter cold and blazing heat. “We’d love to have the processing power of a PC,” said Dr. Herrtwich.
The focus is on two types of autonomy. One is based on vehicle-to-infrastructure (V2I) communications, rather like what Bel Geddes had in mind. Cars orient themselves, navigate, talk to each other (V2V) and avoid collisions as part of a dedicated electronic network.
“The problem with V2I is penetration,” said Dr. Herrtwich. For such systems to be even minimally effective, a large percentage of vehicles would have to subscribe, and they would rely on an infrastructure not yet engineered, built nor paid for. Beyond that, many of the promises of V2I—the fuel-saving platooning of vehicles on major highways—can be accomplished using the mobile broadband network.
“We’re bullish on both technologies,” Mr. Strickland said. “With V2I, cars we’ll be able to see around the corner and nine blocks away. Vehicles will be able to predict scenarios and avoid them. They’ll be able to deal with the chaos of traffic much faster than humans.”
The architecture for an inter-automotive network is up in the air. NHTSA has reserved 75MHz in the 5.9 GHz band of the communication spectrum for auto makers who are doing research in V2X (the X stands for either infrastructure or vehicle), known as Intelligent Transportation Systems. This technology would involve equipping cars with various kinds of black box-like devices, to communicate with other cars and the infrastructure as well as to record and share telemetry from the vehicle.
In an age of connectivity based on mobile broadband, the more disruptive notion is one in which the rolling stock is connected on the “Mobility Internet,” as mentioned in Columbia University’s 2010 Earth Institute report. This would do “for the movement of people and goods what the Internet has done for the movement of information by coordinating large amounts of real-time spatial and temporal connectivity and infrastructure data.”
That’s right, John Connor. It’s Skynet.
Regardless of safety, convenience, energy and transportation advantages of autonomous cars, are you really ready to turn over control to somebody’s science project?
Seven years ago, Nissan did a study describing to consumers over-the-horizon features of autonomous cars. “Then it was…’Oh, you mean the car is going to take control? I’m not comfortable with that,” said Nissan’s Ms. Nguyen. “Now it’s almost a complete 180 in attitude. And I think that’s because they are doing other stuff in their car,They have a personal alliance with their smartphone, and the car is the last place where the Internet is not part of their lives.”
Traffic-safety regulators in Europe and the U.S. are struggling to catch the tail of this comet. Several states, including Nevada and California, have adopted rules governing research and testing of autonomous vehicles on public roads. The policy debate to come is how far to trust our new robot chauffeurs.
“Consumer acceptance will be driven by reliability,” said Mr. Burns.
Whatever direction this technology takes, NHTSA’s director Mr. Strickland vowed, “We’ll be ready.”
The promise of autonomy is this: It doesn’t have to be perfect, at first, as long as it is better than the faulty, flimsy wetware currently occupying the driver’s seat. “Human error is behind almost 100% of car crashes,” noted Dr. Herrtwich.
Ford Focus Active Park Assist feature
The Ford Focus is the least expensive car available with a parallel-parking assistance feature. The driver switches on the system while the car is moving at low speed. The car scans for a place to park (the driver is responsible for determining legal parking). Once the ultrasonic sensors have located a space, the driver brings the vehicle to a stop, relinquishes control of the wheel and watches as the car swings into place (the driver controls the brakes and throttle). Other systems enable nose-first and tail-first parking.
Infiniti Q50 Lane Departure Prevention with Active Lane Control/Predictive Forward Collision
Infiniti’s flagship sedan is available with an optics-based lane-keeping system that will intervene with steering if it senses the car drifting out of lane. Also, the Q50 has what it calls Predictive Forward Collision Warning. This function, piggybacked on the car’s adaptive cruise-control radar system, is able to judge the relative speeds of the cars ahead, and it can alert the driver of imminent danger caused by a potential rear-end collision and intervene to lessen the impact.
Volvo City Safety
Ever egalitarian, the Swedish company Volvo has made pedestrian safety an important part of its message. The company’s Pedestrian Detection system sweeps the area ahead of the car with radar. If a pedestrian wanders into the path of the vehicle, the system will automatically brake, avoiding, or at least minimizing, pedestrian injury.
Audi Piloted Parking
At this year’s Consumer Electronics Show, Audi demonstrated a late-prototype system that would allow drivers to exit their cars and send the cars off to park themselves. Drivers could then summon their cars remotely with a phone app, to return when and where needed.