By Dr. Holger Quast, Product Strategy Core Tech & Innovation
This summer, I was driving to a customer meeting with a colleague who had picked me up at the airport. The freeway was congested, and we were in dense stop-and-go traffic. While we were engaged in an intense conversation, we were suddenly kicked into our safety belts as the car forcefully braked to a full stop, just inches away from the car in front of us. This not only saved my colleague’s car and us from a hazardous situation on the freeway, but it also served as a strong illustration of the benefits of using sensor data and smart, fast processing to make the driving experience safer.
Multimodal, Multi-Sensory Perception of the Environment
With all of the sensors available on the inside and outside of a car nowadays – cameras, radar, ultrasound receivers and microphones, etc. – and the artificial intelligence to interpret these and act accordingly, there are actually very few, if any, situations in which an accident can not be avoided. The car has become incredibly smart, with the ability to proactively ensure driving safety. At Cerence for instance, we are using microphone arrays to detect an approaching emergency vehicle by its siren sound, determine from where that vehicle is approaching, and alert drivers, possibly before they even hear it, so they can take the appropriate action to remain safe.
Next to these huge benefits for safety, we can also boost convenience with the technology available on the outside of the car. With vehicles that are able to perceive what is happening around them, the automotive assistant can extend its reach to help its users when they are outside of the car.
Example Functions for Exterior Speech Recognition
A scenario that’s quite common for me: when picking up my kids from their hobbies, we arrive at home, they dash to the door, and I’m left picking up their sports gear. With my hands full, it would be very convenient if I could direct the car to “close the trunk” using my voice. Similarly, arriving at the car, carrying grocery bags in both hands, being able to open the trunk or doors via a voice command would be immensely helpful. You may also want to adjust the temperature of the car before entering, say, because it’s been sitting in the sun in the summer, or you want to heat it before starting your drive in the winter.
Walking towards or away from your car in a gloomy area – ask it to turn on the lights for you to illuminate the space around you. And if something does go wrong, it’ll sound the alarm and call for help if you ask for it.
With shared vehicles, you can approach the car and ask if it’s available, and handle registration, setting the destination, unlocking the car, and personalizing the environment to your user profile, all with your voice, before you even enter the car. Lastly, consider at least partially autonomous vehicles: you can ask your car to park itself or pull out of a space and come towards you so you don’t need to squeeze in between a tight spot.
All this may sound futuristic, but all ingredients of these killer functions are available today with Cerence Exterior Vehicle Interaction.
Other Functionality
With the same sensors on the car outside – microphone arrays, possibly cameras and distance sensors – it’s also possible to create other helpful features. I already mentioned Emergency Vehicle Detection above, detecting from where a vehicle with a siren is approaching.
We also offer what we call Interior – Exterior Communication, or IEC for short. If you are in your car and want to talk to someone outside, but it’s too cold/hot/unsafe/inconvenient to lower the window or step out of the car, you can use the speakers and microphones on the outside and inside of your car like an intercom. Say when you are ordering at a drive through in the rain or asking for directions. Given the right hardware setup, you can even point to a direction into which you want to speak and hear.
Privacy
Now that cars have eyes and ears, we want to make sure that these are not used for spying on the people around them.
Our voice solutions have data protection built in. All the voice functionality described above can be created to work fully offline, in the car only. No information needs to be transmitted to the cloud. We also offer design guidance to help our partners create assistants that make the use of sensors outside of the car convenient and respecting of personal privacy.
For more information on how we are creating smart solutions on the vehicle exterior for our users, check out this video about Cerence Exterior Vehicle Interaction.