The vehicles we drive are becoming progressively more complex, with increasingly advanced technologies being added to each new generation of cars. These systems help enhance performance, efficiency and safety in our cars, but it is perhaps the convenience-focused features that are having the biggest impact, making our lives easier every day. This has led to many auto manufacturers adding an increasing number of sensors to their vehicles.
Parking sensors were first introduced nearly four decades ago. These small electromagnetic or ultrasonic sensors meant we no longer ran the risk of damaging our vehicles during slow-speed maneuvers, because they would give an audible warning before an accident happened. This was the start of the integration of these valuable sensor-based technologies.
Today, sensor-based technologies are highly prevalent: night vision systems help us see more clearly in poor light, adaptive cruise control takes the stress out of motorway driving, blind spot monitoring prevents us from making dangerous maneuvers when surrounded by traffic, park assist systems park our cars both in parallel and perpendicular spaces automatically, and driver drowsiness detection systems warn us when we should take a rest.
In-vehicle interaction
Along with these convenience and safety features, interaction inside the cabin is growing too: gesture recognition, handwriting capabilities, and voice control mean there are numerous ways to interact with the vehicle. And information can be presented in many ways, including central HMI screens, digital instrument clusters or head-up displays, and natural-sounding voice output (Text-to-Speech) making it easier to digest information.
Current, state-of-the-art systems utilize natural and varied inputs rather than stereotypical scripted commands. So, if a driver says, “it’s too hot in here,” the assistant knows to reduce the in-car
temperature. Not only this, the natural language understanding recalls what a driver has said and can understand references to things that were said in the past. So, you might ask “what’s the weather like in London?” then follow up with “how about in Manchester?” The assistant has the intelligence to follow the conversation and respond to such requests.
Interconnectivity as key to improved UX
But, as useful as these systems are, they have always worked in a very isolated, independent and use-case-centric manner. In a changing world and in a time of greater connections, where information and data are combined and fused, technology can offer a deeper suite of functionality and provide a clearer picture of our surroundings.
At Cerence, we continuously strive to integrate new intelligence into our Cerence Drive portfolio, taking data from different aspects of the car and occupants to elevate the user experience. By doing so, we are progressing beyond the isolated integration of sensors and the information they provide – bringing intelligence and semantic fusion into the cabin and making better use of integrated technologies as they become ever more connected and automated.
As Cerence continues to work on the future, we want to build systems that bring sensor and semantic fusion together into a multi-modal world – the ability to bring together information and resolve various references points between sensor data to provide valuable information to the driver. For example, the driver can look at shops and buildings and ask questions about them from within the car. The system is enabled to provide the user with useful information about the objects in question by combining the information in the user’s utterance with gaze and vehicle sensor information (e.g. positioning) and the 3D city.
As a result, Cerence Drive turns the vehicle into a mobility assistant, that includes Natural Language Understanding (NLU) and Natural Language Generation (NLG) as well as other modalities such as gaze, gesture and handwriting recognition, allowing drivers and passengers to interact with onboard systems and the environment intuitively – e.g. by looking at objects and asking about them.
Bringing the outside world on the user’s mind
Because of this, the integration of sensor and AI platforms will grow, bringing a plethora of more advanced systems that blend not only onboard sensor information, but receive and react to data from other vehicles and their surroundings. This evolutionary multi-modal and multisensory approach will merge information from the car, street, home and humans, creating a mobility assistant that goes beyond the car and can provide not only intuitive interaction between the vehicle and user but also serve as a proactive assistant, making useful offers and suggestions that fit the current driving situation. For example, considering the interaction with outside environment mentioned above, this can be expanded into a full-fledged natural form of interaction between the user and the environment including the car. The in-car mobility assistant will act as an intelligent medium in-between, that understands user’s needs and the environmental context. So, the user can point at a button in the car and ask for its functionality or look at a movie advertisement outside and ask the system to reserve tickets at the cinema close to the destination.
As autonomous vehicle technologies develop, more sensors are added to our cars, pressure is removed from the driver, and the opportunities for these systems grows yet again, opening the possibility for a myriad of new functionality.