SmartDeviceLink Application to Intelligent Climate Control
Oleg Gusikhin, Omar Makke, Jeffrey Yeung and Perry MacNeille
Research and Advanced Engineering, Ford Motor Company, 20300 Rotunda Drive, 48121, Dearborn, Michigan, U.S.A.
Keywords: Air Quality, Climate Control, Machine Learning, Intelligent Control, Connected Vehicles.
Abstract: SmartDeviceLink (SDL) is an open-source software development kit (SDK) that enables a smart-device to
connect to the vehicle, providing functions for safe and easy access to the vehicle human-machine interface
(HMI) and the ability to programmatically control vehicle functions. This paper discusses a framework for
developing intelligent control applications that implement personalized and context-aware features for
automotive climate control systems. There is also a discussion of integration of wearables, internet-of-things
(IoT) sensors, cloud and mobile machine learning.
1 INTRODUCTION
Connected cars and Internet of Things (IoT)
technologies are at the frontier of the automotive
industry innovation (Lu, 2014; Swan, 2015). In recent
years, significant progress has been made in the area
of brought-in connectivity. The increasing ubiquity of
smartphones created a need to provide a safe and easy
access to smartphone apps while driving. Ford Motor
Company’s SYNC
TM
Applink was the first wide scale
implementation of such technology followed by the
rest of the industry generating a multitude of different
solutions. This situation presented a significant
challenge for the app development community as it
requires adaptation of a given app to each
automaker’s interface.
To address this challenge, Ford contributed
Applink to the open source SmartDeviceLink project
to promote the development of an industry wide
standard. MirrorLink is another phone linking open
standard that has been developed in parallel with
Applink. It projects the app’s display of a MirrorLink
enabled phone to MirrorLink enabled vehicle head
unit. This standard, however; is not universally
adopted by OEMs and phone providers. For instance,
iOS does not support MirrorLink, leaving out the
iPhone users who are a substantial portion of the
addressable market.
Apple and Google also introduced their phone
projection technologies, CarPlay and Android Auto
respectively, that provide in-car access to selected
approved apps through a familiar Android and iPhone
display (Shelly, 2015). In contrast, SDL allows
seamless integration of mobile apps into the vehicle
head unit while maintaining the look and feel of the
given OEM human machine interface (HMI) design.
In addition, SDL implements the capability to
programmatically manage the vehicle sub-system
settings, including radio, climate control, navigation
and other user-configurable infotainment and
convenience features. Consequently, it provides a
powerful platform for cyber-physical systems that
integrate wearables, IoT sensors and cloud data into
the intelligent vehicle control (Smirnov, 2016). The
paper discusses the opportunities for the SDL
application with automotive climate control.
Climate comfort in the vehicle cabin is achieved
through a system of integrated heating, ventilation
and air conditioning (HVAC), either controlled
manually or automatically (Daly, 2011). The basic
HVAC system maintains a manually set temperature
and speed of air flowing from the ventilation system.
On the other hand, an automatic climate control keeps
a set temperature within the space of the cabin by
regulating a blower speed and direction based on
temperature, humidity and sun-load sensors. The state
of the art automotive climate control features multi-
zone automatic climate control. Many high end
vehicles include separate climate control for driver,
front passenger and rear passenger zones. A number
of luxury vehicles have a four-zone automatic climate
control system with infrared sensors that monitor
occupants’ surface temperature. Further advancement
of climate control systems includes the development
of air quality control. Many automakers have now
begun to install air quality sensors in the vehicle that
234
Gusikhin, O., Makke, O., Yeung, J. and MacNeille, P.
SmartDeviceLink Application to Intelligent Climate Control.
DOI: 10.5220/0005997002340240
In Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2016) - Volume 1, pages 234-240
ISBN: 978-989-758-198-4
Copyright
c
2016 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
measure different pollutants in the outside air, such as
PM
2.5
(particulate matter less than 2.5 micron in
diameter), carbon monoxide (CO), and hydrocarbons
(HC). When a high level of pollution is detected,
climate control switches to recirculation mode to
prevent polluted air from entering the cabin.
Factory-installed climate control equipment is an
automotive grade system that must meet a wide range
of requirements for safety, robustness, and
manufacturability. Automotive grade components
withstand significant physical forces, function under
a wide range of ambient conditions, and have a
lifespan substantially longer than typical consumer
devices. The ability to effectively integrate consumer
devices into vehicle control systems not only reduces
the cost of the feature, but also ensures that customers
can leverage the latest technological innovations
available in the market. For example, biometrics from
wearable devices could provide more accurate
prediction of the perceived comfort levels of the
passengers.
The IoT revolution is a game changer for the
control approach. Home climate control has seen such
changes with products like the Nest Learning
Thermostat that implement model predictive control
(MPC) algorithms. IoT technology provides
significant opportunities and challenges for
automotive control applications. Potentially
significantly advances in personalization and
intelligence are achieved more rapidly and at lower
cost than development of the automotive grade
technology. It supports the trend in mobility for on-
demand transportation and a shared car economy, as
personalization is not tied to the vehicle but stays with
a user’s smartphone and can be dynamically
integrated into vehicle control. The ability to collect
and analyse data in the cloud can provide new data
from crowdsourcing of environment information,
remote prognostics and diagnostics and provide new
insights into consumers’ behaviour and climate
control usage. However, it raises a number of
concerns regarding security, safety and robustness of
the applications that rely on external information.
SDL allows integration of IoT technology in a
safe and secure way. It also ensures that the OEM is in
control of what application is used within the system.
This paper examines the way wearable devices,
IoT sensors, personal smart mobile devices and cloud
information can be integrated into automotive climate
control using SDL. These examples show how the
IoT approach can implement intelligent control for
basic HVAC units or augment the existing automatic
climate control system with additional sensory input
and personalization. In the next section, we describe
the SDL APIs relevant to build climate control apps.
Section 3 provides an example of the intelligent
climate control integrating wearables and machine
learning. Section 4 describes air quality control
leveraging brought-in sensors and cloud data. Section
5 provides a summary and discussion of benefits.
2 SmartDeviceLink FOR VEHICLE
CLIMATE CONTROL
SmartDeviceLink is an open source project under
GENIVI Alliance. It comprises head unit software
and mobile SDKs for Android and iOS, as well as
cloud configuration. It supports several transport
protocols: Bluetooth, WiFi and USB.
SmartDeviceLink supports both media and non-
media apps. Media apps are dedicated to audio
streaming and provide alternative user interface (UI)
to the native media UI, which usually include
FM/AM/XM, and CD. Non-media apps normally
read vehicle data and provide added functionality to
the driver. The head unit defines four states called
HMI_LEVEL for each app: FOREGROUND,
LIMITED, BACKGROUND, and NONE.
When the app is selected from the head unit, it
opens a UI and is put in FOREGROUND state, which
gives the app all its permissions. Once opened, the
driver may switch to a different screen on the head
unit, such as the navigation screen, which puts the app
in LIMITED, and thus limiting some of the app’s
permissions.
An app in NONE state is idle and is only allowed
to be discovered and started. The app is put in
BACKGROUND state if its functionality interferes
with a higher priority function, such as an incoming
phone call during audio streaming, and can be treated
as a temporary NONE state for many cases. Mobile
applications can communicate with SDL core once
they implement the SDL software development kit
(SDK), which is available for Android and iOS
platforms. The SDK makes the app discoverable by
the vehicle’s head unit. It exposes a set of remote
procedure calls (RPCs) through a defined set of
application programming interface (API).
In brief, the app instantiates an instance of SDL
proxy class which handles the communication
between the app and the vehicle. The RPCs are
methods of the proxy class. Moreover, the proxy class
intercepts the vehicle’s notifications and makes them
available for the mobile app.
The proxy class also allows the app to query the
head unit for capabilities, since the app can be
SmartDeviceLink Application to Intelligent Climate Control
235
Table 1: SDL API’s for reading and writing module parameters.
API Parameters Description
getInteriorVehicleCapabilities()
Zone
Returns supported modules the vehicle is equipped
with (Radio, Climate Unit…).
getInteriorVehicleData()
Zone, Module
Reads module data. Modules are obtained from
getInteriorVehicleCapabilities.
setInteriorVehicleData()
Zone, Module,
Data
Control API. Sets the data of the Module for the
specified zone
running on different vehicles designed by different
OEMs with different capabilities. Although the RPC
implementation is not directly exposed to the app
developer, it is worth noting that the RPC protocols
are implemented as JSON strings.
A remote control extension for SDL is created and
is available in the public repository. The extension
consists of additions to SDL core inside the vehicle
and to the mobile SDK for Android and iOS. Three
major RPC’s responsible for remote control, and their
corresponding APIs are shown in Table 1. These APIs
provide enough abstraction for mobile apps to control
vehicle modules with different capabilities inside
vehicles by different OEMs.
SDL opens new opportunities to bring IoT to the
vehicle using remote control APIs. Figure 1 shows a
general configuration of how this can be achieved.
Two consumer grade sensors are brought by the
driver: the mobile device hosting the mobile
application, and the optional sensors. The sensors
may be embedded directly on the device hosting the
mobile application, such as an accelerometer or skin
temperature sensor on a wearable, or it may be a
completely separate device, such as a PM
2.5
sensor,
which can broadcast its values over Bluetooth.
Figure 1: SDL-based sensor integration.
The remote control SDK allows reading and
controlling the parameters shown in Table 2 (except
for ambient temperature which is read only) for the
climate control unit. Other modules, such as radio,
also have defined set of parameters which can be read
and controlled. For each and every parameter,
permissions to read and write to the parameters are
controlled by the OEM through a cloud server.
Table 2: SDL parameters for climate control.
Parameter Description
acEnable
Toggles AC ON/OFF
desiredTemp
User input (the set
temperature in the
head unit).
fanSpeed
Blower speed in %
currentTemp
Outside Ambient
Temperature
temperatureUnit
Unit of the
temperatures
circulateAirEnable
Air recirculation
ON/OFF
autoModeEnable
Auto mode value
ON/OFF
defrostZone
Front, Rear defrost
ON/OFF
dualModeEnable
Dual mode is
ON/OFF for units
supporting zone
control
In order to provide control over which apps can be
discovered by SDL and what the app can execute, the
OEM implements a policy table in the cloud. Each
approved app is assigned an application ID (App ID)
by the OEM. Each App ID is associated with an
explicit set of RPCs which the app can execute in each
of the four HMI_LEVEL states of the app. If the app
attempts to execute an RPC that is in incorrect state,
it will be denied. The policy table contains this
information for every approved app in the cloud.
When an SDL app connects to a new vehicle, it will
send its App ID, and the vehicle will search its local
policy table for it. If the App ID is not found, the
vehicle requests the mobile app (the mobile software
development kit specifically) to obtain encrypted
policy information specifically for the App ID. This
request is also initiated regularly to update the local
policy table for all discovered apps which give the
OEM ultimate control of permissions.
ICINCO 2016 - 13th International Conference on Informatics in Control, Automation and Robotics
236
3 PERSONALIZED CLIMATE
CONTROL
The goal of intelligent climate control is to maintain
the comfort level of the user with minimal user
interactions. Most existing automatic climate control
systems maintain a preset temperature by blowing hot
or cold air until the temperature reaches the preset
level. This mode is designed to work for the vast
majority of the population. However, user climate
comfort preferences vary based on physiological and
psychological factors and may change during driving
(Rosenfeld, 2015). In Kuang, 1995, the authors have
demonstrated the perception of comfortable
temperature correlates more with skin temperature
rather than with the ambient temperature of the cabin.
The paper describes experimental control system that
uses an IR sensor to obtain skin temperatures.
Nowadays, we can leverage many wearable devices,
such as the Seraphim Sense Angel Sensor (Seraphim,
2016), that provide a real-time reading of skin
temperature among other biometrics. SDL-enabled
application integrating wearable’s data can be written
to personalize climate settings using cloud services
and machine learning algorithms.
Figure 2: Personalized climate control.
Figure 2 shows the main components of such a
system. It has been demonstrated that neural networks
have been an efficient and effective approach to
implement complex non-linear models for
personalized climate control (Thomas, 2007; Kajino,
2000). The modern smartphone’s hardware provides
sufficiently powerful computational platform to
implement machine learning, and neural networks in
particular. Lane, 2015, discusses a feasibility of
implementation of low power consumption cloud-
free deep neural networks for smartphones for audio
processing.
Recently, Google open-sourced its TensorFlow
engine, which can be used to implement deep learning
and neural networks on many platforms, including
locally on smartphones.
The proposed system implementation diagram is
shown in Figure 3. The input includes vehicle climate
control parameters shown in Table 2 which are
obtained via SDL, combined with the skin
temperature from the wearable device, and the output
is the target temperature, which can be set via SDL.
With a proper learning model, the cabin settings could
be adjusted seamlessly to the user’s desires without
user input.
4 CABIN AIR QUALITY
CONTROL
Air quality is increasingly becoming a concern
particularly as urban areas continue to grow. It is
typically described by an Air Quality Index (AQI),
which is a measure of the concentration of various
gases and particulates in the air over a specific time
span and an indicator of the health impact of the air.
Cabin air quality can be improved through proper
management of the climate control system (Müller,
2011). For instance, if the external air quality is
poor, the vehicle should recirculate internal air.
However, recirculating air too long can cause fogging
of windows and drowsiness.
Conversely, if the internal air quality is poor, the
cabin air should be purged with external air by turning
recirculation off and increasing the air blower speed.
Since this air is processed through the vehicle’s air
filter, this will improve the air in the cabin even if the
external air is not clean.
With a growing interest in cabin air quality
management, automakers are actively seeking
implementation of internal and external air quality
sensors with the climate control system. While there is
substantial progress in the development of air quality
sensors for the consumer market, their availability for
automotive applications is currently limited. In lieu of
an embedded system, SDL offers an efficient and
effective approach leveraging brought-in consumer
sensors and cloud-based AQI data.
In the prototype, we used a sensor from
ChemiSense (ChemiSense, 2016) that detects several
analytes using proprietary sensor arrays and machine
learning, including: NO
2
, NH
3
, CO, CHOH
(Formaldehyde), Humidity, VOCs (volatile organic
SmartDeviceLink Application to Intelligent Climate Control
237
Figure 3: Architecture of intelligent climate control.
compounds); temperature with a thermistor, and
PM
2.5
with an IR smoke detector. An AQI value is
determined by calculating the indices for CO and
PM2.5. The higher value is taken as overall internal
cabin AQI.
The brought-in air quality sensor is not
necessarily tied to any particular provider, except that
the sensor must communicate with a mobile device or
the HMI with a low-energy radio such as Bluetooth
LE to reduce power draw so the device can be self-
standing. In fact, this platform can work with any
many air quality sensors, provided it meets sufficient
standards for reaction time, for specificity and
accuracy. The interface with the AQI value from the
in-vehicle sensor (and the sensor itself) built with
Applink is shown in Figure 4.
Figure 4: Air Quality Index App and Sensor.
Estimation of the air quality outside the vehicle
was done using the API provided by AirNow
(AirNow, 2016), a website operated by the U.S. EPA
to provide real-time and forecasted data on the air
quality in the U.S. This API provides the calculated
AQI for O
3
, PM
2.5
, and PM
10
. The AQI values are
also assigned a category, ranging from “Good” to
“Hazardous”.
For the purposes of this application, we are
focused on PM
2.5
, since this is of the most interest in
China. Thus, wherever AQI is concerned for the
external (outside) readings, it is based solely on
PM
2.5
. AirNow uses weather stations scattered
throughout the U.S. to provide their measurements.
The API determines the closest station to the latitude
and longitude coordinates requested, which we
determine from the GPS of our device. For
deployment in China, a similar governmental service
can be used instead. Obtaining data from AirNow or
similar government data services in other regions is
easy and reliable. However, it is not sufficiently
granular for effective air quality management.
Figure 5 provides the results of the field test
conducted in a Bay Area. During the field test we
identified 3 areas with elevated level of pollution
corresponding to poor air in an AQI chart. At
Dumbarton Bridge (1), a pungent smell similar to
rotting fish and sulfur was noticed. Near the San
Francisco International Airport (2), the air quality
worsened. In downtown San Francisco (3),
measurements were taken on a street by street basis
to formulate a true air quality map during a high-
traffic period. Note that the entire region listed as
good air quality on AirNow even though large areas
of noxious odors were observed.
One way to capture the fixed local areas of
elevated pollution is to remember GPS coordinates of
those areas during repeated drives. These coordinates
can be used to switch to recirculation. Furthermore,
combining government air quality data from the fixed
sources and crowdsourcing from the vehicles together
ICINCO 2016 - 13th International Conference on Informatics in Control, Automation and Robotics
238
with weather, traffic, maps, machine learning and
comprehensive analytics can enable development of
granular and accurate air quality maps that can be
used for air quality management. Even for vehicles
with external air quality sensors, this map can provide
significant benefits: it can provide advance warning
to close windows, start recirculation in advance vs
reacting to AQI reading that due to delay may cause
some polluted air entering the cabin, and provide an
advice on the least polluted routes.
5 CONCLUSIONS
The paper discusses the opportunities to augment
automotive climate control with a SmartDeviceLink
enabled smartphone app.
Figure 5: SF Bay Area Air Quality Test.
Such an approach leverages cloud data, IoT
sensors, and wearable devices with opportunities for
intelligent algorithms personalized to individual
drivers. This paper reviews two potential
applications: integration of wearable devices to
enable personalization of climate control and an air
quality management system based on brought-in air
quality sensors.
Such an approach leverages cloud data, IoT
sensors, and wearable devices with opportunities for
intelligent algorithms personalized to individual
drivers. This paper reviews two potential
applications: integration of wearable devices to
enable personalization of climate control and an air
quality management system based on brought-in air
quality sensors.
Integration of wearables that measure biometric
data adds a new dimension to climate control by
implementing a passive feedback loop requiring
minimal user input. Many wearable devices today
monitor skin temperature in real-time. With this data,
along with information about the current climate
control settings and external weather data, the system
can learn the user's preferences. For example, when
the user increases the cabin temperature, it may relate
to their skin temperature being cold. After this occurs
several times, the system can learn this preference and
automatically increase the cabin temperature when
the wearable detects a cold skin temperature.
Combining this with machine learning, we can create
an intelligent, personalized climate control system.
Correlation of other parameters monitored by
wearables, including metabolic rate, heart rate, stress
levels, breathing, or even blood oxygen levels can
give insight into how climate control affects patients
with chronic illnesses such as asthma.
Cabin air quality management is gaining attention
especially in regions with high pollution levels, such
as China. Despite the innovation of consumer air
quality sensors in recent years, the availability of
automotive grade sensors remains limited. The
proposed solution augments the climate control
system with consumer-grade sensors, cloud-based
information, and applications that manage air
recirculation and refresh.
SDL’s ability to dynamically integrate consumer
devices with vehicle climate control makes this
approach effective within emergent mobility models,
such as car sharing and on-demand ride services.
Passenger preferences are readily passed between
vehicles as they are carried between vehicles on the
passenger’s mobile device. With on-demand services
such as Uber or Lyft, the commuter's mobile device
can send climate comfort preferences with other
information like location and destination. The driver's
SDL enabled application would modify vehicle
climate control settings to precondition passenger
zone. A remote control feature of SDL allows the
backseat passenger to manage climate settings using
personal device directly paired with the vehicle
electronics.
Other opportunities arise from cloud connectivity
where data collected from sensors and wearables
enables cloud analytics. There is the opportunity for
crowdsourcing of air quality sensor data continuously
from vehicles at many locations to improve real-time
air quality maps. Finally, insights from climate
control usage patterns under different contexts will
SmartDeviceLink Application to Intelligent Climate Control
239
facilitate future advances leading to optimization of
climate control system design.
ACKNOWLEDGEMENTS
We would like to thank the ChemiSense team
(chemisense.co), especially Mr. Charlie Choi and Mr.
Dev Mehta, for their invaluable input in this research
project.
REFERENCES
ChemiSense. 2016. Advanced Air Quality Monitoring.
[ONLINE] Available at: http://chemisense.co/.
[Accessed 26 April 2016].
Daly, S., 2011. Automotive Air Conditioning and Climate
Control Systems. Butterworth-Heinemann, p 432.
AirNow, 2016. Environmental Protection Agency.
[ONLINE] Available at: https://airnow.gov/. [Accessed
26 April 2016].
GENIVI. 2016. SmartDeviceLink. [ONLINE] Available
at: http://projects.genivi.org/smartdevicelink/home/.
[Accessed 26 April 2016].
Kajino, Y., Sugi, H., Kawai, T., Ito, Y., Tateishi, M.,
Samukawa, K., 2000. Development of Automatic
Climate Control with Neural Control. SAE Technical
Paper 2000-01-0978, pp. 1- 6.
Kuang C. W. and Dage, G. A. An Intelligent Automotive
Climate Control System, 1995. IEEE International
Conference on Systems, Man and Cybernetics.
Intelligent Systems for 21st Century, Vol. 4, pp. 2977-
2982.
Lane, N. D., Georgiev, P., Qendro, L., 2015. DeepEar:
Robust Smartphone Audio Sensing in Unconstrained
Acoustic Environments using Deep Learning.
Proceedings of the 2015 ACM International Joint
Conference on Pervasive and Ubiquitous Computing,
pp. 283-294.
Lu, N., Cheng, N., Zhang, N., Shen, X., and Mark, J.,
August 2014. Connected Vehicles: Solutions and
Challenges. IEEE Internet of Things, Vol. 1, No. 4.
Müller, D., Klingelhöfer, D., Uibel, S., Groneberg, D. A.,
2011. Car indoor air pollution - analysis of potential
sources. Journal of Occupational Medicine and
Toxicology, 6:33.
Rosenfeld, A., Azaria, A., Kraus, S., Goldman, C. V.,
Tsimhoni, O., 2015. Adaptive Advice in Automobile
Climate Control Systems. Proceedings of the 2015
International Conference on Autonomous Agents and
Multiagent Systems, (AAMAS), Istanbul, Turkey, May
4-8, 2015, pp. 543-551.
Seraphim Sense. 2016. Angel Sensor. [ONLINE] Available
at: http://angelsensor.com/. [Accessed 26 April 2016].
Shelly, P., 2015. Addressing Challenges in Automotive
Connectivity: Mobile Devices, Technologies, and the
Connected Car. SAE Int. J. Passeng. Cars – Electron.
Electr. Syst. 8(1):161-169.
Smirnov, A., Shilov, N., & Gusikhin, O., 2016. Socio-
cyberphysical system for parking
support. International Journal of Future Computer and
Communication, 5(1), pp. 27-32.
Swan, M., Feb 2015. Connected Car: Quantified Self
becomes Quantified Car. Journal of Sensor and
Actuator Networks, Vol. 4, Issue 1, pp. 2-29.
Thomas, B., Soleimani-Mohseni, M., Jan 2007. Artificial
neural network models for indoor temperature
prediction: investigations in two buildings. Neural
Computing and Applications, Volume 16, Issue 1, pp
81-89.
ICINCO 2016 - 13th International Conference on Informatics in Control, Automation and Robotics
240