Method and System for Gender-oriented Targeted Advertising
Dmitry Mikhaylov, Timur Khabibullin, Andrey Starikovskiy, Alexander Smirnov, Mikhail Froimson,
Maxim Aristov and Vladimir Konev
Engineering Centre of the National Research Nuclear University “MEPhI”,
Kashirskoye highway 31, 115409, Moscow, Russian Federation
Keywords: Gender-oriented Targeted Advertising, Face Detection, Person Detection, Advertising Rendering.
Abstract: This paper provides an original and convenient rendering of advertisements targeted to a specific audience.
The system includes a detection unit for detecting a person, a control module connected to the information
rendering panel and to the detection unit. The control module detects age, gender and/or emotional state of a
detected person. This data is provided to the information rendering panel for selecting and displaying
advertisements appropriate for the detected person.
1 INTRODUCTION
Many conventional advertising systems have an
information rendering panel with a surface that is
used (entirely or partially) for rendering
advertisements (Ramer, 2013); (Van Datta, 2007);
(Tang, 2012); (Tung, 2011) when a presence of a
person is detected. Such systems include a detection
device for detecting a presence of a person and a
control module that connects and synchronizes the
information between rendering panel and the
detection device. These advertisement systems are
disclosed in the following publications: US
2006/024867A1 (Paik, 2006), JP 2000-112418
(Miyagi, 2000), etc.
A main common shortcoming of these
conventional systems is that the advertising is
directed towards a broad audience and is not
targeted to a particular person. This makes
advertisements significantly less effective.
Furthermore, the advertiser cannot keep track of the
advertisements that were shown to potential
customers. The advertiser prefers to pay for each
time the advertisement is shown. The conventional
systems do not allow for pay-per-show billing.
Accordingly, there is a need in the art for an
efficient and effective system for detecting a person
and rendering advertisements that are targeted
specifically for this person.
2 GENDER-ORIENTED
TARGETED ADVERTISING
The present invention deals with the system and
method for detecting a presence of a person in a
public area and providing targeted advertisement of
goods and services to the detected person according
to gender.
The information rendering panel is located in a
public place. When the advertisements are not
displayed, the information rendering panel appears
as a regular mirror and serves as a part of a room
ambience. However, once a person appears in a field
of the video camera`s view, an image is acquired
and sent to the control module (i.e., a computer) in
real-time. The control module determines personal
parameters (i.e., gender, age, race and an emotional
state) (Aristov, 2012; Pronichkin, 2013; Nosik,
2013).
Subsequently, based on the personal parameters,
appropriate advertisement is selected for rendering
to the detected person. The selection of
advertisement is made according to pre-stored data
reflecting potential (most likely) preferences of a
person of a detected age, race, gender and/or
emotional state. Then, the selected advertisement is
rendered to the person via a monitor located behind
the information rendering panel.
For example, if a potential customer is
determined to be a young woman, the advertisement
of cosmetics, make up or beauty products are
153
Mikhaylov D., Khabibullin T., Starikovskiy A., Smirnov A., Froimson M., Aristov M. and Konev V..
Method and System for Gender-oriented Targeted Advertising.
DOI: 10.5220/0004734401530158
In Proceedings of the 3rd International Conference on Operations Research and Enterprise Systems (ICORES-2014), pages 153-158
ISBN: 978-989-758-017-8
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
rendered. Yet, if a potential customer is determined
to be a middle-aged man, an advertisement of wrist
watch, beer or alcoholic beverage may be selected
and rendered. As soon as the person exits the
camera’s field of view (or based on a pre-set time
out), the rendering of the advertisement is
terminated and the information rendering panel
begins to serve as a regular mirror again.
Those skilled in the art will appreciate that a
surprise effect from a sudden advertisement
displayed from a regular mirror (once a person looks
at it) combined with an original and convenient way
of delivering the targeted advertisement messages
significantly increases the advertisement effect (and
impression) on the person. Thus, the effectiveness of
the overall advertisement campaign is increased, as
well.
Furthermore, an additional data about potential
customers are collected. For example, an aggregate
data reflecting age and gender (or race) of patrons of
a particular establishment can be used for selecting
the advertisements that are of interest to this group
of people, including new people within a camera’s
field of view.
A system includes an information rendering
panel having a mirror or similar transparent surface
that is used (entirely or partially) for rendering
advertisements upon detection of a person in a
vicinity of the system.
The system comprises a detection unit for
detecting a person and a control module connected
to the information rendering panel and to the
detection unit. The detection unit includes a video
camera. The control module detects at least age,
gender and emotional state of a detected person.
This data are provided to the information rendering
panel for displaying advertisements appropriate for
the detected person.
The control module includes a storage medium
with recorded advertisement data. A camera can be
in a form of a web-cam and the control module can
download advertisement data from a web-server.
The control module determines data related to a
detected person and stores these data for future
references. In particular, the data can be used for
analyzing the needs of potential customers and for
developing future advertising campaigns.
The information rendering panel has a mirror
surface that deflects optical rays received from the
outside. At least a portion of the information
rendering panel is implemented as a display, which
can render the advertisements upon detection a
person within a field of view of the video camera.
The information rendering panel allows through
the optical rays generated by an advertisement
source (e.g., a special monitor) located behind the
information rendering panel (or within the system).
The control module can be in a form of a computer
that runs applications for determining person’s age,
gender and emotional state based on a person’s
image(s). For example, applications developed by
agisoft.ru can be used (Agisoft, 2014).
Additionally, voice analysis alone, or in
combination with visual analysis, can be applied to
identify a person’s gender, for example, and, the
subject matter of conversation between several
people, to further target the advertisement. Also
social network data (particularly, photographs in
people’s profiles, which can be matched to the
image received by the camera) can be used to
identify the person, and to assist in targeting the
advertisement based on data in the profile. Then, the
targeted advertisement is rendered to the person via
a monitor. The system can also ask the user
questions, and, in response, provide him or her with
both advertisements and relevant information (e.g.,
from the server), based on his responses, geographic
location and identified personal characteristics.
The control module can include a storage
medium for storing pre-recorded advertisements
(i.e., for example, a hard drive of a computer). Then,
the personal data of the detected persons can be also
stored onto the hard drive by the control module.
Figure 1 illustrates a system for targeted
advertising. A media source is connected to a control
server over the Internet and includes a web-camera
with an infra-red (IR) filter for detecting persons and
acquiring their images. The IR filter allows through
only the rays from the infra-red illuminated zone
that are produced by an IR projector. The IR lighting
allows for camera to operate in low (or dimmed)
lighting and increases the accuracy of person
recognition and determination of gender, race and
age.
The media source includes a computer which has
the following applications (components) installed:
a person recognition system;
a system for loading advertisement clips from the
control server;
a billing system for billing an advertiser for the
clips shown by a rendering device.
The computer allows for classification of a person
and selection of advertisement clips in real-time, on
the order of tens of milliseconds.
The media source also includes a rendering
device, which can be implemented in a form of an
LCD or plasma display for showing advertisement
video clips to potential customers. The rendering
ICORES2014-InternationalConferenceonOperationsResearchandEnterpriseSystems
154
Figure 1: Diagram of a system for targeted advertising.
device controller selects and loads the advertisement
video clips from the computer.
The computer receives information from a sensor
panel, which acquires parameters of a person. The
sensor panel can include a microphone, an X-ray
source, a distance measuring module, a presence
detector, a smell detector, a weight platform, a
detector of ultra sound volume, an infrared-ray
measuring device, Kinect or similar systems, etc.
With additional system improvement it will be
able to determine parameters such as dress style
(brands of clothing etc.), body type, degree of
intoxication, hear style (color), presence of
cigarettes, injuries, scars, accessories (hat, glasses,
jewelry, ear rings etc.), special features (beard,
mustache, degree of shaving, etc.), mobile phone
(tablet computers, smart-phones, laptops, Bluetooth
devices, etc.), walking characteristics, presence of
pets, person’s speed of movement and a number of
people in front of the media source (a mirror).
Figure 2 illustrates a method for teaching the
targeted advertising system. In order for the
recognition system of the media source to be
effective, it needs to be trained (or taught) for
creation of accurate classifiers for determining
gender, age and emotional state of persons.
Various sets of test images (containing facial
features) need to be provided.
For example, in order to teach an age classifying
Figure 2: A method for teaching the targeted advertising
system.
module, a number of images of children’s faces,
adult faces and elderly people faces are provided
along with their classifying properties.
Subsequently, the age classifying module learns
what properties classify a particular age group. The
more test sets are processes, the more accurately the
age recognition is performed by the system. The
same training principle is applied to gender
classification, as well as emotional state
MethodandSystemforGender-orientedTargetedAdvertising
155
classification.
A teaching process is initiated in the step where a
test set of images is loaded into a recognition
system. A test set of images is normalized by a
method of principal component analysis, that allows
to reduce significantly the data dimensions by
converting the image into a set of uncorrelated
vectors called “properties”. Then, properties
corresponding to the test images are loaded into a
classifier module of the recognition system. After
that the teaching process is completed.
The teaching process can be repeated with a new
set of test images. Thus, separate teaching cycles can
be performed for gender, age and emotional state
definitions.
A recognition process, shown in Figure 3, is
initiated by detecting a person and acquiring a
person’s image. Definition of face, mouth and eyes
within the image is performed using a feature
recognition algorithm (for example, Viola-Jones
algorithm). Then facial image rotation is normalized.
Figure 3: A flow chart of a person recognition method.
Normalization is performed as an affinity
transformation, when a new coordinate system is
selected and every point from one system to another
is mapped, which rotates, moves and resizes the
image using three pairs of given points.
After the transformation, coordinates of a first
point from each pair coincide with coordinates of a
second point from this pair. Coordinates of detected
eyes and mouth are used as the first point
coordinates, while coordinates of these features
calculated for an average face (out of a test set) are
used as the second point coordinates. The average
face is positioned horizontally and the eyes are
directed forward. (Minin, 2012)
Then lighting is normalized as well, using, for
example, a known method of histogram
transformation. The facial image is decomposed into
components and after that – into a set of images that
represent certain typical areas of the facial image. A
property vector is generated and provided to a
classifier. The image is classified based on the
property vector reflecting properties defining age,
gender and emotional state. (Froimson, 2013)
Different advertisements can be provided to the
same person based on his emotional state (or,
alternatively, the same advertisements running at
different speeds, since an angry person is likely to
more positively react to a slow-moving
advertisement, while a calm person may be more
likely to want to see a fast-moving ad, or,
alternatively, different advertisements for the same
subject matter selected based on the person’s
emotional state).
A centralized system having a plurality of
interconnected media sources is implemented. The
media sources are connected over the Internet. One
or several control servers (i.e., web-servers) are used
for supplying the media sources with advertisement
media data stored in a distributed database.
The information provided by a video system can
include information on where an advertised product
can be purchased, directions and traffic. A person’s
reflection in the mirror panel of the media system
can use “added reality” virtual features. For
example, a person can be displayed wearing a hat,
glasses or a different hairstyle provided by local
vendors. Alternatively, only an audio advertisement
can be provided.
Based on personal data detected by sensors, a
person can be offered information about local food
places, pharmacy, hair salon, fashion boutique, etc.
Additionally, brand names of items that a person is
wearing can be identified (for example, many
jewelry or accessories or clothing items have a
distinctive look, or display logos, etc.), and other
items of that brand can be displayed to that person
(or, alternatively, items of direct competitors).
Video feeds from another media source (located
at another point of a shopping mall or a city) are
provided, optionally showing the nearby address of
the location where the goods or services can be
purchased.
Multiple people can be detected, and advertise
ment can be selected, for example, based on highest
likelihood of purchase, highest cost of goods being
ICORES2014-InternationalConferenceonOperationsResearchandEnterpriseSystems
156
Figure 4: Scheme of an exemplary computer system that can be used for implementation of the invention (Aboba, 2009).
advertised, highest price paid by a particular
advertiser, highest likelihood of interest to an
average person, etc.
As a further option, different advertisements can
be simultaneously displayed to viewers who see the
screen from different angles. Also, the system can
keep track of which advertisement is shown to
whom and when, including counters of how many
times the ad was shown to the same person, and
overall and optionally any reaction from the target
audience, permitting an analysis of the effectiveness
of the advertising. The system can track whether the
person is looking at the advertisement, for example,
by analyzing the eye position (as opposed to away
from the advertisement).
The media system can accept payments for
product orders using a touch screen. Payments can
be made by SMS, cash, debit or credit card, PayPass,
PayWave, PayPal, etc. The media system can
display QR-code or bar-code for reading it by a
mobile phone after which a link to a mobile site for
purchasing the product is provided. It can provide
product information via NFC and NSDT
technologies and also offer a person to provide
product-related data to his mobile phone via
Bluetooth or Wi-Fi.
With reference to Figure 4, a system for
implementing the invention comprises a general
purpose computing device in the form of a computer
or a server or the like, including a processing unit, a
system memory, and a system bus that couples
various system components as well as the system
memory to the processing unit. (Aboba, 2009; Jones,
2010; Johnson, 2013)
The media system can be placed on a bus, metro,
plane, train, elevator, or in a hotel. For any of these
locations, the system determines an advertisement
that can be of interest to an average person in the
vicinity of the media panel.
In case of a bus or a train, geo-location is taken
into consideration. A tracker determines the location
of the bus (or train) using the geo-location data from
cell stations, satellite navigation systems and Wi-Fi
access points and the advertisement may be directed
to stores, or other places based on their location.
In case of a gym or a fitness center the mirror of
the media system can be placed in front of a training
machine and, based on acquired images, a person
can be given exercise advises or advertisements of
fitness-related products (weight belts, gloves,
weights, vitamins, etc.).
3 CONCLUSIONS
The described above media system for gender-
oriented targeted advertising provides:
person detection in a vicinity of an advertising
system;
acquirement of a facial image of the detected
person;
MethodandSystemforGender-orientedTargetedAdvertising
157
provision of the facial image to a classification
module;
classification of the detected person using the
image based on pre-stored properties;
selection of an advertisement based on the
detected person’s classification;
provision of the advertisement to the advertising
system; and
rendering of the advertisement to the detected
person.
This system will improve the efficiency of
advertising in public places saving money for
advertisers.
The testing experiments and further system
improvements are underway to increase the quality
of data analysis and providing the person with
appropriate gender-oriented advertisement.
REFERENCES
Ramer, J., Soroca, A., Doughty, D. 2013, Rendering
targeted advertisement on mobile communication
facilities. United States Patent 8364521. Pub. date
01/29/2013. URL:
http://www.freepatentsonline.com/8364521.html.
Van Datta, G., Zalewski, G., 2007, Targeted advertising.
WIPO Patent Application WO/2007/041022. Pub. date
04/12/2007. URL: http://www.freepatentsonline.com/
WO2007041022A2.html.
Tang, K. 2012, Advertisement delivery system with
location based controlled priority mechanism and
method of operation thereof. United States Patent
Application 20120150853. Pub. date 06/14/2012.
URL: http://www.freepatentsonline.com/y2012/
0150853.html.
Tung, K. W., Chan, R. E. 2011, Advertising system and
method. United States Patent Application
20110071911. Pub. Date 03/24/2011. URL:
http://www.freepatentsonline.com/y2011/0071911.htm
l.
Paik, W. S. 2006, Method of applying electrical stress to
low-temperature poly-crystalline thin film transistor.
United States Patent Application 2006/024867A1.
Pub. Date 02/02/2006. URL: http://www.
freepatentsonline.com/y2006/0024867.html.
Miyagi, Y. 2000, Video appearing mirror. Japanese Patent
JP2000112418. Pub. Date 04/21/2000. URL:
http://www.freepatentsonline.com/JP2000112418.html
M. S. Aristov, D. M. Mikhaylov, L. A. Panfilov, V. N.
Konev 2012, Program for recognition of the
individual's gender in the facial image. Scientific
Session of MEPhI 2012. Abstracts. In 3 volumes. V.2
Problems of fundamental sciences. Strategic
Information Technology. Moscow, MEPhI. – 388 pp.
Page 254.
A. S. Pronichkin 2013, The system for determining the age
of the mobile device user to restrict access to content
and implementation of targeted advertising
technology. XVI International Telecommunication
Conference of Young Scientists and Students "Youth
and Science". Abstracts. In 3 volumes. V.3. Moscow,
MEPhI. – 236 pp. Page 221.
O. A. Nosik 2013, System of monitoring of psychological
state of a person based on the analysis of visual
activity. XVI International Telecommunication
Conference of Young Scientists and Students "Youth
and Science". Abstracts. In 3 volumes. V.1. Moscow,
MEPhI. – 184 pp. Page 173.
Image-based 3D modeling. Agisoft, 2014. URL:
http://agisoft.ru.
Minin P. Е. 2012, One-Shot Similarity assessment
application to the problem of gender recognition in the
image. Scientific and technical Journal “Specialized
machinery and communication”№5-6. Moscow. P 47-
53.
Froimson М. I., Мichaylov D. М., Коrsakova А
. I.,
Sorokina М. А., Коndratyev М. D. 2013, System of
real time determining of user visual direction.
Scientific and technical Journal “Specialized
machinery and communication”№3. Moscow. P 32-
34.
B. Aboba, P. Bahl, A. S. Miller, F. Sun 2009, System and
method for performing client-centric load balancing of
multiple globally-dispersed servers. Patent EP
2088744 A1. Pub. date 12/08/2009. URL:
http://www.google.co.in/patents/EP2088744A1?cl=en.
G. G. Jones, S. W. Fisher 2010, Removable wireless
keyboard with base. United States Patent US 7663602
B2. Pub. date 16/02/2010. URL:
https://www.google.ru/patents/US7663602.
Johnson, T. H., Johnson, C. E. 2013, Networked freezer
stocking management. United States Patent 8545113.
Pub. date 10/01/2013. URL: http://www.
freepatentsonline.com/8545113.html.
ICORES2014-InternationalConferenceonOperationsResearchandEnterpriseSystems
158
