XR4DRAMA Knowledge Graph: A Knowledge Graph

for Media Planning

Alexandros Vassiliades

, Spyridon Symeonidis

, Sotiris Diplaris

, Georgios Tzanetis,

Stefanos Vrochidis

and Ioannis Kompatsiaris

Information Technologies Institute, Center for Research and Technology Hellas, Thessaloniki, Greece

Keywords:

Knowledge Graph, Media Planning, Points of Interest, POI Management Mechanism.

Abstract:

In the previous two decades, knowledge graphs have evolved, inspiring developers to build even more context-

related Knowledge Graphs. Because of this development, artiﬁcial intelligence applications can now access

open domain-speciﬁc information in a format that is both semantically rich and machine comprehensible. In

this paper, we introduce the XR4DRAMA Knowledge Graph, which can serve as a representation of media

planning information. The XR4DRAMA knowledge graph can speciﬁcally represent data about the following:

(a) Observations and Events (for example, data information from photos and text messages); (b) Spatial and

Temporal data, such as coordinates or labels of locations and timestamps; and (c) Tasks and Plans for media

planning. In addition, we provide a mechanism that allows Points of Interest to be created or updated based

on videos, photos, and text messages sent by users. For improved media coverage of a remote location, Points

of Interest serve as markers to journalists.

1 INTRODUCTION

The next step toward making Knowledge Graphs

(KGs) the primary knowledge representation format

for the Web, looks to be the development of context

related KGs, i.e., KGs that can only be used in partic-

ular environments (Berners-Lee et al., 2001). In this

work, we focus on representing information for me-

dia planning, more speciﬁcally information about: (a)

Observations and Events (for example, information

from photos, and information from text messages);

(b) Spatial and Temporal data, such as coordinates or

labels of locations and timestamps; and (c) Tasks and

Plans for media planning. Information representation

in Linked Open Data format aids in the data’s reuse

and linkage with other KGs (Ehrlinger and W

oß,

2016; Villazon-Terrazas et al., 2021). When covering

a recording of a documentary in a remote location,

unknown to the media production team, a journalist

should have access to geospatial data that gives de-

tails on the location they will be visiting. This infor-

https://orcid.org/0000-0003-4569-503X

https://orcid.org/0000-0003-3170-1750

https://orcid.org/0000-0002-9969-6436

https://orcid.org/0000-0002-2505-9178

https://orcid.org/0000-0001-6447-9020

mation will enable him or her to accurately and cost-

effectively setup the production. For this reason, we

offer a tool that allows users to add or update Points-

of-Interest (POIs)

. We use the term POI manage-

ment mechanism to refer to the process for creating

and updating POIs, throughout the paper. A POI, ac-

cording to the ofﬁcial deﬁnition, is a particular place

or location point on a map that a user would ﬁnd use-

ful or interesting. In our situation, POIs also com-

prise geospatial data that includes details from user-

provided videos, photos, and text messages on the

condition of a location. As a result, POIs contain data

that can assist journalists in actual situations involv-

ing media preparation.

The XR4DRAMA KG was created, to serve as

the knowledge representation for the XR4DRAMA

project

. Through the use of many technologies,

including eXtended Reality (XR), XR4DRAMA is

committed to enhancing situation awareness. Media

planning is one of the XR4DRAMA project’s primary

use cases. In short, XR4DRAMA project stands in

three key-points: (a) Facilitating the gathering of all

necessary (digital) information for a particular, dif-

https : / / xr4drama.eu / 2022 / 07 / 07 / xr4drama - pois -

virtual-whiteboards/

https://xr4drama.eu

124

Vassiliades, A., Symeonidis, S., Diplaris, S., Tzanetis, G., Vrochidis, S. and Kompatsiaris, I.

XR4DRAMA Knowledge Graph: A Knowledge Graph for Media Planning.

DOI: 10.5220/0011621600003393

In Proceedings of the 15th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2023) - Volume 3, pages 124-131

ISBN: 978-989-758-623-1; ISSN: 2184-433X

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

ﬁcult or even dangerous situation that a media team

faces, (b) Utilizing extended reality technologies to

simulate an environment ”as if on site” in order to ac-

curately predict an event or incidence, and (c) Estab-

lishing a shared understanding of an environment and

giving users of the project’s platform (ﬁrst responders

in the ﬁeld/control room, citizens, and journalists) the

option to update representations of locations as events

change, allowing them to comprehend and re-evaluate

the effects of particular actions/decisions. As a result,

the XR4DRAMA KG may incorporate the ﬁndings

of several advanced analysis components that process

multimodal data and depict the structures they pro-

duce (in this project, for the media use case, we in-

tegrate visual and textual analysis messages). Addi-

tionally, the XR4DRAMA KG provides a innovative

method through its POI management mechanism that

may generate or update POIs, which contain essential

geospatial data that can make it easier for journalists

to cover the production recordings.

As part of their day-to-day business journalists

and other media houses produce news coverage in

various locations. Despite thorough research and

preparations, remote production planning very often

runs into challenges and difﬁculties. Many depend

on the characteristics of the individual location and

the situation on the ground. These challenges can be

circumstantial and organisational - like accessibility,

noise, the presence of people, the lack of infrastruc-

ture (from electricity supply to parking space), the

wrong choice of equipment or other ﬁlming restric-

tions. Hence, it is crucial for journalists and media

houses to access information about the state of a loca-

tion, such as the accessibility of the location, among

others, in order to plan their media coverage.

The challenge is to combine all of this informa-

tion into a coherent image to give everyone a precise

picture of the location and the situation on the ground

in order to prepare themselves for a smooth and safe

production. The XR4DRAMA KG is capable of ﬁll-

ing the aforementioned gap in distribution of crucial

knowledge to journalists, in order for them to be able

to plan more efﬁciently the news coverage.

Our contribution in this paper, is on the one hand

the XR4DRAMA KG, which can represent multi-

modal measures and let journalists report the impor-

tant elements in a location as effectively as possible.

Next, the development of the POI management mech-

anism for the XR4DRAMA KG, which can be use-

ful in real-life occasions by creating and maintaining

POIs that will further facilitate journalists’ work by

informing them of the location’s current condition.

The rest of this paper is organized as follows. Sec-

tion 2, contains the related work. The XR4DRAMA

KG, the POI management mechanism that creates or

updates POIs, and the data upon which we built the

KG are all presented in Section 3. The POI mech-

anism system and the evaluation of the KG are both

found in section 4. Lastly, we conclude our paper with

Section 5.

2 RELATED WORK

In this section we present other KGs that are found in

the area of KGs for media planning, fake news detec-

tion, media planning through disasters, and other KGs

which are related with media, media houses and news

in general.

The area of KGs for media planning is not very

rich, as only a handful of studies can be classiﬁed in

this area. For instance, the studies of Opdahl et al.

(Opdahl et al., 2016) and Berven et al. (Berven et al.,

2018) are two similar studies which present mecha-

nisms for media planning. The basic concept for both

these studies is that they offer a news extraction mech-

anism which based on the semantics of a KG, will

extract related posts from social web sites, and other

well-known media houses, about an event. This syn-

ergy of news extraction, and subsequently represen-

tation of knowledge about an event, is set to help a

journalists to see what parts of the event have been

covered, and what are the restrictions for accessing

a location to cover the event. The difference with

the XR4DRAMA KG lies mostly in the POI manage-

ment mechanism, as we offer the most crucial infor-

mation about an event in a POI, and therefore make it

more easily digestible for the journalists, while (Op-

dahl et al., 2016) and (Berven et al., 2018) return in-

formation in the form of text which can be more time

consuming for an individual to process.

The area of media planning in disasters based on

KGs is also not so rich, as to the best of our knowl-

edge, only a few studies exist in this area. KG in me-

dia are mostly used for fake news detection (Pan et al.,

2018) and building event-centric news (Rospocher

et al., 2016; Tang et al., 2019). Some exceptions

are (Wang and Hou, 2018) and (Ni et al., 2019). In

the former, the authors propose a method to construct

a KG for disaster news based on an address tree.

Address Trees, are tree structures which analyze an

address having as root the broader region. For ex-

ample, home address → town district → town, is a

small address tree. In the latter, the authors present

a data driven model which generates storylines from

huge amount of web information and proposes a KG-

based disaster storyline generating framework. For

the work of (Wang and Hou, 2018), comparing to

XR4DRAMA Knowledge Graph: A Knowledge Graph for Media Planning

125

XR4DRAMA KG there is not a mechanism for creat-

ing POIs, and the indication for the location is given

in string descriptions which can be obscure in some

cases, while XR4DRAMA KG represents locations

with coordinates. For (Tang et al., 2019), the issue

of noise in the data inserted in the KG is addressed,

an issue which is not part of the XR4DRAMA KG

(see subsection 4.1).

The following studies (Rospocher et al., 2016;

Tang et al., 2019), present methods and tools to auto-

matically build KGs from news articles. As news arti-

cles describe changes in the world through the events

they report, an approach is presented to create event-

centric KGs using state-of-the-art natural language

processing and semantic web techniques. Such event-

centric KGs capture long-term developments and his-

tories on hundreds of thousands of entities and are

complementary to the static encyclopedic information

in traditional knowledge graphs. Even though these

two studies might not solve exactly the same problem

with XR4DRAMA KG, the crucial information can

be accessed through sophisticated SPARQL queries,

which might not be user-friendly even with an UI. On

the other hand, XR4DRAMA through its POI man-

agement mechanism serves the crucial information

about an event, with a POI, which is more easily un-

derstandable by an individual.

One can have a more detailed view at KGs for me-

dia by reading the survey (Opdahl et al., 2022).

3 XR4DRAMA KNOWLEDGE

GRAPH

The project’s platform’s back-end includes the

XR4DRAMA KG. Because of this, a detailed inves-

tigation of the multimodal mapping mechanism that

accepts messages from the visual and textual analy-

sis components and transmits their content into the

XR4DRAMA KG will be skipped. But one can ﬁnd

a blueprint of these messages here

. Moreover, the

source code of the multimodal mapping mechanism

can be found here

. The idea behind the pipeline

is to map the data into the KG after the multimodal

mapping mechanism has received the message from a

component. The POI management mechanism of the

XR4DRAMA KG will then construct a new POI or

update an existing one based on the information in the

message and the information from the KG, when the

https : / / xr4drama.eu / wp - content / uploads / 2021 /

12 / d3.5 xr4drama semanticrepresentationfusiondss

20211201 v1.2.pdf

https://github.com/valexande/xr4drama-icaart-paper

message is received from the textual or visual analysis

component. In the second case, the premise is that a

generated POI’s status has changed, for instance, the

area has become crowded, necessitating an update of

the POI’s metadata. The pipeline is depicted in Fig-

ure 1, where each circled number denotes a different

step’s sequence.

Figure 1: Pipeline of the XR4DRAMA KG.

3.1 Nature of Data

The use of a semantic KG was necessary to meet

the project’s needs due to the system’s multimodal-

ity, diversity, and need for homogeneity and fusion.

There is an underlying data storage facility for this

purpose, thus the XR4DRAMA KG is not in charge

of archiving and storing raw data ﬁles. Instead, the

XR4DRAMA KG stores raw data metadata, analysis

ﬁndings, and other material with semantic value that

might be mapped and combined with other candidates

to construct the knowledge base.

The main types of information that needed to be

recorded in the XR4DRAMA KG were: general data

about virtual reality experiments, visual analysis re-

sults from images and videos, and textual analysis re-

sults derived from online retrieved content.

The visual analysis component is off-the-shelf

tool, which is result of some of our previous work

(Batziou et al., 2023). In more detail, the visual anal-

ysis is a computer vision mechanism that can identify

objects in an image or video, and also classify the im-

age into a speciﬁc category which is called verge of

the image. The Verge classiﬁer was used to assign the

photos (or video frames) to one or more classes de-

pending on context (Andreadis et al., 2020), while the

model conducting semantic segmentation (Qiu et al.,

2021) on images was trained to extract semantic la-

bels and percentages per pixel on images.

Since it was decided not to deep copy structures

from a SOLR instance

, which do not serve any re-

quirements, only a small number of the generated as-

https://solr.apache.org/

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

126

sets were chosen to be included in the XR4DRAMA

KG. These assets include the text itself, the sentences

that make up the text, and the named entity relation-

ships that may be found there.

3.2 A Knowledge Graph for Media

Planning

In this subsection, we give a high-level overview of

the KG schema’s structure (i.e., the XR4DRAMA on-

tology) and the guiding principles of each class. You

can ﬁnd the KG and the programs created to pop-

ulate it here

. A high-level overview of the main

XR4DRAMA ontology classes is shown in Figure 2.

• InformationOfInterest: The fundamental enti-

ties of interest to aid in decision-making.

• Location: The location of an event is represented

by this class. It may be displayed with the loca-

tion’s name or with its coordinates (e.g., Vicenza).

• Metadata: All the supplementary data that is pro-

vided with the analysis results, and can be used

in the decision-making process is referred to as

metadata.

• MultimediaObject: Indicates the type of the

transmission that is used to transfer information,

it can be either Audio, Textual, or Video.

• Observation: This class, which is utilized by

each individual component, describes the method

of assessing the date.

• Result: This class, represents information about

the outcome of an observation.

• Procedure: This class, represents information

about the procedures that should be taken dur-

ing an observation (i.e., tasks that should be per-

formed).

• Project: Each observation is described in this

class, along with some relevant data.

• RiskReport: This class describes the total out-

come of all risk levels derived from various com-

ponents. We added this class as some locations

can be risky to access due to various reasons, even

in a media planning scenario.

• User: Users are journalists, and each journalist is

given a unique ID.

We also analyze the purpose of the various object

type properties, i.e., properties that connect instances

from one class with instances from another class.

• hasMultimedia: This property identiﬁes if the

observation was provided via a textual, video, or

image post.

• hasResult: This property shows how the event in

the observation turned out.

• usedProcedure: This property identiﬁes the

method used to extract the information from the

observation, such as whether the information was

taken from the visual or textual analysis compo-

nent.

• hasInformationOfInterest: This property iden-

tiﬁes the most important data in an observation,

such as the type of scene, for example airﬁeld,

and the recognized objects, such as building, auto-

mobile, etc. Domain experts suggested what was

deemed important information.

• hasMetadata: Based on the type of the obser-

vation, this property indicates the observation’s

metadata. This property indicates the metadata of

the observation, based on the type of the observa-

tion. If the observation is a result of the: (a) visual

analysis component (see Table 1), and (b) textual

analysis component (see Table 3).

• hasLocation: This property shows where the im-

portant information in an observation is located.

Observe that the location is speciﬁed using lati-

tude and longitude.

• consistsOf: This property lists the observations

that make up a project. A project is a collection of

observations with neighboring coordinates.

• hasProjectLocation: This property indicates the

location of the project.

• includes: This property indicates the name of the

user that created the project.

• hasUserLocation: This property indicates the lo-

cation of the user.

• hasRiskReport: This property indicates the risk

level of the project, i.e., if it is an emergency or

not.

3.3 Point of Interest Creation and

Update

In order to make it easier for journalists and media or-

ganizations to complete a remote production mission,

POIs aim to create some points in a region (i.e., pins

on a map) that convey vital geographical information.

Any user can add or modify POIs using a phone ap-

plication (which is not currently public). The user can

either provide a picture or video that the visual anal-

ysis component analyzes, and some of the important

data in the image or video is then passed to a new

or existing POI (the pipeline for a textual message is

similar). Notice that in Figure 1, we also show that

XR4DRAMA Knowledge Graph: A Knowledge Graph for Media Planning

127

Figure 2: XR4DRAMA KG high level illustration.

the POIs receive information from the KG. Here we

analyze only the information from the messages, as

the information from the KG refers to some IDs that

relate to the projects and the observations (see subsec-

tion 3.2).

It is simple to comprehend why a POI would need

to be formed: if an event had taken place and there

were none already present in the region. The updating

of POIs, on the other hand, takes place when there are

already POIs in the region and part of the information

in them needs to be updated since the event’s state has

changed. For instance, the area has become crowded.

The data from a visual message that is sent to a POI

during creation (see Table 1) or updating (see Table

2) is shown below.

Table 1: Information passed from a visual message to a POI

when created.

Label Value Example

category string Education

subcategory string Universities

current user string journalist 1

objectsDetected list of strings [cabinet,chair]

sceneRecognition string theater

type string Point

coordinates list of ﬂoats [11.55,45.54]

Table 2: Information passed from a visual message to a POI

when updated.

Label Value Example

objectsDetected list of strings [cabinet,chair]

sceneRecognition string theater

One can notice that when a POI is created the in-

formation passed from the visual analysis messages

are: (i) what objects have been recognized, and (ii) the

label of the scene (i.e., the scene is the verge classiﬁ-

cation of the image see subsection 3.1). Since some

of the aforementioned data may be dynamic, the POI

will still be constructed even if some are absent. The

category and subcategory characterize the area which

was recognized, in our running example an Education

area, and more particularly, a University was recog-

nized. In addition, the current user is the name of

the user who sent the message. Last but not least, the

coordinates are also given in the format: [longitude,

latitude]. The current user, the category, the subcate-

gory, and the location are required pieces of informa-

tion. However, only a limited amount of information

can be altered if a POI already exists. The data which

can be updated are: (i) and (ii).

We also analyze the information from a textual

message that is passed to a POI when is created (see

Table 3) or updated (see Table 4).

Table 3: Information passed from a textual message to a

POI when created.

Label Value Example

category string Education

subcategory string Universities

current user string journalist 1

sourceText string

the theater

has become

crowded

objectsDetected list of strings [cabinet, man]

label string theater

type string Point

coordinates list of ﬂoats [11.55,45.54]

Similarly when a POI is created the information

passed from the textual analysis messages are: (i)

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

128

Table 4: Information passed from a textual message to a

POI when updated.

Label Value Example

sourceText string

the theater

has become

crowded

objectsDetected list of strings [cabinet, man]

label string theater

which are the detected objects, (ii) an auxiliary label

that characterizes the location, and (iii) the source text

of the textual message. The aforementioned data can

be dynamic, meaning that even if some are missing

the POI will still be created. The necessary data is the

current user, the category, the subcategory, and the

coordinates. On the other hand, if a POI already ex-

ists only some information can be updated. The data

which can be updated are: (i-iii).

4 EVALUATION

The XR4DRAMA KG was evaluated with two sep-

arate methods. On the one hand, we analyzed the

XR4DRAMA KG’s consistency and completeness

using two separate evaluation techniques. By de-

veloping a set of competency questions that the KG

must be able to answer with the knowledge it con-

tains (subsection 4.1), we evaluated the comprehen-

siveness of the XR4DRAMA KG. Then, we evalu-

ated the XR4DRAMA KG’s consistency by seeing if

it adhered to a particular set of SHACL constraints

(subsection 4.1). On the other hand, by calculating

the precision-recall-F1 scores utilized in information

extraction systems (subsection 4.2), the POI manage-

ment mechanism was evaluated.

4.1 Completness and Consistency of the

Knowledge Graph

Competency Questions (CQs) compiled during the

creation of the ofﬁcial ontology requirements spec-

iﬁcation document (ORSD) were used to assess

the completeness of the XR4DRAMA KG (Su

arez-

Figueroa et al., 2009). For this reason, we asked a

group of specialists to create a series of questions that

they would like the XR4DRAMA KG to answer be-

fore we built it. The experts were authority workers

from Autorita’ di bacino distrettuale delle alpi orien-

tali

and journalists from Deutsche Welle

. A total of

32 CQs were gathered, and we have included a sam-

http://www.alpiorientali.it/

https://www.dw.com/en/news/s-30701

ple of 10 of them in Figure 3. The full list of CQs may

be accessed here

Figure 3: Batch of Competency Questions.

The completeness of the XR4DRAMA KG

was found adequate, as each CQ when translated

into a SPARQL counterpart returned the desired

information. For instance, the fourth CQ from

Figure 3 when translated into a SPARQL counterpart

(see Example 1), for the observation VisualMeta-

data 2c60537511c240c9add7fb2eb4e7459e 0 re-

turned amphitheater. If the observation is visual, the

name will be created from the text VisualMetadata

(if not, TextualMetadata ) and a unique simmoid

value.

Example 1. SELECT DISTINCT ?area WHERE {

xr:VisualMetadata 2c60537511c240c9add7

fb2eb4e7459e 0

xr:hasInformationOfInterest ?info .

?info xr:hasLocation ?location .

?location xr:hasArea ?area. }

In addition to the CQs, we carried out a validation

process to examine the syntactic and structural quality

of the KB’s metadata and to verify their consistency.

Custom SHACL consistency checking rules and na-

tive ontology consistency checking, such OWL DL

reasoning, were used to adhere to the closed-world

criterion. One can ﬁnd constraint violations, such

as cardinality inconsistencies, incomplete, or miss-

ing information, by employing the ﬁrst method. By

employing the latter, the terminological semantics, or

TBox, are taken into account as validation, much like

in the case of class disjointness. Out of 56 SHACL

rules, 21 of which referenced to object type proper-

ties and 35 to data type properties, the consistency of

the XR4DRAMA KG was deemed sufﬁcient because

none of them returned any rule invalidation. We also

looked for instances that belong to the intersection of

classes because we did not want that to happen, but

none were found.

XR4DRAMA Knowledge Graph: A Knowledge Graph for Media Planning

129

4.2 POI Management Mechanism

Evaluation

The standard precision, recall, and F1-score used for

information extraction systems (Equations 1, 2 and 3)

were applied to the evaluation of the POI management

mechanism in order to create or update POIs from

visual and textual messages. The POI management

mechanism can be regarded as an information extrac-

tion mechanism, as a query is received (in our case

a message from the textual or visual analysis com-

ponents), and some information is extracted from the

KG (a POI is created or updated).

precision =

|{RelevantInstance} ∩ {RetrievedInstance}|

|{RetrievedInstance}|

(1)

recall =

|{RelevantInstance} ∩ {RetrievedInstance}|

|{RelevantInstance}|

(2)

F1 = 2 ∗

recall ∗ precision

recall + precision

(3)

Retrieved Instances are considered all the visual

(or textual) messages for which the POI management

mechanism, did not return an error when we casted a

message in order to create or update a POI.

Relevant Instances are considered all the the vi-

sual (or textual) messages for which the POI manage-

ment mechanism, managed to create or update a POI,

when we casted a message with them.

The intuition behind the retrieved and relevant

instances, is that retrieved pairs from the moment

that the POI management mechanism did not return

any error they are capable of retrieving information

(through a POI), while relevant are instance which

managed to create or update a POI and therefore con-

tain information relevant to a project.

The number of retrieved textual and visual mes-

sages are indicated by the variables Retrieved

and

Retrieved

, respectively. The numbers of relevant tex-

tual and visual messages are Relevant

and Relevant

respectively. Next, recall

, recall

, are the recall

scores for the textual and visual messages, F1

, F1

are the F1 scores for the textual and visual messages,

respectively, and precision

, precision

are the preci-

sion scores for the textual and visual messages.

One can ﬁnd the dataset used to test our POI man-

agement mechanism here

. It consists of a set of 1501

text messages and 800 visual messages. Be aware

that the values of each label in each message were

chosen at random from a gold standard dataset as-

sembled by domain experts, in order to tackle poten-

tial biases. It is interesting that all messages—textual

or visual—were considered to have been successfully

retrieved, which means that our POI management

mechanism never returned an error for any given mes-

sage, whether it was textual or visual. The resulting

values are Retrieved

= 1501 and Retrieved

= 800.

The same does not hold for the relevant messages, ei-

ther textual or visual, as there were 1376 Relevant

messages for the textual analysis component and 697

Relevant

messages for the visual analysis compo-

nent.

Table 5 contains the precision, recall, and F1

scores for both textual and visual messages based on

the aforementioned data. The results are rounded to

four decimals.

Table 5: Precision, Recall and F1-scores for textual and vi-

sual messages.

Precision Recall F1

Textual Messages 0.917 1.0 0.956

Visual Messages 0.871 1.0 0.931

5 DISCUSSION AND

CONCLUSION

The XR4DRAMA KG was founded with the goal

of assisting journalists in managing and disseminat-

ing information regarding the condition of a location,

so that they may provide the best coverage possi-

ble for events that took place there. In addition, the

XR4DRAMA KG provides a innovative mechanism,

the POI management mechanism that can create or

update POIs, which contain vital geographical data

required by journalists in order to have a clear view

of the area and plan appropriately for the coverage of

an event. The XR4DRAMA KG was constructed in

order to work as a KG that would assist journalists in

media planning.

Regarding the evaluation our goal was to examine

the POI management mechanism using the precision-

recall-F1 metrics for information extraction systems,

as well as the completeness and consistency of the

XR4DRAMA KG. CQs, which were gathered by ex-

perts, were used to assess the KG’s completeness

(subsection 4.1). More speciﬁcally, we converted

each CQ into a SPARQL equivalent, and we antici-

pated that each one would return results. This is evi-

dence that our KG may deliver signiﬁcant information

in a broader media planning scenario. A series of 56

SHACL restrict rules, of which 21 related to object

type properties and 35 to data type properties, were

used to test the consistency of the KG (subsection

4.1); none of them resulted in the rule being invali-

dated. Additionally, we looked to see whether there

ICAART 2023 - 15th International Conference on Agents and Artiﬁcial Intelligence

130

were any instances that belonged to the intersection

of the classes, but none were found. This demon-

strates the coherence of our KG, proving that it is free

of noise and contradicting information.

Our POI management mechanism received strong

F1 scores for both the visual (93.1%) and textual

(95.6%) message, demonstrating that it can be utilized

independently to create/update POIs in a broad me-

dia planning scenario. Additionally, we can comment

that this occurred when updating POIs, which means

that new information could not be added to POIs that

already existed in the area, and were missing relevant

instances for both textual and visual messages. The

updated messages were straying outside of the bound-

ing boxes of all existing POIs because each POI has

a box around it. Be aware that the POIs’ bounding

boxes are part of a broader bounding box that encom-

passes the area that requires news coverage. It seems

reasonable to take into account a bounding box for the

POIs and the area that requires news coverage; other-

wise, we risk adding POIs to the area that are situated

in a completely other area of the map.

The high Recall scores—100% for both the visual

and textual messages—can also be mentioned. This

essentially indicates that there were no textual or vi-

sual messages that indicated an error. If we examine

the two scenarios in which an error may be returned,

the reason for not doing so is pretty clear: (i) The mes-

sage’s coordinates do not fall within a bounding box

that designates the location where an event has oc-

curred, or (ii) The user will identify a non-matching

category-subcategory tuple. It is difﬁcult for the user

to choose the incorrect selection in both cases since

the user sends messages using a mobile application

(which is now private) that displays the permissible

category-subcategory tuples, and the bounding box

with a blue hue over an area.

In terms of future work, we intend to provide a

method that will make the POIs more beneﬁcial while

making decisions. Additionally, we will provide POIs

with a list of tasks that must be taken in order to com-

plete a remote production mission more accurately.

ACKNOWLEDGEMENTS

This work has been funded by XR4DRAMA Horizon

2020 project, grant agreement number 952133.

REFERENCES

Andreadis, S., Moumtzidou, A., Apostolidis, K., Gkoun-

takos, K., Galanopoulos, D., Michail, E., Gialam-

poukidis, I., Vrochidis, S., Mezaris, V., and Kompat-

siaris, I. (2020). Verge in vbs 2020. In International

Conference on Multimedia Modeling, pages 778–783.

Springer.

Batziou, E., Ioannidis, K., Patras, I., Vrochidis, S., and

Kompatsiaris, I. (2023). Low-light image enhance-

ment based on u-net and haar. In In Proceedings of the

29th International Conference on Multimedia Model-

ing (MMM 2023). Springer.

Berners-Lee, T., Hendler, J., and Lassila, O. (2001). The

semantic web. Scientiﬁc american, 284(5):34–43.

Berven, A., Christensen, O. A., Moldeklev, S., Opdahl,

A. L., and Villanger, K. J. (2018). News hunter: build-

ing and mining knowledge graphs for newsroom sys-

tems. NOKOBIT, 26:1–11.

Ehrlinger, L. and W

oß, W. (2016). Towards a deﬁnition

of knowledge graphs. SEMANTiCS (Posters, Demos,

SuCCESS), 48(1-4):2.

Ni, J., Liu, X., Zhou, Q., and Cao, L. (2019). A knowl-

edge graph based disaster storyline generation frame-

work. In 2019 Chinese Control And Decision Confer-

ence (CCDC), pages 4432–4437. IEEE.

Opdahl, A. L., Al-Moslmi, T., Dang-Nguyen, D.-T.,

Gallofr

e Oca

na, M., Tessem, B., and Veres, C. (2022).

Semantic knowledge graphs for the news: A review.

ACM Computing Surveys (CSUR).

Opdahl, A. L., Berven, A., Alipour, K., Christensen, O. A.,

and Villanger, K. J. (2016). Knowledge graphs for

newsroom systems. NOKOBIT, 24:1–4.

Pan, J. Z., Pavlova, S., Li, C., Li, N., Li, Y., and Liu,

J. (2018). Content based fake news detection us-

ing knowledge graphs. In International semantic web

conference, pages 669–683. Springer.

Qiu, W., Li, W., Liu, X., and Huang, X. (2021). Subjective

street scene perceptions for shanghai with large-scale

application of computer vision and machine learning.

Technical report, EasyChair.

Rospocher, M., van Erp, M., Vossen, P., Fokkens, A., Ald-

abe, I., Rigau, G., Soroa, A., Ploeger, T., and Bogaard,

T. (2016). Building event-centric knowledge graphs

from news. Journal of Web Semantics, 37:132–151.

arez-Figueroa, M. C., G

omez-P

erez, A., and Villaz

on-

Terrazas, B. (2009). How to write and use the

ontology requirements speciﬁcation document. In

OTM Confederated International Conferences” On

the Move to Meaningful Internet Systems”, pages

966–982. Springer.

Tang, J., Feng, Y., and Zhao, D. (2019). Learning to up-

date knowledge graphs by reading news. In Proceed-

ings of the 2019 Conference on Empirical Methods

in Natural Language Processing and the 9th Inter-

national Joint Conference on Natural Language Pro-

cessing (EMNLP-IJCNLP), pages 2632–2641.

Villazon-Terrazas, B., Ortiz-Rodriguez, F., Tiwari, S. M.,

and Shandilya, S. K. (2021). Knowledge graphs and

semantic web. Springer.

Wang, Y. and Hou, X. (2018). A method for constructing

knowledge graph of disaster news based on address

tree. In 2018 5th International Conference on Systems

and Informatics (ICSAI), pages 305–310. IEEE.

XR4DRAMA Knowledge Graph: A Knowledge Graph for Media Planning

131