loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Affiliation

Council for Scientific and Industrial Research, South Africa
Context (please use complete words only)
Field
Sort by

15 Papers

Belief Re-Use in Partially Observable Monte Carlo Tree Search

Topics: Planning and Scheduling; Uncertainty in AI ; State Space Search; Autonomous Systems

Authors: Ebert Theeuwes 1 , Gabriele Venturato 1 and Gavin Rens 2

Affiliations: 1 KU Leuven, Dept. of Computer Science, B-3000 Leuven, Belgium , 2 Computer Science Division, Stellenbosch University, Stellenbosch, South Africa

Digital Object Identifier: 10.5220/0013265200003890

Proceedings of ICAART 2025, ISBN: 978-989-758-737-5

Collapse Button Quick Abstract
Partially observable Markov decision processes (POMDPs) require agents to make decisions with incomplete information, facing challenges like an exponential growth in belief states and action-observation histories. Monte Carlo tree search (MCTS) is commonly used for this, but it redundantly evaluates identical states reached through different action sequences. We propose Belief Re-use in Online Partially Observable Planning (BROPOP), a technique that transforms the MCTS tree into a graph by merging nodes with similar beliefs. Using a POMDP-specific locality-sensitive hashing method, BROPOP efficiently identifies and reuses belief nodes while preserving information integrity through update-descent backpropagation. Experiments on standard benchmarks show that BROPOP enhances reward performance with controlled computational cost.

Proposing Hierarchical Goal-Conditioned Policy Planning in Multi-Goal Reinforcement Learning

Topics: Task Planning and Execution; Cognitive Robotics

Author: Gavin Rens

Affiliation: Computer Science Division, Stellenbosch University, Stellenbosch, South Africa

Digital Object Identifier: 10.5220/0013238900003890

Proceedings of ICAART 2025, ISBN: 978-989-758-737-5

Collapse Button Quick Abstract
Humanoid robots must master numerous tasks with sparse rewards, posing a challenge for reinforcement learning (RL). We propose a method combining RL and automated planning to address this. Our approach uses short goal-conditioned policies (GCPs) organized hierarchically, with Monte Carlo Tree Search (MCTS) planning using high-level actions (HLAs). Instead of primitive actions, the planning process generates HLAs. A single plan-tree, maintained during the agent’s lifetime, holds knowledge about goal achievement. This hierarchy enhances sample efficiency and speeds up reasoning by reusing HLAs and anticipating future actions. Our Hierarchical Goal-Conditioned Policy Planning (HGCPP) framework uniquely integrates GCPs, MCTS, and hierarchical RL, potentially improving exploration and planning in complex tasks.

Learning Optimal Behavior in Environments with Non-stationary Observations

Topics: Machine Learning; Planning and Scheduling; Uncertainty in AI

Authors: Ilio Boone and Gavin Rens

Affiliation: DTAI group, KU Leuven, Belgium

Digital Object Identifier: 10.5220/0010898200003116

Proceedings of ICAART 2022, ISBN: 978-989-758-547-0

Collapse Button Quick Abstract
In sequential decision-theoretic systems, the dynamics might be Markovian (behavior in the next step is independent of the past, given the present), or non-Markovian (behavior in the next step depends on the past). One approach to represent non-Markovian behaviour has been to employ deterministic finite automata (DFA) with inputs and outputs (e.g. Mealy machines). Moreover, some researchers have proposed frameworks for learning DFA-based models. There are at least two reasons for a system to be non-Markovian: (i) rewards are gained from temporally-dependent tasks, (ii) observations are non-stationary. Rens et al. (2021) tackle learning the applicable DFA for the first case with their ARM algorithm. ARM cannot deal with the second case. Toro Icarte et al. (2019) tackle the problem for the second case with their LRM algorithm. In this paper, we extend ARM to deal with the second case too. The advantage of ARM for learning and acting in non-Markovian systems is that it is based on well-understood formal methods with many available tools.

Online Learning of non-Markovian Reward Models

Topics: Machine Learning; Planning and Scheduling; Model-Based Reasoning

Authors: Gavin Rens 1 , Jean-François Raskin 2 , Raphaël Reynouard 2 and Giuseppe Marra 1

Affiliations: 1 DTAI Group, KU Leuven, Belgium , 2 Université Libre de Bruxelles, Belgium

Digital Object Identifier: 10.5220/0010212000740086

Proceedings of ICAART 2021, ISBN: 978-989-758-484-8

Collapse Button Quick Abstract
There are situations in which an agent should receive rewards only after having accomplished a series of previous tasks, that is, rewards are non-Markovian. One natural and quite general way to represent history- dependent rewards is via a Mealy machine. In our formal setting, we consider a Markov decision process (MDP) that models the dynamics of the environment in which the agent evolves and a Mealy machine synchronized with this MDP to formalize the non-Markovian reward function. While the MDP is known by the agent, the reward function is unknown to the agent and must be learned. Our approach to overcome this challenge is to use Angluin’s L∗ active learning algorithm to learn a Mealy machine representing the underlying non-Markovian reward machine (MRM). Formal methods are used to determine the optimal strategy for answering so-called membership queries posed by L∗. Moreover, we prove that the expected reward achieved will eventually be at least as much as a given, reasonable value provided by a domain expert. We evaluate our framework on two problems. The results show that using L∗ to learn an MRM in a non-Markovian reward decision process is effective.

A Modal Logic for the Decision-Theoretic Projection Problem

Topics: Knowledge Representation and Reasoning; Uncertainty in AI ; Knowledge-Based Systems

Authors: Gavin Rens 1 , Thomas Meyer 1 and Gerhard Lakemeyer 2

Affiliations: 1 University of KwaZulu-Natal and CSIR Meraka, South Africa , 2 Aachen University, Germany

Digital Object Identifier: 10.5220/0005168200050016

Proceedings of ICAART 2015, ISBN: 978-989-758-074-1

Collapse Button Quick Abstract
We present a decidable logic in which queries can be posed about (i) the degree of belief in a propositional sentence after an arbitrary finite number of actions and observations and (ii) the utility of a finite sequence of actions after a number of actions and observations. Another contribution of this work is that a POMDP model specification is allowed to be partial or incomplete with no restriction on the lack of information specified for the model. The model may even contain information about non-initial beliefs. Essentially, entailment of arbitrary queries (expressible in the language) can be answered. A sound, complete and terminating decision procedure is provided.
Collapse Button Related Ontology Subjects/Areas/Topics (5)
Artificial Intelligence

Knowledge-Based Systems

Uncertainty in AI
Knowledge Representation and Reasoning

Symbolic Systems

Hybrid POMDP-BDI - An Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels

Topics: Agent Models and Architectures; Task Planning and Execution

Authors: Gavin Rens and Thomas Meyer

Affiliation: University of KwaZulu-Natal and CSIR Meraka, South Africa

Digital Object Identifier: 10.5220/0005185000050014

Proceedings of ICAART 2015, ISBN: 978-989-758-073-4

Collapse Button Quick Abstract
Partially observable Markov decision processes (POMDPs) and the belief-desire-intention (BDI) framework have several complimentary strengths. We propose an agent architecture which combines these two powerful approaches to capitalize on their strengths. Our architecture introduces the notion of intensity of the desire for a goal’s achievement. We also define an update rule for goals’ desire levels. When to select a new goal to focus on is also defined. To verify that the proposed architecture works, experiments were run with an agent based on the architecture, in a domain where multiple goals must continually be achieved. The results show that (i) while the agent is pursuing goals, it can concurrently perform rewarding actions not directly related to its goals, (ii) the trade-off between goals and preferences can be set effectively and (iii) goals and preferences can be satisfied even while dealing with stochastic actions and perceptions. We believe that the proposed architecture furthers the theory of high-level autonomous agent reasoning.
Collapse Button Related Ontology Subjects/Areas/Topics (4)
Agent Models and Architectures

Artificial Intelligence

Task Planning and Execution
Agents

Speeding up Online POMDP Planning - Unification of Observation Branches by Belief-state Compression Via Expected Feature Values

Topics: Planning and Scheduling; Uncertainty in AI ; Reactive AI

Author: Gavin Rens

Affiliation: University of KwaZulu-Natal and CSIR Meraka, South Africa

Digital Object Identifier: 10.5220/0005165802410246

Proceedings of ICAART 2015, ISBN: 978-989-758-074-1

Collapse Button Quick Abstract
A novel algorithm to speed up online planning in partially observable Markov decision processes (POMDPs) is introduced. I propose a method for compressing nodes in belief-decision-trees while planning occurs. Whereas belief-decision-trees branch on actions and observations, with my method, they branch only on actions. This is achieved by unifying the branches required due to the nondeterminism of observations. The method is based on the expected values of domain features. The new algorithm is experimentally compared to three other online POMDP algorithms, outperforming them on the given test domain.
Collapse Button Related Ontology Subjects/Areas/Topics (10)
Artificial Intelligence

Planning and Scheduling

Soft Computing

Formal Methods

Reactive AI

Symbolic Systems

Informatics in Control, Automation and Robotics

Simulation and Modeling

Uncertainty in AI
Intelligent Control Systems and Optimization

ACCEPTANCE OF ENTERPRISE RESOURCE PLANNING SYSTEMS BY SMALL MANUFACTURING ENTERPRISES

Topics: HCI on Enterprise Information Systems; Human Factors; Enterprise Resource Planning; Accessibility and Usability; Business-It Allignment

Authors: Rubina Adam 1 , Paula Kotzé 2 and Alta van der Merwe 3

Affiliations: 1 SAP Research Pretoria/Meraka Unit of Technology Development and University of South Africa, South Africa , 2 Meraka Institute of the CSIR and Nelson Mandela Metropolitan University, South Africa , 3 Meraka Institute of the CSIR and University of South Africa, South Africa

Digital Object Identifier: 10.5220/0003500302290238

Proceedings of ICEIS 2011, ISBN: 978-989-8425-53-9

Collapse Button Quick Abstract
ERP systems are widely used by large enterprises for managing functional areas of the enterprise. However, recently ERP systems have also been introduced to the small enterprise market. ERP systems are now considered an important small enterprise management aid that may contribute to the sustainability and growth of the small enterprise. Although there are several acceptance factors that may impact on the acceptance of ERP systems, limited research has been done to understand the acceptance of ERP systems by small enterprises. This paper addresses this gap by considering the strategic, business, technical and human factors that influence the acceptance of ERP systems in small manufacturing enterprises in South Africa. The consultative list of acceptance factors flowing from this research may guide future initiatives aiming to ensure the acceptance of ERP systems by small manufacturing enterprises.
Collapse Button Related Ontology Subjects/Areas/Topics (12)
Accessibility and Usability

Enterprise Software Technologies

Physiological Computing Systems

Databases and Information Systems Integration

HCI on Enterprise Information Systems

Simulation and Modeling

Enterprise Information Systems

Human Factors

Simulation Tools and Platforms

Enterprise Resource Planning

Human-Computer Interaction

Software Engineering

SECURE VIRTUALIZATION - Benefits, Risks and Controls

Topics: Cloud Risk, Challenges, and Governance; Virtualization Technologies

Authors: Mariana Carroll 1 , Paula Kotzé 2 and Alta van der Merwe 1

Affiliations: 1 Meraka Institute of the CSIR and University of South Africa, South Africa , 2 Meraka Institute of the CSIR and Nelson Mandela Metropolitan University, South Africa

Digital Object Identifier: 10.5220/0003390400150023

Proceedings of CLOSER 2011, ISBN: 978-989-8425-52-2

Collapse Button Quick Abstract
Cloud computing is changing the IT delivery model to provide on-demand self-service access to a shared pool of computing resources (physical and virtual) via broad network access to offer reduced costs, scalability, flexibility, capacity utilization, higher efficiencies and mobility. In many instances cloud computing builds on the capabilities of a virtualized computing infrastructure enabling multi-tenancy, scalability and a highly abstracted cloud model. Even though cloud computing provides compelling benefits and cost-effective options for IT hosting and expansion, security of applications and data remains a number one business objective. It is therefore essential to ensure adequate security not only for cloud computing, but also for the underlying technologies enabling cloud computing. Management should understand and analyse risks in order to safeguard systems and data. The focus of this paper is on mitigation for virtualization security risks as a fundamental step towards secure cloud computing environments.
Collapse Button Related Ontology Subjects/Areas/Topics (5)
Cloud Computing

Cloud Risk, Challenges, and Governance

Virtualization Technologies
Cloud Computing Enabling Technology

Fundamentals

USING FORMAL ONTOLOGIES FOR THE DEVELOPMENT OF CONSISTENT AND UNAMBIGUOUS FINANCIAL ACCOUNTING STANDARDS

Topics: Ontology Engineering; Knowledge Representation; Domain Analysis and Modeling

Authors: Marthinus C. Gerber 1 , Aurona J. Gerber 2 and Alta van der Merwe 3

Affiliations: 1 University of Pretoria, South Africa , 2 Meraka-CSIR and University of Kwazulu Natal, South Africa , 3 Meraka-CSIR, South Africa

Digital Object Identifier: 10.5220/0003658404190424

Proceedings of KEOD 2011/IC3K, ISBN: 978-989-8425-80-5

Collapse Button Quick Abstract
The purpose of accounting is to gather financial data of a business or entity, to interpret this data and to report the results in financial statements to the different users thereof. The interpretation of financial data is regulated by financial accounting standards including a conceptual framework that were developed to facilitate the reporting of financial information of entities. Due to the history of standards development as well as the mechanisms used, inconsistencies in the standards, framework and interpretations are part of the common legacy accountants are confronted with every day. The development of unambiguous and principle based financial accounting standards is therefore a key initiative at present of international accounting standards bodies such as the FASB and the IASB. This paper is concerned with the question of how recently developed computer science technologies could assist in dealing with and eliminating inconsistencies and ambiguities within and between different financial accounting standards. In our research we developed a formal ontology for some of the basic elements, and in this paper we report on our findings as well as make some suggestions for a formal approach to the conceptual framework and financial accounting standards development.
Collapse Button Related Ontology Subjects/Areas/Topics (11)
Artificial Intelligence

Information Systems Analysis and Specification

Ontologies and the Semantic Web

Data Engineering

Knowledge Engineering and Ontology Development

Ontology Engineering

Domain Analysis and Modeling

Knowledge Representation

Symbolic Systems
Enterprise Information Systems

Knowledge-Based Systems

Context (please use complete words only)
Field
Sort by

Topic(s):

Filter listed results, by

Publication date

From:
To:
Author
Locate Author in list below
Data pager
Data pager
12
 Page 1 of 2, items 1 to 10 of 17.
Add Author(s) to filter:
Conference
Locate Conference in list below
Add Conference(s) to filter:

- Science and Technology Publications, Lda.
RESOURCES

Proceedings

Papers

Authors

Ontology

CONTACTS

Science and Technology Publications, Lda
Avenida de S. Francisco Xavier, Lote 7 Cv. C,
2900-616 Setúbal, Portugal.

Phone: +351 265 520 185 (National fixed network call)
Fax: +351 265 520 186
Email: info@scitepress.org

EXTERNAL LINKS

PRIMORIS

INSTICC

SCITEVENTS

CROSSREF

PROCEEDINGS SUBMITTED FOR INDEXATION BY:

dblp

Ei Compendex

SCOPUS

Semantic Scholar

Google Scholar

Microsoft Academic