Improving Public Sector Efficiency using Advanced Text Mining in the Procurement Process

Nikola Modrušan, Kornelije Rabuzin, Leo Mršić


The analysis of the Public Procurement Processes (PPP) and the detection of suspicious or corrupt procedures is an important topic, especially for improving the process’s transparency and for protecting public financial interests. Creating a quality model as a foundation to perform a quality analysis largely depends on the quality and volume of data that is analyzed. It is important to find a way to identify anomalies before they occur and to prevent any kind of harm that is of public interest. For this reason, we focused our research on an early phase of the PPP, the preparation of the tender documentation. During this phase, it is important to collect documents, detect and extract quality content from it, and analyze this content for any possible manipulation of the PPP’s outcome. Part of the documentation related to defining the rules and restrictions for the PPP is usually within a specific section of the documents, often called “technical and professional ability.” In previous studies, the authors extracted and processed these sections and used extracted content in order to develop a prediction model for indicating fraudulent activities. As the criteria and conditions can also be found in other parts of the PPP’s documentation, the idea of this research is to detect additional content and to investigate its impact on the outcome of the prediction model. Therefore, our goal was to determine a list of relevant terms and to develop a data science model finding and extracting terms in order to improve the predictions of suspicious tender. An evaluation was conducted based on an initial prediction model trained with the extracted content as additional input parameters. The training results show a significant improvement in the output metrics. This study presents a methodology for detecting the content needed to predict suspicious procurement procedures, for measuring the relevance of extracted terms, and for storing the most important information in a relational structure in a database.


Paper Citation