Real‑Time Big Data Analytics for Cross Sector Decision Intelligence:

A Scalable Framework for Transforming Enterprise Data into

Strategic Action

Mohanraj P.

, Chidambaram

, Bharath K.

, V. Karthika

Shanmugapriya D.

and Kalpesh Rasiklal Rakholia

Department of MBA, Faculty of Management, SRM Institute of Science and Technology, Ramapuram Campus,

Ramapuram, Chennai - 600 89, Tamil Nadu, India

Department. of MBA, Faculty of Management, SRM Institute of Science and Technology - Ramapuram Campus,

Ramapuram, Chennai - 600 89, Tamil Nadu, India

Department. of MBA, School of Commerce and Management, Sanjivani University, Kopargaon, Maharashtra, India

Department of Management Studies, Nandha Engineering College, Vaikkalmedu, Erode - 638052, Tamil Nadu, India

Department of CSE, New Prince Shri Bhavani College of Engineering and Technology, Chennai, Tamil Nadu, India

Parul Institute of Engineering and Technology, parul university vadodara, vadodara‑391760, Gujarat, India

Keywords: Real‑Time Analytics, Decision Intelligence, Cross‑Sector Big Data, Strategic Insights, Scalable Data

Framework.

Abstract: In the time of digital change, companies are being overrun with data, but few can devote the necessary time

and resources to turn this into operations! In this work, we propose a scalable big data analytics framework

that generates actionable and real-time insights in various industry domains. Combining sophisticated

machine learning models and streaming data pipeline, this system performs raw data ingestion to enable real-

time, intelligent decision making. Unlike earlier efforts that were either static, focused solely on a narrow

industrial sector or both these, this framework utilizes cloud- based analytics, edge computing and intelligent

visualization to support the adaptive and cross functional decision intelligence. Findings show greater

organizational responsiveness, operational capability, and strategic flexibility by exploiting data to make

decisions.

1 INTRODUCTION

In the world of businesses today dominated by data,

extracting meaningful and actionable insights from

the haywire can be the significant differentiator

between competitors. Big Data analytics is

increasingly being adopted by organizations across

industries to improve decision-making, optimize

processes, and predict market trends. Yet, the gap

between data accessibility and the use of it to make

real-time, organization-wide decisions is still large

despite the flood of data and analysis tools.

Some of the drawbacks for the prior art analytical

systems are that scalability, responsiveness, and

adaptability in variant industrial environments are

sometimes poor. Existing solutions are often limited

to after-the-fact analysis or domain-specific si- los

that do not empower decision-makers with real-time,

action- able insights. The lack of integration across

platforms, the delay in insights and a misalignment

with strategic business goals, however, continue to

temper what can be achieved with big data.

In this study, such limitations are tackled by

proposing an intelligent and scalable big data

analytics framework. The model utilizes a

combination of real-time data ingestion, cloud-edge

synergy, and cutting-edge machine learning to

provide real-time decision intelligence that is flexible

across various industries. By enabling actionable

insights, the approach allows a company’s

previously passive data assets to become a strategic

mechanism for informed decision-making and long-

term business transformation. This research paves the

way for organizations to help facilitate the process

from the collection of data to its practical use in order

P., M., Chidambaram, , K., B., Karthika, V., D., S. and Rakholia, K. R.

Real-Time Big Data Analytics for Cross Sector Decision Intelligence: A Scalable Framework for Transforming Enterprise Data into Strategic Action.

DOI: 10.5220/0013870400004919

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st International Conference on Research and Development in Information, Communication, and Computing Technologies (ICRDICCT‘25 2025) - Volume 1, pages

625-631

ISBN: 978-989-758-777-1

625

to utilize big data to its maximum potential in the

constantly changing business world.

2 PROBLEM STATEMENT

With the explosion of data produced across

industries, companies are still struggling to convert

that data into real-time, actionable insights that drive

strategic decisions. Current big data analytics

toolboxes are usually not scalable, restricted in the

domain and provide no Realtime decision support.

These are bottlenecks causing reaction lags, partial

intelligence, and not utilizing all the potential

available data facilities. What’s more, without a

consolidated platform for dynamic data processing,

cross-platform accessibility, and adaptive

intelligence, organizations are unable to get a full

picture of both their operational and strategic

landscapes. It’s is not that the data isn’t there, the

point of customer data and the challenge is creating

an effective, scalable and industry agnostic analytics

infrastructure that connects raw data to business

decisions in real time.

3 LITERATURE SURVEY

Converting raw data into business-driving insight has

long since been a challenge, and an ever-changing

key focus of big data analytics. Previous studies,

including Papineni et al. (2021), highlighted the

combination of multi-criteria decision-making

models and deep learning for more accurate

analyses; however, the question of scalability was

raised. Schmitt analysed the automation of machine

learning in analytics-driven business processes but

found few empirical assemblages in corporate

environments. The study by Tawil et al. (2023)

provided useful perspective on data-driven practices

in the UK SME sector, with adaptability and

interpretation of data emerging as recurrent themes.

A number of academics have tried to relate big

data initiatives with strategic business objectives.

Akter et al. (2021) and Ren et al. (2019), which

developed models to externally assess organisation

performance in the context of analytics capability,

highlighting the required strategic alignment and real-

time feedback dimensions. Wamba et al. (2021) built

on this view by investigating the ways in which

dynamic capabilities shape analytics-driven

organizational outcomes. Nevertheless, from the

above literature, limited studies have been found on

the second-tier suppliers, one limitation that was

alleviated to some extent by Gunasekaran et al.

(2021) who linked big data to predictive supply chain

performance.

The convergence of big data with decision

intelligence systems has been of continued interest.

Orjatsalo et al. (2025) examined perceptions of

analytics at the managerial level and suggested that

even though tools exist, often they are not

strategically employed due to a lack of strategic

perception. Further, Tiwari (2024) and Ats Tsaniyah

et. (2025) pointed to conceptual models for deciding,

however their studies were mostly abstract as they

did not produce empirical values. Fanelli et al. (2023),

support of the notion that organisational and technical

obstacles to implementation have both a health care

specific impact as well as an impact on the broader

enterprise.

Another primary issue addressed in the literature

is the real-time generation of insights. Trinh (2025)

also looked into the use of deep neural networks in

business prediction problems, suggesting a hybrid

model as a solution for adaptive analytics. Abu-Salih

et al. (2021) showed how machine learning can be

applied to social big data but also pointed out the

shortcomings of using unstructured public data for

enterprise-specific decisions. Dubey et al. (2021) and

Mikalef et al. (2021) investigated the mediation of

analytics capabilities in firm performance,

confirming the importance of such dynamic, adaptive

systems.

Ahmed et al. (2024) presented a comprehensive

review of business intelligence tools in decision

support, and Orji et al. (2023) delivered regional

cases to emphasise the significance of local data to

strategic issue for organisation. Both Sabri (2021) and

Ayokanmbi (2021) reiterate the role of organizational

readiness and digital culture in analytics integration

success. Meanwhile, Kaviani et al. (2022) researched

an area of big data in project management,

associating data flow with real-time planning.

The recent works such as Hsieh et al. (2024) are

among those who have tried to converge advances on

machine learning and analytics, suggesting

technologically sound frameworks that are still

unexploited in the business settings. The work of

Mariani and Fosso Wamba (2020) shed light on the

ways in which consumer goods firms are exploiting

the opportunities of digital innovation, pointing

towards an increasing importance of industry-specific

applications of analytics. Akter et al. (2021) also

presented qualitative models for analytics-driven

decision-making in services and noted that there is a

ICRDICCT‘25 2025 - INTERNATIONAL CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION,

COMMUNICATION, AND COMPUTING TECHNOLOGIES

626

need for research on translating these concepts into

practice.

Wamba et al. (2021) and Akter et al. (2021)

persistently noted that analytical tools are evolving

rapidly on one hand, but how they operate within real

time business situations is quite restricted by

infrastructure, challenges of integration and without

executive sponsorship. Such gaps underpin the

purpose of the research proposed in this document

that aims not only to integrate the strong aspects of

available models into a comprehensive, scalable and

cross-industry framework, but also to facilitate real-

time, strategic decision intelligence.

4 METHODOLOGY

The research method of this paper is to design,

implement and evaluate a real time big data analytics

framework specially used for converting raw

enterprise-level data into strategic decision

intelligence in various business fields. It combines

elements of system architecture design, machine

learning model integration, cross-industry data

emulation, and real-world system validation so as to

provide credible scaling and practical relevance.

The first phase was the discovery and collection

of heterogeneous data, ranging from e-commerce to

hospital transactions (anonymized) and logistics and

supply chain logs, as well as financial performance.

These were created from a variety of data sources

with differences in structure, volume, and velocity (3

core properties associated with big data). We utilized

Apache Hadoop HDFS to store the data by utilizing a

data lake architecture, Apache NiFi to orchestrate

real-time flow of data, and pre-process the flow

pipelines. Figure 1 show the Real-Time Big Data

Analytics Framework for Enterprise Decision

Intelligence.

Figure 1: Real-Time Big Data Analytics Framework for

Enterprise Decision Intelligence.

Tokenization was used for textual input while

categorical variable features are label-encoded,

numerical property features are z-score standardized

and anomaly detection by isolation forest were

performed for data pre-processing to normalize the

input data before being fed to the model. Missing data

imputation were addressed with K-Nearest

Neighbours and time-series interpolation to provide

data regularities and improve the generalization

capability.

After pre-processing, the data was piped into an

analytics engine developed on top of Apache Spark

with its hooks connected to Python based ML

libraries such as Scikit-learn, XGBoost and

TensorFlow. The analytics layer was an ensemble of

multi-algorithm approach for classification,

regression and clustering type models, executed in

parallel as per need of the use-cases. For example,

customer churn prediction was performed using

gradient boosting classifiers and revenue prediction

was done following an LSTM neural network based

on learned temporal pattern. Market Segmentation

Unsupervised market clustering was performed by

using K-Means and DBSCAN. Table 1: show the

Data Pre-processing Techniques Applied

Table 1: Data Preprocessing Techniques Applied.

Dataset

Type

Missing

Value

Strateg

Outlier

Handli

Normali

zation

Method

Feature

Engineer

ing

Trans

action

Logs

KNN

Imputat

ion

Z-

Score

Thresh

olding

Min-

Max

Scaling

One-Hot

Encodin

Medic

Recor

Time-

Series

Interpol

ation

Isolatio

Forest

Z-Score

Scaling

Principal

Compon

ent

Analysis

Finan

cial

Logs

Mean

Substitu

tion

IQR

Filterin

Standard

Scaling

Lag

Feature

Creation

The solution included decision orchestration layer

built on Kafka Streams with the capability for real

time decision intelligence delivered using Kubernetes

clusters to auto scaling the workload. This layer

orchestrated system health, raised alerts, and

dynamically adjusted model selection based on

confidence thresholds, business context, and

prediction recency. The insights generated were

included as a summary in a single business

intelligence dashboard using Dash (Plotly) and

Grafana, enabling stakeholders with actionable KPIs,

trends, forecasts, and alerts.

Real-Time Big Data Analytics for Cross Sector Decision Intelligence: A Scalable Framework for Transforming Enterprise Data into

Strategic Action

627

The deployment was in a hybrid computing

environment to trade off the latency cost and

computational cost. Edge devices, such as Raspberry

Pi clusters and industrial gateways, were employed to

process sensor data on-site and run localized

analytics in poor connection condition. Model

training, storage, and high-throughput computing

were performed using cloud services (AWS and

Google Cloud). It served to provide resilience,

elasticity and global access to decision data.

The framework was tested and validated in

offline benchmarking, in simulation-based stress

testing, and in the pilot deployment in live

environment. We gathered various metrics including

throughputs (records per sec), prediction accuracy,

response time, down time of the system, user

interpretability and, the impact of the decision

(which was measured through A/B testing in

operational workflows). Comments from industry

experts in finance, healthcare, and retail were

included to enhance dashboard usability, model

explain ability, and data traceability functionality.

Figure 2 show the Feature Correlation Heatmap for

Financial Data.

Figure 2: Feature Correlation Heatmap for Financial Data.

With such a comprehensive approach, the research

arrives at a strong realization that mitigates root

shortcomings in previous work lagged insights, siloed

analytics and limited scalability whilst providing a

generic, flexible and intelligent approach to decision-

making across industries.

5 RESULT AND DISCUSSION

The performance of the proposed big data analytics

framework was tested on three different industrial

setup scenarios: retail analytics, healthcare decision

support, and financial forecasting. All deployments

were used for the system to process heterogeneous

and varied streams of data, assimilate real-time

insights and enable dynamic/high volume decisions.

Table 2 show the Predictive Model Performance

Across Sectors

Table 2: Predictive Model Performance Across Sectors.

Sector

Model

Used

Accur

acy

(%)

Precisi

on (%)

Laten

(sec)

F1-

Sco

Reta

Gradient

Boostin

91.2

89.6

2.1

0.9

Heal

thcar

LSTM

Network

88.4

87.2

4.8

0.8

Fina

nce

LSTM +

Isolation

93.7

96.1

2.6

0.9

Figure 3: Accuracy of Predictive Models Across Sectors.

In the retail case, the system provided an average

insight latency of less than 5.5 seconds with over one

million transaction messages being processed in real

time. Figure 3 show the Accuracy of Predictive

Models Across Sectors The predictive models

efficiently detected customer churn patterns and

boosted inventory suggestions based off of the

company’s baseline models and achieved an average

prediction accuracy of 91.2%. Customers using the

dashboard experienced significantly improved

confidence in decision making due to greater

visibility into customer behavior and operational

outliers. Table 3 show the Evaluation of Machine

Learning Models on Unified Dataset.

Anonymized patient data were streamed from a

hospital database in the healthcare setting to simulate

clinical decision-making. The system combined

historical records and sensor data to suggest

treatment prioritization and allocation of resources.

Decision latency is still less than the critical threshold

of 5 seconds and the model reaches an accuracy of

88.4% in predicting risk of readmission for the

patient, a vital metric for hospital management.

Incorporating edge computing in patient-monitoring

locations eliminated the overburden of central

ICRDICCT‘25 2025 - INTERNATIONAL CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION,

COMMUNICATION, AND COMPUTING TECHNOLOGIES

628

servers, and thus continuous analytics, even when

bandwidth was insufficient. Table 4 show the

Performance Comparison with Existing Analytics

Systems

Table 3: Evaluation of Machine Learning Models on Unified

Dataset.

Model

Accur

acy

(%)

Precisi

on (%)

Rec

all

(%)

Scor

Training

Time

(sec)

Random

Forest

88.5

87.9

86.4

0.90

12.3

Gradient

Boosting

91.2

89.6

90.1

0.93

18.6

LSTM

(Deep

Learning)

93.7

91.8

92.5

0.96

45.2

Table 4: Performance Comparison With Existing Analytics

Systems.

Metric

Baseline

System

Proposed

Framework

Improve

ment (%)

Insight

Accurac

74.3

91.2

+22.7

Decision

Latency

9.5 sec

2.3 sec

−75.8

Alert

Sensitivi

81.0

94.8

+17.1

User

Satisfact

ion

6.1/10

8.7/10

+42.6

There were promising results on a financial sector

pilot (revenue forecasting and fraud detection)

demonstrating considerable gains in the performance

of real time forecasting. The model, with deep

learning approach: LSTM, achieved 93.7% accuracy

in quarterly trend prediction whereas 96.1% true

positive rate for anomaly detection to spot potential

fraud events. This had a significant effect on the

finance organization, which were able to glean these

results directly into strategic planning and risk

management processes. Feedback from the decision-

makers suggested that the visual analytics interface

was intuitive and provided better insights into the

predictions making it easier to respond faster and

better informed. Figure 4 show the Real-Time System

Latency Under Varying Throughput

Figure 4: Real-Time System Latency Under Varying

Throughput.

Table 5: Use Case Scenarios and Framework Responses.

Use Case

Trigger

Event

Framework

Response

Resp

onse

Time

Retail -

Cart

Abandonm

ent

Custom

inactivit

Triggered

discount

recommendat

ion

1.8

sec

Healthcare

- Patient

Risk Alert

Drop in

vitals

detected

Alert sent to

ICU

dashboard

3.9

sec

Finance -

Suspicious

Geoloca

tion

anomal

Flagged

transaction

and notified

user

2.5

sec

And the framework had been widely deployed

beyond single deployments. The physical architecture

of BCDSS supported data formats, volumes and

business logic that were interchangeable without

reprogramming, emphasizing the universal nature of

the solution. Table 5 show the Use Case Scenarios

and Framework Responses Across all use cases, A/B

testing demonstrated the average benefit of using

real-time analytics to inform decisions leads to a 27%

improvement in accuracy and outcome efficiency

over purely intuition-driven decision making.

Figure 5: Distribution of Dashboard Evaluation Metrics.

Real-Time Big Data Analytics for Cross Sector Decision Intelligence: A Scalable Framework for Transforming Enterprise Data into

Strategic Action

629

On the whole, this finding accepts the research

hypothesis that the real-time, cross-industry, scalable

big data analytics environment immensely improves

the decision intelligence. The model enhances

analytical capabilities and organizational agility,

flexibility, and competitive value in information

abundant circumstances. These results lay the

groundwork for wider dissemination and further

development, such as incorporation with explainable

AI, self-deciding agents, and predictive governance

toolkits. Figure 5 show the Distribution of Dashboard

Evaluation Metrics

6 CONCLUSIONS

In today’s online-driven economy, where data is

growing exponentially, this represents a huge

opportunity and a huge challenge for organisations

looking to make informed, data-led decisions. The

lack of this is the missing bridge between data

generation and making use of it and we addressed this

in our research by creating a scalable and real-time

big data analytics framework to support decision

intelligence in various industries. Extensive

experimentation and deployment on real retail,

healthcare and finance streams indicate that the

framework is capable of handling complex, high

volume data streams, generate accurate predictions,

and obtain timely actionable insights.

This is in sharp contrast with most prior work

which is either bound to a static model, industry-

specific constraints or cannot process requests in a

timely manner. By combining edge computing and

cloud-based analytics and ML the approach is able to

reduce latency and increase the relevance of the

insights provided to decision-makers. Intelligent

dashboards and orchestration layers guide the insights

to be not only correct, but interpretable and actionable

in strategic and operational settings as well.

The research validates the value of live analytics

in speeding response times within a business and

within the larger digital business ecosystem. It also

underscores the necessity for flexible frameworks

adaptable across sectors that remain performance-

optimal, regardless of data and infrastructural

heterogeneity. As businesses become more complex

and data-dependent, frameworks like these are going

to be key to translating raw data into competitive

advantage.

This paper paves the way for future developments

in the domain of big data analytics, such as the

inclusion of explainable AI, autonomous decision-

making agents, and adaptive learning systems. The

study serves to extend the theoretical and practical

knowledge base in the big data-driven business

intelligence domain by addressing current limitations,

and offering a viable and scalable solution.

REFERENCES

Abu-Salih, B., Wongthongtham, P., Zhu, D., Chan, K. Y.,

& Rudra, A. (2021). Predictive analytics using social

big data and machine learning. arXiv .https://arxiv.org/

abs/2104.12591

Ahmed, S., Rizvi, S. W. H., & Laghari, S. H. (2024).

Transforming data into insights: The impact of business

intelligence on enhancing decision-making and

achieving organizational success. International Journal

of Data Science and Big Data Analytics, 4(2), 59–70.

https://www.researchgate.net/publication/387341450

ResearchGate

Akter, S., Wamba, S. F., Gunasekaran, A., Dubey, R., &

Childe, S. J. (2021). How to improve firm performance

using big data analytics capability and business strategy

alignment? International Journal of Production

Economics, 211, 1–14.

Ats Tsaniyah, F., Ningsih, S. P., Cyntia, D. Y., Kusumasari,

I. R., & Hidayat, R. N. (2025). The role of big data

analytics in supporting decision-making theories in

companies. Jurnal Bisnis dan Komunikasi Digital, 2(2),

1– 10. https://www.researchgate.net/publication/38774

7117:contentReference[oaicite:5]{index=5}

Ayokanmbi, F. M. (2021). The impact of big data analytics

on decision-making. International Journal of

Management, IT & Engineering, 11(4), 1– 15. https://

www.researchgate.net/publication/354968015

ResearchGate

Fanelli, S., Pratici, L., Salvatore, F. P., Donelli, C. C., &

Zangrandi, A. (2023). Big data analysis for decision-

making processes: Challenges and opportunities for the

management of health-care organizations. Management

Research Review, 46(3), 369–389.

https://doi.org/10.1108/MRR-09-2021-0648

Gunasekaran, A., Papadopoulos, T., Dubey, R., Fosso

Wamba, S., & Childe, S. J. (2021). Big data and

predictive analytics for supply chain and organizational

performance.

Hsieh, W., Bi, Z., Chen, K., Peng, B., Zhang, S., Xu, J., ...

& Liu, M. (2024). Deep learning, machine learning,

advancing big data analytics and management. arXiv.

https://arxiv.org/abs/2412.02187arXiv

Kaviani, P., Vanjale, S., & Dhotre, S. S. (2022). Big data

analytics in project administration for planning and

real-time decisions. Journal of Optoelectronics Laser,

41(1), 1–10.ResearchGate

Orjatsalo, J., Hussinki, H., & Stoklasa, J. (2025). Business

analytics in managerial decision-making: Top

management perceptions. Measuring Business

Excellence, 29(1), 1–17. https://doi.org/10.1108/MBE-

09-2023-0130

ICRDICCT‘25 2025 - INTERNATIONAL CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION,

COMMUNICATION, AND COMPUTING TECHNOLOGIES

630

Orji, U., Obianuju, E., Ezema, M., Ugwuishiwu, C.,

Ukwandu, E., & Agomuo, U. (2023). Using data

analytics to derive business intelligence: A case study.

arXiv. https://arxiv.org/abs/2305.19021arXiv

Papineni, S. L. V., Yarlagadda, S., Akkineni, H., & Reddy,

A. M. (2021). Big data analytics applying the fusion

approach of multicriteria decision making with deep

learning algorithms. arXiv. https://arxiv.org/abs/2102.

02637

Ren, S. J.-F., Fosso Wamba, S., Akter, S., Dubey, R., &

Childe, S. J. (2021). Modelling quality dynamics,

business value and firm performance in a big data

analytics environment. International Journal of

Production Research, 59(1), 1–17.

Sabri, M. S. (2021). The impact of big data analytics on

decision-making. International Journal of

Management, IT & Engineering, 11(4), 1– 15. https://

www.researchgate.net/publication/354968015

ResearchGate

Schmitt, M. (2022). Automated machine learning: AI-

driven decision making in business analytics. arXiv.

https://arxiv.org/abs/2205.10538

Tawil, A.-R., Mohamed, M., Schmoor, X., Vlachos, K., &

Haidar, D. (2023). Trends and challenges towards an

effective data-driven decision making in UK SMEs:

Case studies and lessons learnt from the analysis of 85

SMEs. arXiv. https://arxiv.org/abs/2305.15454&#820

3;:contentReference[oaicite:4]{index=4}

Tiwari, V. (2024). Role of data analytics in business

decision making. Knowledgeable Research: A Multidi

sciplinary Journal, 3(1), 18–27.

https://www.researchgate.net/publication/383588986

Trinh, V. (2025). A comprehensive review: Applicability of

deep neural networks in business decision making and

market prediction investment. arXiv .https://arxiv.org/

abs/2502.00151

Wamba, S. F., Gunasekaran, A., Akter, S., Ren, S. J.-F., &

Dubey, R. (2021). Big data analytics and firm

performance: Effects of dynamic capabilities. Journal

of Business Research, 131, 1–10.

Real-Time Big Data Analytics for Cross Sector Decision Intelligence: A Scalable Framework for Transforming Enterprise Data into

Strategic Action

631