CLIP-LLM: A Framework for Autonomous Plant Disease Management in Greenhouse

Muhammad Salman, Muhayy Ud Din, Irfan Hussain

2025

Abstract

Agricultural disease detection and intervention remain challenging due to complex crop health variations, dynamic environmental conditions, and labor-intensive fieldwork. We introduce an end-to-end, platformagnostic robotic pipeline for autonomous disease detection and treatment systems, with a specific focus on cassava leaves as an example. The pipeline integrates a vision-language perception module based on a pretrained Contrastive Language-Image Pre-training (CLIP) model, fine-tuned on an augmented dataset of cassava leaf images for disease detection. High-level task planning is performed by a Generative Pre-trained Transformer 4 (GPT-4), which interprets perception outputs and generates symbolic action plans (e.g., navigate to target, perform treatment). The low-level control system is implemented in the PyBullet dynamic simulator. We evaluated a vision-language model (VLM) perception and a Large Language Model (LLM) based planning system (in a virtual environment with predefined 3D coordinates for plant and spray positions). The VLM achieved 83% classification accuracy in simulation and real-time tests with a static camera produced classification accuracies of 70% Cassava Brown Streak Disease (CBSD), 65% Cassava Mosaic Disease (CMD) and 52% Cassava Bacterial Blight (CBB), and under dynamic camera it achieve the accuracy of 65% (CBSD), 52% (CMD), and 32% (CBB). Currently, our low-level controller executes the LLM-generated plans with high precision (less than ±2 mm positioning error). These results demonstrate the viability of our platform-agnostic modular architecture for precision agriculture that supports closed-loop robustness and scalability.

Paper Citation

in Harvard Style

Salman M., Din M. and Hussain I. (2025). CLIP-LLM: A Framework for Autonomous Plant Disease Management in Greenhouse. In Proceedings of the 22nd International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO; ISBN 978-989-758-770-2, SciTePress, pages 202-210. DOI: 10.5220/0013674200003982

in Bibtex Style

@conference{icinco25,
author={Muhammad Salman and Muhayy Din and Irfan Hussain},
title={CLIP-LLM: A Framework for Autonomous Plant Disease Management in Greenhouse},
booktitle={Proceedings of the 22nd International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO},
year={2025},
pages={202-210},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013674200003982},
isbn={978-989-758-770-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 22nd International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO
TI - CLIP-LLM: A Framework for Autonomous Plant Disease Management in Greenhouse
SN - 978-989-758-770-2
AU - Salman M.
AU - Din M.
AU - Hussain I.
PY - 2025
SP - 202
EP - 210
DO - 10.5220/0013674200003982
PB - SciTePress