Authors:
Shorabuddin Syed
1
;
Adam Jackson Angel
2
;
Hafsa Bareen Syeda
3
;
Carole France Jennings
4
;
Joseph VanScoy
5
;
Mahanazuddin Syed
1
;
Melody Greer
1
;
Sudeepa Bhattacharyya
6
;
Meredith Zozus
7
;
Benjamin Tharian
8
and
Fred Prior
1
Affiliations:
1
Department of Biomedical Informatics, University of Arkansas for Medical Sciences, U.S.A.
;
2
Department of Internal Medicine, Washington University, U.S.A.
;
3
Department of Neurology, University of Arkansas for Medical Sciences, U.S.A.
;
4
Department of Internal Medicine, Tulane University, U.S.A.
;
5
College of Medicine, University of Arkansas for Medical Sciences, U.S.A.
;
6
Department of Biological Sciences, Arkansas State University, U.S.A.
;
7
Department of Population Health Sciences, University of Texas Health Science Centre at San Antonio, U.S.A.
;
8
Division of Gastroenterology and Hepatology, University of Arkansas for Medical Sciences, U.S.A.
Keyword(s):
Colonoscopy, Natural Language Processing, Deep Learning, Word Embeddings, Clinical Concept Extraction.
Abstract:
Colonoscopy is a screening and diagnostic procedure for detection of colorectal carcinomas with specific quality metrics that monitor and improve adenoma detection rates. These quality metrics are stored in disparate documents i.e., colonoscopy, pathology, and radiology reports. The lack of integrated standardized documentation is impeding colorectal cancer research. Clinical concept extraction using Natural Language Processing (NLP) and Machine Learning (ML) techniques is an alternative to manual data abstraction. Contextual word embedding models such as BERT (Bidirectional Encoder Representations from Transformers) and FLAIR have enhanced performance of NLP tasks. Combining multiple clinically-trained embeddings can improve word representations and boost the performance of the clinical NLP systems. The objective of this study is to extract comprehensive clinical concepts from the consolidated colonoscopy documents using concatenated clinical embeddings. We built high-quality annota
ted corpora for three report types. BERT and FLAIR embeddings were trained on unlabeled colonoscopy related documents. We built a hybrid Artificial Neural Network (h-ANN) to concatenate and fine-tune BERT and FLAIR embeddings. To extract concepts of interest from three report types, 3 models were initialized from the h-ANN and fine-tuned using the annotated corpora. The models achieved best F1-scores of 91.76%, 92.25%, and 88.55% for colonoscopy, pathology, and radiology reports respectively.
(More)