Dry eye disease (DED) is a prevalent ocular condition characterized by discomfort, visual disturbances and tear film instability due to insufficient tear production or rapid tear evaporation.1 With an increasing global incidence (i.e. up to 50%) and its significant impact on the quality of life, there is a growing need for accurate and efficient diagnostic methods to aid in its timely identification and management.2 Traditional diagnostic approaches rely on subjective assessments and clinical evaluations, often leading to variability and suboptimal accuracy.
In recent years, machine learning (ML) techniques have emerged as potential tools in revolutionizing healthcare, including in ophthalmology.3 ML models offer the possibility of improving accuracy, objectivity and speed in medical diagnosis by leveraging extensive real-world clinical data.4 These algorithms analyze complex patterns and relationships within data sets, extracting valuable insights not readily apparent to human clinicians. This potential could significantly benefit the diagnosis and management of DED.
Recent studies have showcased the potential of ML models in ophthalmology. For instance, Ting et al. developed a deep learning system for the diagnosis of diabetic retinopathy.3 While not specific to dry eye, this study illustrated how ML models can effectively use real-world clinical data to classify and diagnose complex eye conditions. Leveraging a large data set of retinal images, the authors achieved an AUC-ROC of 0.936, underscoring the power of ML techniques in medical image analysis.
In this study, we explore the application of ML models for the diagnosis of DED using real-world clinical data. We examine various approaches and methodologies used by researchers to harness the potential of ML in identifying and classifying DED. The insights gained from this study have the potential to contribute to the evolution of the diagnosis of dry eyes, providing more accurate and efficient methods that may lead to improved patient outcomes and an enhanced quality of life.
Background
The study aims to enhance diagnostics in DED by specifically examining the type and severity of the disease using ML in a typical clinical setting. The objectives include the use of real-world clinical data to create ML models for assessing the type and severity of DED, focussing on facilitating the implementation by clinicians in a clinical setting.
Real-world clinical data collected over 18 months were used to develop two ML diagnostic models for the severity and type of dry eyes. The data set from a Canadian clinic comprised 1,313 well-structured samples, each annotated by domain experts, showcasing numerous features.
Methods
Real-world data was collected in Canadian clinics for over 28 months from December 1, 2018 to March 31, 2021 and then anonymized. These data were used to develop two ML diagnostic models for assessing the severity and type of DED. The data set consisted of 1,313 patient samples, each carefully annotated by domain specialists, showcasing various features. The use of such extensive real-world clinical data aims to enhance the generalizability and relevance of the models in practical settings.
To optimize the performance of the ML models, we applied a correlation feature selection technique.5 This method systematically identified and eliminated redundant and irrelevant features, refining the input variables to focus on the most pertinent aspects for accurate diagnosis. This step streamlined the models and improved their interpretability.
To assess the performance and robustness of the models comprehensively, we used a stratified 10-fold cross-validation methodology.6 This technique divides the data set into subsets, maintaining the distribution of the original data to minimize bias and generate a more reliable assessment of the predictive capabilities of the models.
Among the various ML techniques, support vector machines (SVMs) demonstrated proficiency in handling the intricacies of the data set.7 SVMs excel in categorizing the data into distinct groups, making them well-suited for discerning different levels of severity and types of dry eyes.
We developed and fine-tuned two distinct SVM models, the dry eye severity model (SM) and the dry eye type model (TM). We used the p-value as a statistical measure to validate our hypothesis against the observed data and ensure that our proposed models achieve sufficient statistical power. The SM quantifies the severity of DED, offering insights into its progression and intensity. The TM classifies dry eye types, aiding clinicians in making more precise treatment decisions. Importantly, both models use the same set of features for predictions, ensuring consistency and comparability between severity and type assessments. This uniformity in feature selection enhances the applicability of the models and simplifies their integration into clinical practice.
This study was performed in accordance with the Helsinki Declaration of 1964 and its later amendments. Written informed consent was received from the patients to participate in this study and for the publication of this article. This article does not contain any identifying information about the participants.
Results
The SM demonstrated satisfactory performance in predicting the severity of dry eyes in various cases, with an area under the receiver operating characteristic (AUC-ROC) curve of 0.79 (p-value = 1.41e−05) (Figure 1) and an area under the precision-recall (AUC-PR) curve of 0.61 (p-value = 1.34e−03) (Figure 2). These metrics offer a comprehensive evaluation of the capability of the models to differentiate between different severity levels and effectively identify true-positive cases while managing false positives.
Figure 1: The area under the receiver operating characteristic curve per class of the dry eye severity model
AUC-ROC = area under the receiver operating characteristic.
Figure 2: The precision-recall micro-averaged curve per class of the dry eye severity model
On the other hand, the TM showed reliable predictive capabilities in distinguishing various dry eye types, as indicated by an AUC-ROC value of 0.91 (p-value = 1.28e−04) (Figure 3) and an AUC-PR value of 0.83 (p-value = 1.37e−04) (Figure 4). These values suggest the effectiveness of the model in accurately identifying different dry eye classifications. The notable metrics highlight the strength and capacity of the models to capture true-positive cases while minimizing false positives.
Figure 3: The area under the receiver operating characteristic curve per class of the dry eye type model
AUC-ROC = area under the receiver operating characteristic.
Figure 4: The precision-recall micro-averaged curve per class of the dry eye type model
The robustness and performance of both the SM and TM were confirmed through a thorough comparison with nine baseline ML methods. These baseline methods included various traditional algorithms commonly used in classification tasks. The results of the comparative analysis emphasized the consistent performance of the SM and TM. Across both AUC-ROC and AUC-PR metrics, both models consistently outperformed all baseline methods, indicating their effectiveness in terms of predictive accuracy, sensitivity and specificity.
Discussion
The AUC-ROC and AUC-PR values achieved by the SM and TM, coupled with their consistent outperformance over baseline methods, emphasize their usefulness in accurately diagnosing the severity and type of dry eyes. These findings suggest the potential of ML techniques to contribute to ophthalmology and refine diagnostic procedures.
In the realm of the diagnosis of dry eyes, the ML models introduced in this study serve as a validated prediction framework. Their ability to predict both severity and type of dry eye reflects their robustness and potential impact. By providing clinicians with a tool for reliably anticipating dry eye conditions, these models have the potential to improve health outcomes. Their predictive capabilities may play a role in initiating timely interventions, potentially preventing the progression of DED.
Beyond diagnostic accuracy, these models could proactively contribute to patient care by offering early alerts. This could enable healthcare professionals to implement preventive measures or tailor treatment strategies promptly, potentially mitigating the discomfort and impairment associated with dry eyes and enhancing the quality of life of patients.
While already demonstrating predictive capabilities, the trajectory of these ML models is set for further refinement and augmentation. The integration of future data, encompassing a broader array of clinical scenarios and patient profiles, has the potential to enhance these models further. As these models continue to learn from diverse data sets, their precision and reliability are expected to improve, allowing them to adapt to the nuanced complexities inherent in DED.
To implement the findings of this study in clinical practice, we have taken a proactive step by seamlessly integrating the SM and TM with the widely adopted CSI Dry Eye Software (CSI Dry Eye Inc., Calgary, Alberta, Canada). This strategic integration enables a streamlined incorporation of our ML models into the existing workflows of clinicians globally who rely on the CSI Dry Eye Software. By leveraging this established platform, we aim to facilitate the accessibility and adoption of our predictive models, ensuring that clinicians worldwide can readily incorporate these models into their routine diagnostic practices. This collaborative approach not only enhances the practical applicability of our models but also fosters a more efficient and widespread integration of advanced diagnostic capabilities into real-world clinical settings. As a result, clinicians using the CSI Dry Eye Software can now benefit from the enhanced predictive accuracy and tailored insights provided by our SM and TM, contributing to more precise and personalized management of DED for patients worldwide.
Conclusion
By using ML and incorporating real-world clinical data, the models presented in this study contribute to the field of dry eye diagnosis and highlight the potential of ML to support medical decision-making processes. The validation of these ML models as the initial framework for predicting the severity and type of dry eyes marks a significant advancement. With an ongoing focus on improvement through the integration of future data, these models have the potential to bring changes to the landscape of dry eye diagnosis and treatment.
