Ophthalmology has long been an early adopter of new and innovative technologies in patient care. The advent of optical coherence tomography in the early 2000s for evaluation of the retina quickly revolutionized daily clinical practice. In the past few years, the use of viral vectors to enable in vivo gene alterations to cure genetic disease became available in ophthalmology well before any other field with the U.S. Food and Drug Administration approval of Luxturna (voretigene neparvovec-rzyl, Spark Therapeutics) in 2017. Most recently, artificial intelligence technologies entered the field of ophthalmology. In April 2018, the FDA approved the first fully-autonomous AI-enabled screening device for diagnosing ophthalmic disease, the IDX-DR.
AI is a burgeoning field, and as the capabilities of machine learning and AI in ophthalmology continue to expand, new technologies will undoubtedly continue to emerge that will augment our practices and enable clinical insights never before thought possible. In this article, we’ll take a look at the benefits and limitations of this technology.
What is Artificial Intelligence?
Artificial intelligence is a broad discipline under the umbrella of computer science that aims to enable computers to perform tasks usually done by humans.1 AI algorithms allow computers to function intelligently and independently. Machine learning is a subset of AI that uses computer algorithms known as neural networks to allow computers to learn from datasets and subsequently edit their own coding with the goal of making future predictions about new data.2 The ability of computers to learn mimics the capabilities of human intelligence, hence the use of the terms “artificial intelligence” and “machine learning.”
Ophthalmology is uniquely capable of capitalizing on the promise of AI. Ophthalmologists generate robust amounts of data during routine clinical practice, such as visual acuity, intraocular pressure and the cup-to-disc ratio, along with ancillary imaging data from fundus cameras, OCT machines and visual fields. Machine learning algorithms require large amounts of data in order to learn. While machine learning and AI algorithms can be applied to a myriad of data sources (i.e., written text, audio, images, video), one of the most well-established use cases is screening for and diagnosing disease using clinical images (i.e., fundus photos) paired with clinical data.
To date, researchers have used machine learning algorithms to screen for and diagnose multiple ophthalmic diseases such as diabetic retinopathy, age-related macular degeneration, macular edema, glaucoma, keratoconus, post-LASIK corneal ectasia, retinopathy of prematurity, and cataracts.1 It has also proven useful for predicting the prognosis of various ophthalmic diseases.
Commercially Available AI
As mentioned earlier, the IDx-DR (IDx, LLC, Coralville, Iowa, USA) was the first autonomous AI system capable of screening for disease independent of a physician. IDx-DR uses a machine-learning-algorithm-equipped standalone fundus camera to screen for DR in diabetic patients; it’s meant to be used by primary care physicians to identify patients requiring a referral to an ophthalmologist for further management.3-6 Years of research by the University of Iowa’s Michael Abramoff, MD, PhD, and colleagues established the clinical efficacy of machine learning to use retinal photos to detect referable DR.4 Studies of the technology’s implementation in various primary care office settings supported the clinical trials, leading to eventual FDA approval. Of note, the IDx-DR algorithm only lets the primary care provider know if the patient has DR worrisome enough to be referred to an eye doctor, termed referable DR, which equates to moderate non-proliferative DR or worse. It doesn’t report on the absence of diabetic retinopathy or the presence of mild DR not requiring referral.
Another technology on the horizon is a home-based OCT device from Notal Vision (Notal Vision Ltd, Tel Aviv, Israel), which uses a machine learning algorithm trained on patients with exudative retinal diseases such as DR and AMD.7 Notal’s home-based OCT machine is intended to be used by high-risk patients, and may autonomously detect changes in retinal morphology with accumulation of new fluid. If such worsened disease activity is detected, the patient’s ophthalmologist is directly contacted. Notal received a “breakthrough device” designation from the FDA in 2018, and expects the device to be commercially available by 2020.
Research in Retinal Disease
AI in ophthalmology has focused most heavily on the field of retina. This makes sense, given the large number of images captured by retina specialists, the variety of imaging modalities capable of evaluating the retina, and the vast burden and vision-threatening nature of retinal disease. Multiple fundus photograph databases exist, and clinical use of OCT is prevalent, making the accumulation of these images easy for AI researchers.
Diabetic retinopathy screening and diagnosis is a major focus within AI. Most AI-based diabetic retinopathy screening programs have focused on identifying diabetic patients with referable DR.4-6,8-12 As a benchmark, a 2004 study found that ophthalmologists have a 73-percent sensitivity and 91-percent specificity in their ability to detect diabetic retinopathy on a dilated fundus examination.13 By comparison, in 2016, Google published a novel study in JAMA showing that referable DR could be identified with a sensitivity of 97.5 percent and a specificity of 98.5 percent using fundus photos alone.8 A separate larger study which validated its algorithm on a more varied multi-ethnic patient population reported sensitivities and specificities as high as 91 percent and 92 percent.9
It’s worth noting that more robust algorithms more applicable to real- world populations can result from the use of datasets with more diverse metrics, such as more heterogeneous populations and varied fundus camera modalities. Additional studies have shown promise in stratifying between various stages of DR, such as mild, moderate or severe non-proliferative DR and proliferative DR.8,9,14 Lastly, researchers recently reported the ability to reliably detect the presence of diabetic macular edema using color fundus photos alone.15
Screening and diagnosis for AMD have also been a focus of AI researchers. Using fundus photographs, OCT, or a combination of the two, researchers have been able to identify normal patients as well as those with evidence of AMD, to delineate areas of pathologic retinal fluid, and to grade the severity of AMD present.7,16-18 Fundus autofluorescence imaging has also been used to automatically discern areas of geographic atrophy.19
Perhaps more interesting than disease detection is the ability of machine learning to prognosticate on various clinical outcomes in patients with retinal disease. With 96-percent accuracy, a neural network was able to use color fundus photographs to predict which diabetic patients would need laser or surgical intervention versus no intervention at all.20 A second AI algorithm could use OCT retinal imaging to predict the need for an anti-VEGF injection in patients with neovascular AMD.21 Yet another machine learning algorithm could predict which eyes with intermediate AMD were most likely to progress to advanced disease (i.e., neovascular AMD or GA) based on OCT findings in combination with demographic and genetic factors with relatively high accuracy.22
While predicting disease progression can be useful to the clinician, predicting future visual acuity for patients with eye disease could be extremely beneficial for both the clinician and the patient. In one study, using the baseline OCT characteristics and demographic information of patients with diabetic macular edema, an AI algorithm could reliably predict visual acuity within 6.4 letters at one year and 6.81 letters at two years following an injection of ranibizumab.23 In AMD, a separate AI algorithm using OCT images and clinical data could predict a patient’s VA within 8.6 letters at one year.24
 |
AI Beyond the Retina
Even though the majority of AI-related research has focused on the posterior pole, anterior segment pathology and glaucoma have seen advances as well.
• Anterior segment disease. Automated cataract grading systems using AI have been developed for both adults and children, the latter of which can even discern which pediatric cataracts are most likely to require surgical intervention to preserve vision.25,26
Corneal ectatic disease has also received the attention of AI researchers, given the relative difficulty of detecting diseases such as keratoconus early on and the relatively recent availability of corneal cross-linking in the United States to help slow or halt the progression of ectasia, as well as these conditions’ potential for long-term visual morbidity. Using Scheimpflug data, corneal OCT imaging or a combination of the two, multiple algorithms have demonstrated reliable detection of keratoconus, with sensitivities and specificities of up to 100 percent.27-30 Others have reliably identified patients who are at risk of developing corneal ectasia after LASIK surgery, an AI application which stands to benefit refractive surgeons immensely.31
Lastly, using retinal photographs alone, a Google-led team of machine learning researchers showed success in predicting a patient’s spherical equivalent refractive error with a mean absolute error of 0.56 D.32 This finding, along with many others listed above, highlight some of the ways in which computer-based AI harnesses the power and potential of machine learning to improve clinical decision-making, which can augment the clinical abilities of physicians.
• Glaucoma. Glaucoma has also received considerable attention.33-36 Given the multiple data sources used by clinicians to test for glaucoma, machine learning is an ideal tool for assimilating the various testing modalities available to clinicians to screen for and diagnose glaucoma, such as visual fields, OCTs of the retinal nerve fiber layer and intraocular pressure readings. However, algorithms requiring much less data have also shown utility in accurately detecting glaucoma. Widefield OCT of the retina encompassing the optic nerve, as well as fundus photography, have each shown utility in detection of glaucoma and may simplify screening if users can make a diagnosis using the simplicity of single-image testing.35,36
Future Promise of AI
AI stands to support ophthalmologists’ clinical decision-making by supporting screening efforts, diagnostics and the prediction of diseases and their prognoses. It isn’t hard to imagine a future where the plethora of data collected from the medical record are combined with new data generated during a clinical visit, with the result being an AI report that helps with clinical decision-making. If computers are able to process the large volume of clinical data for each patient, the physician can spend more time considering the diagnosis and management plan and less time with data analysis. Improved information at the fingertips of ophthalmologists stands to improve patient outcomes and streamline clinical practice.
Machine learning may also be able to detect the presence or risk of developing both ophthalmic and systemic disease using eye imaging. Ophthalmic imaging is unique in that it allows doctors to directly assess blood vessels, neural tissue and connective tissue in living patients with high image quality and without the need for surgical pathologic specimens. Already, ocular images have proven useful in making medical assessments beyond eye-specific disease and have provided insights into patients’ overall health. For example, researchers have reported the ability to predict a patient’s age, gender, smoking status and systolic blood pressure based solely on retinal fundus photos.37
AI may also improve patients’ access to eye care, since remote and tele-ophthalmic screening initiatives could be combined with AI-enabled technologies to help diagnose and treat eye disease, thereby preventing vision loss. AI technologies, once developed, require fewer resources to operate than laborious, time-intensive and expensive human-led screening programs. Additionally, AI-based systems can
realize cost savings when compared to in-person clinical visits and manual image grading.38
Limitations
While there is considerable promise from AI, it’s not without potential risk.
First, there is a risk of deskilling the workforce with implementation of AI technologies. For example, if clinicians are aided to the point of not needing to diagnose disease on their own, then they may become reliant on the technology and may lose or blunt their diagnostic abilities.
Second, despite the impressive diagnostic accuracy of AI programs, some algorithms result in relatively high false-negative rates of detection of disease, which means that the algorithms incorrectly classify eyes as being disease-free or not requiring further evaluation. False negatives of this nature could be clinically disastrous for patients’ vision, highlighting the need for continued improvement in AI technology and appropriate interpretation of AI algorithm results.
Additionally, gaining patient trust in the use of automated AI screening and diagnosis is likely to be a hurdle. Studies show that many patients are willing to embrace machine learning in medicine, but that there are still patients who don’t trust computer-aided diagnosis and prefer in-person ophthalmic visits.39 Lastly, it’s not always obvious how a computer algorithm came to its clinical conclusion. Little insight into the “thought process” behind AI causes a “black-box” problem, whereby clinicians are left to trust the AI system blindly without being able to evaluate the value of the metrics used by the computer program.
In conclusion, artificial intelligence and machine learning in ophthalmology are enabling computer-assisted screening, diagnosis and prognostication of ophthalmic disease. Ophthalmology is uniquely poised to capitalize on the benefits of machine learning, given the abundance of clinical data and multimodal imaging generated in the field. Machine learning algorithms have been used in commercially available products, and have been applied in research applications focused on both anterior and posterior segment diseases. Advances in ophthalmology-specific AI stand to increase patient access to clinical screening and diagnosis as well as decrease health-care costs, especially when applied to high-risk populations, low-resource communities or when combined with telemedicine initiatives. REVIEW
Dr. Armstrong is chief resident in ophthalmology at Harvard University Medical School (AY 2019-2020) and director of the Ocular Trauma Service at the Massachusetts Eye and Ear Infirmary. Dr. Rahimy is a vitreoretinal specialist at the Palo Alto Medical Foundation.
Dr. Armstrong is a founding member of a tele-medicine ophthalmology device company, OculAR Technologies (OculAR, Boston). Dr. Rahimy is a consultant for Google.
You can contact Dr. Armstrong at: 243 Charles St., Boston, MA 02114. Phone: (617) 523-7900; fax: (617) 573-4028; email: grayson_armstrong@meei.harvard.edu.
1. Armstrong G, Lorch A. A(eye): A review of current applications of artificial intelligence and machine learning in ophthalmology. Int Ophthalmol Clin 2019. In press.
2. Consejo A, Melcer T, Rozema J. Introduction to machine learning for ophthalmologists. Seminars in Ophthalmology 2019;34:1:19-41.
3. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 2018;1:1:39.
4. Abràmoff MD, Folk JC, Han DP, et al. Automated analysis of retinal images for detection of referable diabetic retinopathy. JAMA Ophthalmol 2013;131:3:351-357.
5. van der Heijden AA, Abramoff MD, Verbraak F, van Hecke M V, Liem A, Nijpels G. Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System. Acta Ophthalmol 2018;96:1:63-68.
6. Abràmoff MD, Lou Y, Erginay A, et al. Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Investig Ophthalmol Vis Sci 2016;57:13:5200-5206.
7. Chakravarthy U, Goldenberg D, Young G, et al. Automated identification of lesion activity in neovascular age-related macular degeneration. Ophthalmology 2016;123:8:1731-1736.
8. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA - J Am Med Assoc 2016;316:22:2402-2410.
9. Ting DSW, Cheung CYL, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 2017;318:22:2211-2223.
10. Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 2017;124:7:962.
11. Li Z, Keel S, Liu C, et al. An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care. 2018;41:12:2509-2516.
12. Wilkinson CP, Ferris FL, Klein RE, et al. Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology 2003;110:9:1677-1682.
13. Lawrence MG. The accuracy of digital-video retinal imaging to screen for diabetic retinopathy: An analysis of two digital-video retinal imaging systems using standard stereoscopic seven-field photography and dilated clinical examination as reference standards. Trans Am Ophthalmol Soc 2004;102:321-340.
14. Ren F, Cao P, Zhao D, Wan C. Diabetic macular edema grading in retinal images using vector quantization and semi-supervised learning. Technol Heal Care 2018;26:S389-S397.
15. Arcadu F, Benmansour F, Maunz A, et al. Deep learning predicts OCT measures of diabetic macular thickening from color fundus photographs. Investig Ophthalmol Vis Sci 2019;60:4:852.
16. Venhuizen FG, van Ginneken B, Van Asten F, et al. Automated staging of age-related macular degeneration using optical coherence tomography. Investig Ophthalmol Vis Sci 2017;58:4:2318-2328.
17. Treder M, Lauermann JL, Eter N. Automated detection of exudative age-related macular degeneration in spectral domain optical coherence tomography using deep learning. Graefe’s Arch Clin Exp Ophthalmol 2018;256:2:259-265.
18. Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 2018;172:5:1122-1124.
19. Treder M, Lauermann JL, Eter N. Deep learning-based detection and classification of geographic atrophy using a deep convolutional neural network classifier. Graefe’s Arch Clin Exp Ophthalmol 2018:2053-2060.
20. Takahashi H, Tampo H, Arai Y, Inoue Y, Kawashima H. Applying artificial intelligence to disease staging: Deep learning for improved staging of diabetic retinopathy. PLoS One 2017;12:6:1.
21. Prahs P, Radeck V, Mayer C, et al. OCT-based deep learning algorithm for the evaluation of treatment indication with anti-vascular endothelial growth factor medications. Graefe’s Arch Clin Exp Ophthalmol 2018;256:91-98.
22. Schmidt-Erfurth U, Waldstein SM, Klimscha S, et al. Prediction of individual disease conversion in early AMD using artificial intelligence. Invest Ophthalmol Vis Sci 2018;59:8:3199-3208.
23. Chen S-C, Chiu H-W, Chen C-C, Woung L-C, Lo C-M. A novel machine learning algorithm to automatically predict visual outcomes in intravitreal ranibizumab-treated patients with diabetic macular edema. J Clin Med 2018;7:12:475.
24. Schmidt-erfurth U, Bogunovic H, Sadeghipour A, et al. Machine learning to analyze the prognostic value of current imaging biomarkers in neovascular age-related macular degeneration. Ophthalmol Retin 2018;2:1:24-30.
25. Cheung CY, Li H, Lamoureux EL, et al. Validity of a new computer-aided diagnosis imaging program to quantify nuclear cataract from slit-lamp photographs. 2018;52:3:1314-1319.
26. Long E, Lin H, Liu Z, et al. An artificial intelligence platform for the multihospital collaborative management of congenital cataracts. Nat Biomed Eng 2017;1:2:1-8.
27. Smadja D, Touboul D, Cohen A, et al. Detection of subclinical keratoconus using an automated decision tree classification. Am J Ophthalmol 2013;156:2:237-246.
28. Arbelaez MC, Versaci F, Vestri G, Barboni P. Use of a support vector machine for keratoconus and subclinical keratoconus detection by topographic and tomographic data. Ophthalmology 2012;119:11:2231-2238.
29. Cheboli D, Ravindran B. Detection of keratoconus by semi-supervised learning. Workshop on Machine Learning for Health-Care Applications, Helsinki, Finland; 2008.
30. Hwang ES, Perez-Straziota CE, Kim SW, Santhiago MR, Randleman JB. Distinguishing highly asymmetric keratoconus eyes using combined scheimpflug and spectral-domain OCT analysis. Ophthalmology 2018;125:12:1862-1871.
31. Lopes BT, Ramos IC, Salomão MQ, et al. Enhanced tomographic assessment to detect corneal ectasia based on artificial intelligence. Am J Ophthalmol 2018;195:223-232.
32. Varadarajan A V, Poplin R, Blumer K, et al. Deep learning for predicting refractive error from retinal fundus images. Invest Ophthalmol Vis Sci 2018.
33. Haleem MS, Han L, van Hemert J, et al. A novel adaptive deformable model for automated optic disc and cup segmentation to aid glaucoma diagnosis. J Med Syst 2018;42:1:1-18.
34. Kim SJ, Cho KJ, Oh S. Development of machine learning models for diagnosis of glaucoma. PLoS One 2017;12:5:1-16.
35. Muhammad H, Fuchs TJ, De Cuir N, et al. Hybrid deep learning on single wide-field optical coherence tomography scans accurately classifies glaucoma suspects. J Glaucoma 2017;26:12:1086-1094.
36. Li Z, He Y, Keel S, Meng W, Chang RT, He M. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 2018;125:8:1199-1206.
37. Poplin R, Varadarajan A V, Blumer K, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng 2018;2:3:158-164.
38. Tufail A, Rudisill C, Egan C, et al. Automated diabetic retinopathy image assessment software: Diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology 2017;124:3:343-351.
39. Keel S, Lee PY, Scheetz J, et al. Feasibility and patient acceptability of a novel artificial intelligence-based screening model for diabetic retinopathy at endocrinology outpatient services: A pilot study. Sci Rep 2018;8:1:8-13.
