Abstract:
Convolutional Neural Networks (CNNs) have emerged as powerful tools in ophthalmology by exhibiting human-level performance in the classification of various ocular conditions and diseases. However, the decisions of these models are opaque and hard to interpret, which limits their trustworthiness and applicability in clinical settings. In this thesis, we address the challenges through: an inherently interpretable model architecture called BagNet and counterfactual explanations. BagNets use small receptive fields to identify local features in an image that contribute significantly to the model's decision. On the other hand, counterfactual explanations show the changes required on a given input image to alter the decision of the classifier to a particular class. Importantly, both of these explanation methods share similarities with processes employed by humans to offer explanations for their decisions. Intriguingly, CNNs demonstrate remarkable accuracy in predicting gender from retinal fundus images even though it was previously unknown to ophthalmologists that retinal fundus images encoded gender information. Here, it would be beneficial to explain the CNN model's decisions in order to identify features that the model uses for distinguishing between male and female fundus images. To this end, we utilized the local feature importance estimates from BagNets to produce saliency maps that highlight informative patches in fundus images. Our analysis revealed that patches from the optic disc and macula contribute significantly, with the former favoring detection of male fundus images and the latter, female. We conclude that BagNets are feasible alternatives to standard CNN architectures which have the potential to serve as an effective approach to provide explanations in medical image analysis tasks. Following our study on explanations from BagNets, we investigated the generation of counterfactual images from CNN classifiers to provide explanations. Specifically, we assessed various counterfactual generation techniques across a range of retinal disease classifiers in ophthalmology. The first technique relied on the generation of counterfactual images solely using the gradients of a classifier with respect to the input. Here, adversarially robust models offered more interpretable gradients than a standard classifier although at the expense of reduced accuracy. We combined the strengths of both approaches by ensembling a standard CNN with an adversarially robust one. Our ensemble method achieved high accuracies comparable to the standard CNN while also generating meaningful visual counterfactuals. However, a notable limitation of this classifier-only approach is a lack of realism of the generated counterfactuals. To achieve realism, the second technique employed a diffusion model alongside adversarially robust and plain classifiers trained on retinal disease classification tasks from color fundus photographs and optical coherence tomography (OCT) B-scans. The gradients of the classifiers guide the diffusion model effectively, enabling it to add or eliminate disease-related lesions in a realistic manner. In a user evaluation, domain experts rated the counterfactuals generated using this approach as significantly more realistic than those produced by the classifier-only method and found them indistinguishable from real images. We conjecture that such realistic counterfactual explanations hold significant promise for assisting clinicians in decision-making processes. To summarize, BagNets provide saliency map based explanations by highlighting image regions that have a substantial impact on the model's final decision. In contrast, counterfactuals illustrate the actual visual features that are relevant to the classifier's decision making process by generating varied versions of the input image corresponding to each class in the task. Overall, both of these methods offer visual explanations pertaining to the model's decisions albeit through different mechanisms.
Explainable and interpretable AI models