In this PhD, the student will develop an approach combining eye tracking with deep learning to use the resulting attentional maps both in place of expensive segmentation, and to identify where expert’s look to arrive at a decision. To achieve this, new software tools and algorithms will be developed to allow the use and application of these systems.
Deep learning systems are promising to revolutionise medical image analysis, achieving incredible performance at segmentation and scoring tasks. There are two challenges to adoption, though:
First, large, well-annotated datasets are required to train the networks. Although we would like to only have to label an image with a diagnostic outcome for training, in practice an intermediate segmentation is often required. Without first labelling the regions of interest, the deep network training problem is too unconstrained – the networks do not know implicitly which areas within an image set are most important for arriving at a diagnostic decision or producing an image score. Whilst providing manual labelled regions as part of the training process can help with this, this is very time consuming and labour intensive.
Second, there is a concern about how decisions are arrived at. This is true with human experts as well as AI systems; if three clinicians make a particular diagnosis from an image, but a fourth makes a contradictory diagnosis, we should be concerned with what information was used to make the outlying decision. With human experts this can, to some extent, be resolved by discussion, but not always – justifying a decision making process can be complex. With AI systems, at the moment the ability to justify a decision is still in its infancy.
In this studentship, the student will directly address both of these challenges by developing a new technology-based approach. By using modern eye tracking systems, we propose to track the attention of clinicians as they make decisions. This can then be used to generate an attention map of where the experts have looked when making a clinical decision.
This attention map can then be used in two ways:
1) As an additional input to an AI system to help neural networks focus on the important areas in an image.
2) The attention map can also be used to examine where experts looked when arriving at a decision.
To ensure success of the work, whilst we will be training a computer science student, we have formed a collaboration between the schools of Computer Science, Psychology and Medicine to: develop the computational methods, understand how we can use visual attention to best effect, and integrate with a clinical radiological investigation respectively. Therefore there is much cross-discipline training potential for the student.
The student will benefit from access to the deep learning development hardware in the computer science Computer Vision Lab (https://www.nottingham.ac.uk/research/groups/cvl/
), as well as embedding in the existing group of PhDs and postdocs in the lab.