Guiding Deep Networks for Expert Image Labelling Using Human Gaze

Deep learning has proven to be extremely effective in supervised tasks, surpassing the state-of-the-art in most areas, including segmentation, classification and object localisation. This has quickly built up a reliance on high-quality annotation data. In some domains such as general object classification, manual annotation of images is still cost-effective, even when transferring to completely new domains. In biological imaging, this is often not the case. Experts examining images will consider numerous separate metrics, weighing them together before arriving at an image-level decision. These complex and often ambiguous problems pose a real challenge to deep networks, where there are many possible ways to interpret an image and weigh up its features. This inevitably places an extremely heavy burden on annotators, who must label even more features in each image in order to provide effective supervision.

This PhD will explore techniques to leverage human gaze and fixation information, captured while annotation takes place, to more effectively guide the training of deep neural networks. Tools will be developed to allow experts to quickly analyse large sets of images, while information on where and when they look is recorded. A core part of the PhD will be the development of deep networks able to exploit this information through novel attention-driven techniques. A key measure of success will be the general nature of the approaches; the datasets and images used during this project will be widely varied. These will range from new large-scale datasets plants under heat and drought stress, through to generalised problems over widely-used public datasets. This work will have wide impact in a variety of fields.

This PhD will be based in the School of Computer Science. For further details, please contact Dr. Michael Pound: [Email Address Removed]