Don't miss our weekly PhD newsletter | Sign up now Don't miss our weekly PhD newsletter | Sign up now

  Visual Learning and Understanding with Multi-task and Limited Supervision

   Department of Informatics

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr Miaojing SHI  Applications accepted all year round  Funded PhD Project (UK Students Only)

About the Project

Driven by the increasing availability of large, annotated datasets and faster computational platforms, deep learning has been progressively employed for a broader spectrum of computer vision applications. In some scenarios, one might find that a machine performs even better than a human. On the other hand, a human, though sometimes less proficient at performing certain tasks, can perform a broader range of tasks, superior to any existing computer vision algorithms. Moreover, a machine has to learn with adequate data/labels to perform visual tasks, while a human effortlessly interprets visual scenes. To equip the machine perception with abilities that a human has, this PhD studies visual learning and understanding with multi-task and limited supervision.

Multi-task learning is common in deep learning, where clear evidence shows that jointly learning correlated tasks can improve on individual performances [1-2]. Notwithstanding many tasks are processed independently. The reasons are manifold: 1) many tasks are not strongly correlated, benefits might be obtained for only one or none of the tasks in joint learning [3]; 2) the scalability of learning multiple tasks is limited with the number of tasks [4]. Having a scalable and robust multi-task learning strategy is of substantial potential in many real applications, i.e., autonomous vehicle, robotic surgery.

Learning with limited supervision concerns both the lack of visual data and labels. The former addresses issues like rare classes or unseen classes which are featured as the long-tail problem [5] or few-shot problem [6], respectively. The latter is often referred to as weakly-supervised learning or semi-supervised learning where labels come in a weak form [7] or part of data come without labels [8]. Existing studies are mainly focused on the image classification task while for more complex visual tasks are just unfolding. 

This project will first push forward the current studies in the above two regimes; next, a combinatorial view of studying them together will be proposed. The project will be mainly focused on high-level visual understanding tasks, i.e., object detection and semantic segmentation, image captioning. More tasks might be involved when applying the developed paradigms to real applications, e.g., autonomous driving.  

The candidate should have a degree in Computer Science, Applied Mathematics or Electrical Engineering; solid mathematical background and programming skills; preferably, prior experience in computer vision, machine learning and deep learning. 

The studentship is funded for 3.5 years and includes tuition fees, a stipend at the [UK Research Council Rate][] plus London weighting, and allowance for research consumables and travel.

This studentship is valid for home students or international students who have a pre-settled status in the UK. The target starting date is Jun/Oct 2022. The PhD will be supervised by Dr Miaojing Shi and Dr Michael Spratling. Work will be carried out within the Department of Informatics, King’s College London.  

Application Instructions: Candidates are requested to send an initial expression of interest to Miaojing Shi ([Email Address Removed] ) preferably with updated CV and research proposal. 

Computer Science (8) Engineering (12) Mathematics (25)

Funding Notes

The studentship is funded for 3.5 years and includes tuition fees, a stipend at the [UK Research Council Rate][] plus London weighting, and allowance for research consumables and travel.


[1] Y. Yang, T.M. Hospedales, (2017). Trace Norm Regularized Deep Multi-Task Learning. In ICLR workshop 2017.
[2] Ishan Misra∗ Abhinav Shrivastava∗ Abhinav Gupta, et al. Cross-stitch Networks for Multi-task Learning. In CVPR 2016.
[3] T. Standley, A. R. Zamir, et al., Which tasks should be learned together in multi-task learning? arXiv, 2019
[4] A. R. Zamir, A. Sax, et al., Taskonomy: Disentangling task transfer learning. In CVPR, 2018.
[5] L. Xiang, G. Ding, J. Han. Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification. In ECCV 2020.
[6] C. Simon, P. Koniusz, et al., Adaptive subspaces for few-shot learning. In CVPR, 2020.
[7] Z. Zeng, B. Liu, et al., Wsod2: Learning bottom-up and top-down objectness distillation for weakly-supervised object detection. In ICCV, 2019.
[8] A. Tarvainen and H. Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NIPS, 2017.
Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.