Multi-Modal Representation Learning for Computer Vision Applications

Recent advances in imaging, networking, data processing and storage technology have resulted in an explosion in the use of multi-modal sensory data in many fields, including medical imaging, urban monitoring, robot vision and many others. Integration of data, such as audio, text, image, video from multiple channels, can provide complementary information, thus increasing the accuracy of the overall decision-making process. Currently, seeking an efficient way to analyse, mine and understand such large-scale, multi-modal and noisy data is a challenging and interesting research topic, where the core problem is learning representations from the data.

About the project

In this PhD work, we will formalize the ideas of representational geometry/conceptual spaces in a tractable, deep probabilistic framework, and use it to develop a new type of bio-inspired models capable of several novel aspects of human-like learning, including learning 1) from few examples, 2) from multiple modalities (visual, spatiotemporal, and text data), 3) grounding concept representations in perceptions and action possibilities. The targeted applications include healthcare, smart environment, video monitoring, and robot vision.

For information and informal enquiries, please contact: Prof. Jungong Han ([Email Address Removed]).

Supervisor bio

Prof. Jungong Han’s CV can be found at https://www.sheffield.ac.uk/dcs/people/academic/jungong-han

About the Department and Research Group

The student will work in the computer vision group within the department of computer science at the University of Sheffield.

99 percent of our research is rated in the highest two categories in the REF 2021, meaning it is classed as world-leading or internationally excellent.

Candidate requirements

• A minimum 2:1 undergraduate and/or postgraduate masters’ qualification (MSc) in a science and technology field: Computer Science, Engineering, Mathematics, with specialisation in Computer Vision, Machine Learning and AI
• Familiarity with machine learning and probabilistic models
• Relevant software knowledge and experience, for example Python and tensor frameworks (PyTorch or TensorFlow), C++, etc
• Excellent written and verbal communication skills
• The English language requirements must also be met. Information on this can be found here: https://www.sheffield.ac.uk/postgraduate/english-language

How to apply

To apply for a PhD studentship, applications must be made directly to the Department Of Computer Science.

University of Sheffield using the Postgraduate Online Application Form. Make sure you name Jungong Han as your proposed supervisor. Information on what documents are required and a link to the application form can be found here - https://www.sheffield.ac.uk/postgraduate/phd/apply/applying

The form has comprehensive instructions for you to follow, and pop-up help is available.