Human Action Recognition using Machine Learning and Multimodal Data

With the increased availability of video and depth sensors, such as Xbox Microsoft Kinect, the automated analysis of human activity has now become an essential area of research in machine learning. Applications for such technology include visual surveillance, sports analysis, gaming and human-computer interactions. Variability in human shape, appearance, posture and individual style in performing motions makes the unified description of a given action difficult. In addition, sensor position, perspective, scene environment and operational conditions have a critical impact on the quality of recorded data. Given the complexity of the task, accurate automatic action recognition is currently restricted to a limited number of actions performed in very controlled environments. Deployment of such technology in real-world applications requires addressing many challenges including real-time constraints, data heterogeneity, noise and incompleteness.

Since it has been observed that most human activities can be described by intrinsically low dimensional data lying in a very high dimensional space [1], the last 15 years has seen the development of a diversity of manifold-based machine learning algorithms. Based on mathematically sound models, they have aimed at not only reducing in a nonlinear manner data dimensionality to make computations easier, but also normalising heterogeneous data in a common framework, reducing noise and biases. Although recently those techniques have been applied to visual surveillance data with very promising results [2-4], they are not yet operationally ready due to under-constrained data structure representations and insufficient removal of noise and bias.

The aim of this project is to design and implement the next generation of manifold-based learning algorithms able to handle the rich but noisy data generated by real-time multimodal surveillance systems. Successful completion of the project requires addressing the following scientific objectives:
1. Manifold construction constrained by neighbourhood graphs based on global optimisation processes integrating meta-data and training data bias
2. Hierarchical manifold learning including noise removal by taking advantage of the ‘two-manifold solution’
3. Incremental manifold learning appropriate to continuous data streaming

Applicants should have, at least, an Honours Degree at 2.1 or above (or equivalent) in Computer Science or related disciplines. In addition, they should have excellent programming skills in Matlab, Java and/or C++ and fundamental knowledge of machine learning.

Qualified applicants are strongly encouraged to informally contact the supervising academic, Dr Nebel ([Email Address Removed]), to discuss the application. More on Dr Nebel’s research group and activities can be found on his personal website: https://sites.google.com/site/jeanchristophenebel/