In the middle of applying to universities? | SHARE YOUR EXPERIENCE In the middle of applying to universities? | SHARE YOUR EXPERIENCE

Development of Change Transformer—Attentive Deep-Learning Approaches for Abnormal Change Detection

   Faculty of Computing, Engineering and the Built Environment

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr Y Bi  No more applications being accepted  Competition Funded PhD Project (Students Worldwide)

About the Project

Deep learning is a hot topic in machine learning and it is playing increasingly important roles in intelligence systems for our daily lives, such as computer vision, autonomous car driving, earth observation, etc. Deep learning is typically data driven, there is little domain knowledge utilised in training neural network models and determining decision boundaries. Learning process mainly focuses on searching and optimizing network models by gradient descent functions, without accounting for inherent discriminant and dependent characteristics of data distribution. Such a learning paradigm often makes it more difficult to achieve the convergence of training models, and therefore the resulting network models are not easy to generalize, typically in machine translation. These observations have motivated the recent development of deep Transformer Networks. 

Transformer Networks make extensive use of attention mechanisms to discriminate the representative parts of data distribution based on contextual information and fade out the rest, thereby devoting more learning processes to deal with that small but representative part of the data for classification, recognition or prediction tasks. Currently ‘the two most common attention techniques used are dot-product attention, which uses the scalar product between vectors to highlight important parts of data, and multi-head attention, which uses combined attention to direct the overall attention of a network or sub-network’.  

The proposed project will study existing attention techniques and develop new attention mechanisms that will be used as heuristics in a transformer network model. The project will incorporate one of two application scenarios into the development of attention: 1) detecting abnormal change from images, and 2) anomaly detection in electromagnetic satellite signals, which are part of the ongoing work undertaken in the supervisory team. For the former case, the development of an attention mechanism could be inspired by human visual attention, in which humans often focus on an object of interest in an image with high resolution while perceiving the surrounding of the object at low resolution over time. Whereas for the latter case, as electromagnetic satellite signals might contain a range of events, not all parts of signals are equally relevant for a specific event. This observation could motivate development of an attention based on the contextual characteristics of an event to determine the relevant portions of signals. 

The transformer networks developed may be applicable to various real world application scenarios, such as monitoring abnormal singale change in Earth Observation, detecting faults in assemble lines of manufacturing processes, or human’s health deterioration in the context of healthcare. 


Xiangzeng Kong, Yaxin Bi, David H. Glass. Detecting anomalies in sequential data augmented with new features. Artif. Intell. Rev. 53(1): 625-652 (2020)
Stuart J. Blair, Yaxin Bi, Maurice D. Mulvenna. Aggregated topic models for increasing social media topic coherence. Appl. Intell. 50(1): 138-156 (2020)
Vyron Christodoulou, Yaxin Bi, George Wilkie. A Fuzzy Shape-Based Anomaly Detection and Its Application to Electromagnetic Data. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 11(9): 3366-3379 (2018)
Yaxin Bi, Jiwen Guan, David A. Bell. The combination of multiple classifiers using an evidential reasoning approach. Artif. Intell. 172(15): 1731-1751 (2008)

How good is research at Ulster University - Belfast Campus in Computer Science and Informatics?

Research output data provided by the Research Excellence Framework (REF)

Click here to see the results for all UK universities
Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.

PhD saved successfully
View saved PhDs