Don't miss our weekly PhD newsletter | Sign up now Don't miss our weekly PhD newsletter | Sign up now

  Enhancing 3D Control of Robotic Limbs Using Brain-Computer Interfaces with Reinforcement Learning


   Department of Computer Science

  , , ,  Sunday, March 09, 2025  Funded PhD Project (Students Worldwide)

About the Project

About the Project

The Institute for the Augmented Human at the University of Bath is inviting applications for the following fully funded PhD project, which will commence 29 September 2025. An earlier start date in the 2024/25 academic year is possible, if agreed with the supervisors and Doctoral College.

This advert may close early if a suitable candidate is identified. It is therefore strongly recommended to contact the lead supervisor before applying and submit a formal application as soon as possible.

Project

The potential to control a prosthetic limb by decoding 3D limb movement intention or imagination directly from the brain using an electroencephalography (EEG) brain computer interface (BCIs) has recently been demonstrated [1],[2],[3],[4]. Decoding movement trajectories directly from EEG presents challenges, such as variable correlation (0.2<R<0.8) between movement and decoded movement trajectories, along with substantial training and calibration demands. EEG's challenges include low signal-to-noise ratio (SNR), non-stationarity, and high inter-person variability. Addressing these issues requires robust AI approaches. Roy et al. [5] systematically assessed deep learning in EEG-based BCIs highlighting its promise but also the field’s lag behind AI advancements in text, speech, and vision. The impact of large, labelled datasets and generative AI models on decoding 3D limb trajectories remains unclear however it is clear that a cooperative learning strategy within the prosthetic controller is essential to reduce reliance on the BCI user and decoder.

This project will focus on reinforcement learning (RL) [6],[7],[8],[9],[10],[11],[12] to develop a cooperative learning controller for upper-limb prostheses or exoskeletons, addressing inaccuracies in decoding neural signals. RL may be deployed in several ways to address these challenges:

  1. Adaptation to Noisy Signals: RL algorithms adapt to noise by identifying consistent signal-response patterns, maximizing rewards, and developing noise-tolerant policies through training in noisy environments.
  2. Personalized Calibration: RL enables personalized control strategies by optimizing a reward function reflecting the user’s ability to perform smooth, accurate movements, enhancing usability.
  3. Real-Time Feedback and Adaptation: RL agents refine control policies dynamically using real-time feedback. Algorithms like Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) support continuous updates during operation.
  4. Robustness to Environment Changes: RL systems trained with domain randomization generalize across diverse environments, building robust policies.
  5. Multi-Signal Integration: RL learns to optimally integrate EEG and EMG signals, prioritizing inputs based on reliability. Hierarchical RL assigns sub-policies for signal-specific challenges within a unified framework.
  6. Reducing Calibration Time: Transfer learning allows pre-trained RL models to serve as baselines, reducing personalization time. Online RL further adapts during use, minimizing calibration needs.
  7. Reward Shaping: RL rewards partial successes, encouraging gradual improvement. Incorporating user feedback into reward signals aligns the system with user preferences.
  8. Safety and Error Correction: Safe RL methods, such as Constrained Policy Optimization (CPO), ensure learned policies respect safety constraints. Reward penalties discourage unsafe or unstable movements.

The project will evaluate these possible augmentations provided by RL and deploy RL through a two-phase framework: (1) simulation-based offline training and (2) human-in-the-loop RL, incorporating user feedback via VR-based hand simulations and prosthetic exoskeletons. This hybrid system combines supervised learning for initial signal decoding with RL for fine-tuning and error correction, improving decoding accuracy and movement control for better user experience and integration into daily life.

Aligned with ongoing research, the project leverages software, AI, hardware, and experimental paradigms for trials involving augmented reality, virtual reality, and real-world prosthetic limb and/or exoskeleton manipulation. The goal is to create a robust, user-friendly system that bridges the gap between decoding accuracy and practical application, advancing BCI-driven assistive and rehabilitation technologies. 

Candidate Requirements

We encourage applicants from any science or engineering background.

Applicants should hold, or expect to receive, a First Class or good Upper Second Class Honours degree (or the equivalent). A master’s level qualification would also be advantageous.

Non-UK applicants must meet our English language entry requirement.

Enquiries and Applications

Informal enquiries are encouraged! Direct these to Prof Damien Coyle -

Please make a formal application should via the University of Bath’s online application form for a PhD in Computer Science.

When completing the form, in the 'Funding your studies' section, please select 'Donor or Alumni Funded Studentship' from the first drop-down list and specify Nick Hynes PhD Scholarship in the text box. In the 'Your PhD project' section, please state the project title and supervisor's name in the appropriate boxes.

Failure to complete these steps will delay the processing of your application and may cause you to miss the deadline.

More information about applying for a PhD at Bath may be found on our website.

Equality, Diversity and Inclusion

We value a diverse research environment and strive to be an inclusive university, where difference is celebrated and respected. We encourage applications from under-represented groups.

If you have circumstances that you feel we should be aware of that have affected your educational attainment, then please feel free to tell us about it in your application form. The best way to do this is a short paragraph at the end of your personal statement.

The Disability Service ensures that individuals with disabilities are provided the support that they need. If you state if your application that you have a disability, the Disability Service will contact you as part of this process to discuss your needs.

Keywords: AI, Reinforcement learning, EEG, EMG, neurotechnology, brain-computer interface, exoskeleton, prosthetic limb, motion trajectory prediction/decoding estimation 

Computer Science (8) Engineering (12)

Funding Notes

Applicants will be considered for a fully funded 4 year Nick Hynes PhD studentship for the augmented human. This will cover the tuition fee at the Home or Overseas rate, a maintenance stipend at the UKRI rate (£19,237 in 2024/25) and a training and research budget of £5000 per annum. More info here: View Website


References

[1] A. Korik, R. Sosnik, N. Siddique, and D. Coyle, “Decoding imagined 3D hand movement trajectories from EEG: Evidence to support the use of mu, beta, and low gamma oscillations,” Front. Neurosci., vol. 12, no. MAR, 2018, doi: 10.3389/fnins.2018.00130.
[2] A. Korik, R. Sosnik, N. Siddique, and D. Coyle, “Decoding Imagined 3D Arm Movement Trajectories From EEG to Control Two Virtual Arms — A Pilot Study,” vol. 13, no. November, pp. 1–22, 2019, doi: 10.3389/fnbot.2019.00094.
[3] N. M. Shane, D. K. McCreadie, D. D. Charles, D. A. Korik, and P. D. Coyle, “Online 3D Motion Decoder BCI for Embodied Virtual Reality Upper Limb Control: A Pilot Study,” 2022 IEEE Int. Work. Metrol. Ext. Reality, Artif. Intell. Neural Eng. MetroXRAINE 2022 - Proc., pp. 697–702, 2022, doi: 10.1109/MetroXRAINE54828.2022.9967577.
[4] N. McShane, A. Korik, K. McCreadie, D. Charles, and D. Coyle, “Decoding Motion Trajectories in an Online Upper Limb BCI: Linear Regression Vs Deep Learning,” in IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)., 2023, p. accepted.
[5] Y. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and J. Faubert, “Deep learning-based electroencephalography analysis: a systematic review,” J. Neural Eng., vol. 16, no. 5, p. 051001, Aug. 2019, doi: 10.1088/1741-2552/ab260c.
[6] J. DiGiovanna, B. Mahmoudi, J. Fortes, J. C. Principe, and J. C. Sanchez, “Coadaptive Brain–Machine Interface via Reinforcement Learning,” IEEE Trans. Biomed. Eng., vol. 56, no. 1, pp. 54–64, Jan. 2009, doi: 10.1109/TBME.2008.926699.
[7] S. Reddy, S. Levine, and A. D. Dragan, “First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization,” May 2022, [Online]. Available: http://arxiv.org/abs/2205.12381.
[8] C.-R. Phang and A. Hirata, “Multiagent Copilot Approach for Shared Autonomy between Human EEG and TD3 Deep Reinforcement Learning,” Dec. 2023, [Online]. Available: http://arxiv.org/abs/2312.14458.
[9] X. Gao, J. Si, Y. Wen, M. Li, and H. H. Huang, “Knowledge-Guided Reinforcement Learning Control for Robotic Lower Limb Prosthesis,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), May 2020, pp. 754–760, doi: 10.1109/ICRA40945.2020.9196749.
[10] Q. Li, T. Zhang, G. Li, Z. Li, H. Xia, and C.-Y. Su, “Neural-Dynamics Optimization and Repetitive Learning Control for Robotic Leg Prostheses,” IEEE/ASME Trans. Mechatronics, vol. 27, no. 2, pp. 811–822, Apr. 2022, doi: 10.1109/TMECH.2021.3071936.
[11] X. Wang, J. Xie, S. Guo, Y. Li, P. Sun, and Z. Gan, “Deep reinforcement learning-based rehabilitation robot trajectory planning with optimized reward functions,” Adv. Mech. Eng., vol. 13, no. 12, Dec. 2021, doi: 10.1177/16878140211067011.
[12] M. Mohammedalamen, W. D. Khamies, and B. Rosman, “Transfer Learning for Prosthetics Using Imitation Learning,” Jan. 2019, [Online]. Available: http://arxiv.org/abs/1901.04772.

Open Days


Register your interest for this project



How good is research at University of Bath in Computer Science and Informatics?


Research output data provided by the Research Excellence Framework (REF)

Click here to see the results for all UK universities

Where will I study?

Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.