Optimising dynamic scheduling in smart manufacturing through reinforcement learning PhD

This PhD aims to develop an adaptive self-organising solution for dynamic scheduling in job-shop manufacturing. The research will introduce an innovative approach that integrates deep reinforcement learning and multi-agent systems to rapidly respond to emergencies and disruptions.

This approach will adapt scheduling strategies based on real-time data, considering collaborative utilisation of resources (e.g. machines, operators, materials) and human factors (e.g. fatigue, skill levels) to minimise tardiness in production tasks. The proposed approach will enhance decision-making in job sequencing, machine selection and worker assignment, leading to improved operational efficiency, enhanced worker satisfaction and reduced costs within dynamic manufacturing environments.

In today's rapidly evolving manufacturing world, efficient coordination of tasks/jobs is vital. Unforeseen dynamic events, such as sudden job insertion, machine breakdowns or operator unavailability, can unexpectedly disrupt production plans, leading to inefficiencies. These occurrences often introduce conflicting objectives, underscoring the need for dynamic multi-objective rescheduling methods that effectively balance time constraints and solution quality. Thus, dynamic scheduling, enabling schedules to be promptly adjusted in response to unexpected events, is crucial for maintaining production efficiency. While prior research has explored aspects of dynamic scheduling, the focus has mainly centred on classical shop scheduling and dynamic scheduling with new order insertions. Traditional methods rely on genetic algorithms and dispatching rules, providing near-optimal results but lacking efficiency and long-term efficacy.

Challenges also arise in handling the scheduling of collaborative resources (e.g. machines, operators, etc.), while optimising production time, resource utilisation and meet the desired production goals. Achieving optimal scheduling in such scenarios is challenging due to the interdependencies between different resources, the presence of uncertainties and the need to address various conflicting objectives simultaneously. Furthermore, recent literature in the field of dynamic scheduling has witnessed a shift in emphasis towards understanding the influence of human factors and workforce scheduling on manufacturing processes. Factors like worker fatigue and skill levels can significantly impact task efficiency. However, current approaches often overlook these human-centric aspects, limiting their adaptability in coping with uncertain dynamic disruptions within complex manufacturing environments. 

The primary focus of this project is to address the limitations of existing dynamic scheduling approaches in the context of complex manufacturing systems, characterised by unforeseen disruptions and multi-resource collaboration. The study seeks to address the collaborative scheduling of various resources, while considering human factors like worker fatigue and skill levels to minimise production task delays. To address these challenges, the project will focus on developing an innovative dynamic scheduling approach that integrates multi-agent systems and deep reinforcement learning techniques. Multi-agent systems consist of autonomous units that can make decisions collaboratively, allowing quick responses to disruptions. Deep reinforcement learning leverages advanced algorithms and neural networks to learn optimal decision-making strategies from trial and error. By combining these approaches, the project aims to develop a self-organising system that can dynamically adapt scheduling strategies based on real-time data. This approach will consider collaborative resource utilisation and human factors to minimise makespan, optimise resource utilisation and provide real-time responsiveness to unforeseen disturbances. The research will enable flexible decision-making for tasks such as job sequencing, machine selection and worker assignment.

The anticipated outcomes of this research hold the potential to enhance dynamic manufacturing scheduling and operational efficiency in complex manufacturing environments. Through the integration of deep reinforcement learning and multi-agent systems, the research anticipates the development of an adaptive self-organising solution capable of promptly responding to unexpected disruptions. This approach will not only optimise scheduling strategies based on real-time data but will also consider collaborative resource utilisation and human factors to mitigate tardiness in production tasks. The implementation of this innovative approach is expected to lead to impactful results: 

• A self-organising multi-agent system will be developed for resource scheduling, enabling manufacturers to swiftly address emergencies and disturbances. This will enhance production responsiveness and adaptability and contribute to reduced downtime, improved resource utilisation and overall optimised production schedules. 
• The development of a deep reinforcement learning model for job sequencing and machine selection will introduce a novel algorithm for tardiness estimation. Incorporating a reward mechanism that combines short-term and long-term returns, the model aims to enhance the convergence and effectiveness of reinforcement learning algorithms. The result will be more robust decision-making processes and efficient scheduling outcomes. 
• Through the development of an attention-based network, a deep reinforcement learning model will be created for worker assignment decisions. This network's innovative approach to effective decision-making is expected to enhance predictive capabilities. It is also expected to enhance the system's responsiveness considering human factors such as fatigue and competencies. 

Through case studies and numerical experiments, the research aims to demonstrate the superiority and effectiveness of the proposed method in addressing complex scheduling challenges in dynamic manufacturing environments. 

This self-funded PhD program offers a range of compelling advantages. It centres on applied research that not only advances your academic journey but also contributes to solving real-world challenges. The program offers diverse training experiences, both internally and externally, enriching your skill set and expanding your knowledge base. Pursuing this PhD at Cranfield University, renowned for its academic excellence, holds the potential to unlock promising career pathways. Moreover, the opportunity to interact with experts from academia and industry not only fosters extensive networking but also offers exposure to cutting-edge insights. This collaborative environment nurtures personal growth and equips you with valuable connections within your field. 

The student will gain from the experience in numerous ways, whether it be transferable skills in the technical area of optimisation and machine learning, or soft skills including presentation skills, project management, and communication skills. There are also numerous employability opportunities that the PhD will offer whether it be in Industry or in Academia.  

Entry requirements

 We are inviting applicants with a First or upper Second Class degree equivalent qualification in an engineering background, or an alternative quantitative focused discipline.  

Funding

This is a self-funded PhD; open to UK, EU and International applicants.

How to apply

To apply for this PhD opportunity please complete the application form below. 

Apply now

For further information please contact Christina Latsou

Email: Christina.latsou@cranfield.ac.uk?subject=PhD