Multimodal Artificial Intelligence (23042)

Four-Year HOME Funded Studentship. Application Deadline: 21st March 2025

Multimodal Artificial Intelligence for Surgical Scene Understanding in Robotic-Assisted Surgery

Primary Supervisor: Dr. Evangelos Mazomenos (UCL MPBE)
Additional Supervisor: Prof. Danail Stoyanov (UCL CS)

Introduction

A four-year HOME funded PhD studentship is available in the UCL Department of Medical Physics and Biomedical Engineering. This position will be hosted at the Wellcome / EPSRC Centre for Interventional and Surgical Sciences (WEISS) and involve interdisciplinary work with clinical teams and human-computer interaction researchers under a recently awarded EPSRC project. Funding will be at least the UCL minimum. Stipend details can be found here. 

The successful candidate will join our Research Degree in Medical Physics and benefit from the activities and events organised by the department

Project Background

The Wellcome / EPSRC Centre for Interventional and Surgical Sciences (WEISS) at UCL has been awarded £1.4M funding from EPSRC for developing Human-centred Machine Intelligence for optimising Robotic Surgical Training (HuMIRoS). The HuMIRoS project is a collaboration between WEISS, the UCL Interaction Centre and the Griffin Institute at Northwick Park Hospital.  

Robotic-assisted surgery (RAS) is well-adopted with over 1.8M RAS procedures performed every year. RAS is complex, highly variable, and as it is carried out under restricted access to the internal anatomy, operating without risk of injuring critical structures presents technical challenges and requires skill. Approximately 10-15% of UK surgical patients experience adverse events, of which 50% are preventable, while 10,624 adverse events relating to robotic procedures were reported in the US between 2000-13. 

The proposed research will develop surgical video analysis technology based on Artificial Intelligence (AI), focused on detecting and recognising intraoperative errors and assessing surgeon’s performance. Such technology has the potential to elevate the perception and understanding of the surgical environment, by identifying and highlighting critical anatomy and surgical tools and detecting surgical errors in real-time. Successful automation of these tasks can power computer-assisted navigation systems to reducing the risk of intraoperative errors, thereby increasing safety and efficiency during the delivery of RAS.

Research aims

Aim 1: Multimodal AI for skill estimation and error detection 
To develop machine learning methodologies to detect and characterise surgical errors and assess surgical performance by assigning performance scores. The aim will allow the exploration of fusion of video and robot kinematic data alongside semantic information (tool segmentation, task recognition) for developing multimodal AI architectures. Development and benchmarking will be supported by fully anonymised clinical datasets and experiments in dry-lab settings with realistic artificial models.  

Aim 2: Explainable AI for machine interpretation 
Error detection and skill analysis are complex tasks, requiring an understanding of human performance and contextualisation of surgical actions. To increase explainability and allow AI outputs to be appropriately communicated to the surgeon-user this aim will incorporate explainable AI methodologies to rationalise error detection/skill analysis outputs. 

Aim 3: Intelligent computer-assisted guidance in RAS training
This final objective will integrate outputs from Aims 1 and 2 for designing intelligent computer assistance in RAS. This will initially focus on surgical training experiments targeting at producing an automated feedback report highlighting the surgeons’ performance and providing guidance for skills improvement. Developments and initial deployments with open-source robotic platforms will take place in lab settings with phantom models and ex-vivo tissue. Translation of the technology in clinical settings will follow.

Person Specification and Requirements

This studentship is open to Home Fee-Paying candidates only. More information is available in the Funding section of this webpage.

Candidates must have a UK (or international equivalent) first class or 2:1 honours degree preferably in computer science, mathematics, engineering, or a comparable subject.

The ideal applicant will have an MSc in data science, computer vision or automatic control. The student is expected to have the desire to work in an interdisciplinary environment and a keen interest in biomedical engineering research that has a positive impact on the delivery of interventional healthcare. 

A good level of mathematical and computing skills and solid experience in computer programming (e.g. Python, MATLAB or similar) for data processing and algorithm development are essential. The student is also expected to demonstrate creative and critical thinking; excellent writing and oral communication skills; good working habits; ability to take initiative and work both in an independent and collaborative environment.

Experience with any data modelling and analysis, computer vision, machine learning or control engineering, particularly with prior exposure to complex medical datasets would be advantageous but not essential.

Funding

This is a full studentship available to Home-fee-paying students only. 

The successful student will receive a stipend starting from at least the UCL minimum (£21,237 in 2024/25) as well as the cost of tuition fees for Home fee students (£6,035 in 2024/25). 

The stipends awarded to PhD students at UCL are tax-free and incur no income tax or national insurance contributions. The amount received increases each year throughout the studentship. 

UCL’s fee eligibility criteria can be found at this link.

Application Process

How to Apply

  1. Send an expression of interest and current CV to Dr Evangelos Mazomenos (e.mazomenos@ucl.ac.uk) and medphys.pgr@ucl.ac.uk, quoting Project Code 23042 in the email subject line.
  2. Make a formal application via the UCL application portal. Please select the programme code RRDMPHSING01 (Research Degree: Medical Physics) and enter Project Code 23042 under ‘Name of Award 1’.

Application Deadline and Timeline

The deadline for this application is 21st March 2025. The position is anticipated to start in October 2025. Applications will be assessed on a rolling basis so please do apply as early as possible.

  • After the deadline, all applicants who expressed their interests and specified Project 23042 in their Portico application will be considered for interview.
  • If shortlisted, candidates are normally invited for an interview three weeks after the deadline. Unfortunately, if you have not been contacted within this period, you have not successfully been shortlisted.
  • The interview panel will normally consist of the supervision team on the project. 
  • Note that applications without specifying the project they are applying for and making a formal Portico application will be automatically rejected.
  • If you are offered and accept a studentship position, a formal UCL Offer of Admission will be sent to you as well as an offer of studentship funding.