Multimodal Artificial Intelligence for Surgical Scene Understanding (25007)

Multimodal Artificial Intelligence for Surgical Scene Understanding in Robotic-Assisted Surgery (25007)

Primary Supervisors

Dr Evangelos Mazomenos -  Medical Physics and Biomedical Engineering, UCL, e.mazomenos@ucl.ac.uk

Subsidiary Supervisors:  Prof. Danail Stoyanov, Computer Science, UCL

Introduction

A four-year HOME funded PhD studentship is available in the UCL Department of Medical Physics and Biomedical Engineering. This position will be hosted at the UCL Hawkes Institute and will involve interdisciplinary work with clinical teams and human-computer interaction researchers under a recently awarded EPSRC project. Funding will be at least the UCL minimum. Read more details on funding here. 

The successful candidate will join our Research Degree in Medical Physics (application portal here), and benefit from the activities and events organised by the department.

Project Background

The Hawkes Institute at UCL has been awarded £1.4M funding from EPSRC for developing Human-centred Machine Intelligence for optimising Robotic Surgical Training (HuMIRoS). The HuMIRoS project is a collaboration between the Hawkes Institute, the UCL Interaction Centre and the Griffin Institute at Northwick Park Hospital.  

Robotic-assisted surgery (RAS) is well-adopted with over 1.8M RAS procedures performed every year. RAS is complex, highly variable, and as it is carried out under restricted access to the internal anatomy, operating without risk of injuring critical structures presents technical challenges and requires skill. Approximately 10-15% of UK surgical patients experience adverse events, of which 50% are preventable, while 10,624 adverse events relating to robotic procedures were reported in the US between 2000-13. 

The proposed research will develop surgical video analysis technology based on Artificial Intelligence (AI), focused on detecting and recognising intraoperative errors and assessing surgeon’s performance. Such technology has the potential to elevate the perception and understanding of the surgical environment, by identifying and highlighting critical anatomy and surgical tools and detecting surgical errors in real-time. Successful automation of these tasks can power computer-assisted navigation systems to reducing the risk of intraoperative errors, thereby increasing safety and efficiency during the delivery of RAS.

Research aims

Aim 1: Multimodal AI for skill estimation and error detection 
To develop machine learning methodologies to detect and characterise surgical errors and assess surgical performance by assigning performance scores. The aim will allow the exploration of fusion of video and robot kinematic data alongside semantic information (tool segmentation, task recognition) for developing multimodal AI architectures. Development and benchmarking will be supported by fully anonymised clinical datasets and experiments in dry-lab settings with realistic artificial models.  

Aim 2: Explainable AI for machine interpretation 
Error detection and skill analysis are complex tasks, requiring an understanding of human performance and contextualisation of surgical actions. To increase explainability and allow AI outputs to be appropriately communicated to the surgeon-user this aim will incorporate explainable AI methodologies to rationalise error detection/skill analysis outputs. 

Aim 3: Intelligent computer-assisted guidance in RAS training
This final objective will integrate outputs from Aims 1 and 2 for designing intelligent computer assistance in RAS. This will initially focus on surgical training experiments targeting at producing an automated feedback report highlighting the surgeons’ performance and providing guidance for skills improvement. Developments and initial deployments with open-source robotic platforms will take place in lab settings with phantom models and ex-vivo tissue. Translation of the technology in clinical settings will follow.

Person Specification & Requirements

• This studentship is only open to Home Fee-paying candidates. More information about fee status criteria.
• The ideal applicant will have an MSc in data science, computer vision or automatic control. The student is expected to have the desire to work in an interdisciplinary environment and a keen interest in biomedical engineering research that has a positive impact on the delivery of interventional healthcare.
• A good level of mathematical and computing skills and solid experience in computer programming (e.g. Python, MATLAB or similar) for data processing and algorithm development are essential. The student is also expected to demonstrate creative and critical thinking; excellent writing and oral communication skills; good working habits; ability to take initiative and work both in an independent and collaborative environment.
• Experience with any data modelling and analysis, computer vision, machine learning or control engineering, particularly with prior exposure to complex medical datasets would be advantageous but not essential.
• Candidates should hold a UK (or international equivalent) first or upper-second class Bachelor’s degree, preferably in computer science, mathematics, engineering, or a comparable subject. 

Funding

This is a full studentship available to Home-fee-paying students only. 

The successful student will receive a stipend starting from at least the UCL minimum (£22,780 in 2025/26) as well as the cost of tuition fees for Home fee students (£6,215 in 2025/26). 

The stipends awarded to PhD students at UCL are tax free and incur no income tax or national insurance contributions. The amount received increases each year over the duration of the studentship. 

UCL’s fee eligibility criteria

How to Apply

Please complete the following steps to apply: 

• Send an expression of interest and current CV to Dr Evangelos Mazomenos and medphys.pgr@ucl.ac.uk, quoting Project Code 25007 in the email subject line. e.mazomenos@ucl.ac.uk
• Make a formal application via the UCL application portal: Apply | Prospective Students Graduate - UCL – University College London. Please select the programme code ‘Medical Physics RRDMPHSING01’ and enter Project Code 25006 under ‘Name of Award 1’
• If shortlisted, candidates will be invited for an interview. 

Application Timeline:

• After the deadline, all applicants that expressed their interest and specified Project 25007 in their Portico application will be considered for interview.
• Candidates will normally be invited for interview within three weeks of the deadline. If you have not been contacted within this time-period, you have unfortunately not been successful in being shortlisted.
• The interview panel will normally consist of the supervision team on the project.
• Note that applications without specifying the project they are applying for and/or making a formal Portico application will be automatically rejected.
• If you are offered and accept a studentship position, a formal UCL Offer of Admission will be sent to you as well as an offer of studentship funding.