UCL Centre for Medical Image Computing


Rema Daher & Juana Gonzalez Bueno Puyal- CMIC/WEISS Joint Seminar Series

16 February 2022, 1:00 pm–2:00 pm

Rema Daher & Juana Gonzalez Bueno Puyal- talks as part of CMIC/WEISS Joint Seminar Series

Event Information

Open to





UCL Centre for Medical Image Computing and Wellcome/EPSRC Centre for Interventional and Surgical Sciences

Speaker: Rema Daher

TitleA Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence


Video streams are utilised to guide minimally-invasive surgery and diagnostic procedures in a wide range of procedures, and many computer assisted techniques have been developed to automatically analyse them. These approaches can provide additional information to the surgeon such as lesion detection, instrument navigation, or anatomy 3D shape modeling. However, the necessary image features to recognise these patterns are not always reliably detected due to the presence of irregular light patterns such as specular highlight reflections. In this paper, we aim at removing specular highlights from endoscopic videos using machine learning. We propose using a temporal generative adversarial network (GAN) to inpaint the hidden anatomy under specularities, inferring its appearance spatially and from neighbouring frames where they are not present in the same location. This is achieved using in-vivo data of gastric endoscopy (Hyper-Kvasir) in a fully unsupervised manner that relies on automatic detection of specular highlights. System evaluations show significant improvements to traditional methods through direct comparison as well as other machine learning techniques through an ablation study that depicts the importance of the network's temporal and transfer learning components. The generalizability of our system to different surgical setups and procedures was also evaluated qualitatively on in-vivo data of gastric endoscopy and ex-vivo porcine data (SERV-CT, SCARED). We also assess the effect of our method in computer vision tasks that underpin 3D reconstruction and camera motion estimation, namely stereo disparity, optical flow, and sparse point feature matching. These are evaluated quantitatively and qualitatively and results show a positive effect of specular highlight inpainting on these tasks in a novel comprehensive analysis.

Speaker: Juana Gonzalez Bueno Puyal

TitleSpatio-temporal information for colorectal cancer detection 


Colorectal cancer is the third most prevalent cancer worldwide and early detection and treatment can significantly improve the patient’s prognosis. During colonoscopies, the bowel is inspected, and detection and characterisation of pre-cancerous polyps is carried out. Deep learning has been used as a support tool and has shown improved performance. However, analysis of colonoscopy videos can be extremely challenging. Humans aggregate temporal information from videos to make decisions, and in the same way we have explored the use of spatio-temporal techniques for colorectal cancer detection. In this talk we present different approaches to incorporate temporal information into a CNN with methods that include an efficient Hybrid 2D/3D network, LSTM modules or simple postprocessing.  


Chair: TBC