UCLIC - UCL Interaction Centre


Simulating Interaction Movements with Optimal Feedback Control and Deep Reinforcement Learning

03 July 2024, 3:15 pm–4:15 pm


Event Information

Open to



Professor Duncan Brumby – UCL Interaction Centre


Room G01
66-72 Gower Street
United Kingdom

The increasing use of VR and AR applications gives rise to the development of new interaction techniques. Inconveniently, the indispensable evaluation and proper fine-tuning of such techniques requires extensive user studies, which are costly and time-consuming. In particular, when interactions are performed using the whole body, factors such as ergonomics and fatigue become critical. However, these are particularly difficult to measure in vivo, and are not predictable with most previous approaches that focused on quantitative models such as Fitts’ Law or simple end-effector models such as Minimum Jerk.

However, recent advances in forward biomechanical models, optimal control and deep reinforcement learning enable the simulation of real user behaviour, incorporating both muscle control and perception. By predicting the whole movement of users, ergonomic factors of interaction techniques can be evaluated in vitro, even for a large number of different settings. This facilitates the optimisation of parameters to enhance interaction.

To this end, we formulate the interaction of humans with computers as an optimal control problem. We then explore different optimal feedback control methods in their ability to predict user behaviour, in particular, LQR, LQG, and MPC. In contrast to these classic approaches, we also evaluate and discuss deep reinforcement learning agents trained on different interaction tasks, and outline how these simulations could be applied to the optimisation of interaction technique parameters.

The seminar is also available for online attendance via Zoom: https://ucl.zoom.us/j/91801473411.

About the Speaker

Markus Klar

Post-Doctoral Research Associate at University of Glasgow


Photo of Markus Klar
Markus Klar is a post-doctoral research associate at the University of Glasgow where he is part of the DIFAI project. His research focuses on Computational Interaction, in particular, simulating interactions with optimal feedback control, deep reinforcement learning and active inference. Read more here: https://mkl4r.github.io/


More about Markus Klar