Computational learning and decision making

We study human learning, inference and decision making with behavioural experiments, neuroscience, and computational modelling.

The goal of our research is to understand the learning and decision processes which allow humans to function effectively in an uncertain and dynamic world. By developing computational models and assessing how they compare to human behaviour in experimental tasks, we hope to gain deeper insight into how information is processed and represented during learning and decision making.

Decisions in an uncertain and changing world

Of particular interest is how people learn to make decisions from experience. Traditional research on decision-making is often focussed on how people make "decisions under risk" in well-described situations (e.g., gambles). For instance, someone might be asked whether they prefer to play Gamble A, in which there is an 20% chance of winning 8 pounds, or Gamble B, in which there is a 30% chance of winning 6 pounds. In every-day life, decision problems are rarely encountered in such a neatly packaged format. Instead, people need to experience the outcomes of their actions and learn which actions are usually better than others. This usually leads to an exploration/exploitation trade-off, where people can either make a decision to obtain the best outcome, or make a decision in order to obtain more information about which alternative course of action is best.

Many decisions can be supported by evidence. We are interested in how people evaluate and integrate different sources of information and how this relates to characteristics of the evidence sources such as their reliability. Perceptual decision making is an area in which these characteristics can be precisely controlled. While it is known that the perceptual system integrates information (close to) optimally, the precise mechanisms which allow this performance are not fully understood yet.

Most real life situations are not static. Over time, the outcomes of actions, and preferences for the various outcomes, may change drastically and rapidly. How do we adapt our decision strategies in an uncertain and volatile world?

Active learning

Of related interest is the field of "active learning", which looks at how people choose between different sources of information in order to optimise their learning or decision making. For example, consider a doctor who needs to determine whether a patient is ill, and if so, which illness the patient suffers from. The doctor has some prior belief regarding the probability of various diseases and needs to determine which tests to run in order to strengthen these beliefs so she can come to a diagnosis. There are different ways in which to determine what the best possible test is. An important one is based on information theory and takes as the best test the one which is expected to maximally reduce the uncertainty in the doctor's beliefs. The question is whether real people (including doctors) follow such formal principles when searching for information. This question is not only interesting in its own right, but assessing how people search for information can also provide information about how people learn, and the types of representations that underly this learning. Computationally, active learning is closely related to (Bayesian) optimal experimental design, which is another topic of interest.

Adaptive design

Experiments are usually designed "statically", before they are conducted. While this allows a large degree of experimenter control, in order to design an experiment optimally, we usually need to know quite a bit about the expected behaviour of participants in an experiment. Moreover, even if we succeed in designing an experiment optimally for an average participant, when there are large individual differences, the design will be suboptimal for a large proportion of participants. In adaptive design, an experiment is designed "on the fly", while it is running. Using computational models, we can determine relevant characteristics of a participant from their earlier behaviour, and tailor the design to the participant. This can have substantial benefits when conducting studies to e.g. compare different theories.

Location

UCL Experimental Psychology
26 Bedford Way
London WC1H 0AP
United Kingdom