Analytical Connectionism 2023

Analytical Connectionism is a 2-week summer course on analytical tools for probing neural networks and higher-level cognition. The course brings together neuroscience, psychology and machine-learning communities, and introduces attendees to analytical methods for neural-network analysis and connectionist theories of higher-level cognition and psychology.

Connectionism, a key theoretical approach in psychology, uses neural-network models to simulate a wide range of phenomena, including perception, memory, decision-making, language, and cognitive control. However, most connectionist models remain, to a certain extent, black boxes, and we lack a mathematical understanding of their behaviours. Recent progress in theoretical neuroscience and machine learning has provided novel analytical tools that have advanced our mathematical understanding of deep neural networks, and can help us unravel these ‘black boxes’.

This course will introduce (1) mathematical methods for neural-network analysis, providing a solid theoretical and analytical overview of the tools available to understand neural-network models, and (2) key connectionist models with links to experimental observations, and which provide important targets for analytical results. To provide context for the applications of these methods, we will discuss phenomena in psychology that still lack an explanation. During the 2-week course you will:

• attend lectures on theoretical methods and applications, key connectionist models, and experimental observations
• participate in tutorials, Q&A sessions and panel discussions
• take part in activities such as poster and networking sessions

The course will run full-day Mondays-Fridays and end with a 1.5-day workshop, during which you will hear about current state-of-the-art and the limits of our understanding.

Course Lecturers and Organisers

Lecturers

Athena Akrami Sainsbury Wellcome Centre	Maria Eckstein DeepMind	Florent Krzakala EPFL
Jay McClelland Stanford University	Sebastian Musslick Brown University	Anna Rogers IT University of Copenhagen
Andrew Saxe Gatsby Unit; SWC	Haim Sompolinsky Hebrew University; Harvard University
Matt Lambon-Ralph University of Cambridge	Christopher Summerfield Oxford University	SueYeon Chung New York University; Flatiron Institute

 

Organisers

• SueYeon Chung, New York University; Flatiron Institute
• Stefano Sarao Mannelli, Gatsby Unit
• Andrew Saxe, Gatsby Unit; Sainsbury Wellcome Centre
• Peter E. Latham, Gatsby Unit
• I-Chun Lin, Gatsby Unit

Course Schedule

Week 1

• Florent Krzakala - Exact methods for the study on neural networks.
• Jay McClelland and Andrew Saxe - Neural network models of human cognition.
• Haim Sompolinsky - Neural networks as a model for neuroscience.

Week 2

• Discussion with SueYeon Chung, Matt Lambon-Ralph and Chris Summerfield, moderated by Andrew Saxe.
• Athena Akrami - Understanding memory at a neuroscientific level.
• Maria Eckstein - Computational cognitive modeling, reinforcement learning, and neural networks.
• Sebastian Musslick - A graph-theoretic analysis of parallel processing in neural network architectures.
• Anna Rogers - natural language processing.

For course schedule, see here.

List of Posters

1. Máté Aller - Efficiency and (lack of) flexibility in a deep learning model of human spoken word recognition
2. Jan Philipp Bauer - Quantifying rich and robust inductive biases in chaotic recurrent neural networks
3. Anna-Lea Beyer - The relationship between behavioural tasks and brain space
4. Victoria Bosche - The brain can’t copy-paste: End-to-end topographic neural networks as a way forward for modelling cortical map formation and behaviour
5. Chi-Ning Chou - Probing biological and artificial neural networks with task-dependent neural manifolds
6. Marianne de Heer Kloots - What components of NLP models drive similarity to brain activity in language processing? Layer- and head-level analyses
7. Mani Hamidi - Using representation-learning to guide efficient exploration  
8. Michael Hanna - Understanding subject-verb agreement in pre-trained language models: A circuits approach
9. Eghbal Hosseini - Teasing apart the representational spaces of ANN language models to discover key axes of model-to-brain alignment
10. Jaedong Hwang - Efficient exploration via fragmentation and recall
11. Akshay Kumar Jagadish - Using large-language models to meta-learn human inductive biases
12. Maximilian Mittenbühler - Human resource-rational planning: A neural network approach
13. Turan Orujlu - VividDreamer: Tokenized world model with stochastic attention
14. Mitchell Ostrow - Beyond geometry: Comparing the temporal structure of computation in neural circuits with dynamic mode representational similarity analysis
15. Alexandra Proca - Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks
16. Safura Rashid Shomali - Revealing hidden neuronal microcircuits from correlations among spiking neurons
17. Jirko Rubruck - Learning dynamics of semantic knowledge in humans and neural networks
18. Ábel Ságodi - An interpretable language for robust neural computation
19. Quilee Simeon - Dimensionality and dynamics of abstract representations
20. Sushrut Thorat - Characterising representation dynamics in recurrent neural networks for object recognition
21. Elia Turner - The simplicity bias in multi-task RNNs: Shared attractors, reuse of dynamics, and geometric representation
22. Sven Wientjes - Strategic cognitive control is bound to representations of temporal context

Information for Applicants - Application is now closed

Target Audience

This course is appropriate for graduate students, postdoctoral fellows and faculty in a variety of fields, from psychology and neuroscience, to physics, computer science, and mathematics. Attendees are expected to have a strong background in one discipline and to have made some effort to introduce themselves to a complementary discipline.

The course is limited to 40 attendees, who will be chosen to balance the representation of different fields. In circumstances where all other things are equal, priority will be given to applicants from underrepresented groups in STEM fields, using positive action under the Equality Act 2010 where appropriate.

Course Fees

There are no course fees, but attendees are expected to cover their own travel, accommodation and subsistence expenses.

Financial assistance may be available for successful applicants who find it difficult to take up a place for financial reasons. If funding becomes available, successful applicants will be asked to complete a financial aid request form if they need assistance. The amount of financial aid available will depend on the course funding from grants and sponsors.

Outcome Notification

We aim to notify applicants of the outcome by email w/c 10 July 2023. Due to the high volume of applications that we may receive, we are unable to provide individual feedback. Unfortunately we will not be able to offer everyone a place. Competitive applicants may be invited to join a waiting list in the event that a place becomes available.

How to Apply to Attend the Workshop Only

There are limited spaces available for researchers who are not taking the course to attend the workshop. Please visit the page 'Workshop' for more information and how to apply.