Objective Structured Professional Assessments (OSPAs)

An OSPA is an Objective Structured Professional Assessment similar to an OSCE (Objective Structured Clinical Examination), used to assess clinical skills in medicine. The purpose of an OSPA is to provide trainee educational psychologists with the opportunity to demonstrate the attainment of the knowledge, skills, behaviours and attitudes necessary for competent professional practice.

OSCEs were introduced into medical training in the 1970s. They have been described by the originators as a “timed examination in which medical students interact with a series of simulated patients, in stations that may involve history taking, physical examination, counselling or patient management” (Harden & Gleeson, 1979). They have become widely used within medicine and allied health professional training as well as other disciplines such as engineering. The format involves students rotating around a number of “stations” where they spend a specified amount of time carrying out a particular professional skill. At each station the student is presented with a task, which often involves responding to a short written scenario in the presence of a trained, simulated patient. Students' performance is assessed against a checklist of criteria related to the components of the task.

OSCEs are considered to increase fairness in assessment as they include a wider number of skills than written examinations, increase the number of examiners and are marked against explicit criteria which are standardised. Simulations can be used to assess procedural competence as well as knowledge.

An OSPA is...

Objective: at each station the trainee is presented with a different professional task presented as a scenario. All trainees are presented with the same scenarios and interact with an actor who has been provided with clear guidance on how to react and respond.

Structured: the assessment scheme for each station is structured. Specific competence modalities that are likely to be included are communication skills, perspective taking, information gathering and synthesis, management and professional integrity.

Professional Assessment: Each activity is a test of performance of professional skills. Trainees have to demonstrate competence in dealing with the scenarios that they are presented with.

Why use OSPAs in assessment of professional practice and performance?

Using OSPAs as a method of assessment allows careful specification of content and observation of a range of professionally authentic scenarios. The interaction between assessor and trainee is structured. An assessor is present at each station to monitor and score performance. The scoring is carried out according to pre-set criteria. Because trainees are being assessed using exactly the same scenarios and marked against clear and explicit criteria it increases the reliability of the observations and the consistency of the feedback. Assessment will be formative, defined by Cowie and Bell (1999) as “the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning”.

How are OSPAs to be carried out?

Each trainee educational psychologist engages with four standardised scenarios in turn, with an actor playing a role and with a standardised script. A different scenario is presented at each of the four stations:

1. Initial Consultation (history)

2. Assessment and explanation (data interpretation)

3. Action planning (psychological advice)

4. Communication and ethics

Scenarios are based around practice issues and tap the competencies required for practice with young people aged between 16-25 years. There is a 10 minute time limit for each scenario. An assessor (a university tutor) is present who rates the trainee's performance using a standardised marksheet. Assessors meet and review the scenarios beforehand and agree how the assessment system will operate. This process, known as calibration, is essentially a standard setting process, and is critical to the fair and consistent conduct of the assessment. All scenarios are video-taped. At the end of the 10 minute scenario, trainees are asked to make some comments on their self-assessed performance and provided with formative feedback in the form of strengths and improvement suggestions.

Reliability of OSPAs

There will be a subsequent assessor training day for local authority supervisors. Following a workshop on assessment of competence within an OSCE framework, supervisors will be invited to view a selection of OSPA videos and to provide a second rating of performance against the same criteria as the university assessors. This will provide valuable data that will be used to establish the reliability of the assessment system, across candidates, examiners and course centres.

Placement Assessment

OSPAs are designed to supplement learning and assessment on professional placements. Although placements are a costly element of professional psychology training (Gonsalvez & Milne, 2010), they are credible, straightforward, versatile and have ecological validity when judgements of competence are based on a range of observations over time. However, there is some evidence that placement supervisors demonstrate systematic halo and leniency biases (Gonsalvez & Freestone, 2007) which can reduce opportunities for trainees to receive the specificity of feedback necessary to develop professional competence. The addition of OSPAs ensures that trainees are assessed against authentic, standardised scenarios and receive consistent feedback across assessors and course centres.

References and further reading:

Cowie, B. & Bell, B. (1999). A model of formative assessment in science education. Assessment in Education, 6, 101-116.

Doran T. & O'Neill P. (2006). Core Clinical Skills for OSCEs in Medicine. Churchill Livingstone Elsevier.

Gonsalvez, C. J., & Milne, D. L. (2010). Clinical supervisor training in Australia: A Review of current problems and possible solutions. Australian Psychologist, 45, 233-242.

Gonsalvez, C. J., & Freestone, J. (2007). Field supervisors' assessments of trainee performance: Are they reliable and valid? Australian Psychologist, 42, 23-32.

Harden, R. & Gleeson, FA. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 30 (1): 38-43.

Nicol, M. & Freeth, D. (1998). Assessment of clinical skills: a new approach to an old problem. Nurse Education Today, 18, 601-609.

Ranga, R. (Ed). (2005). OSCEs in Psychiatry. Royal College of Physicians.

Yap, K. Bearman, M. Thomas, N. & Hay, M. (2012). Clinical psychology students' experiences of a pilot objective structred clinical examination. Australian Psychologist, 47 165-173.

Information and Guidance for Trainee Educational Psychologists about OSPAs

Objective Structured Professional Assessments (OSPAs) have been designed to support your learning and development of professional competence in training to be an educational psychologist. Structured simulated assessments are a tried and tested framework and will be used in a formative (not summative) way. They are designed to help you identify your strengths and areas where you would benefit from additional supervised experience. OSPAs are supplementary to existing course assessments of placement competency and performance on these will have no impact on current course assessment. You will be presented with scenarios at four different stations.

Preparing for the day:

You will receive the scenario outlines by email one working day before the OSPA Read though these

There is no need to spend a lot of time preparing

The resources that you have developed and accessed through participation in PBL will provide you with sufficient information to undertake the OSPA

Your management of the scenario will be more natural and you will perform better if you don’t over prepare or try to adhere to a pre-determined script

On the day:

Dress professionally

Make sure you arrive in good time as the OSPAs operate to a very strict time schedule

OSPAs can generate some anxiety, but be reassured that you have had all the preparation you need and the intention is not to catch you out but to support your learning in a meaningful and constructive way

You will have exactly 10 minutes to interact with the role player (an actor) and will be given a two minute warning before the end

After the OSPA is over, the assessor (a university tutor) will give you feedback in relation to strengths and improvement suggestion

Basic considerations:

The role player has had a briefing and the case information that they can reveal will be the same for everyone. It is up to you to elicit this.

Assume consents have been agreed

Assume that all of the clients in the scenarios have been explained and understand the role of an EP. There is no need for further explanation.

Ensure that any advice or interventions proposed are realistic and feasible

Do not assume that clients understand jargon or technical terms

The role players will respond to you in a way that they consider to be realistic e.g. A role player who feels comfortable and listened to will reveal more information than when they feel defensive

It is important to respond to the role player in a natural way to ensure best quality interaction

General principles to take into account:

Always introduce yourself with your name and job title

Where appropriate, check client’s identity, relationship to young person and involvement in the problem situation before you tackle the problem

Try to keep the discussion on topic and maintain focus

Aim to start with open questions

Adjust your manner and use of language as appropriate for the situation

Use summaries to check your understanding of the problem is the same as the client’s

If you have to ask difficult or sensitive questions you should prepare the client and ensure they are clear they do not have to respond if they are not comfortable

Top tips for managing each of the four stations:

1. Initial consultation

The aim is to collect background information and gain a clear overview of the problem/situation from the perspective of the client(s). This can include their views of the perspective of others. This station does not require you to explain anything to the client This station does not require you to plan future actions If the client is not the young person, consider how you can access the young person’s views. Try to focus on the current situation rather than spending time taking detailed background information that is not directly relevant

2. Assessment and explanation

The aim is to explain psychological and technical information in a way that is accessible to a lay person

Check the client’s expectations about the interaction

Establish the baseline knowledge of the client before embarking on an explanation

It may appropriate to seek the perspective of the client and any additional relevant information in relation to the focus of the assessment

Clearly and slowly explain key information and terms, avoiding jargon

Do this in small chunks and check the client’s understanding throughout

Reiterate important points

Allow time for feedback, questions and checking the client’s understanding

Know the limitations of your own knowledge and experience. It’s okay to say you don’t know but explain what you will do to find out and feedback

Offer to provide a written summary if appropriate

3. Action planning

The aim is to agree clear actions and next steps with the client in relation to the issue presented

You should only discuss realistic and appropriate actions; do not assume unlimited resources

Establish both yourself and the client wish to manage the same problem and there are no hidden agendas

You should allow the client to suggest solutions that they would favour and consider feasible

You should present clear and non-biased information about potential interventions and be prepared to justify these

Seek the client’s evaluation of the options you provide

If you don’t know key information, it is appropriate to let the client know that you need to find out more

At the end summarise agreed actions and check understanding

State how you will follow up and ensure the client has realistic expectations of the time frames

4. Communication and Ethics

The issues presented in this station will relate to ethical decision making. Familiarity with the BPS Code of Ethics and Conduct (2009) and the HCPC Standards of conduct, performance and ethics (2012) will be central to the issues at hand

If relevant, seek information about the young person’s mental capacity

Consider the case presented in relation to the following elements:

Respect (including confidentiality and its limitations, informed consent, self-determination)

Competence (including ethical decision making, recognition of limits of competence)

Responsibility (including duty to avoid harm; termination and continuity of care)

Integrity (honesty and accuracy; addressing professional misconduct)

Make sure you adapt your communication style to the needs of the client. e.g. through an interpreter if the client is deaf or has limited English.

Be sensitive to communication and ethical differences between cultures

After the OSPAs:

After the OSPAs you will be given a DVD with the videos of your OSPAs including the verbal feedback which you were given by the two assessors on the day. The feedback will be linked to the aims of each station and the competencies targeted by each station. This year, you will also be given copies of the blank mark sheets for each scenario, as used by the assessors on the day. We strongly encourage you to use the videos and mark sheets to complete your own self-assessment of your performance. You might like to consider the extent to which you agree with the feedback from the assessors. As well as offering a valuable opportunity for self-reflection, your work can also be used to illustrate how you have met training competencies.

To provide an idea of what it is like to participate in an OSPA, we have provided an example of an OSPA taking place. Here is the advice that trainees are given beforehand. The trainee EP in the video will have had this in mind when she undertook her online consultation.

Station 3 video: Action Planning

The aim is to agree clear actions and next steps with the client in relation to the issue presented.

You should only discuss realistic and appropriate actions; do not assume unlimited resources.

Establish that both yourself and the client wish to manage the same problem and there are no hidden agendas.

You should allow the client to suggest solutions that they would favour and consider feasible.

You should present clear and non-biased information about potential interventions and be prepared to justify these.

Seek the client’s evaluation of the options you provide.

If you don’t know key information, it is appropriate to let the client know that you need to find out more.

At the end summarise agreed actions and check understanding.

State how you will follow up and ensure the client has realistic expectations of the time frames.

Below are the assessment criteria for OSPA stations. Watch the video and reflect on how these criteria were demonstrated.

Professional Skill 1. Communication Skills Clarity of purpose/ structure Encourages client contribution and elicits important information Uses appropriate language and explains key terms if necessary Summarises

2. Perspective Taking Seeks, detects, acknowledges and attempts to address concerns Demonstrates awareness of client’s perspective Listens Empathic

3. Information gathering and synthesis Obtains sufficient information to make decisions Presents synthesis of new and existing information

4. Management Works in partnership to develop a shared plan Suggests next steps Manages client’s emotions and expectations

5. Professional Integrity Addresses the rights of young person Specifies limits of knowledge, competence and role Where appropriate, challenges assumptions and perspectives Does not collude

There are a number of publications linked to the HEA funded OSPA project, which you can access below.

Dunsmuir, S., Atkinson, C., Lang, J. & Wright, S. (2020). The value of practice simulations and Objective Structured Professional Assessments (OSPAs) for school psychology training: Participant perspectives. International Journal of School and Educational Psychology. https://www.tandfonline.com/doi/full/10.1080/21683603.2019.1605953

Dunsmuir, S., Atkinson, C., Lang, J., Warhurst, A. & Wright, S. (2017). Objective Structured Professional Assessments for Trainee Educational Psychologists: an Evaluation. Educational Psychology in Practice, 33(4), 418-434. https://www.tandfonline.com/doi/full/10.1080/02667363.2017.1352490

Dunsmuir, S., Atkinson, C., Lang, J. & Wright, S. (2015). Use of Delphi methodology to define competencies in professional educational psychology training. Assessment and Development Matters, 7(1), 22-25. https://www.research.manchester.ac.uk/portal/files/28879040/POST-PEER-REVIEW-PUBLISHERS.PDF

Atkinson, C., Dunsmuir, S., Lang, J. & Wright, S. (2015). Developing a competency framework for the initial training of educational psychologists working with young people aged 16-25. Educational Psychology in Practice, 31(2), 159-173. https://www.tandfonline.com/doi/full/10.1080/02667363.2015.1004038

You can access other documents by clicking on these links:

OSPA marksheet

OSPA national dissemination event 2014