A A A
Portico

Procedure for the Conduct of Augmented Annual Monitoring

contact: Sandra Hinton, Quality Assurance Manager, Academic Services, Student and Registry Services

Procedure

Key to abbreviations:

AC Academic Committee
AM Annual Monitoring
AMR Annual Monitoring Report
AS Academic Services
AugAM Augmented Annual Monitoring
AugAMR Augmented Annual Monitoring Report
DTC Departmental Teaching Committee
FTC Faculty Teaching Committee
HoD Head of Department
IQR  Internal Quality Review
JSSC
Joint Staff Student Committee
PS Programme Specification
PSRB Professional, Statutory and Regulatory Body
QAA Quality Assurance Agency for Higher Education
QME Quality Management and Enhancement
QMEC Quality Management and Enhancement Committee
SEQ
Student Evaluation Questionnaire
SES Self-evaluative Statement
SSCC Staff-Student Consultative Committee

1 Introduction

1.1 This document sets out the procedure for Augmented Annual Monitoring (AugAM). Before beginning this procedure, you should read UCL’s policy on AugAM which sets out the purposes and principles of AugAM and can be found at:

1.2 You are also advised to read UCL’s policy and procedure for Annual Monitoring (AM) at:

1.3 AM at UCL involves, in an agreed cycle (every fifth year), an ‘augmented’ element known as AugAM and enables Programme Organisers to evaluate, comment upon and monitor the impact on their taught programmes of cumulative, incremental change over a longer review period and to identify further opportunities for enhancement of the student learning experience.

2 The Main Steps of Augmented Annual Monitoring

2.1 The AugAM, procedure is conducted every fifth year in the year before a department’s Internal Quality Review (IQR). All Programme Organisers in the department will produce an augmented report which will summarise developments in the programme over the review period by drawing on the evidence provided by the four previous AM reports (AMRs).

2.2 The AugAM procedure also involves a person external to UCL, (the ‘External Scrutineer’) who will be asked to comment upon the Programme Organiser’s augmented report and accompanying documentation.

2.3 The main steps of the AugAM procedure are as follows. They are identical to those for routine AM except where in bold.

Steps Action Date People Hyperlinks Notes
Step One Statistical Data for the previous academic session is published on the Academic Services (AS) website. End of July 2013 Gary Smith in Student Data Services. ANNUAL MONITORING STATISTICAL DATA A notifying email is sent to all Heads of Department (HoDs), Faculty Tutors and relevant administrative staff by the Chair of the Quality Management and Enhancement Committee (QMEC).
Step Two The Module Organiser, using the relevant proforma, produces a report on each module for which he/she is responsible. August 2013 Module Organiser PROFORMA FOR MODULE ORGANISER REPORT  
Step Three Module Organiser forwards his/her report (s) to the relevant Programme Organiser. August 2013 Module Organiser   Where modules are offered in more than one programme, the module report will be sent to the Programme Organiser of all programmes where the module is offered.
Step Four The Programme Organiser, using the relevant proforma, produces an augmented report on the programme for which he/she is responsible, reflecting and commenting on developments in the programme over the previous four years by drawing on the evidence provided by the four previous AM reports (AMRs)*. August/September 2013 Programme Organiser PROFORMA FOR PROGRAMME ORGANISER AUGMENTED REPORT
Step Five The Programme Organiser forwards his/her augmented report and accompanying documentation* to an External Scrutineer** September 2013

*See section 5 below for a note on accompanying documentation.

**See section 6 below for guidance on appointment of External Scrutineer.

Step Six The External Scrutineer, using the relevant proforma, produces a report on, and makes recommendations for, the programme in the light of the Programme Organiser’s augmented report and accompanying documentation and returns this to the Programme organiser*. September 2013 PROFORMA FOR EXTERNAL SCRUTINEER REPORT This process will normally be conducted by email.
Step Seven The Programme Organiser forwards his/her augmented report, plus the External Scrutineer’s report to the HoD/Chair of the Departmental Teaching Committee (DTC) October 2013 Programme Organiser  
Step Eight The HoD/Chair of DTC, using the relevant proforma, produces a report reflecting and commenting on the information contained in the Programme Organisers’ and External Scrutineers’ reports. October 2013 HoD/Chair of DTC. PROFORMA FOR HOD/CHAIR OF DTC REPORT HoDs/DTC Chairs are also asked to complete the proforma attached to the report which summarises its discussion of the results of the SEQs.
Step Nine The HoD/Chair of DTC submits (1) his/her report (2) the Programme Organisers’ augmented reports and documentation, (3) the External Scrutineer's report and recommendations to the DTC and Departmental Staff-Student Consultative Committee (DSSCC) for discussion. October 2013 HoD/Chair of DTC    
Step Ten The HoD/Chair of DTC submits his/her report, (including any comments arising from discussion by the DTC and DSSCC and the proforma summarising its discussion of SEQs) to the Faculty Teaching Committee (FTC). Early December 2013/End of Autumn Term HoD/Chair of DTC    
Step Eleven The FTC considers the complete AugAM reports, HoD/Chair of DTCs report(s)* and documentation. If it is satisfied that any outstanding issues have been addressed by the department** it should endorse/approve the AugAMR (s). January 2014

*Please note that more than one department in a faculty may be scheduled for AugAM in that year.

**If the FTC identifies any issues which require resolution before approval of the AugAMR (s), the FTC should first draw these matters to the attention of, and invite comments from, the department (s) concerned, via the DTC (s).

Step Twelve

The Chair of the FTC, using the relevant proforma, prepares a faculty summary report of issues arising from the HoD/Chair of DTC’s reports. January 2014 Chair of the FTC PROFORMA FOR FACULTY SUMMARY REPORT  
Step Thirteen The Chair of the FTC submits the faculty summary report to the FTC for discussion. January 2014 Chair of the FTC   FTC Chairs are also asked to complete the proforma attached to the faculty summary which summarises its discussion of the results of SEQs.
Step Fourteen The Chair of the FTC forwards the faculty summary report, (including any comments arising from discussion by the FTC and the proforma summarising the SEQs) to the QMEC Secretary*. End of January 2014 Chair of the FTC   *The QMEC Secretary will pass the SEQ proforma to the Secretary of the JSSC for action.

3 Reporting and Dissemination of Issues arising from Augmented Annual Monitoring Reports

3.1 UCL operates a pyramidal structure of committees at department, faculty and institutional-level to assure quality and standards, which reflects UCL’s academic-led and devolved approach to QME. The procedure at institutional level for the reporting of issues arising from AugAM reports, the identification of any action needed and dissemination of this back to faculties is identical to routine AM and is as follows:

Step Fifteen The QMEC Secretary prepares a summary of issues or themes arising from the faculty summary reports for reporting to QMEC. QMEC Secretary February 2014
 
Step Sixteen QMEC discusses the summary of issues or themes arising from the faculty summary reports. QMEC March 2014
 
Step Seventeen The QMEC Chair makes an oral report to the Spring Term meeting of Academic Committee (AC), highlighting the key issues arising from the recent AM round. QMEC Chair Late March 2014  
Step Eighteen QMEC requests responses from relevant departments/committees/officers to those issues. QMEC Chair/Secretary April 2014  
Step Nineteen QMEC submits a written report to the first meeting of AC in the Summer Term of the following session which includes responses received from relevant departments/committees/officers to issues raised. QMEC Chair/ Secretary June 2014  
Step Twenty Following discussion at AC QMEC Officers will forward the QMEC written report to Faculty Tutors and Deans with instructions to submit the report to the next scheduled meeting of their own FTC for discussion.     Discussion of QMEC’s summary report by AC also provides Deans of Faculty and Faculty Tutors with an opportunity to see what is happening in other faculties and to share good practice.

4 Contents of Augmented Annual Monitoring Reports

4.1 As can be seen from Steps Four to Seven above, the AugAM process involves the preparation of an augmented report by a Programme Organiser and a report from an External Scrutineer, using a proforma for each report.

4.2 A set of core institutional reporting requirements has been developed for each of these reports and the proforma have been designed to meet these requirements. To see the core reporting requirements for each report, please click on the link to the relevant proforma in the chart above, or below:

5 Accompanying Documentation to be sent to the External Scrutineer

5.1 The following information should be provided for the External Scrutineer by the Programme Organiser in order to enable him/her to complete his/her report. All documentation should, where possible, be provided as email attachments or as hyperlinks within an email:

  • A Programme Organiser’s augmented report (see proforma above for contents);
  • AMRs for the preceding four years including the reports of the Chairs of Boards of Examiners;
  • A hyperlink to the AS website containing an up to date PS for each programme;
  • A hyperlink to the student handbook for each programme;
  • A hyperlink to the AS website containing the AM statistical data (see section 10 below);
  • A hyperlink to Faculty and Departmental Learning and Teaching Strategies.

6 Role and Appointment of the External Scrutineer

6.1 The External Scrutineer for AugAM will be asked to consider the information in the Programme Organiser’s AugAMR accompanied by the PS and relevant supporting documentation and having reviewed this will complete the AugAM proforma which invites comment on the programme as a whole in the light of this information. The proforma also invites the External Scrutineer to make any recommendations for enhancement he/she considers necessary and to comment on good practice.

6.2 Departments should consult as the External Scrutineer the current or a recent External Examiner for the programme unless:

  • that External Examiner acted as the External Scrutineer when the programme was originally approved;
  • the current or recent External Examiner is not available.

In these instances, departments are advised to approach instead another academic of appropriate experience and qualifications from another Higher Education Institution (HEI).

6.4 External Examiner appointments are made by the Chairs of Boards of Examiners in consultation with those Boards. If External Scrutineers to AugAM are the same peers as those currently appointed to act for the programme as External Examiners then they will have already been through UCL’s process for the appointment of External Examiners for Taught Programmes and there is no need for additional approval in order for the External Examiner to act as an External Scrutineer.

6.5 Where an individual other than the External Examiner is to act as External Scrutineer (ie in the case of the exceptions noted at 6.2 above), the appointment process is as follows:

  • A nomination is made by the Programme Organiser to the Chair of the DTC;
  • The nomination is then endorsed by the Chair of the DTC, in consultation with the DTC;
  • The appointment is approved by the FTC or the FTC Chair by Chair’s action.

6.6 In order to reduce the burden on the programme and the External Scrutineer, it is recommended that where there are a number of very similar programmes in a department, the External Scrutineer may complete a single report which covers them all. He/she is not obliged to complete a separate proforma for each individual programme.

7 Payment of External Scrutineer

7.1 Where the External Scrutineer is a current External Examiner they may be paid an additional honorarium for scrutinising the AugAM documentation. A sum of between £100-150 may be appropriate but this is entirely at the discretion of the HoD concerned. New External Scrutineers appointed for the purpose of AugAM can be paid for a ‘one off’ service. A figure of £250 might be appropriate.

8 Issues Concerning the Validity of Programmes Arising in Augmented Annual Monitoring

8.1 External Scrutineers’ reports may raise issues concerning the currency and validity of programmes reviewed. It should be noted, however, that AugAM is not intended to be a procedure for approval or non-approval of programmes; indeed, there is no notion within AugAM of any programme being approved (or not approved) to continue as a result of the monitoring process. It is considered that a faculty’s overall endorsement of the academic viability of its programmes is appropriate and sufficient and FTCs will therefore be asked to endorse/sign off the AugAMR as at Step Eleven above.

9 Articulation with Internal Quality Review

9.1 The AugAM process is articulated with the IQR process as follows:

  • by ensuring that the AugAMRs of all programmes within a department are completed in the year preceding the department’s IQR;
  • by the incorporation of selected information from each Programme Organiser’s augmented report and the External Scrutineer’s comments and recommendations into the Departmental Self Evaluative Statement (SES) which will be required for the following year’s IQR;
  • by the submission by departments of the Programme Organisers’ augmented reports and External Scrutineer’s reports in full as part of the core documentation for the IQR.

9.2 AS will notify departments when they are due to produce AugAMRs ie in the year prior to their scheduled IQR.

10 Data to Support Annual and Augmented Annual Monitoring

10.1 AugAM uses the same statistical dataset as AM. The Programme Organiser’s augmented report will reflect on the data for the current year and on that used for the four previous AMRs. All statistical data will be generated by Student Data Services, Student and Registry Services, and published on the AS website by the end of June each year. Departments will be notified of this by email. Student Data Services is happy to discuss the accuracy or completeness of the data with individual departments, although departments are advised that this is easier if done as soon as possible after the data has been put on to the site.

11 Publication of Augmented Annual Monitoring Reports

11.1 AugAMRs should be published on either departmental or faculty intranet sites, for viewing by UCL staff and students. Each faculty has now decided on the most appropriate site for publication of its own AugAMRs. Programme Organisers should therefore check with the Faculty Tutor which site has been chosen before publishing their AugAMRs.

11.2 Before publication, it is requested that colleagues exclude anything from their AugAMRs that could identify individuals. They are therefore asked not to name course tutors or individual students.

12 Annual Monitoring and Student Evaluation Questionnaires (SEQs)

12.1 In 2011-12, proformas have been designed to assist in the summary of the SEQ data for consideration by the various committees, in order to fulfill the requirements of the various UCL policies and practices as referred in the Academic Manual. The SEQ proformas are attached to the AM reporting proforma for the HoD/DTC's report and the proforma for the FTC summary and have been designed to provide a clear overview of the main matters of interest arising from the analyses of the SEQs and any action taken.

12.2 The following points should be considered in completion of the proformas for departmental and faculty consideration of the SEQ data:

  • it is advisable that analysis of the SEQs is conducted by departments and academic units in time for the beginning of the following academic session;
  • ithe Departmental proformas should be submitted to the first meetings of the DTC and the DSSCC in the academic session;
  • ithe faculty SEQ proforma should be submitted to the first meeting of the FTC in the academic session;
  • ithe completed faculty SEQ proforma should be submitted to the QMEC Sceretary by January each year.  This information will then be passed to the Secretary of the JSSC and considered alongside other sources of student feedback data (NSS, ISB and the DSSCC minutes).        


July 2013