XClose

Academic Manual

Home
Menu

Section 3: Internal Quality Review (IQR)

Published for 2019-20


3.1 Introduction3.7 The IQR Visit
3.2 Summary of IQR3.8 The IQR Report
3.3 The Review Team3.9 Follow-up
3.4 Departmental Contact3.10 IQR Panel/Academic Committee
3.5 Preliminary Briefing3.11 Dissemination of Findings of IQR Report
3.6 The Self-Evaluative Statement and supporting material 

3.1 Introduction

1.

Internal Quality Review is UCL’s central academic quality management and enhancement process. IQR is a rolling programme of peer review, in which all academic units of UCL*  (as well as a small number of interdepartmental degree programmes) are reviewed on a six-yearly cycle.

*Except where otherwise indicated, 'Department', in the context of these guidelines, means 'the unit of activity being reviewed'; this will in most cases mean an academic department of UCL or an interdepartmental degree programme, although ‘Department’ in these guidelines also subsumes relevant academic units which are not formally academic Departments established by Council.

2.An important purpose of IQR is to review the effectiveness of a Department’s mechanisms for the management of its programmes, ensuring that UCL’s policies and procedures are operating as intended to safeguard academic standards and to provide a high-quality learning experience for students. A data-informed approach to agenda preparation for the visit day also ensures that review teams focus on priority strategic areas for each department, as well as on quality and standards. In advance of each review an institutional IQR Panel will advise on the agenda and themes for exploration.
3.IQR is also concerned with reviewing the academic content of a Department’s programmes, including the continuing currency and validity of programmes in light of developments in research, professional and industry practice and pedagogy (including the use of technology in learning and teaching), changes in the external environment such as the requirements of professional, statutory and regulatory bodies (PSRBs) and continued alignment with UCL’s strategy and mission.
4.IQR also aspires to be a genuinely developmental process which provides an opportunity for Departments to review and, in partnership with the review team, identify ways of enhancing their existing QME structures and systems.

3.2 Summary of IQR

 Each IQR comprises seven main stages:
1.

The IQR Panel, chaired by the Vice-Provost (Education and Student Affairs), supported by UCL’s Pro-Vice-Provosts sets Review Teams specific issues and themes for follow-up, based on:

  • Annual Student Experience Review (ASER) plans.
  • reports of outcomes from the series of ASER follow-up meetings being held by the Office of the Vice-Provost Education and Student Affairs with all academic departments.
  • Teaching Excellence Framework (TEF) metrics.
  • data on student achievement and employability.
  • National Student Survey (NSS).
  • Postgraduate Taught Experience Survey (PTES).
  • Postgraduate Research Experience Survey (PRES).
  • New to UCL Survey.
  • Student Experience Surveys.
  • An annual report from UCL Estates concerning what it is doing to improve the teaching estate overall and in particular in respect of each of the departments scheduled for review.
2.Submission by the Department to the review team of a self-evaluative statement, with a list of supporting documentary evidence (which should, in order to minimise any burden on the Department, be made available electronically e.g. on the Department’s website or Moodle).
3.Scrutiny of the SES and supporting evidence by the review team.
4.A visit by the review team to the Department, normally lasting 1.5 working days when interviews with relevant staff and students of the Department take place. See Annex 9.3.2  IQR Sample Timetable.
5.Production of an IQR Report.
6.Preparation by the Department of a preliminary plan of action to be taken in response to the recommendations contained in the IQR Report.
7.Subsequent consideration by the Internal Quality Review Panel of (i) the IQR Report and (ii) the Department's action plan.

3.3 The Review Team

1.The review team will normally comprise six reviewers, a Review Manager and an Administrative Secretary. Three reviewers will be members of staff of UCL, one will be a student reviewer and two will be external reviewers. The members of review teams will be appointed by the Internal Quality Review Panel as follows:
 i)A Senior Academic Member of staff i.e.: a Vice-Dean Education or Faculty Tutor (from a different Faculty to the Department under review), who will act as Chair.
 ii)Academic or other senior practitioner. Teaching Fellow or Research Fellow (at least Grade 9).
 iii)One Head of Department (from a different Faculty to the Department under review).
 iv)

One external reviewer** who is a senior member of staff, either academic or administrative, of another institution of higher education with expertise in quality management and enhancement.

**This reviewer will only form part of the Team where the IQR Panel decides that the evidence provided by the SES and supplementary documentation make it necessary.

 v)One external reviewer who is an external subject specialist, with appropriate expertise, nominated by the Head of Department to be reviewed. This must NOT be the current or recent former External Examiner. 
 vi)One student reviewer who will be nominated by the UCL Union.
2.The role of the team leader.
 i)To chair the review team’s planning meeting, which will begin the Review Visit. In the planning meeting, the team leader will confirm the areas being explored by each member of the team. The team leader is not, however, expected to be solely responsible for, e.g., reading the SES or other briefing material or asking questions during interviews on the review visit; these are all shared responsibilities of all members of the review team.
 ii)During the review visit, to introduce other members of the team and explain briefly the purpose of the visit at the start of each interview with staff or students.
 iii)At the end of the review visit, to develop, in conjunction with the other members, a summary of the review team’s main findings and conclusions.
 iv)After the review visit, formally to approve the IQR Report once a draft of this has been agreed by all other members of the review team and by the Department reviewed.
3.The role of the External Reviewers.
 

External QME expert***

  • One External Reviewer will be a senior member of staff, either academic or administrative, of another institution of higher education. S/he will be able to demonstrate expertise in and experience of QME structures and systems and to comment impartially on these aspects of the Department’s provision.

***This reviewer will only form part of the Team where the IQR Panel decides that the evidence provided by the data makes it necessary.

External Subject Expert

  • The other External Reviewer will normally be a senior academic of another institution of higher education. S/he should have knowledge of the discipline concerned and, where possible, should have experience of Periodic Review in his/her own institution or external review by the QAA or relevant professional body.  S/he should not have had any formal links with the Department under review within the previous 5 years. S/he will produce a structured report of maximum 1.5 sides of A4. It will set out any lines of enquiry required by the External Subject Expert.
 Both External Reviewers play a vital role in assisting the Team to identify key issues to be explored during the visit to the department and play a full part in the Team’s meetings with staff and students. In particular, External subject reviewers are able to identify excellence in provision and in the case of the External Subject expert, they are able to make comparisons with similar provision at other institutions and comment on the currency of a department’s programmes in the context of developments in the discipline. They can also offer feedback on the appropriateness of aims and learning outcomes to future career development.  External Reviewers will be invited to comment on the Report of the Review and the department’s response to the Report.
 Each External Reviewer will be provided with an outline of their role as a member of an IQR team and receives a fee from UCL.
 The External Subject Expert will, prior to the visit, conduct a desk-based review of programme documentation in order to establish (and, when established, to make a statement to this effect) that any incremental programme/course changes made since the last review have not detracted from the aims of the programmes, affected their standing in relation to the National Qualifications Framework or where appropriate the Subject Benchmark Statement. See Annex 9.3.3 External Subject Expert proforma for report.
 The External Subject Expert will be nominated by the Department to be reviewed. Academic Services will then take responsibility for his/her appointment and remuneration. S/he will be paid £300 for a one off duty. If a department wishes to appoint more than one External Subject Expert to the Review, it will be expected to pay for any additional reviewer/s. The External Subject Expert cannot be the current or recent former External Examiner. See Annex 9.3.4 External Subject Expert nomination form.
4.All internal members of the review team, including the administrative secretary and the Review Manager, will have received formal briefing in advance of undertaking an IQR.
5.All reviewers will undertake:
 i)To read all supporting documentation and, in the case of the External Subject Expert, to conduct a desk-based review;
 ii)To participate fully in interviewing staff and students during the review visit;
 iii)To make an appropriate contribution to the preparation of the IQR Report.
6.The Review Manager
 

The role of the Review Manager is to:

  • provide high level advice and support to the Team and Chair on UCL’s education strategy;
  • To advise the team on educational strategy (how the department are meeting it well and how else they could be implementing it to strengthen their student experience);
  • Suggest areas the team may wish to investigate further;
  • Work with the secretary and Team Leader to empower the review team.
7.

The Administrative Secretary.

  • The administrative secretary will liaise with the Department concerned on behalf of the review team in advance of the visit.
  • In consultation with the Department and reviewers agree a date for the review team’s visit to the Department and then devise the overall timetable for the conduct of the IQR, including deadlines for, or dates of, the key stages in the process.
  • Receive from the Department its SES in electronic form, (e.g. on the departmental website, Moodle or a memory stick) which will either (i) contain within the text of the document links to supporting material which is available on the Department’s website or (ii) contain a separate index of links to supporting material.
  • Ensure that for authorised users this electronic departmental information is saved and electronically archived so that handbooks and other items from previous years are still accessible for the IQR team. The websites for each Department must be archived or a Sharepoint site used to store all documentation reviewed for the purposes of maintaining an audit trail.
  • Discuss and confirm during the departmental briefing, the most efficient means of providing the SES and supporting material with the Department concerned.
  • Discuss and confirm with the Department in advance how access to any departmental intranet sites for (i) UCL staff and (ii) external reviewers will be obtained.
  • Commission the External Subject Expert’s Report. This will be a structured report of maximum 1.5 sides of A4 and will set out any lines of enquiry required by the External Subject Expert.

3.4 Departmental Contact 

1.The Head of Department/Programme Co-ordinator may nominate a colleague (either academic or administrative) as a Departmental Contact. The Contact’s essential role will then be to co-ordinate preparations within the Department for the review visit on behalf of the Head of Department. This will include practical arrangements for the visit, such as the provision of documentation, the drawing up of the visit timetable etc. The Departmental Contact can also help to ensure that the review team has an adequate understanding of the particular nature of the Department in advance of the visit to the Department.

3.5 Preliminary Briefing

1.

At the start of the IQR process each year, Academic Services officers and the Office of the Vice-Provost (Education and Student Affairs) will make arrangements to brief each of the following groups:

  • Heads of Department and/or Departmental Contacts, etc., in the academic units to be reviewed during the coming academic year.
  • New IQR reviewers, including new external reviewers, student reviewers and reviewers acting as Team Leaders for the first time.
  • Administrative secretaries to IQR teams.
  • Review Managers.

3.6 The Self-Evaluative Statement and supporting material 

1.The SES should discuss both strengths and weaknesses in the Department’s provision.
2.The SES should be completed by the Department according to the format set out at Annex 9.3.5 Guidance on the Composition of the SES. It should be submitted electronically e.g. via a departmental website or on a memory stick to the administrative secretary to the review team, for receipt at least three working weeks before the date of the review team's visit to the Department. See Annex 9.3.5 Guidance on the Composition of the SES.
3.Departments should issue the draft SES to the Departmental Teaching Committee for approval, before the document is submitted to the review team. Departments should also consult staff and students more widely in the process of developing the SES. Departments should ensure that the final version of the SES is received by the DTC and the Departmental Staff-Student Consultative Committee before the review team’s visit to the Department and is made available to all staff and students in the Department.
4.The Administrative Secretary should also send a copy of the SES  once received, to the Quality Assurance Manager. On receipt of the SES the Quality Assurance Manager will copy the SES to: the Dean of Faculty concerned; the Faculty Tutor concerned; the Faculty Graduate Tutor concerned, inviting them, if they wish, to send comments on the SES to the IQR team, via the administrative secretary.
5.The review team will read the SES and all supporting documentation in conjunction with the External Subject Expert’s desk-based report.
6.The SES will be seen by UCL colleagues other than members of the review team and will form part of a documentary record which may also be seen by external reviewers in the context of Higher Education Review or other external review such as accreditations by PSRBs. Departments who, with this in mind, feel they need advice on the inclusion in the SES of potentially sensitive material are encouraged to contact the Quality Assurance Manager.
7.The SES will consist of two sections (in addition to supporting statistical data and other briefing material). These two sections are expected to comprise a total of around 5 pages. [See Annex 9.3.5]
8.Student Data
 Departments will be supplied with the ASER data for the previous session through Tableau. A list of this data is at Annex 9.3.6 IQR Data Set. See Annex 9.3.6 IQR Data Set.
9.Programme Handbooks and Programme Summaries
 The External Subject Expert on the IQR review team will need to consider special documentation in addition to the Self-evaluative statement (SES). This will normally be the programme handbooks and Programme Summaries. Programmes will be asked to provide External Subject Experts, via the Administrative Secretary to IQR with the same programme-related documentation they give their External Examiners. Programme Summaries will be downloaded and given to Externals via the Secretary.
10.Sampling of Programmes
 For large/complex provision there will be sampling of programmes. The amount will be the subject of negotiation between the review team and the Department. These should be chosen by the Department. A statement will be required in the SES about why these programmes have been chosen. Any academic partnership should automatically be included in any sample of programmes.
11.The other supporting material made available to the review team by the Department through its web pages or via the other methods listed [see above] should consist of documentation which the Department believes provides relevant evidence of its QME processes and its programmes. A list of the core documentation which IQR teams normally expect Departments to submit, together with the SES, in advance of their IQR visit is at Annex 9.3.7 IQR Core Documentation. See Annex IQR 9.3.7 Core Documentation.

3.7 The IQR Visit 

1.The IQR team’s visit to the Department should normally last 1.5 working days. However, in the case of particularly small numbers of students and/or a limited number of programmes the visit may take one working day. This will still be conducted by a full Review Team.
2.

The review team will agree with the Department in advance of the visit a detailed timetable of interviews to be conducted on the visit. It is suggested that this be:

  • Day 1 12.30-5.30pm
  • Day 2 9am-3.30pm 
3.Please see Annex 9.3.2 IQR Sample Timetable for an example. This also provides an indication of the types of meetings that the IQR team will wish to hold during the course of the IQR visit.
4.

Interviewees should always include:

  • The Dean of Faculty concerned. 
  • The Faculty Tutor concerned.
  • The Vice-Dean Education concerned. 
  • Students (including both undergraduate and taught graduate students, wherever the Department teaches at both levels, as well as graduate research students). 
  • The Head of Department.
  • A range of academic staff including senior education leads and professional services staff. 
  • Key Education Staff including programme leads.
5.Where the subject of a review is an interdepartmental degree programme, those interviewed should normally include the Programme Co-ordinator and the Chair of the relevant Board of Examiners.
6.Attendance at each interview session should normally be restricted to those being interviewed within that particular session. Departments should bear in mind the need to provide, as far as possible, a fully representative and balanced sample of staff and students for interview.

3.8 The IQR Report 

1.The administrative secretary will normally have main responsibility for drafting the Report in consultation with the Review Manager, Chair and other members of the team as appropriate.
2.

The IQR Report should normally include (in the following order):

  • The composition of the review team for the current IQR.
  • A list of good practice in the Department. Review teams should seek out and record good practice where there is clear evidence that it has contributed to outstanding achievement in one or more areas of recruitment, progression, student satisfaction, student achievement and employability.
  • A list of recommendations for improvement in the Department's operations - the list should clearly distinguish improvements as either ‘essential’ or ‘advisable’ or ‘desirable’. An essential’ action point will be either (i) dictated by policy as defined in the UCL Academic Manual or (ii) concern an issue which the review team considers to represent a significant risk to the vision or direction of travel of the department, sufficient to warrant immediate action by the Department. An ‘advisable’ action point will be where the Department is not actually contravening UCL policy but the Team feels strongly that practice should be improved. A ‘desirable’ action point reflects a suggestion for improvement based on the personal views of the review team but which is not (at present) prescribed in the Academic Manual. In the case of ‘essential’ recommendations, it is expected that explicit timescales should be set for their implementation. These should be appropriate and achievable.
  • A statement from the External Subject Expert concerning whether the learning outcomes etc. of any sampled programmes are valid and current, whether academic standards are measured appropriately, and student achievement is equally appropriate and whether the academic staffing profile is diverse and able to cover the current programme requirements. Any major/minor programme amendment needed as a consequence of the External Subject Expert’s statement will be dealt with via the existing programme amendment process.
  • Before the draft IQR Report is referred to the Department concerned, the administrative secretary to the IQR team should submit the list of recommendations included in the team’s draft Report to the Quality Assurance Manager for confirmation that the proposed grading of recommendations as ‘essential’ or ‘advisable’ or ‘desirable’ is appropriate.
3.External Subject Experts’ statements may raise issues concerning the currency and validity of programmes reviewed. It should be noted, however, that IQR is not intended to be a procedure for approval or non-approval of programmes. There is no notion within IQR of any programme being approved (or not approved) to continue as a result of the IQR process.
4.When contextualising the recommendations, a Report should detail specifically why the recommendation is being advised, and how this would, in the team’s view, improve departmental performance. A responsible officer must be assigned by role to each recommendation in order to ensure a direct link between the recommendation and the action proposed and to promote accountability to ensure that it is performed. Recommendations should therefore not be made to ‘the Faculty’ or ‘the Department’ but to the specific role of a member (or members) of staff. However, this will be done by the Department as part of its action planning, as it is best placed to know who would be most suitable to implement a particular recommendation. A template will be provided for this purpose by the Administrative Secretary to the review.
5.Any recommendations in IQR Report which are to be addressed by the Faculty, another Department or bodies within UCL, rather than by the Department which is the subject of the review, should be clearly indicated as such in the concluding list of recommendations under the heading ‘Matters for attention outside the Department’.
6.The Review Team would not normally make explicit recommendations for additional resources such as space.  However, this may sometimes be necessary. Briefing will emphasise to Review Teams that they must scrutinise departments closely and they should be challenged to provide evidence that the root cause of an issue affecting the student experience is not within its control. Teams will insist on triangulation between what the Head of Department, the students and active academics are telling them. If the issue is genuinely outside departmental control but impacts on students, the IQR Panel will be able to advocate on the department’s behalf. However, the criteria for intervention will be strict and the issue must be one which is both persistent (i.e., not a ‘one-off’) and particular to the department and/or discipline specific.
7.Where a sensitive or potentially confidential issue has arisen, the review team should, through the team leader and/or administrative secretary, seek guidance on how to address the issue in the IQR Report from the Quality Assurance Manager.
8.

The IQR Report should normally include as appendices:

  • The Department's SES (with a list of the items of supporting documentation).
  • A list of the individuals or groups interviewed on the visit.
9.The full draft IQR Report should be agreed by all members of the review team and should normally be forwarded to the Head of Department, for receipt within fifteen working days of the end of the review visit, with an invitation to notify any factual corrections needed to the Report.
10.The review team will consider the Department's comments on the factual accuracy of the draft Report and will then decide what changes, if any, to make to the report in the light of these comments. The final version of the report will be submitted by the administrative secretary to the review team to: (i) the Head of Department; (ii) the Quality Assurance Manager.

3.9 Follow-up

1.Once the final version of the IQR Report has been submitted to (i) the Head of Department or Programme Co-ordinator and (ii) the Quality Assurance Manager by the administrative secretary to the review team, the administrative secretary will provide the Department with a template for preparing its action plan setting out how it intends to respond to the recommendations contained in the IQR Report. The template comprises juxtaposed lists of:
 i)Recommendations in the IQR Report.
 ii)Action taken or planned in respect of each of these recommendations.
 iii)Timescale for implementation of the recommendation.
 iv)The officer responsible.
2.The Department will be requested to produce an action plan, using the template provided, for discussion by the IQR Panel.
3.The Arena Centre Teaching Fellow or Senior Teaching Fellow appointed to each Faculty will assist the IQR department in the formulation of its action plan in response to any recommendations made by the Review Team. A specific engagement with the Arena Centre to this effect may be undertaken.
4.Recommendations outside the Department.
 Where the IQR Report makes recommendations concerning the Faculty, another Department, Professional Services or other bodies within UCL, the Quality Assurance Manager will write to the other Head(s) of Department(s) concerned, asking them to submit, by a specified deadline, a similar summary of action taken or planned.
5.Departments must:
 i)Ensure that they make the final IQR Report and action plans accessible to students in the Department, e.g. by making these public on departmental intranets. It is proposed that reports and action plans also be circulated to students directly but the feasibility of this is currently under active consultation with the UCL Students’ Union.
 ii)Submit the IQR Report and action plan to the relevant DTC for discussion.
6.The IQR Report will be sent by the Quality Assurance Manager to the officers of the Faculty concerned, with a note which makes clear the Faculty’s particular responsibilities to:
 i)Submit the IQR Report and action plans to the FTC for discussion.
 ii)Note and disseminate within the Faculty good practice and/or recommendations for improvement identified in the IQR Report.

3.10 IQR Panel/Academic Committee 

1.The Quality Assurance Manager will write to the Heads of Departments concerned, asking them to submit the final summary of action taken or planned by the Department in response to the recommendations of the IQR Report for submission to the IQR Panel.
2.On receipt of the action plans, responses and comments requested, the Quality Assurance Manager will refer these for consideration by the IQR Panel, in conjunction with the IQR Reports to which they refer.
3.Sustained dialogue between the Department which has been reviewed and those responsible for oversight of the review process is an essential element of IQR. Consequently, after the review visit has taken place, the Head or a nominated representative of the Department will attend the meeting of the IQR Panel which considers the IQR Report and action plan. The Head or other representative of the Department will be invited to discuss with the Panel at that meeting the perceived usefulness of the IQR process, the Report and recommendations, and the progress made by the Department in implementing the action plan.
4.If, having reviewed the Report and action plan and interviewed the Head of Department, the Panel judges that the Department has not made satisfactory progress in implementing the recommendations, the Quality Assurance Manager (as Secretary to the IQR Panel) may request further information or clarification. Only when the Panel is satisfied that the Department has implemented the recommendations will the Panel recommend that the IQR Report, the Department’s action plan and any other responses to the Report be formally approved.
5.Following the Panel’s approval of responses to an IQR Report, the Quality Assurance Manager will confirm approval in writing to the Departments concerned.
6.The Pro-Vice-Provost of the Doctoral School, as a member of the IQR Panel will be expected to refer PGR issues to Research Degrees Committee and the Vice-Provost (Education and Student Affairs) will refer taught student issues (UG and UGT) to Education Committee. A report on outcomes of the session’s IQRs will be made to Academic Committee.
7.Longer term follow-up of IQR actions in respect of taught provision (UG and PGT) will be undertaken via ASER plans and tracked and monitored by the Quality Review Sub-Committee (QRSC). From session 2019-20 Departments undergoing IQR will be excused ASER in the same year as their IQR and the process of following up UG and PGT actions arising from IQR will also replace ASER the following year. Any longer-term actions should then be rolled up into continuing ASER development and enhancement plans. Follow up of PGR-related actions will be undertaken through the Doctoral Planning Process.

3.11 Dissemination of Findings of IQR Report 

1.Following the Panel’s approval of responses to all the IQR Reports the Quality Assurance Manager will prepare an annual report on that year's IQR programme for submission to and formal approval by Academic Committee.
2.The Quality Assurance Manager will prepare a Summary of Good Practice arising from IQR in the previous session. The IQR Panel will, in the course of its annual discussion of this Summary, refer these to the Arena Centre for wider dissemination and implementation. The Quality Assurance Manager will also circulate the Summary to Vice-Deans Education, Faculty Tutors, Faculty Graduate Tutors with the suggested that they promulgate this more widely by (for example) sharing it with Heads of Department/Division and Departmental Teaching Committees with an invitation to consider whether there are items which could be of wider relevance or application to their own disciplines. The Summary is also copied to the Arena Centre who will publish it on the Teaching and Learning Portal.
3.Recommendations concerning research student issues arising from IQR during the previous session are noted in a separate section and these recommendations and any progress noted are then discussed at the autumn meeting of the Research Degrees Committee. A summary of all other recommendations arising from IQR will also be reported to the Academic Committee.