XClose

The Nahrein Network

Menu

Peer Review Process

Overview of the peer review process for assessing Research Grant Awards applications

Overview 

The Nahrein Network's peer review process for Research Grant Awards abides by UKRI's principles of assessment and decision making and is based on AHRC-UKRI's peer review guidance. All members of our Peer Review College are international experts, working outside Iraq. Management Committee members are ineligible to serve as peer reviewers.

Our process is as follows: 

  • Immediately after a Research Grant Awards round closes, all applications are checked for eligibility and completeness 
  • All eligible Expressions of Interest are assigned to at least one peer reviewer for independent assessment ahead of the next Management Committee meeting. Reviewers are typically assigned 3–5 applications each; every application is also read by the Director and Deputy Director 
  • Reviewers make independent assessments of the projects they are assigned and complete a short, secure online evaluation form, to an agreed deadline ahead of the Management Committee meeting 
  • The reviewers’ rankings are collated in order to determine the top ten applications. If a clear top ten does not emerge by this process, peer reviewers may be asked to join a short online meeting in early December to discuss and agree the shortlist. 
  • The shortlist is sent to the Nahrein Network’s Management Committee for ratification at its next meeting 
  • Applicants are notified of the outcome of this process as soon as possible after that meeting 

  1. The role of the peer reviewer 

Peer reviewers are provided with a small number of eligible Expresssions of Interest and an assessor’s form. In advance of starting work on reviewing them we advise them to:  

  • familiarise themselves with these guidelines and assessment criteria for the scheme 
  • alert the Nahrein Network administrator to any conflicts of interest, including potential conflicts. 

In reviewing the applications they must:  

  • use their knowledge, judgement and expertise in order to reach clear, sound, evidence-based decisions. 
  • treat all applications, and the discussions about them, as strictly confidential at all times. 
  • strive to be fair and objective  
  • adhere to the Nahrein Network’s equal opportunities policy   

Safeguarding decision making 

We are committed to ensuring that those who make funding decisions recognise the factors that introduce risk into the decision making process. To do this, it is important to be aware of and take steps to remove any impact of unintentional bias in our processes, behaviours and culture. We know that pressure to make decisions, time pressures, high cognitive load and tiredness all create conditions that introduce the risk of unintentional bias. 

To minimise these risks the reviewers are advised to consider the following: 

  • All applications must be assessed on equal terms and objectively assessed on their merits using the evluation form 
  • Decisions must be based on all the information provided 
  • Question and challenge cultural stereotypes and bias 
  • Be aware that working with a high cognitive load, with time pressures and the need to make quick decisions, creates conditions for bias  
  • Slowing down the speed of decision making, allowing sufficient time for considering each application 
  • Reconsider the reasons for decisions, recognising that they may be post-hoc justifications 
  • Question cultural stereotypes and be open to seeing what is new and unfamiliar 
  • Remember you are unlikely to be fairer and less prejudiced than the average person 
  • One can detect unconscious bias more easily in others than in yourself, so all panel members should feel able to call out bias when they see it 

For further information, the Royal Society has issued a Briefing and video on unconscious bias. 

Proposals are submitted to the Nahrein Network in confidence and may contain confidential information and personal data belonging to the applicant (and other researchers named in the proposal). Peer reviewers are asked to make sure that all proposals are treated confidentially. 

  1. Conflicts of interest 

It is vital that peer reviewers are seen to be completely impartial at all stages of the review process.   

Peer reviewers should not take part in the assessment of any proposal where a conflict of interest is at play. Conflicts of interest need to be declared at nahrein@ucl.ac.uk. 

If any Management Committee member is in conflict with a proposal, they will be required to leave the meeting whilst the proposal is being discussed.  

  1. Approach to assessment 

In order to fully understand the quality and content of the proposals, all peer reviewers must ensure their judgements are based solely on the scheme requirements, the Nahrein Network’s Aims and the assessment criteria for the Research Grant Awards Scheme (see below), as well as the information that is provided in the application form. 

Reviewers and Management Committee members should not allow private knowledge of the applicant or the proposed research to influence their judgement and panellists are expected to decline invitations to review if their private views, knowledge or relations will affect the judgement of applications. 

Before starting the review process, reviewers are advised to:  

  • read the entire proposal thoroughly.  
  • familiarise themselves with the Nahrein Network’s Aims and the scheme assessment criteria (below). 
  • contact the Nahrein Network administrator nahrein@ucl.ac.uk if anything is unclear. 

Review  

  • Observations must always be accompanied by evidence to support them. Reviewers must use only the information provided in the application form.  
  • Reviewers should take into account the information waht they are asked to provide under each heading or item in the scheme assessment criteria and ensure sufficient detail is provided for each one.  
  • Reviewers should give a clear assessment of strengths and weaknesses of the proposal and indicate whether these are major or minor concerns.  
  • Reviewers should provide an evaluation of the risks associated with the project.  
  • Reviewers should contextualise the proposal that they are assessing within current work in the field, and comment on its relative importance/significance.  
  • Reviewers should be receptive to new ideas and approaches to thinking within thier discipline as well as methodology. 
  • Reviewers should identify any inconsistencies and contradictions in the proposal.  
  • Reviewers should scrutinise the budget and justification of resources for appropriate level of detail and value for money 
  • In the case of interdisciplinary applications, reviewers should assess if the different disciplines meet up in a coherent way. 
  • Reviewers should provide enough information to enable a judgement on the relative quality of this proposal compared to other applications. 

General points 

Reviewers should: 

  • provide an impartial, objective, fair and analytical assessment of the proposal under review
  • ensure they are providing an evaluation, not a description of the work proposed
  • ensure the grade is justified by, and consistent with, any comments submitted.  

  1. Grading proposals 

In the Research Grant Awards evaluation form, reviewers should comment briefly on the strengths and weaknesses of the proposal, against each of the following criteria: 

  1. The research question / problem and context are relevant and well explained 
  2. The work could create new knowledge about Iraqi history, heritage or a related area  
  3. The applicant is qualified to do this work 
  4. The proposed project partners are suitable 
  5. The proposed methodology is appropriate and viable 
  6. The proposed outputs are suitable and viable 
  7. The project addresses at least one of the Network’s research aims 
  8. The project has potential to improve social, cultural or economic like in Iraq/KRI 

Reviewers should score each criterion of assessment in the range 1–5, where 1 is Poor and 5 is Excellent. They are also required to give an overall judgement: 

  • STRONG: priority to shortlist for further development 
  • FAIR: lesser priority shortlist for further development  
  • WEAK: do not shortlist for further development 

Completed evalulation forms must be completed by the agreed deadline ahead of the Management Committee. 

The Management Committee will be guided by the reviewer's evaluation and scoring in their ratification, alongside other criteria such as the range and profile of proposals under consideration, and the amount of funding available in each round.  

  1. The role of the Management Committee  

The Nahrein Network Management Committee meets to ratify the peer reviewers’ recommendations, make final decisions on which proposals to fund and, where necessary, to agree broad feedback for applicants.   

Comments and grades will not be used outside the funding decision making process, unless they are subject to specific legal requirements or to be used as the basis of feedback. 

The Management Committee also welcome peer reviewers’ comments and recommendations for improving the application and assessment process for applicants and reviewers in future rounds of the scheme. 

  1. Resubmission Policy 

Resubmission of unsuccessful applications is not permitted except in very particular circumstances, where the Management Committee may exceptionally decide to invite the applicant to resubmit the proposal on one further occasion. 

This will happen only where the Committee identifies an application of exceptional potential and can identify specific changes to the application that could significantly enhance its competitiveness. In this case, the Committee will need to agree specific feedback — based on the reviewers’ comments — to be provided to the applicant. 

In order for a proposal to be invited for resubmission the Management Committee should satisfy itself that it meets all of the following criteria: 

  •  the core research ideas and approach are original, innovative and exciting and the proposal has outstanding, transformative potential. It has clear potential to secure funding if the identified weaknesses can be satisfactorily addressed 
  • there should be clear potential for the revised proposal to significantly increase its overall grading and priority for funding  
  • the Management Committee should be confident that issues identified in deeming a proposal to be unfundable can be addressed through resubmission and that these are surmountable. This does not necessarily mean that the Committee is able to identify how this will be achieved, just that they are confident that it is possible. 
  • the issues should be of sufficient scale and significance that they could not have been adequately addressed through the use of conditions. Requested changes should be of sufficient scale to require the proposal to go through the full assessment process once more  
  • the Management Committee must be able to provide clear guidance on the key issue or issues which need to be addressed in any resubmission.  

When invited resubmissions are submitted they will be assessed in the usual way in competition with all other proposals.  

Invited resubmissions should not be used:  

  • where the identified weaknesses relate to under-development, poor presentation or other problems relating to the preparation of the proposal, which could reasonably have been expected to be addressed in submitting a proposal of this kind
  • for proposals where the core ideas, rationale and foundations, aims and focus or overall design of the project need substantial re-working, since such radically revised proposals could be submitted as a significantly re-worked new proposal rather than as a resubmission. 

  1. Feedback on processes 

If reviewers or Management Committee members have any feedback on Nahrein Network policy, process and/or documentation for this scheme, this will be discussed in the meeting and recorded once all funding decision have been made.  Reviewers are also welcome to submit feedback. All feedback will be formally recorded and used by the Network to inform the future development of processes.    

  1. After the Management Committee Meeting 

It is vital that peer reviewers and Management Committee members do not divulge or discuss the content of applications, evaluations or funding outcomes with any individual who is not directly involved in the assessment and awarding process. Maintaining confidentiality is paramount. 

All announcements of outcomes and funding decisions will be made by the Nahrein Network. Any peer reviewer or Management Committee member who is asked directly for feedback by applicants should refuse and advise applicants to direct all such requests to nahrein@ucl.ac.uk. 

Following the meeting, reviewers will be reminded to delete all copies of applications and evaluations, in compliace with GDPR legislation.