Academic tips from GBSH student – How to read and review relevant research paper

Evaluating academic papers is such an integral skill in master's programmes. I'm certainly no expert, but as a master's student at UCL's GBSH, I've picked up a fair bit about the process. I figured I'd share my two cents in case it helps any newcomers.

When I read a paper, I try to assess each section carefully. I look at the methods - are they solid? I interpret the results and compare them to prior work. I think about the implications and limitations and the overall contribution. It's also helpful to discuss all this with my advisor and peers. We bounce ideas around from different angles.

Of course, I'm still learning myself! However, having a systematic approach helps me better analyse papers as I continue developing this skill. Hopefully, these insights from my experience so far can give new master's students a bit of a head start.

Here is how I would analyse an academic paper in bullet points:
1.    Carefully read the title, abstract, and introduction to get an overview of the topic, research questions, methodology, and main findings.
2.    Skim through the paper, noting the significant sections and structure. Look for literature review, hypothesis development, data and methods, results, and discussion/conclusions.
3.    Read the literature review and make notes on previous studies, gaps addressed, critical theories referenced, and how they relate to the current paper.
4.    Look closely at the hypothesis section and underline the proposed hypotheses. Consider how they build on the literature review.
5.    Examine the data and methods section in depth. Make notes on the sample description, variables used, sources of data collection, and statistical analysis methods. Assess the validity, rigour, and limitations of the methodology.
6.    Review the results, noting the main findings, including numerical results and statistical significance. Do the data analyses appear appropriate to test the hypotheses? Are the limitations of the study discussed?
7.    Read the discussion and conclusion closely. Do they logically follow from the results? Are the research questions answered? Is the contribution to literature and implications clearly described? Are limitations acknowledged?
8.    Consider how the paper relates to broader debates or issues in the field. Suggest future research directions building on this study.
9.    Evaluate the paper's overall quality, rigour, and contribution. Are the methods scientifically sound? Is it well-written and organised logically? Does it significantly advance knowledge in the area?

I will take a paper on ESG that I have read recently (From the Stockholder to the Stakeholder: How Sustainability Can Drive Financial Outperformance) as an example; my analysis is below:

1.    Title and Introduction
•    The title "How Sustainability Can Drive Financial Outperformance" clearly indicates the paper will examine the relationship between sustainability and corporate financial performance.
•    The abstract states the paper reviews over 200 high-quality studies on links between ESG (environmental, social, governance) factors and financial metrics like cost of capital, operating performance, and stock prices.
•    The introduction explains the paper will assess the business case for sustainability and ESG integration by investors. It aims to provide economic evidence on both sides of the debate using a robust sample of studies.

2.    Paper Structure
•    The structure moves from theory to evidence, reviewing the academic literature and then providing empirical analysis on the financial impacts of sustainability.

3.    Literature Review
•    The literature review is mainly in the introduction and the business case section.
•    It summarises previous research showing mixed evidence on whether sustainability practices improve corporate financial performance.
•    The section refers to critical theories like competitive advantage from sustainability initiatives and debates on shareholder versus stakeholder models.
•    The current paper aims to clarify the financial impacts through a comprehensive review of studies. It addresses gaps in definitively determining the business case.

4.    Hypotheses
•    The paper does not have a distinct hypothesis section.
•    The introduction states the paper will assess if sound sustainability practices lower the cost of capital, improve operational performance, and lead to higher stock prices.
•    These implied hypotheses build on the literature review, which showed contradictory evidence on the financial impacts. The paper aims to test them through its robust evaluation.
•    Though devoid of an explicit hypothesis section, the introduction lays the groundwork for implied hypotheses. These conjectures flow from the literature review and intend to substantiate the relationship between sustainability and financial outcomes.

5.    Data and Methods
•    The methodology systematically reviews and analyses existing academic studies on the topic (secondary data).
•    Over 200 studies are categorised by effects on cost of capital, operational performance, and stock prices.
•    Using such a comprehensive sample adds rigour and representativeness. However, reliance on secondary data limits causality assessment in the original studies.
•    Statistical meta-analysis to determine overall effect size is not conducted. The categorisation is used to imply directional effects.
•    Employing a systematic review of existing studies, the methodology classifies over 200 research pieces by their effects on financial metrics. While this approach ensures robustness, its reliance on secondary data limits the potential to infer causality.

6.    Results
•    The results show that 90% of studies find sustainability lowers the cost of capital, 88% find it improves operational performance, and 80% find it increases stock prices.
•    The data analysed appears appropriate to test the hypotheses on the financial impacts of sustainability directly.
•    However, statistical significance of the results is not reported. And meta-analytic techniques are not used.
•    There is no discussion of publication bias in the analysis.
•    Results reveal a notable percentage of studies pointing towards sustainability's positive influence on cost of capital, operational performance, and stock prices. While these findings align with implied hypotheses, the absence of statistical significance indicators and meta-analytical techniques detracts from the rigour.

7.    Discussion and Conclusion
•    The discussion and conclusion argue the findings show a clear economic case for corporate sustainability and ESG integration by investors.
•    It relates to broader debates on CSR and responsible investing but does not mention the limitations of relying on secondary data.
•    The paper calls for ongoing research on material sustainability factors by industry to strengthen the business case.
•    The conclusion logically extrapolates the findings into an economic rationale for corporate sustainability and ESG integration. While tying into broader debates, limitations in relying on secondary data are unaddressed. The study prompts future research directions to bolster the business case.

8.    Broader Issues
•    The paper relates to debates on whether sustainability improves corporate financial performance that have implications for management research and practice.
•    It aims to inform investment processes and active ownership policies based on the financial materiality of ESG factors.
•    Future research could use primary data and meta-analytic techniques to address causality and generalizability further.
•    The paper resonates with ongoing dialogues regarding the financial repercussions of sustainability, carrying implications for both scholarly inquiry and practical corporate strategies. It advocates informed investment practices and emphasises future avenues for research.

9.    Quality and Contribution
•    The large sample size, rigorous categorisation by financial metrics, and neutral perspective add credibility.
•    The methods are appropriate for a review paper, but limitations exist in assessing causality in the underlying studies.
•    It contributes significantly to the academic literature on the financial case for ESG issues. The analysis appears to resolve previous contradictory findings.
•    The paper's robust categorisation, comprehensive approach, and neutral standpoint bolster its credibility. While methodologically suitable for a review paper, limitations in causal inference remain. Nevertheless, its significant contribution to the discourse on ESG's financial dimensions is palpable, seemingly reconciling prior disparities.

The process I typically review an academic paper as a master's student. My process involves all the content above and taking extra time to discuss the paper with my supervisor and peers. My perspective may not be the expert since I am still learning. I suggest looking at relevant materials from UCL's library website for the most professional guidance. There, you can access expert resources vetted by UCL librarians.