7.3 Critically Appraising the Literature

Now that you know the parts of a paper, we will discuss how to critically appraise a paper. Critical appraisal refers to the process of carefully and methodically reviewing research to determine its credibility, usefulness, and applicability in a certain context.⁶ It is an essential element of evidence-based practice. As stated earlier, you want to ensure that what you read in the literature is trustworthy before considering applying the findings in practice. The key things to consider include the study’s results, if the results match the conclusion (validity) and if the findings will help you in practice (applicability). A stepwise approach to reading and analysing the paper is a good way to highlight important points in the paper. While there are numerous checklists for critical appraisal, we have provided a simple guide for critical appraisal of quantitative and qualitative studies. The guides were adapted from Epidemiology by Petra Buttner (2015) and How to Read a Paper [the basics of evidence-based medicine and healthcare (2019);  papers that go beyond numbers- qualitative research (1997)] by Trisha Greenhalgh to aid your review of the papers.^5,7,8

A guide to reading scientific articles – Quantitative studies

What is the title of the study?

• Does the title clearly describe the study focus?
• Does it contain details about the population and the study design?

What was the purpose of the study (why was it performed)?

• Identify the research question
• Identify the exposure and outcome

What was the study design?

• Was the design appropriate for the study?

Describe the study population (sample).

• What was the sample size?
• How were participants recruited?
• Where did the research take place?
• Who was included, and who was excluded?
• Are there any potential sources of bias related to the choice of the sample?

What were data collection methods used?

• How were the exposure and outcome variables were measured
• How was data collected- instruments or equipment? Were the tools appropriate?
• Is there evidence of random selection as opposed to systematic or self-selection?
• How was bias minimised or avoided?

For experimental studies

•  How were subjects assigned to treatment or intervention: randomly or by some other method?
•  What control groups were included (placebo, untreated controls, both or neither)
•  How were the treatments compared?
•  Were there dropouts or loss to follow-up?
•  Were the outcomes or effects measured objectively?

For observational studies

• Was the data collection process adequate (including questionnaire design and pre-testing)?
• What techniques were used to handle non-response and/or incomplete data?
•  If a cohort study, was the follow-up rate sufficiently high?
•  If a case-control study, are the controls appropriate and adequately matched?

How was the data analysed?

• Is the statistical analysis appropriate, and is it presented in sufficient detail?

What are the findings?

• What are the main findings of the study? Pay specific attention to the presented text and tables in relation to the study’s main findings .
• Are the numbers consistent? Is the entire sample accounted for?

Experimental study

•  Do the authors find a difference between the treatment and control groups?
•  Are the results statistically significant? If there is a statistically significant difference, is it enough of a difference to be clinically significant?

Observational study

•  Did the authors find a difference between exposed and control groups or cases and controls?
•  Is there a statistically significant difference between groups?
•  Could the results be of public health significance, even though the difference is not statistically significant? (This may highlight the need for a larger study).
• Are the results likely to be affected by confounding? Why or why not?
• What (if any) variables are identified as potential confounders in the study?
• How is confounding dealt with in this study?
• Are there any potential confounders that the authors have not taken into account? What might the likely impact be on the results?

Summing it up

A guide to reading scientific articles – Qualitative studies

What is the title of the study?

• Does the title clearly describe the study focus?
• Does it contain details about the population and the study design?

What is the research question?

Was a qualitative approach appropriate?

• Identify the study design and if it was appropriate for the research question.

How were the setting and the subjects selected?

• What sampling strategy was used?
• How were participants recruited?
• Where was the study conducted?

Was the sampling strategy appropriate for the approach?

• Consider the qualitative approach used and decide if the sampling strategy or technique is appropriate

What was the researcher’s position, and has this been taken into account?

• Consider the researcher’s background, gender, knowledge, personal experience and relationship with participants

What were the data collection methods?

• How was data collected? What technique was used?

How were data analysed, and how were these checked?

• How did the authors analyse the data? Was this stated?
• Did two or more researchers conduct the analysis independently, and were the outcomes compared (double coding)?
• Did the researchers come to a consensus, and how were disagreements handled?

Are the results credible?

• Does the result answer the research question?
• Are themes presented with quotes and do they relate to the research question or aim?

Are the conclusions justified by the results?

• Have the findings been discussed in relation to existing theory and previous research?
• How well does the interpretation of the findings fit well with what is already known?

Are the findings transferable to other settings?

• Can the findings be applied to other settings? Consider the sample.

Summing it up

Now that you know how to critically appraise both quantitative and qualitative papers, it is also important to note that numerous critical appraisal tools exist. Some have different sub-tools for different study designs, while others are designed to be used for multiple study designs. These tools aid the critical appraisal process as they contain different questions to prompt the reader while assessing the study’s quality.⁹ Examples of tools commonly used in health professions are listed below in Table 7.2. Please note that this list is not exhaustive, as numerous appraisal tools exist. You can use any of these tools to appraise the quality of an article before choosing to use their findings to inform your own research or to change practice.

Table 7.2 Critical appraisal tools