Reliability and Validity

5.5 Reliability and Validity Checks for Quantitative Research

Assessing the goodness of data is determining that how we are conducting our research is actually measuring the variables supposed to be measured and measuring them accurately. To achieve this, we look at reliability, this is accuracy in measurement. Our measurement instrument consistently measures what it is supposed to be measuring. We also look at validity to determine we are measuring the right thing, and how well our instrument measures what it is supposed to be measuring. Research can be reliable but not valid, but when valid, it is reliable, hence, both are extremely important in research (Sekaran & Bougie, 2013).

Reliability

Reliability is the extent to which:

• the measure is error-free (without bias)
• measurements are consistent across time and items within the instrument
• you can replicate the research design and achieve the same results. (Sallis et al., 2021; Sekaran & Bougie, 2013)

There are several tests of reliability:

• Test-retest

  • The measure remains stable over time.
  • Regardless of changes in uncontrolled testing conditions, and any changes in the state of the respondents, the measure has low vulnerability to change over time.
  • Determining using correlation of responses obtained from the same respondents using the same measures across time, for example, application one, and then at least 2 weeks later, application two.

• Alternative form

  • Different forms measuring the same construct are equivalent.
  • Determined using correlation of responses obtained from the same respondents using the same measures across time, for example, application one, and then at least 2 weeks later, application two.

• Split-halves

  • items of measures in the instrument are divided into 2 sets
  • considers the correlation of the 2 halves of the instrument as measuring the same construct.

• Internal consistency

  • the homogeneity of items used in the measure
  • items should ‘hang together’ as a set and be capable of independently measuring the same construct
  • a variety of algorithms can be used to estimate how reliable a measure is at one point-in-time. (Forza & Sandrin, 2024; Sekaran & Bougie, 2013)

Validity

Validity relates to the appropriateness of measures used, the accuracy of the analysis of results, and the generalisability of the findings. That is, measures must accurately measure what the researcher(s) intend them to measure for them to be valid (Newcomer et al., 2015).

• Content validity
  • Each measure’s items should come within the concept’s theory, or in qualitative studies, the theoretical domain.
  • Each measure’s items should not be too similar; they need to capture different elements of the concept in a balanced way. (Forza & Sandrin, 2024)
• Face validity
  • measures used should be easy to understand; it should be clear what is being measured, and on the ‘face of it’, they are measuring the concept. (McLaughlin & Jordan, 2015)
• Construct validity
  • measured through convergent validity and discriminant validity
  • convergent validity is when 2 different instruments measuring the same concept produce highly correlated results
  • Discriminant validity can be established when, on a theoretical basis, where 2 items (variables) are predicted to be distinct, that is, they are not correlated. Here, scores produced when measuring the two items (variables) show they are not correlated. (Forza & Sandrin, 2024)
• Criterion-related validity
• • To establish criterion-related validity, the measure must differentiate participants on a criterion it is expected to predict. To achieve this concurrent or predictive validity must be achieved.
  • Concurrent validity is when the scale discriminates between participants who are known to be different.
  • Predictive validity is the ability of the measuring instrument to differentiate among participants with reference to future criterion. (Forza & Sandrin, 2024; Sekaran & Bougie, 2013)

5.6 Reliability and Validity Checks for Qualitative Research

Testing the reliability and validity of data in qualitative research is not as simple and straightforward as with quantitative research, and there is also debate over the use of the terms reliability and validity in relation to qualitative research. Whereas in quantitative research we seek reliability, in qualitative research this is seen as consistency. This is how much methods used by qualitative researchers can be trusted and followed, decisions made can be clearly followed and an independent researcher’s findings would differ very little (Noble & Smith, 2025). Validity in qualitative research is all about the truth and its value. This relates closely to how the researcher situates themselves in reality, that is, their paradigm. The researchers are upfront in relation to any pre-conceived views or biases they may have in relation to the research conducted (Noble & Smith, 2025).

References

Forza, C., & Sandrin, E. (2024). Surveys. In C. Karlsson (Ed.), Research methods for operations and supply chain management (3rd ed., pp. 76-158). Routledge. https://doi.org/10.4324/9781003315001

McLaughlin, J. A., & Jordan, G. B. (2015). Using logic models. In K. E. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of practical program evaluation (4th ed., pp. 62-87). Jossey-Bass.

Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (2015). Planning and designing useful evaluations. In K. E. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of practical program evaluation (4th ed., pp. 7-35). Jossey-Bass.

Noble, H., & Smith, J. (2025). Ensuring validity and reliability in qualitative research. Evidence Based Nursing. Advanced online publication. https://ebn.bmj.com/content/ebnurs/early/2025/01/21/ebnurs-2024-104232.full.pdf

Sallis, J. E., Gripsrud, G., Olsson, U. H., & Silkoset, R. (2021). Research methods and data analysis for business decisions: A primer using SPSS. Springer. https://doi.org/10.1007/978-3-030-84421-9

Sekaran, U., & Bougie, R. (2013). Research methods for business: A skill-building approach (6th ed.). John Wiley & Sons.