Thinking critically about psychology research
Critical thinking is often taught in undergraduate psychology degrees, and is a key marking criteria for higher marks in many assignments. But getting your head around how to write critically can sometimes be difficult. It can take practice. The aim of this short blog is to provide an introduction to critical evaluation, and how to start including evidence of critical evaluation in your psychology assignments.
So what does “critical evaluation” really mean?
Broadly speaking, critical evaluation is the process of thinking and writing critically about the quality of the sources of evidence used to support or refute an argument. By “evidence“, I mean the literature you cite (e.g., a journal article or book chapter). By “quality of the evidence“, I mean thinking about whether this topic has been tested is in a robust way. If the quality of the sources is poor, then this could suggest poor support for your argument, and vice versa. Even if the quality is poor, this is important to discuss in your assignments as evidence of critical thinking in this way!
In the rest of this blog, I outline a few different ways you can start to implement critical thinking into your work and reading of psychology. I talk about the quality of the evidence, a few pointers for critiquing the methods, theoretical and practical critical evaluation too. This is not an exhaustive list, but hopefully it’ll help you to start getting those higher-level marks in psychology. I also include an example write-up at the end to illustrate how to write all of this up!
The quality of the evidence
There are different types of study designs in psychology research, but some are of higher quality than others. The higher the quality of the evidence, the stronger the support for your argument the research offers, because the idea has been tested more rigorously. The pyramid image below can really help to explain what we mean by “quality of evidence”, by showing different study designs in the order of their quality.
Not every area of psychology is going to be full of high quality studies, and even the strongest sources of evidence (i.e., systematic reviews and/or meta-analyses) can have limitations! Because no study is perfect, it can be a good habit to tell the reader, in your report, (i) what the design of the study is that you’re citing, AND, (ii) how this affects your argument. Doing so would be evidence of critical thought. (See an example write-up below for implementing this, but do not copy and paste it!)
But first, what do I mean by “design”? The design of the study refers to how the study was carried out. There are sometimes broad categories of design that you’ll have heard of, like a ‘survey design’, ‘a review paper’, or an ‘experimental design’. Within these categories, though, there can be more specific types of design (e.g. a cross-sectional survey design, or a longitudinal survey design; a randomised controlled experiment or a simple pre-post experiment). Knowing these specific types of design is a good place to start when thinking about how to critique the evidence when citing your sources, and the image below can help with that.
Image source: https://thelogicofscience.com/2016/01/12/the-hierarchy-of-evidence-is-the-studys-design-robust/
In summary, there are various types of designs in psychology research. To name a few from the image above, we have: a meta-analysis or a systematic review (a review paper that summarises the research that explores the same research question); a cross-sectional survey study (a questionnaire that people complete once – these are really common in psychology!). If you’re not familiar with these, I would highly suggest doing a bit of reading around these methods and some of their general limitations – you can then use these limitation points in your assignments! To help with this, you could do a Google Scholar search for ‘limitations of a cross-sectional study’, or ‘why are randomised control trials gold standard?’. You can use any published papers as further support as a limitation.
Methodological critical evaluation
- Internal validity: Are the findings or the measures used in the study reliable (e.g., have they been replicated by another study, and is the reliability high)?
- External validity: Are there any biases in the study that might affect generalisability(e.g., gender bias, where one gender may be overrepresented for the population in the sample recruited)?
Lack of generalisability is a common limitation that undergraduates tend to use by default as a limitation in their reports. It’s a perfectly valid limitation, but it can usually be made much more impactful by explaining exactly how it’s a problem for the topic of study. In some cases, this limitation may not be all that warranted; for example, a female bias may be expected in a sample of psychology students, because undergraduate courses tend to be filled mostly with females! - What is the design of the study, and how it a good or bad quality design (randomised control trial, cross-sectional study)?
Theoretical critical evaluation
- Do the findings in the literature support the relevant psychological theories?
- Have the findings been replicated in another study? (If so, say so and add a reference!)
Practical critical evaluation
- In the real world, how easy would it be to implement these findings?
- Have these findings been implemented? (If so, you could find out if this has been done well!)
Summary points
In summary, there are various types of designs in psychology research. To name a few from the image above, we have: a meta-analysis or a systematic review (a review paper that summarises the research that explores the same research question); a cross-sectional survey study (a questionnaire that people complete once – these are really common in psychology!). If you’re not familiar with these, I would highly suggest doing a bit of reading around these methods and some of their general limitations – you can then use these limitation points in your assignments! To help with this, I would do a Google Scholar search for ‘limitations of a cross-sectional study’, or ‘why are randomised control trials gold standard?’. You can use these papers as further support as a limitation.
You don’t have to use all of these points in your writing, these are just examples of how you can demonstrate critical thinking in your work. Try to use at least a couple in any assignment. Here is an example of how to write these up:
An example write-up
“Depression and anxiety are generally associated with each other (see the meta-analysis by [reference here]). For example, one of these studies was a cross-sectional study [reference here] with 500 undergraduate psychology students. The researchers found that depression and anxiety (measured using the DASS-21 measure) were correlated at r = .76, indicating a strong effect. However, this one study is limited in that it used a cross-sectional design, which do not tell us whether depression causes anxiety or whether anxiety causes depression; it just tells us that they are correlated. It’s also limited in that the participants are not a clinical sample, which does not tell us about whether these are clinically co-morbid constructs. Finally, a strength of this study is that it used the DASS-21 which is generally found to be a reliable measure. Future studies would therefore benefit from using a longitudinal design to gain an idea as to how these variables are causally related to one another, and use more clinical samples to understand the implications for clinical practice. Overall, however, the research generally suggests that depression and anxiety are associated. That there is a meta-analysis on this topic [reference here], showing that there is lots of evidence, suggests that this finding is generally well-accepted.”
- Notice how I first found a review paper on the topic to broadly tell the reader how much evidence there is in the first place. I set the scene of the paragraph with the first sentence, and then the last sentence I brought it back, rounding the paragraph off.
- Notice how I then described one study from this paper in more detail. Specifically, I mentioned the participants, the design of the study and the measure the researchers used to assess these variables. Critically, I then described how each of these pieces of the method are disadvantages/strengths of the study. Sometimes, it’s enough to just say “the study was limited in that it was a cross-sectional study”, but it can really show that you are thinking critically, if you also add “… because it does not tell us….”.
- Notice how I added a statistic there to further illustrate my point (in this case, it was the correlation coefficient), showing that I didn’t just read the abstract of the paper. Doing this for the effect sizes in a study can also help demonstrate to a reader that you understand statistics (a higher-level marking criteria).
Are these points you can include in your own work?
Thanks for reading,
Rosie
Rosie Psychology: Your online tutor 