SORTING THROUGH ISSUES WITH A CRITICAL LENS

WSRA ADVOCACY TOOL: 

WSRA values strong advocacy as one of its primary goals. Literacy educators need to be well-informed to better advocate for what is in the best interests of their individual students. Besieged by numerous literacy reports, newspaper articles, research, documentaries, podcasts, policies, legislation, social media posts, and hearsay, it is often difficult to sort through what is fact versus what is fiction.  

The key tool for becoming a critical literacy advocate is to review information through a critical lens. Here’s what to do as a critical consumer generally, and when reviewing research:

• Ask questions
• Determine fact versus someone’s opinion
• Look for evidence to support the ‘facts’ being presented
• Consider whether the ‘facts’ could be interpreted in a different way
• Reflect on the ‘whole picture’ and determine that nothing critical is missing or being deliberately left out.

QUESTIONS TO ASK

A.    BACKGROUND

• Do I have the background necessary to understand the issue? What kind of background do I need? What facts do I need?

• When reading blog posts, books, articles, and listening to the media, podcasts, whose perspective is foregrounded?
• Whose voice is marginalized?
• What is the author’s intent? What does the author want me to walk away with? Does the author want an emotional response?

• Is the information too good to be true?
• What is the credibility of the evidence provided?
• Is the “evidence” from an author’s opinion, blog post, media, article, book, or hearsay? Are there citations for the evidence? 
• Is data from a credible source?
• Is the “evidence” or research from a publisher or a peer-reviewed journal? Whose journal?
• Are there inconsistent statements?

• Do I know the research or am I relying on someone else’s interpretation of the research?
• Who is conducting the study?
• Is there a conflict of interest disclosed at the end of the study?
• Does the study establish causation through randomized controlled assignment of comparison groups? 
• When, for whom, and in what conditions does this effect occur?
• Is the causal effect nonspurious? This means the effect was not caused by a third variable. 
• Has the study been successfully replicated?
• Is the evidence from a meta-analysis* of research, publisher’s “evidence/research”, hearsay, or actual research studies?
• Is there peer-reviewed analysis for what a publisher/researcher/journalist, is purporting?
• Is the “evidence” what the research study really says?
• What are the generalizations that can be made and limitations of the study?
• Are there overgeneralizations?
• Does the study call for future research to better understand other variables and/or to deepen understanding of the current research?

*Limitations of a meta-analysis include:

• “a statistical combination of results of several trials to produce a summary effect, has been subject to criticism in the past, mainly for the reasons of poor quality of included studies, heterogeneity between studies meta-analyzed and failing to address publication bias. These limitations can cause the results to be misleading, which is important if policy and practice decisions are based on systematic reviews and meta-analyses.” https://pubmed.ncbi.nlm.nih.gov/27217394/
• the stripping away of the context or “set of interrelated circumstances in which a particular outcome should be understood.” (Chambliss & Schutt)
• the lack of consistent definitions for particular key terms used in the individual studies
• “its quantitative results should be interpreted with caution even when the analysis is performed according to rigorous rules” https://pubmed.ncbi.nlm.nih.gov/8934372/

 REFERENCES

Chambliss, D.F., & Schutt, R.K. (2019.) Making Sense of the Social World: Methods of Investigation. Thousand Oaks, CA. Sage Publications.

 Champeau, K. & Zarling, D. (2023.) The Science of Reading: Sorting Fact From Fiction, Looking Through a Critical Lens. WSRA Conference. Milwaukee. 

 Esterhuizen TM, Thabane L. Con: Meta-analysis: some key limitations and potential solutions. Nephrol Dial Transplant. 2016 Jun; 31(6):882-5. National Library of Medicine. doi: 10.1093/ndt/gfw092. PMID: 27217394.

 Fagard RH, Staessen JA, Thijs L. Advantages and disadvantages of the meta-analysis approach. J Hypertens Suppl. 1996 Sep;14(2):S9-12; discussion S13. National Library of Medicine. doi: 10.1097/00004872-199609002-00004. PMID: 8934372.