Inroduction

Undergraduate biology education is moving away from traditional lecture and textbook based models to a more welcoming and authentic experience. Part of this movement relies on using primary literature as a focal point for in-class discussions. However, scientific research articles are written for an audience of specialist within a field, not for students.

Learning how to read these articles is an essential skill for undergraduate science students to develop as future scientists and consumers of scientific knowledge. Students develop critical thinking skills and gain a deeper understanding of relationship between scientific evidence and conclusions from reading scientific articles. They also learn science in the way that scientists learn about science -in the context of observations, hypotheses, experiments, data, and analysis (Campbell 2004).

Several outstanding efforts to teach students how to read scientific literature have been published. These approaches can be classified as either data- or text-eccentric (Round, Campbell, and Siegel 2013). Figure Facts is a data eccentric approach that identifies the results section of a paper as the most difficult section of a paper to understand. Consequently, the Figure Facts approach redirects the focus of student reading away from the textual elements of a paper to the inscriptions such the figures, tables, images. (Round, Campbell, and Siegel 2013). The CREATE method is another data-centric approach that incrementally guides students through the scientific process by examining how scientific discoveries build upon each other by following the progress of a series of research articles from the same research group (Hoskins, Stevens, and Nehm 2007; Gottesman and Hoskins 2013).

A text-based approach shifts students attention away from surface-level reading to deeper reading that often involves identifying the persuasive elements of the text. These can be as straight as forward asking students to distinguish between the experimental narrative, interpretive, and critical elements of a text (Gillen 2006), or it could involve asking students to identify a core set of rhetorical moves (Lacum, Ossevoort, and Goedhart 2014). After identifying these persuasive elements or rhetorical moves, the next step is to have students develop interpretations and criticisms that are independent from the authors. (Gillen 2006).

The data centric and text centric approaches can work independently, but I think a combination of both approaches creates some interesting possibilities. For less experienced or less motivated students, the text-centric approach is easier to implement, but they are not really learning much science by focusing only on rhetorical elements. However, once student become proficient at identifying the most common rhetorical moves, the data-centric approach can lead to the formation of wonderful classroom discussions about science, which is an important goal for me.

To help students learn how to read a scientific article, I started assigning my students the task of collaboratively annotating the text using hypothes.is.

Tagging annotations

Tagging annotations is a good practice. Students should use the terms highlighted in grey below to tag their annotations according to the type of annotation.

Taxonomy of Annotations

Student annotations can be classified into three classes: didactic, dialectic, and rhetorical annotations.

Didactic Annotations

Didactic annotations are focused on the information required to understand a text.

  • Glos: Define a word or phrase
  • Background: Generate an outline of the topics you would provide in a presentation explaining the background knowledge needed to understand the research. This list should be created as page note instead of being anchored to a specific highlighted text. Items in the list can be hyper-linked to other sources of information such as Wikipedia entries, related review articles, or videos. Explanations or commentary on items in the list should be avoided. If the list needs to be explored more extensively, then a formal slide presentation covering the background knowledge can be embedded in this section as a presentation, info graphic, or short video.
  • Summary: Using the page note function, write a paragraph using your own words to retell important details

Dialetic Annotations

Dialectic annotations are more conversational in tone.

  • Confusion: Select words or phrases that you find confusing and identify the source of your confusion.
  • Question: Ask a question about the text.
  • Opinion: Share your opinion about something that is controversial.
  • Answer: Responding to a question asked by the text’s author.
  • Class Disucssion: Annotate passages that you want to discuss in class. Explain why and how it should be discussed in class.

Rhetorical Annotations

Rhetorical annotations examine the structural elements of a scientific text. Identifying these and evaluating these elements are an essential part of critically reading a scientific research article.

  • Motive: The motive provides describes the reasons why the additional research is needed. In a research article, the motive justifies the research. Motives accomplish this goal by establishing the importance of the topic, or by identifying a problem, controversy, or knowledge gap. Annotations about the motive should evaluate if the rationale for the work is clearly developed in the form of a logical argument.
  • Objective: Statement about what the authors wanted to know. The objective may be formulated as a research question, research aim, or hypothesis. Annotations should evaluate if the objectives are reasonable, specific, testable, novel, and parsimonious.
  • General Approach: Statement describing the strategy and general methods. The general approach should be limited to one sentence. Avoid experimental details: Commentary about the validity of the approach could be included in the annotation.

  • Results: The results section of a paper is the heart of a research article. Annotations within the result section should focus on the inscriptions. These extra-textual elements present the evidence in the form of figures, tables, or equations. As science progresses, the conclusions and background information needed to read and understand a research article will change. Therefore, it is important to understand the conclusions only in the context of the supporting evidence. We may discover additional information that changes the conclusions of an article, but the results are permanent and re-interpetable in light of new information. Annotating the results section is a repetitive process of annotating each inscription. Each annotation is directly anchored to an inscription and contains the elements described below.

    • Statement of Finding: A one sentence summary of the important conclusion of the inscription.
    • Specific Question: What was the specific gap in knowledge?
    • Specific Approach: Describe the logic of the study design. Evaluate how well this approach answers the specific question.
    • Organization: Identify what is being measured and the controls. Describe the notations and layout of the inscription.
    • Literal Intepretation: What does the evidence unambiguously mean. Annotations about the literal interpretation will only be about the research outcomes and often involves making comparisons between experimental and control groups.
    • Authors’ Interpretation: What did the authors’ say about what the evidence demonstrates. Often times, authors make claims that the evidence supports or refutes a hypothesis or model.
    • Evaluation: Evaluate how well the results answer the specific question. Compare the literal interpretation with the authors’ interpretation. Gauge the validity and reliability of the data.
  • Main Conclusions: What are the main outcomes of the research. Evaluate how well the outcomes are related to the objective presented in the Introduction section. Do the main conclusions answer the research questions? Are the hypotheses supported by the evidence?
  • Support: The main conclusions are supported by evidence and logic. Support for the main conclusions comes from the result section and prior literature.
  • Implications: Why are the results important or interesting? Annotations about implications can focus on the importance of the research, follow-up questions, consequences, or recommendations.
  • Counterargument: Statements that weaken or discredit the main conclusion. For example, possible methodological flaws, anomalous data, results that contradict previous studies, or alternative explanations. Counterarguments are sometimes presented as limitations. Annotations should focus on the strength of the counterargument

  • Refutation: Statements that weaken or refute a counterargument. Annotations should focus on the validity of the refutation.

Grading Rubric

This grading rubric is designed so that students can, to some degree, choose their route to success depending on their strengths, interests, and inclinations. A total of 16 points is possible, but earning 12 points results in a perfect score. I don’t want annotations to be viewed as busywork. Instead, I use this exercise to help students get prepared for in-class discussions.

Interactions with the Text

  1. No or sparse evidence of reading the text.
  2. Little evidence of reading the text. Token annotations are randomly distributed.
  3. Some evidence of reading the text. Annotations are unevenly distributed.
  4. Strong evidence of reading the text. Annotations are evenly distributed and extensive
  5. Strong evidence of rereading the text. Annotations are evenly distributed, extensive, and connected to earlier or later passages of the text.

Style

  1. Annotations are absent, incomprehensible, or objectionable.
  2. Annotations have no coherence; and major grammatical, formatting, or factual problems. Tagging is absent.
  3. Annotations have poor coherence; minor grammatical, formatting, or factual problems that do not interfere with understanding. Tagging is present but inconsistent or incorrect.
  4. Annotations have good coherence; and are free of grammatical, formatting, or factual errors. Tagging is consistent and correct.
  5. Annotations have strong coherence, and are free of grammatical, formatting, or factual errors. Tagging is consistent and correct.

Didactic and Dialetic Annotations

  1. No or sparse evidence of comprehension. Annotations are merely copied and pasted from the text or another source.
  2. Little evidence of comprehension. Annotations are original but are not additive.
  3. Some evidence of basic comprehension but understanding is incomplete. Predictions, questions, and answers are obvious. Annotations focus on less important aspects of the text, but they are additive.
  4. Evidence of with nearly complete comprehension. Annotations demonstrate critical thinking and focus on important or difficult aspects of the text, but lacks clarity.
  5. Evidence of complete comprehension. Annotations demonstrate critical thinking and focus on important or difficult aspects of the text.

Rhetorical Annotations

  1. No or sparse evidence of engagement with the persuasive elements of scientific discourse. Annotations about the evidence do not refer to the figures.
  2. Little or incorrect evidence of engaging persuasive elements of scientific discourse. The student attempts to annotate figures, but the annotation has limited coherence and only points to evidence (“see figure”).
  3. Some evidence of engaging the persuasive elements of scientific discourse. Identifies simple rhetorical moves with limited commentary. Figures are summarized but not evaluated in terms of the claims made.
  4. Evidence of being able to correctly identify and evaluate some persuasive elements of scientific discourse. Figures are asserted to support a claim but without an explanation of how the evidence supports the claim.
  5. Evidence of being able to critically evaluate scientific claims based on the evidence presented in the results section. The organization of a figure is discussed and the relationship between claims and the evidence are discussed.

Grading Scheme

Points are from the previous four areas are added up and translated to a numerical grade as described in the table below.

Total Points Numerical Grade Letter Grade
> 12 4.0 A
11 3.7 A
10 3.3 A
9 3.0 B
8 2.7 B
7 2.3 C
6 2.0 C
5 1.7 D
4 1.3 D
3 1.0 F
2 0.7 F
1 0.3 F
0 0 F

References

Campbell, A. Malcolm. 2004. “Open Access: A PLoS for Education.” PLOS Biology 2 (5): e145. doi:10.1371/journal.pbio.0020145.

Gillen, Christopher M. 2006. “Criticism and Interpretation: Teaching the Persuasive Aspects of Research Articles.” CBE Life Sciences Education 5 (1): 34–38. doi:10.1187/cbe.05-08-0101.

Gottesman, Alan J., and Sally G. Hoskins. 2013. “CREATE Cornerstone: Introduction to Scientific Thinking, a New Course for STEM-Interested Freshmen, Demystifies Scientific Thinking Through Analysis of Scientific Literature.” CBE Life Sciences Education 12 (1): 59–72. doi:10.1187/cbe.12-11-0201.

Hoskins, Sally G., Leslie M. Stevens, and Ross H. Nehm. 2007. “Selective Use of the Primary Literature Transforms the Classroom into a Virtual Laboratory.” Genetics 176 (3): 1381–9. doi:10.1534/genetics.107.071183.

Lacum, Edwin B. Van, Miriam A. Ossevoort, and Martin J. Goedhart. 2014. “A Teaching Strategy with a Focus on Argumentation to Improve Undergraduate Students’ Ability to Read Research Articles.” CBE-Life Sciences Education 13 (2): 253–64. doi:10.1187/cbe.13-06-0110.

Round, Jennifer E., A. Malcolm Campbell, and Vivian Siegel. 2013. “Figure Facts: Encouraging Undergraduates to Take a Data-Centered Approach to Reading Primary Literature.” CBE—Life Sciences Education 12 (1): 39–46. doi:10.1187/cbe.11-07-0057.