There’s zero value in invalid measurements

Anyone who has a child on the autism spectrum, particularly on the lower end formerly known as Asperger’s Syndrome, understands what I mean when I say these children have quirks and obsessions. My son wasn’t diagnosed with Asperger’s Syndrome until mid-way into the fifth grade, but in retrospect, he navigated through a number of obsessions, beginning at a very young age. One of his earliest obsessions was elephants. By age three, he could draw hundreds of variations of elephants. He could throw his arm up in front of his face, like a trunk, and let out the most realistic elephant sound you’ve ever heard. The image shown above was created by my son, at the age of three, on a Windows computer using the Paint program and a trackball. I keep it on my phone as reminder of his potential.

I have mentioned many times that by the time he entered Pre-K, at the age of four, he was a capable reader. By then, his elephant obsession had expanded to wild animals that lived in all of the far corners of the world. On trips to Book-A-Million, he would ask us to buy books on this topic. He would go on virtual visits of zoos all over the world via the internet. He could tell you the habitat and diet of any mammal you could possibly name and tell you which continent they lived on. He obviously had a very high level of reading comprehension for his age.

When my son reached 3rd grade, he was introduced to the concept of preparing all year for a state assessment. At the end of the year, he scored “Mastery” in three of the four content areas. This was no surprise to me. He was a very good reader.

The following year, on the 4th grade test, he only scored “Mastery” in Science. His ELA and Social Studies scores dropped to “Basic.” As a concerned parent, I chose to exclude my son from the state assessment in both 5th and 6th grade. In the 7th grade, the new hybrid LEAP 2025 (he refers to it as “Faux LEAP) was implemented. We discussed the assessment with him and gave him the choice to take the test, or not. He chose to take the test. He scored “Approaching Basic” in ELA. Nothing has changed. His reading skills and comprehension are years above his age. His vocabulary rivals mine. How could this be?

On Monday, April 2, the Louisiana Department of Education (LDOE) released a statement saying that Louisiana had submitted an application to the U.S. Department of Education for an “Innovative Assessment” grant. The press release goes on to explain how LDOE is going to develop an assessment that combines the ELA assessment with the Social Studies assessment.

The very next day, an OpEd written by John White was published by The Hill. In his piece, White further explains that for years teachers have complained that they can’t effectively prepare for reading comprehension tests because the content of the test is unknown. He talks about how the present testing practices don’t effectively measure a student’s knowledge and comprehension. The conceptual assessment will combine the two subjects. Teachers in both subjects will utilize texts related to Social Studies, interim assessments will be given throughout the year, and at the end of the year, students will be prepared because they know the content of the test.

White’s presentation makes total sense. I was intrigued by this concept, and wanted to know what others thought about it. I posted the information on FaceBook and Twitter, asking for feedback. I received many responses. They ranged from excitement from teachers who thought that a path to preparation had been delivered on a silver platter to suspicion that anything conceived by White is evil to hostile declarations that this idea cannot work.

On April 4th, the Fordham Institute’s Robert Pondiscio published his article titled Louisiana’s remarkable reading test. In his opening remarks, he invokes decades of reading and literacy research by acknowledging that Ed Hirsch Jr. (creator of Core Knowledge) had just celebrated his 90th birthday and says that LDOE has given Hirsch the best gift EVER, in its creation of this new innovate assessment. The problem with this is that the assessment described by White is 100% unequivocally in direct conflict with Hirsch’s research.

For decades, Hirsch has not only researched, and published findings, regarding reading, reading comprehension, and the learning process; but also, how to effectively measure progress with valid assessments.  In March 2009, the New York Times published an Op-Ed by Hirsch titled Reading Test Dummies. In his piece, Hirsch provides the exact same complaints given by teachers about reading comprehension tests that White relies on, in his OpEd published by The Hill. Hirsch even goes so far as to imply that focusing test content on what students are actually learning in the classroom is important to measuring comprehension. White mirrors that sentiment. That is the extent of White’s assertion. Hirsch goes even further and insists that the text utilized in a reading comprehension assessment must contain a broad range of subject matter to be valid.

Going back even farther to Spring 2003, Hirsch published a paper for the American Federation of Teachers (AFT) titled, Reading Comprehension Required Knowledge-of Words and the World which explains, in detail, the process of developing reading skills and literacy, and how to measure that. In Spring 2009, Hirsch wrote a brief summary of this process and how it applied to the states that were implementing new state assessments. AFT published this piece, What Do Reading Comprehension Tests Measure?, on its website.

The handful of writings by Hirsch, that I have linked to thus far, are sufficient evidence of Hirsch’s theory on literacy and comprehension. If you would like more evidence, here is a link to a Google search for “Hirsch Measuring Reading Comprehension.” You’ll be overwhelmed. For that reason, I will summarize.

Preparation for achieving, and excelling, in reading involves and requires mastery of the following literacy foundational skills:

• phonetic awareness
• word decoding
• reading fluency
• meaning and usage of age appropriate vocabulary

These are skills that are developed through silent sustained reading and reading aloud, as well as, instruction in spelling and vocabulary. It is worth noting that under the current Louisiana Student Standards (aka Common Core) the very standards that contribute to the development of these skills were deleted from the K-2 standards in the move from Grade Level Expectations (GLEs) to Common Core. In fact, with little emphasis on “letter identification and sounds,” students in K-2 are expected to learn words by sight with zero understanding of phonemes, no phonemic awareness and no life experience in which to draw context.

Hirsch further explains the intent of reading comprehension assessments (the perception vs the reality) and what makes an assessment valid. Again, I’ll summarize Hirsch’s sentiment regarding measuring reading comprehension.

• The first, and most common, mistake teachers make is that reading comprehension is a skill that can be taught.
• The second mistake is the belief that a student can be prepared for a reading comprehension assessment.
• Interest level can drastically skew reading comprehension assessment results. (Think back to my opening story about my son.)
• In order to assure validity, a reading comprehension assessment must include short excerpts of unfamiliar text, across multiple genres and subject matter, that are expected to be well within the life experiences of the child being tested. It is necessary for the texts to include a range of interest level from low to high. A statistical method is then used to determine a level of comprehension.

In the summary posted on the AFT website, Hirsch asserts that properly designed reading comprehension assessments measure what a child knows compared to age appropriate life experiences. He then recognizes the Iowa Test of Basic Skills (ITBS) as the most valid instrument for measuring reading comprehension because it uses nine excerpts across fiction, non-fiction, Science and Social Studies. In his 2003 detailed explanation, Hirsch asserts that the first assessments that students encounter (3rd grade) is a measurement of language usage, mechanics, the skill of reading, and discerning concrete correct answers from the given text. In the fourth grade, students are exposed to the first true baseline measurement of reading comprehension which requires a student to draw from their life experiences and knowledge, and ability to apply reasoning skills to discern answers. He notes that the ITBS assessment is so accurate that its 4th grade assessment can predict success, not only in high school comprehension assessments, but also earnings potential as an adult.

Now, I know that my criticism of White’s plan to introduce an “innovative assessment” will be dismissed by the “powers that be” as an “opinion” and a “personal agenda,” but I believe that I’ve provided enough evidence to support that White’s proposal will not be a valid measurement of reading comprehension. It isn’t my opinion. It is a conclusion drawn from decades of research. I believe that this assessment WILL NOT be in the best interest of students. It will accomplish two things. 1.) It will give the appearance that White is being responsive to complaints about excessive testing by combining tests; therefore, reducing the number of tests. 2.) It will provide uninformed teachers with a method of preparing for a reading comprehension assessment (an invalid one), and ultimately, result in higher scores and the declaration that the innovative assessment has worked.

This assessment will benefit no one but the creators of the assessment. This is nothing more than another step in the education reform movement, and THAT is an adult agenda.