When it Comes to Measuring Historical Thinking, the “Nation’s Report Card” is “Fool’s Gold”


First, here is some background on the National Assessment of Educational Progress report.  It is often described as “the nation’s report card.”

And here is a taste of critique of the NAEP by Sam Wineburg, Mark Smith, and Joel Breakstone:

Students have never fared well on NAEP’s tests in these subjects. The first history test in 1987 found that half of the students couldn’t place the Civil War in the right half-century. Some 15 years later, following a decade of new standards, The Washington Post wrote that students on the 2001 exam “lack even a basic knowledge of American history.” In 2014, the last time history was tested, the New York Times fished into the recycling bin for this headline: “Most Eighth-Graders Score Low on History, Civics.”

But what would happen if instead of grading the kids, we graded the test makers? How? By evaluating the claims they make about what their tests actually measure.

For example, in history, NAEP claims to test not only names and dates, but critical thinking — what it calls “Historical Analysis and Interpretation.” Such questions require students to “explain points of view,” “weigh and judge different views of the past,” and “develop sound generalizations and defend these generalizations with persuasive arguments.” In college, students demonstrate these skills by writing analytical essays in which they have to put facts into context. NAEP, however, claims it can measure such skills using traditional multiple-choice questions.

We wanted to test this claim. We administered a set of Historical Analysis and Interpretation questions from NAEP’s 2010 12th-grade exam to high school students who had passed the Advanced Placement (AP) exam in U.S. History (with a score of 3 or above). We tracked students’ thinking by having them verbalize their thoughts as they solved the questions.

What we learned shocked us.

In a study that appears in the forthcoming American Educational Research Journal, we show that in 108 cases (27 students answering four different items), there was not a single instance in which students’ thinking resembled anything close to “Historical Analysis and Interpretation.” Instead, drawing on canny test-taking strategies, students typically did an end run around historical content to arrive at their answers.

Read the entire piece here.  I need to share this piece with my “Teaching History” class. We are in the midst of reading Wineburg’s Historical Thinking and Other Unnatural Acts.

2 thoughts on “When it Comes to Measuring Historical Thinking, the “Nation’s Report Card” is “Fool’s Gold”

  1. You cannot measure student learning by multiple choice questions. You cannot teach students history by lecturing to them endlessly. The most effective form of learning today is found in interactive learning environments where students encounter history through multiple types of examples. Requiring them to read more than the textbook is utterly necessary. So is putting in place means that evaluate their ability to synthesis the information from multiple sources of data into a written form.

    Paulo Freire pointed out the flaws of what he called the “Banking” system of education in which students are fed only what the educator and others wanted them to learn. In his case he was referring to the military controlled government of Brazil who were literally force feeding the people only what they wanted the people to know. There is no real learning in that system, only the memorization of whatever information they are given. There is no means of developing any critical thinking skills in that system at all.

    Instead, Paulo proposed that educators employ a problem solving system in which students are given problems to solve using the information made available to them. They were to ask questions about that information, which of course would expose false information as such. Obviously this system was not desired by the Brazilian government and they exiled Paulo. Fortunately for us, he took the time to write some books about this, the most famous being “Pedagogy of the Oppressed.”

    This is the basis of an interactive learning environment in which students are given multiple types of sources for information. I prefer that they work in groups as I believe the synergy generated in groups enhances their inquisitive natures. Using a set of questions derived from Bloom’s Taxonomy, educators prompt students to develop interpretations. The questions have varying degrees of difficulty and the ultimate goal of the lesson is to get students to read the information, question it, articulate about it both verbally and in written form, and to develop a group interpretation which they then share with the rest of the class in conjunction with the instructor. During the formative phase of questioning, the instructor works with the groups and helps to keep them on track as well as correcting mistakes and even posing additional questions. Note that the majority of the questions in this lesson are WHY questions.

    I’ve been doing this for several semesters now and it is very effective. The students embrace the system and respond to it enthusiastically. There are however, some that dislike it intensely. My experience with those students show that they prefer the lecture based model where they don’t have to do as much work. None of those students indicated any possibility of striving for a degree in history. The good news is that with this system, I have had several students inform me that they changed their major to history or are now working on a minor in history along with their major in education. Even those that are not majoring or minoring in history comment on how they see history in a different light and can use the skills they are taught in this course in their other courses and daily lives.

    These students generally want to be challenged and to learn. It’s time to stop teaching and start guiding students. Instructors who use the banking model are doing nothing to help their students learn history. They are only helping students memorize facts. That is why the history scores are so abysmal in this nation. Only by changing the way we educators approach our craft will we see positive change in student learning, not only in our field of history, but throughout all of education.


Comments are closed.