TIMSS: What Lessons to Learn?

Gordon J. Aubrecht, II

Department of Physics

Ohio State University

Marion, OH 43302-5695 and Columbus, OH 43210-1106

{Note: Because of space limitations, the Invited Paper in the January, 1999 Science Teacher magazine contains a severely edited version of this position paper. Here is the "unexpurgated" story.}

Performance of American students on the Third International Science Study (TIMSS) was above the world average at the fourth grade level, below average at the middle school level, and significantly below average at the high school level. The results of TIMSS reports should have been a wake-up call. Instead, we have basically ignored the very important question: Why does the performance of U.S. students worsen as they grow older? Because these students may eventually attend my college classes, I am vitally interested in the answer.

The TIMSS report on high school seniors’ performance in physics states, "[The] average performance in Norway was comparable to or even exceeded performance at the 75th percentile in ... countries such as ... the United States." (Mullins et al., 1998) This is a gentle way to characterize it; the accompanying table shows that the U.S. students’ 95th percentile lies at about the Norwegian students’ 25th percentile! What are we to make of these differences? American and Norwegian students cannot be so different, but their performance was so different. To me, the chief culprits behind the poor showing are fractured curricular control, lack of time for teachers (and students) to learn from peers, and our overall superficial approach to science in the schools. At great length, Bowen (Bowen, 1998) has assessed the results from the TIMSS. He reaches similar conclusions.

In a recent talk, Harvard educator Phil Sadler (Sadler, 1998,) spoke about TIMSS results and posited that the major reason for the poor US showing was that it, compared to many of the other countries, has a shallow and broad curriculum. I have some personal knowledge of the German curriculum, and agree that other countries have more focused curricula. In Germany among other countries, science classes are taken for years (at perhaps a small number of hours per week in each discipline). This lets scientific ideas have time to ripen and mature. Also, the German curriculum is national, not local.

State control of the curriculum (as practiced in the U.S.) is not very specific, so that different teachers can do different things, which plays to their strengths. However, it means that there is not an agreed-on set of things to know. In Germany, state control is quite specific, dictating many details. Everyone agrees on what is needed to be learned. The closest thing to a U.S. national curriculum is the College Board test (including the AP tests). Both controls distort the system, but the advantage of the German model is the ability to require continual reinforcement of ideas over a long period of time. We know such repetition works.

Teachers in other countries that generally do well on TIMSS have time to learn from one another. (Stevenson et al., 1986) These teachers have students in class half a day, and so have workday time for grading, class preparation, and discussions about teaching with fellow teachers. U.S. teachers are in charge of students all day. Peer reflection and cooperative approaches to teaching would pay dividends, but would free children to spend the other half day without supervision or would result in the doubling of the number of teachers. Americans seem to be too penny-pinching about education to allow this to happen

U.S. students’ TIMSS performance was good on "recall" questions (Aubrecht and Aubrecht, 1983), but poor on questions that demand self-organization. Answers to recall questions, such as one explaining that steam has a greater volume than water because the molecules are farther apart, U.S. results nearly reached the international mean (60% vs. 64%). However, on "interpret" questions, such as the identification of the trajectory of an electron in a static electric field, U.S. students who had taken physics got 12%, while the international mean was 32% (itself shockingly low). Bowen (Bowen, 1998) sees the same effect. He writes

"Twenty-four of the 33 lowest scoring questions were open-ended, graphical or essay questions. Only 9 of these questions were multiple choice. When I further examined those multiple choice questions (which have been released), it was clear that almost all 9 required a number of steps and relationships, and often the use of a symbolic representation (equations), to generate a correct answer. By contrast the questions which were answered by the largest fractions of U.S. students were predominantly multiple choice and single step or single concept questions."

Perhaps US students are too rule-bound and don’t know how to act when there is no set rule (this transition occurs after fourth grade). We know that the reason isn't excess TV watching by U.S. students or students working, as those occurred at about the same level in all the TIMSS countries.

My daughter, Katarina, now in ninth grade, did well on her fourth-grade math achievement tests (above the 90th percentile in most areas) but emerged from fifth grade feeling terrible about her math skills, a feeling supported by her grades. In speaking to her teachers, I found that they had to follow a set curriculum that included very few assigned problems. We enrolled Katarina in the Kumon math program, which has students do many problems every day at each sequential level of mathematical operation. The ideas are gradually added and the level of difficulty advanced–gradually, almost imperceptibly. After two years of Kumon practice, Katarina was able to take eighth grade algebra with the advanced class. The practice had allowed her to gain confidence in her abilities.

Perhaps the shallow depth to science study results from the distortions inherent in ubiquitous standardized testing. To prevent parent outcry, teachers (and local curriculum writers) have scheduled courses as a brief survey of practically everything they could throw into the book bag. This seems to me to be the opposite of what would be most helpful to students. Katarina’s experience sheds light on what is wrong–on the lack of depth of knowledge transferred. In surveying everything, there is little practice or time to let ideas sink in, little but the most superficial knowledge gets transmitted. Students think science is a collection of unrelated facts, but science is a process. It is more about questions and how to approach them than the fascinating answers that result.

I would prefer that students who come to me had depth in their courses. Let them study the behavior of light the entire year and truly learn how images are formed. Let them really understand kinematics in one dimension, qualitatively as well as quantatively. Or let them learn how to light lamps and how to build a toaster from scratch. If they already have the idea of how science works, by sifting the good models out from the chaff of hypotheses, they are more ready to incorporate the huge number of concepts we throw at them in college. Whatever is studied, students should study in depth enough to feel how the scientific process has worked in it and feel confidence that science is relevant.

References

G. J. Aubrecht and J. D. Aubrecht, "Constructing Objective Tests," Am. J. Phys. 51, 613 (l983). We proposed calling items recall (memorize), interpret (take a novel situation and determine what the result would be), and apply (take a novel situation and determine numerical answers). The latter two categories involve more higher-order thinking, and success comes only when understanding has occurred.

S. P. Bowen, "TIMSS–An Analysis of the International High School Physics Test," in the newsletter of the American Physical Society Forum on Education, Summer 1998.

I. V. S. Mullins et al., Mathematics and Science Achievement in the Final Year of Secondary School (TIMSS International Study Center; Boston, 1998).

Philip Sadler, "Misconceptions in Physics," talk to the Ohio Section of the American Physical Society, Muncie, IN, 3 May 1998.

H. W. Stevenson, C. Chen, and S.-Y. Lee, "Mathematics Achievement of Chinese, Japanese, and American Children," Science 231, 693 (1986). See also ibid., 259, 53 (1993).