PDK International | Phi Delta Kappan: Bracey V88 N7 Mar 2007

Find more Kappan articles in the
searchable PDK Publications Archive

Research: Test Scores and Economic Growth

By Gerald W. Bracey

SOMEWHERE around 2000, I observed that, while people argued vehemently that good schools led to improved national economies, there wasn't much in the way of evidence for that contention. And there was certainly one glaring counterexample: Japan. Kids in Japan continued to ace tests, just as they had during Japan's boom years in the 1980s, but the country had been mired in recession or stagnation for a decade. (People now think that 2006 may have been the year that Japan finally started an expansion that will last for more than a quarter or two.)

Out of curiosity, I correlated the ranking of countries on the Third International Mathematics and Science Study (TIMSS) with the rankings for global competitiveness issued by the World Economic Forum. The resulting number for about 35 countries was quite small and became negative if I removed the bottom five nations, which were quite low on both rankings.

Richard Rothstein and Rebecca Jacobsen's article in the December Kappan observed that the goals for American education have seldom been cast in economic terms. Instead, right from the founding of the nation, they have more often been stated in terms of moral and civic outcomes. The dominance of an economic reference point is new. An aberration, Rothstein and Jacobsen called it. Moreover, the focus on the economy is new elsewhere as well. A Belgian critique of the Programme for International Student Assessment (for details on this critique, see the June 2005 Research column) implied that ranking education systems according to test scores was, well, dumb:

Hit parades have been flourishing here for some years: the best schools, the world's best universities, the top-performing research centers, etc. Some 30 years ago this sort of ranking would have produced a smile as we were of the view that the broad and long-term effects of education cannot be reduced to a few trivial indicators and that every education system could be validly understood only by taking account of its history, its aims, and the complexity of its structures.

Now, in too many places, people seem to think that a good education means only more math and science so students can compete for good jobs in the global economy. Yet there remains very little evidence of the impact of math and science achievement on the economic growth of nations. This makes the powerful rhetoric calling for ever more math and science somewhat hard to understand.

On the day that I began to write this column, for instance, a program called "Preparing U.S. Students for the Global Economy" took place in Washington, D.C. It featured Sen. Christopher Dodd (D-Conn.), Rep. Vernon Ehlers (R-Mich.), and various pundits from think tanks. A Dodd-Ehlers bill calling for the National Assessment Governing Board to establish national standards in mathematics and science has been introduced in Congress.

From a different perspective, in the November 2006 issue of the American Journal of Education, Francisco Ramirez and John Meyer of Stanford University, Xiaowei Luo of the University of Illinois, and Evan Schofer of the University of Minnesota observe the disjuncture between strong rhetoric and weak impact:

The dramatic portrait of achievement and development presupposes a world of enormous variation with respect to curriculum, teaching, and achievement. But cross-national investigations actually show considerably less variation with respect to curriculum, teaching, and achievement than expected in policy discourse. . . . So, from a research perspective, one should not expect robust achievement effects on economic growth.

Or, as I have more crassly put it on occasion, education is critical, but among the developed nations differences in test scores are trivial.

Ramirez and his colleagues go on to look for data that bear on the relationship between achievement and development. Their principal variables were changes in Gross Domestic Product (GDP) in 38 countries over two 20-year periods, 1970-90 and 1980-2000, and various international test comparisons up to the initial TIMSS in 1995.

Because a statistic like GDP is difficult to measure accurately and comparably across many nations, they performed their analyses with a number of alternative indicators as well. They report that their results were consistent across indicators.

For the 1970-90 period, achievement had a positive effect on development. When the researchers dropped four high-scoring nations with much development -- the four "Asian Tiger" nations of South Korea, Hong Kong, Taiwan, and Singapore -- the effect was still positive but much smaller. The data also suggested that

much of the achievement effect can be attributed to the worst performers on international tests. Moving from the "middle of the pack" to the top provides less of an economic boost. This is a striking finding that calls into question the disproportionate attention (and envy) focused on those few countries with the very highest achievement scores.

Such countries "do not experience substantially greater economic growth than countries that are merely average in terms of achievement."

The researchers then lay out an interpretation of their findings that differs from the causal interpretation one usually hears:

We venture, here, the interpretation that much of the achievement "effect" is not really causal in character. It may be, rather, that nation-states with strong prodevelopment policies, and with regimes powerful enough to enforce these, produce both more economic growth and more disciplined student-achievement levels in fields (e.g., science and mathematics) perceived to be especially development related. This idea would explain the status of the Asian Tigers whose regimes have been much focused on producing both economic growth and achievement-oriented students in math and science.

Thus, in the 1970s, when Lee Kwan Yew, an authoritarian ruler of a nominal republic, told his small nation of less than four million Singaporeans (current population 4.5 million) that the country would concentrate on development and emphasize math and science in schools, the country did just that. Presidents in, say, Italy and France would be less successful. (Yew, by the way, also decided early on that English would become the lingua franca of the modern world and that Singapore students would learn English. Singapore students, a majority of whom are Chinese, with a large portion of Malays, take reading tests in English and generally finish a little above average, indicating the power of a strong regime to bring about an outcome in a small country.)

When the authors performed their analysis on the period from 1980 to 2000, they found no achievement effect. The coefficient was positive, but not statistically significant. Again, when the Asian Tiger nations were removed from the analysis, the coefficient was further diminished.

Ramirez and his colleagues then conducted some exploratory analyses of the period 1990-2000, a period in which Japan's bubble burst and the Japanese learned that the emperor's palace and surrounding grounds were not more valuable than the entire state of California. The Asian Tiger nations went into economic free fall. "In these exploratory analyses of economic growth during a period that has been something of a disaster for a number of Asian countries and others, the coefficient associated with academic achievement in science and mathematics entirely disappears."

Those who have argued for a link between achievement and development have used, at least implicitly, a causal chain that looks like this:

National Curricular
and Pedagogical
Modernization

leads to

Math and Science
Interest and
Achievement

leads to

Scientists and
Engineers in Higher
Education

leads to

Scientists and
Engineers in the Labor Force

leads to

National Economic Development

The researchers examine this causal chain by looking at the effect of achievement in math and science on the number of scientists and engineers enrolled in higher education, on the number in the labor force, and on the number of scientific articles published and patents granted. For neither period of time did achievement have an impact on how many people signed up for science and engineering in college. To describe the impact on the other variables, the authors use the technical term "a mixed bag."

Wrapping up their analyses and interpretations, Ramirez and his colleagues write:

From our study, the main conclusion is that the relationship between achievement in science and mathematics in schoolchildren and national economic growth is both time and case sensitive. Moreover, the relationship largely reflects the gap between the bottom third of the nations and the rest; the middle of the pack does not much differ from the rest. . . . Much of the obsession with the achievement "horse race" proceeds as if beating the Asian Tigers in mathematics and science education is necessary for the economic well-being of other developed countries. Our analysis offers little support for this obsession. . . .

Achievement indicators do not capture the extent to which schooling promotes initiative, creativity, entrepreneurship, and other strengths not sufficiently curricularized to warrant cross-national data collection and analysis. Unfortunately, the policy discourse that often follows from international achievement races involves exaggerated causal claims frequently stressing educational "silver bullets" for economic woes. Our analyses do not offer definitive answers, but they raise important questions about the validity of these claims. In an era that celebrates evidence-based policy formation, it behooves us to carefully weigh the evidence, rather than use it simply as a rhetorical weapon.

This might be a good time to remember the words of education historian Lawrence Cremin in his 1990 book, Popular Education and Its Discontents:

American economic competitiveness with Japan and other nations is to a considerable degree a function of decisions made by the President and Congress, the Federal Reserve Board, and the Federal Departments of the Treasury, Commerce and Labor. Therefore, to conclude that problems of international competitiveness can be solved by educational reform, especially educational reform defined solely as school reform, is not merely utopian and millennialist, it is at best a foolish and at worst a crass effort to direct attention away from those truly responsible for doing something about competitiveness and to lay the burden instead on the schools. It is a device that has been used repeatedly in the history of American education. (pp. 102-3)

So, to the Business Roundtable, the National Center on Education and the Economy, the National Association of Manufacturers, and others: knock it off.

* * *

Publication note: The Rotten Apples in Education Awards that used to be a part of the Bracey Reports can be found at
www.america-tomorrow.com/bracey/EDDRA. Other uninhibited expressions of most anything can be seen in the blogs I write for the Huffington Post, www.huffingtonpost.com/gerald-bracey.

GERALD W. BRACEY is an associate for the High/Scope Foundation, Ypsilanti, Mich. His most recent book is Reading Educational Research: How to Avoid Getting Statistically Snookered (Heinemann, 2006). He can be reached at gbracey1@verizon.net.