|
||
The Eighth Bracey Report on the Condition of Public Education | ||
By Gerald W. Bracey This year Mr. Bracey counters some goofy ideas -- among them, the notion that schools control our economic destiny and the notion that the amount of money spent on schools doesn't matter. Illustration © 1998 by Jem Sullivan |
||
BECAUSE THE reporters who normally would have covered the Jonesboro killings were otherwise occupied, Washington Post science writer John Schwartz got the assignment. At a subsequent forum at Arkansas State University, he found that many locals viewed the media as having contributed to the nightmare, being something like "flies on an open wound." He was not surprised. He found that "journalists looking for quick answers out of Jonesboro seemed to have brought them along in their luggage. On a nightly deadline, chaos was molded into compelling story lines. Disaster in the Heartland, logos read. Kids killing kids. Guns in the South. Tragedy was repackaged as entertainment, and authority became a substitute for insight."
On television "self-proclaimed experts blamed everything from TV shows such as 'South Park' to the lack of prayer in schools to ready access to guns to . . . some combination of guns and violent Southern culture." Schwartz dubbed such stories "pat journalism."1
From a scientific standpoint, Schwartz said, the explanations he heard failed the most basic test of epidemiological explanation: your theory must include the afflicted and exclude the well. As an example, Schwartz told a tale from an 1854 cholera epidemic in London. During that crisis, a certain doctor, John Snow, showed that the sick people lived near a certain public water pump. People who lived farther away were not afflicted. Snow concluded that the water was contaminated. He had the pump handle removed, and the disease receded.
Schwartz couldn't find the pump handle in the explanations of Jonesboro. Most kids in the South don't kill other kids, and one of the two arrested in Jonesboro had spent his early years in rural Minnesota. Schwartz found Jonesboro to be the kind of place where he would want to raise his own kids. Even when the reporters wrote insightful stories, he recounted, they were stymied by editors back at the office who wanted the stories to emphasize "young white crackers shooting each other." (An e-mail I received from Abigail Thernstrom, co-author of America in Black and White, argued that our unthinking reporters and editors are themselves evidence that our schools have failed.)
If you were to take Schwartz' tale and replace "Jonesboro" with "TIMSS Final Year data," you would find the media coverage equally predigested. Not one reporter who filed a news story on the Third International Mathematics and Science Study sought any insights into what the data really mean. New York Times education writer Ethan Bronner followed his straight news story2 with a "thought piece," but, rather than questioning the credibility of the TIMSS data (and I presume that Kappan readers know by now that they are not credible), Bronner's second piece focused on why the results weren't all that important because of other, positive outcomes (creativity, problem solving, and so on) that America's education system produces.3
The Washington Post played it straight all the way.4 Education Week was no better. Although I had given Debra Viadero much cause to doubt the TIMSS data, no reservations clouded her story.5 U.S. News & World Report didn't even bother to carry the data as news but told the tale through an abysmal op-ed essay by John Leo.6 The Wall Street Journal unleashed right-wing pit bull and former assistant secretary of education Chester Finn, Jr., who rabidly foamed that the TIMSS 12th-grade results indicted public education as "an ossified government bureaucracy incapable of reforming itself."7
Aside from my editorial in USA Today,8 the only essay declaring that there might be something wrong with the data came not from a journalist but from David Friedman, a fellow at MIT's Japan Project.9 However, he cannot be considered a wholly independent source since one statistic he cited is one that only I have produced. By merging math results from the National Assessment of Educational Progress (NAEP) with data from the Second International Assessment of Educational Progress (1992), I showed that the top third of American schools scored as high in math as the top two nations, Taiwan and Korea. Iris Rotberg came along as fast as lead times at Science allow, demolishing the "impervious" data in the 15 May 1998 issue.10 David Berliner was silenced by the demands of the time-consuming job of being dean at Arizona State University.
In the Fourth Bracey Report, I recounted research showing that 90% of what journalists write is what people in positions of authority tell them to write. This certainly happened with the TIMSS report, and the media were encouraged in their thoughtlessness by the U.S. Department of Education and the TIMSS staff.
It was not until almost two months later that William Schmidt, TIMSS research director, offered what could have been a reasonable introduction to the TIMSS data. But by then it was too late. At a panel at the annual convention of the National Council of Teachers of Mathematics (NCTM) in Washington, D.C., and later at a Brookings Institution seminar (neutrally labeled "Heads in the Sand" by seminar organizer Diane Ravitch), Schmidt proposed that TIMSS had compared not students but systems. Viewed this way, the fact that Norway and Sweden were well ahead of everyone else in physics would become merely an anticipated outcome, not a "devastating" result (a word Schmidt had used earlier). The Scandinavian students performed as they did partly because they were older, but mostly because they had studied physics for three years.
If this fact had been made known during the initial press conference convened by the U.S. Department of Education (ED), then maybe no one would have thought that these students were in any way comparable to American students who had taken only one year of physics. The question might then have become, Should American students have an opportunity in high school to study physics for three years? Or in response to other differences between systems: Should American students wait until age 7 to start school? Should we have a "grade 13" in our system as the Swedish and Norwegian systems do?
If the U.S. Department of Education had presented the results as a comparison of systems, we might have asked whether we should have more focused curricula, as they do in Cyprus. Cyprus came up a lot in media reports because, it seemed, it was especially embarrassing for us to be behind a nation whose schools actually teach Greek kids and Turkish kids to hate one another. But the Cypriot system is not like ours, nor are its results. Cyprus finished 18th of 26 nations in math at the fourth-grade level, 37th of 41 at the eighth-grade level, and 20th of 21 nations on math/science literacy at the 12th-grade level. However, apparently by tracking only a small proportion of students into the study of math and science at the high school level, the Cypriots managed to be sixth among the 21 nations in advanced math and first on the calculus questions. I say "apparently" because I have not ruled out the possibility that they cheated.
Of course, ED's presentation of the results wasn't framed that way. ED portrayed the TIMSS outcomes (and the media thoughtlessly went along) as a comparison not of systems but of kids. As I showed in the September Kappan, everyone played it as "Our best kids went up against their best kids in an apples to apples comparison, and our kids got trounced." But as I also noted in September, if you remove those countries that violated the exclusion rate and participation rate criteria, only five nations remain. Iris Rotberg made the same point in her Science article: "Low participation and high exclusion rates tend to increase a country's rank because lower achieving schools and students are more likely to be excluded from the testing program. Indeed, the very reason that TIMSS provided the guidelines, which were 'more honor'd in the breach than the observance,' was to prevent that occurrence from influencing the rankings" (p. 1030). Incidentally, the U.S. was one nation that did not meet the criteria for participation rates.
I also mentioned in September that the "strict quality controls" claimed by ED did not work. Beyond the participation and exclusion rates, Rotberg declares that, even for the tests of general knowledge, which were supposed to assess a representative sample in each country, "some countries tested a range of diverse schools, whereas others excluded vocational schools, apprenticeship schools, or private schools" (p. 1030). She concludes that the "test score rankings are meaningless as an indicator of the quality of education. . . . The [various international] studies are irrelevant to deliberations about educational reform or as predictors of a nation's scientific or technological strengths" (p. 1031).
Rotberg suggests that, rather than continue to waste money on tests -- TIMSS checked out at $51 million -- we use alternative criteria to decide how well we are doing as a culture. She summarizes these criteria, which were discussed more fully in her December 1990 Kappan article:11
The U.S. would shine on some of these criteria and not on others.
SAT Scores and Advanced Placement Tests
Coming soon from the Educational Research Service is a document I assembled whose working title is The State of Education. This is designed to be a more compact and more user-friendly version of The Condition of Education. It will emphasize numbers, minimize exposition, and be updated annually. One set of statistics that the first edition will contain are state SAT scores from 1982 and 1997.13 In looking at these data, something struck me as quite remarkable. Here are the "national" averages using the recentered scale. (National is in quotes because participation rates in the states range from 4% to 80% of the senior class. And who can say what it means when the College Board lumps them together and calculates a mean?)
V |
M |
|||
| 1982 | 504 |
503 |
||
| 1997 | 505 |
511 |
What could possibly be remarkable about these small increases? The proportion of seniors taking the SAT grew substantially during this 15-year period. If the growth had happened in states where few students took the SAT in 1982, we might expect the scores to rise. That is, if 4% of the students in Utah took the test in 1982 and 12% took it in 1997, we would still be looking at the scores of a select group of Utahans. But this is not where most of the growth occurred. Most of the increase occurred in populous states where many seniors were already taking the SAT. The following chart lists the percentage of students who took the SAT in each of the target years:
| State | 1982 |
1997 |
||
| Connecticut | 69 |
79 |
||
| Maryland | 50 |
64 |
||
| Massachusetts | 66 |
80 |
||
| New York | 62 |
74 |
||
| Pennsylvania | 52 |
72 |
||
| Virginia | 51 |
69 |
All 13 states in which nearly half (or more) of the seniors sat for the SAT in 1982 showed substantial increases in 1997. Given this much deeper dip into the talent pool, it is stunning that scores did not fall.
The number of students taking Advanced Placement (AP) tests continues to soar. Indeed, the numbers have grown so large that some colleges are reconsidering whether they will continue offering course credit for scores of "3." (AP tests are graded on a five-point scale.) In 1997, 581,554 students took 921,801 tests (compared to about 100,000 who took around 150,000 tests 20 years ago.)
There would be more AP candidates, claims Washington Post education writer Jay Mathews, if some schools didn't discourage some students from taking AP courses or taking the tests. Why would schools do so? To look good or in the misguided fear that challenging students with AP courses might affect their self-esteem. To prod schools to encourage students to take AP tests, Mathews devised a "Challenge Index," defined as the number of AP tests taken divided by the size of the graduating class. Mathews feels that anything above 1.0 is good, and the top schools (he rated more than 1,000) have ratios of nearly three tests taken for every graduate.14
Mathews is aware that a single measure is not appropriate for evaluating a high school, but he holds that the index can identify schools that ought to be doing better and other schools that are doing better than anyone would expect. My problem with his Challenge Index is that AP courses are offered in only about half of America's high schools, and I imagine that students in schools without AP courses are less likely to sign up for the tests. Mathews' list also shows few schools in states where the American College Testing (ACT) Program's college entrance exam is the test of choice, and that is a serious weakness.
Education and the Economy: A Continuing Story
Shortly after the TIMSS Final Year results were released, my phone began to ring a lot. Some people just wanted to know my take on the data. Could I refute them? That was easy enough. A larger number of callers accepted the data and wanted me to explain an apparent contradiction: if our students are so bad, how come our economy looks so good? I was dismayed to see how many reporters had bought the notion that schools are tightly linked to the economy -- and to recognize how little influence my writings on the topic had had since I first sought to debunk this notion in the Second Bracey Report in 1992. Since that time I have bolstered my refutation of the idea with more data in each successive Bracey Report.
The goofy idea that schools control our economic destiny was carved into the national consciousness in 1983 by A Nation at Risk. "If only to keep and improve on the slim competitive edge we still retain in world markets," that report intoned, "we must dedicate ourselves to the reform of our educational system."15 Most people bought it, and many still do.
The foolishness reached its apogee (or fell to its perigee, depending on how you look at it) with the publication of Winning the Brain Race, by David Kearns and Denis Doyle. Gushing about the ability of Japanese high school graduates to understand complex instructions, communicate easily with colleagues on the shop floor, solve problems, and continue to learn on the job, these authors wrote that
we cannot compete in a competitive world without the highest levels of academic success. If you doubt it, look at the competition. The Japanese, our most successful competitors, have the highest levels of achievement in the world. . . . Japanese students have the highest test scores in the world. . . . There is a lesson [here] for us, and that is the Japanese treat education as an instrument of national policy. The Japanese are convinced -- as we should be -- that without first-rate schools, they cannot have a first-rate economy.16
In early July, as I began writing this report, mile-high test scores notwithstanding, Japan officially sank into recession: it experienced negative economic growth for two consecutive quarters. The U.S. intervened -- to prop up the yen! The Japanese currency had sunk to an eight-year low before our government stepped in (and it has continued to fall). Fears were rampant that the Japanese economy would go from recession into collapse, taking the rest of Asia and all those wonderful test scores down with it.
Despite all the silly writings of the past about the "Japanese miracle" and the "Asian tigers" and the contribution that high test scores made to these nations' economic achievements, Americans have no reason to gloat. We have plenty of problems, too. The amount of debt in the U.S. is astonishing. The amount of savings is equally astonishing -- by its absence. And if Japan does go deeper into the hole, it can always start selling its U.S. holdings to get dollars.
This might be the best economy that the irrationally ebullient Alan Greenspan has seen, but that doesn't mean there aren't chickens that can come home to roost. Still, in his 21 July 1998 speech to Congress, Greenspan argued that, while the Asian crisis was manifestly slowing our economy, inflation remained a threat.
Indeed, our economic juggernaut steamed along so robustly that New York Times columnist Thomas Friedman imagined this scene: "The nightly news revealed that India and Pakistan had nuked each other off the globe, that Moscow had declared bankruptcy and closed for business, that the Asian economies all descended into the Pacific, and that oil prices fell so low that gas was cheaper than Diet Coke . . . and the Dow Jones hit a record high."17
Robert Samuelson, a columnist for both Newsweek and the Washington Post, conjured up a bundle of potent dangers to the U.S. boom: Asia, the year 2000 problem, the introduction of the Euro, and the overvalued stock market. But, after noting that "everything affects everything else," he said only that there was much uncertainty around and, while contending that all booms end, refused to predict when or how badly this one might go bust.18
Friedman's fanciful reverie is not pure fantasy. Standard & Poor's DRI, an economic forecasting firm, constructed an "Armageddon scenario" that imagined all sorts of terrible things for Asia and South America but left the U.S. in only a mild recession. The problem with DRI's forecast, though, is that it is entirely cast in economic terms. Further economic decline could have major political fallout, including rebellions in Thailand, China, and Indonesia, a nation of 1,100 dialects and 2,000 islands, some of which are talking about secession. Former stockbrokers in Bangkok and Jakarta might join the disaffected masses, putting the U.S. and Europe in a tight spot: Do we shore up the current autocracies or let the rebels and the establishment duke it out?
There is little consensus on the various scenarios, save for one thing: Japan has to get its act together. A permanent tax cut (and let's hope the good citizens don't just put the money into savings) and an end to the "convoy system" of banking, in which the weakest banks are not allowed to fail, are the reforms that are called for most often.
Why am I going on about the economy? This is supposed to be a report about schools. What has all this economy stuff got to do with the condition of public education in these United States? Precious little, and that's the point.
For more proof, look away from Asia to the European Union. The fastest growing economies are Ireland, Spain, and Portugal.19 Only the Irish scored higher than the U.S. in TIMSS.
Pundit Robert Samuelson took a look at the situation beginning with the observation that the resident computer genius at Newsweek had majored in English literature and had never touched a computer in college. There are lots of places other than schools where people learn. With his eye on the TIMSS Final Year data, Samuelson continued, "In isolation, test scores hardly count. What counts -- for the economy at least -- is what people do at work. Do they fully use their skills? Do they develop new ones? Are they engaged?"20
The paean to Japanese high school graduates sung by Kearns and Doyle notwithstanding, opportunities for innovation and the development of new skills occur here more often than in other nations. (I once heard a German commentator on NPR claim that if Bill Gates were German, he would be a middle manager, and Microsoft would not exist.) Moreover, more than in many other countries, businesses here have more freedom to set pay rates, hire and fire employees, and alter work practices. This is a double-edged sword. It results in much greater wage disparities here than in other countries, but the possibility of high wages acts as an incentive for high performance. If you mention the phrase "career ladder" to a Japanese executive or worker, you will be met with a blank stare. The concept does not exist.
Further evidence that the economy and education run on different tracks can be seen in our obsession with math and science, which has led us to overstate the number of people who need high skills in these fields. Borrowing from the study by Anthony Carnevale and Stephen Rose (discussed in "The Statistics from Thin Air Award"), Samuelson concludes that scientists, computer scientists, engineers, accountants, programmers, and business financial officers -- people who need advanced math or science skills -- account for 4% of the work force. (I hasten to add that it is not the Carnevale/Rose report that earned a Rotten Apple.)
Our economy and its accouterments cause admiration, envy, and resentment the world over, according to William Drozdiak of the Washington Post foreign service.21 Drozdiak cites the German weekly, Der Spiegel, as saying, "Never before in modern history has a country dominated the Earth so totally as the United States does today. . . . America is now the Schwarzenegger of international politics: showing off muscles, obtrusive, intimidating." A cartoon in the French daily, Le Monde, depicted the world as a puppet dangling from controls draped in the American flag. "On the brighter side," writes Drozdiak, "even Europeans who worry about the perils of American arrogance say that on many difficult problems, such as making peace in Bosnia, nothing gets done unless the United States assumes a leading role."
Even so, the ebullient economy is not benefiting everyone. We continue to stratify by wealth. William Finnegan, author of Cold New World: Growing Up in a Harder Country, writes that there are 16 million more households in this country than there are adequately paying jobs.22 Entry-level wages for male high school graduates fell 28% in real dollars from 1973 to 1997. College graduates didn't get hit as hard, although they did lose ground, and only about 30% of high school graduates persevere to obtain a bachelor's or better.
The 28% figure above can be a bit misleading. While the proportion of students graduating from high school grew only slightly between 1973 and 1997, the college enrollment rates for these graduates rose from 47% to 65%. Presumably, this increased college attendance left an increasingly less able pool of students available for the workplace. Presumably again, these students would be less able to compete for higher-paying jobs.
What is perhaps more telling than the entry-level wage changes are the changes in the percentage of family income that is discretionary:
1970 |
1993 |
|||
| One to three years of high school | 57 |
34 |
||
| High school graduate | 63 |
56 |
||
| One to three years of college | 68 |
65 |
||
| Four years of college | 73 |
76 |
||
| Five or more years of college | 77 |
81 |
It is also the case that various educational attainments vary with socioeconomic status (SES). For the top quartile of SES, the high school graduation rate is 94%; for the second, 89%; for the third, 80%; and for the fourth, 67%. Thus those who are most in need of at least a high school diploma are least likely to obtain one. Similarly, only 32% of students in the lowest SES quartile who do go to college will have obtained a degree after five years, compared to 47% of those in the middle quartiles and 62% of those in the highest quartile.
The source of these data is Paul Barton of the Educational Testing Service (ETS), who wrote, "When all the negative factors are factored in -- higher cost, stagnating income, declining aid, and high dropout rate [from college] -- the result is growing disparity [among different levels of SES] in students' ability to earn a postsecondary degree."23
A worthwhile publication from the National Center for Education Statistics (NCES) on this topic appeared last year, but it seems to have been little noticed.24 It was written by Paul Decker, Jennifer King Rice, and Mary Moore of Mathematica Policy Research and Mary Rollefson of NCES. And despite its title, Education and the Economy, it does not principally examine how education affects the nation's economic health. It deals mostly with how education affects the economic prospects of individuals. It is a well-organized, well-written compendium of information about education and jobs.
The book is worthwhile in spite of the fact that its first chapter opens with what appears to me to be a major misinterpretation of a trend. Referring to what I have renamed Figure 1, which shows productivity increases from 1947 through 1994, the book says, "It is clear that the growth in output per hour worked [the usual measure of productivity] since 1973 has lagged behind the 1947-1973 trend." This is not true. Or, rather, it is true only if one simply averages the productivity gains from 1973 forward. But such an average does not provide an accurate representation of what's been happening.
Figure 1. Productivity Increases, 1947-94
|
Looking at Figure 1, we see that productivity falls from 1973 to 1974 and then resumes its 1947-73 growth rate until 1977. From 1977 through 1983, productivity growth is essentially zero; after that, it again climbs at the 1947-73 rate until 1987, when growth stops for two years before resuming again. According to the Bureau of Labor Statistics, productivity growth since 1994 has been in line with the 1947-73 rate.25
The book notes that some researchers think that our low placement in international comparisons of math and science is responsible for the "slowing" of productivity growth. But that "slowing" does not really describe what has been happening. What the graph shows is that, since 1973, productivity has grown in fits and starts. When it grows, it grows at the same rate as in 1947-73. Then it stops. Then it starts again. This is hardly the kind of trend line that could be attributed to test scores or to educational attainments or to failings in education. The two periods of slowdown do correspond roughly to periods of national recession. Recessions are part of the economic cycles that economists, if pushed, admit they can't explain.
The first chapter of Education and the Economy discusses "contributions of education to economic productivity" and strikes me as mostly a smokescreen to cover the fact that the econometric techniques for calculating this statistic aren't very useful. At this moment, with unemployment at levels that were considered impossibly low until they happened, it may seem unimportant to look at education and unemployment, but in the long term there is a clear trend. From 1960 through 1994, although there are wide fluctuations, unemployment for both high school dropouts and high school graduates has been increasing.
Dropouts, of course, suffer a much higher rate of unemployment. In the 20-year period from 1974 to 1994, there is no trend for either whites or blacks in the employability of recent high school graduates, but whites have a much lower unemployment rate and show much smaller fluctuations. While the unemployment rates of both groups are affected by recessions, blacks get hit harder. Over the 20-year period, the white unemployment rate varies from 10% to 20%, while the rate for blacks runs from 25% to 58%. "Recent high school graduates" are defined as individuals 16 to 24 years of age who graduated from high school in the year the survey was taken and who were not enrolled in college.
It's unfortunate that the data examined by this publication stop in 1994 because I wonder whether some conclusions are still true. For example, it says that "income inequality may be harmful to the overall economy" (p. 23). Data are presented to show that inequality of income explains about one-fifth of the variability in growth in a group of countries. (Remember, too, that the techniques employed are correlational and that causality cannot be inferred.) Yet income inequality in the U.S. has been increasing rapidly even as the country's economy soars. The U.S. is now the most economically stratified nation in the West (see the Fifth Bracey Report).
Some of the more intriguing data on the relationship between employment and education come from the National Longitudinal Survey of Youth. The data are presented in terms of unemployment rates for people between 19 and 31 years of age, by quartile level on the Armed Services Vocational Aptitude Battery (ASVAB), by subject, and by educational attainment. For people with a high school diploma, math scores show a clear relationship: high scores equal low unemployment and vice versa. For science and reading, though, the results are not so clear. Those in the top quartile have lower unemployment, but there is not much difference between the other quartiles except at the youngest ages. For people with one to three years of college, the relationship is weaker for all three subjects. For those with four to five years of college, there is no relationship at all.
Not all jobs pay equally well, of course. The relationship between test scores and wages is much more regular. Looking again at all people between the ages of 19 and 31, no relationship appears until after age 22, and then the lines for the four quartiles start to diverge and continue to diverge until age 30. Again, math exhibits the strongest relationship of the three subject areas. The difference between the top quartile and all others in math is the largest. For high school graduates, there is no pay difference between first or second quartiles in math and science; for reading, only the fourth quartile has lower pay.
For people with one to three years of college, the differences are nil until about age 24, when the four quartiles separate by a small amount. The divergence continues until age 31, when the largest difference finds the first-quartile scorers in math earning about $13.50 an hour, while the bottom quartile brings in about $10.50.
For people with four to five years of college, placing in the top quartile in math and science means better earnings, and being in the fourth quartile in any of the three areas means lower earnings. For reading, only the bottom quartile is different from the other three.
Data in Education and the Economy, taken from the National Adult Literacy Survey (NALS), are a bit different from those in ASVAB. In NALS, there is a clear relationship between the five levels of literacy and both employment and wages: people with higher literacy levels are more likely to be employed and to make more on their jobs. Among those who have only a high school diploma, those who are employed had higher reading scores.
Education and the Economy expresses concern over the literacy levels of the labor force, with some 40% reading at the lowest two levels. This is certainly a matter of concern, but for the individuals involved, not for the health of the economy, which requires large numbers of unskilled workers (something society refuses to acknowledge). As shown from the First International Adult Literacy Survey (FIALS), in America reading well won't guarantee you a good job, but not reading well virtually guarantees a low-paying occupation.
FIALS also showed that most of our immigrants do not read well. For some reason, FIALS is given short shrift in Education and the Economy, occupying only two pages near the end of the book. It was discussed at length in the Seventh Bracey Report.
Chapter 12 of Education and the Economy, "Education and Literacy of Workers by Occupation," finds that workers in the highest-paying occupations also have the highest literacy scores. This chapter notes too that many of the fastest-growing occupations are in the high-paying categories. This is true, but the difference between rates and numbers must be considered. The fastest-growing jobs -- except for systems analyst -- do not account for many jobs.
Education and the Economy repeats findings seen in SCANS (the Secretary's Commission on Achieving Necessary Skills) reports and the suppressed Sandia Report. Skilled workers and holders of college degrees get much more training than those without skills or degrees, thus increasing the stratification by wealth. Although more people are reporting on-the-job training programs than 15 years ago, only 29% of those with a high school diploma reported receiving such training in a 12-month period, while 61% of those with college degrees reported receiving training. Similarly, those in executive, professional, technical, or administrative support occupations were much more likely to receive job improvement training. In another publication, Anthony Carnevale of ETS found that people earning $25,000 or more a year received 72% of on-the-job training, while those earning $15,000 or less received only 5%.26
The book also reports earlier findings showing that, while secondary school graduation rates are converging in industrialized nations, only Japan comes close to the U.S. in the percentage of college graduates. As reported, the statistics on college completion are a bit misleading. While America graduates men and women in equal numbers, Japan has three male graduates for every female.
It is important to note that, even though some relationships between test scores and earnings are "clear," they are also not particularly powerful; that is, the differences are relatively small and involve large groups of people (quartiles). Henry Levin has observed that
the weak relation between test scores and adult earnings is replicated in virtually every study on the subject. . . . Berlin and Sum found that each additional grade level completed was associated with four times as large a gain in earnings as an additional grade equivalent of basic skills as measured by test scores. And completion of the last year of high school was associated with 10 times the increase in annual earnings as an additional grade equivalent of test scores.27
Levin agrees that "there is no doubt that education, more generally, is an important determinant of earnings. But there is an enormous chasm between this fact and the assertion that new educational performance standards for students will lead to greater economic productivity." Maybe now that someone from a highly regarded university has pointed out this fact, the media and others will pay attention. Hope springs eternal.
Does Money Matter?
Would anyone ask this question in connection with, say, the military? The notion that money doesn't matter is what Alex Molnar, in his revealing and readable Giving Kids the Business, calls "the master myth," the myth that undergirds most privatization incentives.28
In any case, lots of people in 1998 have been saying that money doesn't matter. And when used as described in the June 1998 Kappan Research column, it doesn't. People who claim that money matters assume that the people running schools view those schools as educational institutions. As shown in the June column, that assumption is not valid where schools are seen primarily as community jobs programs. But, as a correspondent put it in a letter quoting Benjamin Barber, "Money can't by itself solve problems, but without money few problems can be solved. Money also can't win wars or put men in space, but it is the crucial facilitator. It is also how America has traditionally announced, 'We Are Serious About This!' "29
The tenacity with which people cling to the idea that money doesn't matter is quite remarkable. In the October 1958 issue of Fortune magazine, one Dan Seligman, apparently a staff writer, constructed a measure he called "teacher-days," which is not clearly described but appears to be the number of days a year a teacher taught multiplied by the number of teachers. Combining this measure and measures of spending, Seligman concluded that efficiency was declining even as per-pupil costs rose. The increasing costs were related in part to declining class size, which, Seligman thought but could not prove, had no bearing on achievement.30
In this 1958 article Seligman drew close to an important insight, but he considered it worthy of only a parenthetical comment: "The steel industry's new facilities have brought it a steadily rising output. The industry's productivity has been rising at an average of 3 or 4 percent a year. . . . No comparable economies are visible in the education industry (nor are they visible, of course, in most of the other service industries)." Of course.
In the 40 years since his article appeared, Seligman has switched magazines, but not mindsets. In the 15 June 1998 issue of Forbes, he declares that there are "solid data telling us that spending more on education does not, by and large, translate into kids learning more."31 Seligman contends that the 1966 study Equality of Educational Opportunity, better known as the Coleman Report, proved this. Coleman, of course, claimed that his report meant that we would have to spend even more money if we wished to improve the achievement of poor and minority children, but Seligman's interpretation is the usual one.32
In 1958 Seligman put his faith in better use of space in schools and in such technologies as Skinner's teaching machines and the like. In 1998 he puts his faith in vouchers. The cult of efficiency has been replaced by prayer.
The contention that money doesn't matter serves as balm for the conscience of business and industry, which have been draining school funds from the tax base through corporate tax breaks. More and more, businesses have been demanding tax breaks as a condition for moving to a given locality. At the same time, they tout their "gifts" to schools. Business hyped the $32 million donated in Florida one year but didn't mention that tax concessions had cost schools more than $500 million in the same year.33 In Wisconsin, by 1994, "corporate tax breaks were taking over $1 billion out of the state treasury every year."34 On a national scale, the contributions that business and industry make to schools would run the schools for less than two hours.35
In the May 1996 issue of School Administrator, I discussed five studies that found relationships between money and achievement, along with several dishonest ones that didn't (e.g., William Bennett's 1993 Report Card on Education in America and George Will's use of that study).36 Even indices remote to daily instruction, such as the SAT, rose with increased spending. To the best of my knowledge, Eric Hanushek has studiously avoided acknowledging that these positive studies exist. At least, I have never seen any references to them in his work, nor did he seem aware of them when we debated the matter at Williams College in January 1998.
Two new analyses provide additional confirmation. The first is from economist Alan Krueger of Princeton University; the other is from David Grissmer, Ann Flanagan, and Stephanie Williamson of the RAND Corporation.37
Krueger, who once studied the effect of education on income by comparing the incomes of identical twins with different educational attainments, takes a new look at Tennessee's Project STAR, an experiment that everyone in the country except Hanushek believes makes a strong case for smaller class size. Krueger notes at the outset that the literature relating money to outcomes is conflicting and that "much of the uncertainty in the literature derives from the fact that appropriate specification -- including the functional form, level of aggregation, relevant control variables, and identification -- of the 'education production function' is uncertain." He also observes that the specification of the appropriate output is in question. Educational researchers favor changes in test scores, whereas economists focus on educational attainment and subsequent earnings.
Krueger found Project STAR interesting because it is the "only large-scale randomized experiment on class size ever conducted in the United States." (For more on Project STAR, see my September 1995 Research column.) At the outset of his analysis, Krueger feared that deviations from randomization could have compromised the outcomes. Students who were randomly assigned to small classes stayed in those classes. However, when parents whose children were initially assigned to regular classrooms complained, the students in regular classes were re-randomized at the end of first grade between regular classes and regular classes with full-time teacher aides. In addition, some behavior problems and other parental complaints resulted in the transfer of about 10% of the students between regular and small classes.
Even allowing for such compromises, Krueger's econometric analyses confirm the essential findings of Project STAR. Some of his new findings contradict contentions from Hanushek. Krueger quotes Hanushek as writing, "If smaller classes [in Project STAR] were valuable in each grade, the achievement gap would widen. It does not. In fact, the gap remains essentially unchanged throughout the sixth grade. . . . The inescapable conclusion is that the smaller classes at best matter in kindergarten."
But Krueger found that, when the same children are tracked over time, the gap does widen, except for those who entered small classes as kindergartners: "For the wave of students who entered kindergarten, the beneficial effects of attending a small class do not appear to increase as students spend more time in their class assignments. For students entering the experiment in first or second grade, however, the test score gap between those in small- and regular-size classes grows as students progress to higher grades."
Krueger also checked for Hawthorne effects (the teachers in the small classes knew they were part of an experiment to show that small classes work) and for John Henry effects (the teachers in regular classes might have worked harder). He found no evidence of either. In his search for Hawthorne and John Henry effects, Krueger did find that, within regular classrooms, large classes (average 25) did not score as well as smaller classes (average 21). Because these comparisons looked at large and small classes within the same building, they cannot be the result of some extraneous cause, such as differences in socioeconomic status.
Overall, Krueger's analyses find that small classes increase test scores by about .22 standard deviations. The effects are larger for minority students and for those receiving subsidized lunches. Is an effect size of .22 big or small? "Unfortunately," writes Krueger, "it is unclear how percentile scores on these tests map into tangible outcomes." But he does note that "in kindergarten the impact of being assigned to a small class is about 64% as large as the white-black test score gap, and in third grade it is 82% as large. By both metrics, the magnitudes are sizable."
Krueger admits, of course, that "no single study, even an experimental one, could be definitive." He also contends, again of course, that "one well-designed experiment should trump a phalanx of poorly controlled, imprecise, observational studies based on uncertain statistical specifications."
The RAND researchers, using different data and different methods of analyzing them, also found indications that smaller classes have a larger benefit for poor and minority students. But before presenting those data, let me observe here that all of these studies operate under the actual variations in spending. While in most states the high-spending districts spend more than twice as much as the low-spending districts, most districts have similar within-district expenditures. If we dropped spending to zero, the effect on achievement would be large and quickly seen (as happened when Prince Edward County, Virginia, closed its public schools rather than integrate them). If we raised spending to $37,000 per pupil, achievement would rise substantially. (This is the average salary nationwide for teachers.) By spending this amount for each child, we could provide each with a personal tutor and, no doubt, observe the well-known effects of one-on-one tutoring.
These researchers observe that the gap between black students and white students on the NAEP mostly decreased from 1971 to 1996, although it remains large -- about four years. The RAND researchers wondered why. In a 1994 study (discussed in the Fifth Bracey Report), they made predictions of test score changes over a 20-year period, using changes in family variables as predictors.38 Although many of the variables improved for blacks -- more education, fewer children, and less mobility for families with children (somewhat offset by increases in teenage pregnancy) -- test scores rose more than predicted. Changes in family could account for only a small amount of the changes in test scores.
In the current analysis, Grissmer and his colleagues examine other variables. Tracking and grouping of elementary school children by skill level within mixed classrooms do not appear to be playing much of a role. Taking more demanding courses might be part of the explanation for middle and high school students: between 1982 and 1992 both blacks and whites reported taking more math courses, but the increase for blacks was greater. As for variables that affect all ages, desegregation is a candidate, but it offers a complex and sometimes contradictory set of data. For example, data from the South support the contention that desegregation had a positive effect, but test scores in the North also rose, despite increasing segregation in northern schools over the period studied.
Looking at the gains for all students nationwide, the researchers suggest three variables that changed while the scores were moving toward one another, each of which could have produced some gain: additional resources, smaller class sizes, and a more experienced teacher force. For the last mentioned, during the period of interest, the proportion of teachers holding a master's degree rose from 25% to 50%, while the percentage of teachers with five to 20 years of experience rose from 45% to more than 60%.
The additional resources allocated to schools over the past 30 years are much less than Hanushek and others have alleged. Conservative critics have generally used the Consumer Price Index to adjust for inflation. But the use of the CPI to measure increases in school funding greatly overestimates how much is actually appropriated. In addition, only about 30% of what was appropriated made it into regular classrooms; the rest went to targeted populations that are often not tested in national, state, or local programs (i.e., students in special education programs).
In discussing the effect of class size, Grissmer and his colleagues present some of the same arguments as Krueger. Incidentally, a summary of class size research written for nontechnical readers can be found in a monograph by Alex Molnar of the University of Wisconsin, commissioned by the Keystone Research Center of Harrisburg, Pennsylvania.39 The monograph also summarizes the conflicting claims made for voucher programs in Milwaukee and Cleveland, as well as the Project STAR data. In addition, it presents some new data from Wisconsin that replicate the STAR findings.
So why, after a period of converging, did the gap between black and white 17-year-olds widen once again? The researchers propose violence as the reason. The murder rate among black teenagers
rose dramatically between 1985 and 1990 and remained high through 1992. If murder is a proxy for other adverse changes in black neighborhoods and schools, it might explain declines in blacks' reading scores. This theory is appealing because there was no parallel increase in murders among white teenagers, so it could help explain why blacks' scores declined while whites' scores remained stable.40
Stephan and Abigail Thernstrom also noticed the widening gap after 1988. "Why this alarming turnaround? What brought about the earlier gains that have now been reversed? These important questions have not attracted the attention of many researchers."41 Although not experimental researchers, the Thernstroms offer the same hypotheses, though they have no data with which to test any of them. Thus they declare themselves "stumped." But to the increased violence hypothesis of Grissmer, Flanagan, and Williamson, the Thernstroms add the observation that crack cocaine and gang-related drug wars both appeared in the mid-1980s, largely in the black community.
The test scores of black students and white students can never converge, though, until the attitudes of many black students change. "A black teacher in the Bronx told me in a despairing tone that she has male students who would rather be paraded in handcuffs before television cameras than be caught reading a book," wrote columnist Bob Herbert in the New York Times.42 Herbert also presented the example of a black 17-year-old girl who worked part time at a mall in Marietta, Georgia, and "was taunted recently by high school classmates who showed up at her job to express their resentment at the high marks she was getting."
The purpose of Herbert's essay was to call attention to the Campaign for African-American Achievement, launched by Hugh Price, president of the Urban League. Price has drawn together 20 other organizations to help. I hope that by now this is not news to Kappan readers. September was designated "Achievement Month" by the campaign, which was to "conduct a month-long series of high-profile events each year celebrating the efforts of black youngsters who are doing well in school."
That the achievement gap between blacks and whites is important outside of school is shown in data provided by Christopher Jencks of Harvard University and Meredith Phillips of UCLA. Their data indicate that in 1964 blacks who scored above the 50th percentile on a test given 12 years earlier earned only 65% of what whites scoring above the 50th percentile earned. By 1993, similar scores yielded earnings for blacks that were 96% of white earnings. "Reducing the black-white score gap would probably do more to promote racial equality than any other strategy that commands broad political support," they write.43
Jencks has changed his position on school effects and now thinks that they are much more powerful than he and his colleagues suggested in 1972 in Inequality. (But see the section on NASDC, below.) Changing school variables probably still won't do much to close the gap, because such a strategy would not command broad support. It would cost too much to be politically feasible. Black schools and white schools receive nearly equal funding, but black schools -- operating largely in poor neighborhoods -- must spend their resources differently: on special education, counseling, security, and so on. Teachers in these schools are generally less qualified than those in suburban schools, and higher salaries would be required to attract more qualified teachers. While the public might accept spending 10% more for poor schools than for others, it probably wouldn't support the large increases that would be needed to make major differences.
Jencks and Phillips acknowledge that the attitudes of black students about achievement in school need to change. However, looking at the fact that black-white test score differences are large when children begin school, they argue that "changing the way parents deal with their children may be the single most important thing we can do to improve children's cognitive skills" (p. 50). They admit that this will be hard and offer no suggestions about how to accomplish it.
Whatever Happened to NASDC?
Well, someone finally said it. Even school critics such as George Will have acknowledged that the 91% of their lives from birth to age 18 that children spend outside of school overwhelms the 9% they spend in school. Unfortunately, it took an educator to say: "Education, Alone, Is a Weak Treatment." That was the title that James Gallagher of the University of North Carolina, Chapel Hill, gave to his commentary in the 8 July 1998 issue of Education Week. I say "unfortunately" because, coming from within the education "establishment," such a comment might be taken as defensive or self-serving. It is not.
It is why no one can find any silver bullets to solve our educational problems. It is why Chris Whittle pared down his Edison Project and adopted curricula developed by others (by educators, no less!) rather than try to construct his own from scratch. It is why Reading Recovery and Success for All can make significant improvements in students' achievement and still leave children years below grade level. It is why most industrialized nations score very much like the U.S. on international assessments of reading, math, and science (at grades 4 and 8, anyway). In the reading study of the IEA (International Association for the Evaluation of Educational Achievement), American 14-year-olds finished eighth among 31 nations, but only Finland had a significantly higher score, and only nations ranked 16th or lower had significantly lower scores. Given that the chances of finding significance increase with sample size and given that the samples were exceptionally large, this means that the differences among nations were tiny indeed.
The "weak treatment" effect of schools should be kept in mind when looking at the programs funded by the New American Schools Development Corporation (NASDC), now known simply as New American Schools (NAS). NAS was introduced with great flourish by President Bush in 1991 and ardently touted by his secretary of education, Lamar Alexander. These were to be "break-the-mold" schools. They would emulate business and show us poor, dumb educationists how to do it. Remember?
By its own criteria, NAS was probably doomed to fail. Examined from the more generous perspective of school reform generally, something good might yet come from the effort. NAS gave itself two years to get fully operational and a total of three years to show significant student improvement. Other studies have suggested that it takes five or six years to improve student achievement; one study even found that it took a small, affluent suburb -- i.e., a place with everything going for it -- eight years just to get its elementary science curriculum revamped.44 In any case, after a much ballyhooed competition, grants were awarded to seven supplicants. Looking at who got the grants, some concluded that the decisions could have been made the day after the competition was announced.
The NAS project is undergoing a long-term evaluation by a RAND Corporation team headed by Susan Bodilly.45 In its second-year report, the team divided the reform effort into eight "elements" that had to be addressed: curriculum, instruction, assessments, student grouping, professional development, community involvement/public engagement, standards, and staff organization. The RAND evaluators concluded that, for improvement in achievement to occur by the end of three years, the first five of these eight elements should be pretty much in place by the end of year two. They studied a nonrandom sample of 40 schools. They concluded that two of these schools were fulfilling their vision and that another 16 were implementing their programs, but the rest were in a piloting or planning stage.
The three-year implementation strategy was not one that all schools have adopted voluntarily. It was an NAS goal. In addition, some of the schools were at risk of state-level takeovers or interventions if they did not show improved performance within three years. The RAND team notes that the time frame was not conducive to positive outcomes:
Districts and NAS had to establish the process [of putting an NAS design in place] at a time [of the year] when school people were busy with testing regimes and when many staff had already determined how they would use their summer leave. Second, districts had to ask schools to make a major commitment at a time of the year when many of the planning and preparation activities for such an undertaking had already occurred. Schools had already developed school improvement plans, school budgets, Title I budgets, and special education IEPs for the coming year. Finally, rhetoric to the contrary, none of the designs were proven at this point in time. Schools were being asked to make choices about designs based on limited data about the effectiveness of the designs in encouraging implementation in like schools and based on limited data about the designs' ability to increase student performance. (pp. 43-44)
One can only imagine the enthusiasm with which school faculties and administrators received these assignments. Indeed, schools that felt they had adopted a design without fully understanding it and those that felt they had been forced to adopt it showed lower levels of implementation. Surprise.
The RAND researchers divided the factors affecting the success of implementation into four categories. Under "school structural factors," elementary schools enjoyed much more success putting the designs in place. At the district level, districts with stable leadership, visibly strong support from the leadership, no crises, a culture of cooperation, school autonomy, assessments aligned with the design, and resources readily available -- teacher planning time, money, etc. -- moved more quickly down the road toward implementation.
The design teams had a strong impact on program implementation as well. Audrey Cohen died, and the single national facilitator of her model retired. The ATLAS (Authentic Teaching, Learning, and Assessment) team could not coordinate the ideas of the four founders. The Modern Red Schoolhouse had initially relied heavily on consultants and had to build a permanent staff quickly. In general, higher levels of implementation were associated with stable design teams that could effectively communicate the design to the schools and market it to districts, that supported implementation with lots of facilitators and extensive training, and that emphasized the first five of the eight elements.
AS USUAL, I come to the end of another report having exhausted the amount of space the Kappan can provide, though far from having exhausted the topics to talk about. This one has said little about vouchers or charter schools, choice or privatization. It's not that nothing happened in these arenas, but few hard data have been reported this year. As economist Richard Rothstein notes, "Praise for charter schools comes in the same form today as criticism of regular public schools: little more than anecdotal puffery and phony statistics."46 On the voucher front, the Wisconsin supreme court ruled that the Milwaukee voucher program could be extended to include church-affiliated schools because the purpose of the voucher was not church-related. This is hardly the last word.
I am currently conducting research in all these areas for People for the American Way. A report will be forthcoming, and I expect it will contain many experiences similar to those of Michael Winerip of the New York Times and of Kevin Smith of the University of Nebraska, Lincoln, and Kenneth Meier of the University of Wisconsin, Milwaukee. In The Case Against School Choice: Politics, Markets, and Fools (a play on John Chubb and Terry Moe's fatally flawed Politics, Markets, and America's Schools), Smith and Meier found that the students in some city schools thought that choice would work, but only if you didn't let the bad kids choose.47 They saw the problems of the schools not in terms of incompetent teachers and inept bureaucrats, but in terms of other kids, especially violent ones. If the bad kids can choose, too, the students said, what is the point? This is skimming in the extreme, but youngsters see in concrete reality and take as a given what has been stated in the abstract by some adult critics of school choice: choice must skim off the best students in order to work.
The Children's Educational Opportunity Foundation (usually referred to as CEO America) listed statistics pertaining to violence in schools.48 It then proposed taking the well-behaved youngsters out by giving them vouchers. It does not seem concerned about the difficult children; it does not seem concerned about what increasing their proportions while reducing resources would do to the schools where they remain. This is not a solution.
For his part, Winerip reports on a meeting he attended in Jersey City, New Jersey, at which parents were encouraged to sign up for a new charter school run by Advantage Schools, Inc., of Boston.49 To some, the presentation sounded like a sales pitch for some new-fangled widget. To others, the presenter seemed to come on more like a preacher at a revival meeting. At this meeting parents learned that at a Rocky Mount, North Carolina, school operated by Advantage, "Our children on third level are reading The Iliad. In fifth grade they read Homer's Odyssey." All the kindergartners could read. This last item was an accomplishment Winerip heard about over and over again.
Like any good researcher but few education reporters, Winerip checked the accuracy of the assertion. He visited the school. A corporate spokesman from Boston flew down to escort him. "All the kindergarten kids read, you know," the spokesman said. Winerip acknowledged that Advantage had done a great job of remodeling the facility into an attractive place, but he began to grow skeptical about its academics when he found the school receptionist teaching a top fifth-grade math group. She told him, "All you have to remember is that you can't go off the script."
Advantage's description of "the script" sounds like a means to prepare children to live in a society organized around principles laid out by George Orwell in 1984 or by Aldous Huxley in Brave New World. In a section of its website that deals with "Frequently Asked Questions," Advantage says that
students will make eight to 12 responses each minute. This means that students will make between 240 and 360 responses in a half-hour lesson. If they have three DI [Direct Instruction, but Drill Instructor works for me] lessons in a day, they will make between 720 and 1,180 responses each day. . . .
Q: Am I right in concluding that all students respond at the same time? Absolutely. It is critical that all students make the correct responses. If a child doesn't respond, the teacher can't know whether the child knows the right answer or not.50
In kindergarten, the script in math finds the teacher saying things like "87 minus 1 equals 86. Everyone, what's the answer?" She snaps her fingers, and all the children say, "87 minus 1 is 86." And TIMSS claimed American public school teachers don't emphasize conceptual understanding!
But when Winerip turned his observations from math to reading, he went deliberately off the script: he brought his own books for the kindergartners to read. Confronted with a book that began, "I am a ghostie," a boy in the class could not cope with "I." Another boy, given a book that opened with "Drip, drip, drop," fared no better. Given repeated hints about the pictures showing falling rain, the boy finally said "Rain drops . . . no, I don't know." A girl managed to sound out a few of these words "in agonizingly slow fashion." Of course, to many of us, "education" means going off the script. Stay tuned.
PDK Home | Site Map
Kappan Professional
Journal
Last updated 6 November 1998
URL: http://www.pdkintl.org/kappan/kbra9810.htm
Copyright 1998 Phi
Delta Kappa International
