{"id":9991,"date":"2005-06-21T00:00:00","date_gmt":"2005-06-21T04:00:00","guid":{"rendered":"http:\/\/localhost\/thenewatlantis.com\/publications\/how-we-measure-up"},"modified":"2020-09-26T14:08:35","modified_gmt":"2020-09-26T18:08:35","slug":"how-we-measure-up","status":"publish","type":"article","link":"https:\/\/www.thenewatlantis.com\/publications\/how-we-measure-up","title":{"rendered":"How We Measure Up"},"content":{"rendered":"

A<\/span>s if coordinated to provoke headlines, top executives at three of the nation\u2019s leading technology firms recently issued bleak appraisals of the American education system, criticizing especially how American students are taught science and mathematics. Microsoft Chairman Bill Gates minced no words at a summit of the nation\u2019s governors: until high schools are redesigned, he declared, \u201cwe will keep limiting, even ruining, the lives of millions of Americans every year.\u201d The chief executives of Intel and Cisco Systems shortly followed suit, suggesting that America\u2019s lackluster schools will increasingly force companies to look overseas for talent.<\/p>\n

Of course, these concerns are hardly new. But the somber prognoses from the heights of high-tech have added high-profile urgency to recent press reports about the declining performance of U.S. students in science and math compared to other nations, and the potential rise of China as a technological and economic superpower. Leading U.S. media outlets have featured major stories on the consequences of China\u2019s rise for America\u2019s future, like the recent Newsweek<\/em> cover story by Fareed Zakaria appealing for a \u201cmassive new focus\u201d on science and technology in the U.S., lest America \u201cfind itself unable to produce the core of scientists, engineers and technicians who make up the base of an advanced industrial economy.\u201d In such a media atmosphere, one could be forgiven for having concluded that the United States is drifting unawares into an educational backwater while the rest of the world paddles furiously past it.<\/p>\n

The truth is more complex. Cross-national studies of scientific and mathematical ability, interpreted rightly, tell a complicated story, giving reason to question how well the tests measure America\u2019s real educational standing in the world. The two tests cited most frequently in press reports are the Program for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). PISA, undertaken by the Organization for Economic Cooperation and Development (OECD), most recently spanned 41 countries and tested 15-year-olds on mathematical word-problems. The latest TIMSS, in 2003, comprised more traditional, textbook-style math and science problems and was administered to fourth- and eighth-graders in 25 countries by an international team of researchers based in Boston and Amsterdam. The Department of Education funded both studies in the U.S., with help from the National Science Foundation.<\/p>\n

Both tests have repeatedly been invoked by sensationalists seeking to cast the United States as unprepared for the high-tech, global economy. When the latest PISA results were released toward the end of last year, for instance, the Christian Science Monitor<\/em> ran with the headline \u201cMath + Test = Trouble for U.S. Economy,\u201d and concluded that the study\u2019s emphasis on \u201creal-life\u201d math skills makes it an accurate and \u201csobering\u201d predictor of students\u2019 performance in \u201cthe kind of life-skills that employers care about.\u201d Federal officials expressed concern about the test results, too. \u201cIf we want to be competitive, we have some mountains to climb,\u201d said Deputy Education Secretary Eugene Hickok.<\/p>\n

To be sure, the results of neither TIMSS nor PISA should send American educators and policymakers rushing to the champagne. In most math areas tested by PISA, the gap between the average U.S. student and the average student in the highest-scoring countries \u2014 often Finland, the Netherlands, Singapore, Japan and Hong Kong \u2014 was roughly equivalent to the gap between the United States and low-scoring countries like Uruguay or Mexico. Where 44 percent of Singapore\u2019s students reached the TIMSS \u201cadvanced international benchmark,\u201d only 7 percent of U.S. students did. And, in general, the longer students had remained in the U.S. school system, the worse they performed relative to their peers abroad.<\/p>\n

The first question that must be asked of such broad results, however, is whether the tests themselves accurately represent the countries\u2019 student populations. International surveys such as these are not given to every student in each participating country; the tests\u2019 organizers pick out statistical samples that are supposed to represent each country\u2019s entire student population. Even so, schools \u2014 especially in the United States \u2014 sometimes decline to participate in the tests, potentially skewing the sample. As far as accurate sampling is concerned, early incarnations of the tests were not encouraging. In the first TIMSS general achievement test, conducted in 1995, only 5 of 21 participating countries met the study\u2019s guidelines for conducting representative samples. While most countries participating in the latest studies have dramatically improved their overall sampling, the United States remains a notable exception. Only 73 percent of U.S. students chosen to be sampled were actually tested, a figure below the \u201cminimum acceptable\u201d rate of 75 percent. In most other countries, that number was well over 90 percent. If the omitted 27 percent of U.S. students were even slightly above or below average, their exclusion casts serious doubts on the accuracy of the U.S. sample.<\/p>\n

The studies also inevitably confront large differences between countries\u2019 school systems. \u201cIn Cyprus, students taking the advanced mathematics test were in their final year of the mathematics and science program; in France, the final year of the scientific track; in Lithuania, the final year of the mathematics and science gymnasia; in Sweden, the final year of the natural science or technology lines; and in Switzerland, the final year of the scientific track of gymnasium,\u201d Professor Iris Rotberg of George Washington University wrote in Science<\/em> concerning the 1995 TIMSS assessment, which tested high-schoolers. \u201cIn contrast, students in several countries, including the United States, attended comprehensive secondary schools. The major differences in student selectivity and school specialization across countries make it virtually impossible to interpret the rankings.\u201d In TIMSS, especially, tests are conducted by grade-level rather than by age. In elementary and middle school, where topics are often covered and learned over the course of a few weeks, the risk of comparing students at incommensurate stages of their education is great.<\/p>\n

Broad curricular differences have probably had a role in deflating U.S. scores. TIMSS and PISA use the same test in every participating country, and the material that makes it onto the test is selected through a winnowing process that leaves the tests considerably narrower than any single country\u2019s general curriculum. Countries that include large amounts of material in their typical curricula are therefore at a disadvantage compared to those countries that focus their curricula more intensely on fewer subject areas. Regardless of its other merits or failings, the American strategy of repeated exposure to a broad range of subjects \u2014 American textbooks are the bulkiest in the world \u2014 is likely to lend itself to unduly poor performance on standardized tests, as full understanding of any single concept takes longer to develop.<\/p>\n

Demographics and culture are also thought to confound the results of cross-national comparisons. In the United States, the tested students come from every socioeconomic rung, while other countries sometimes lack some rungs because of cross-border employment. For example, much of the labor force in Hong Kong (which is treated on the tests as an independent entity) is made up of tens of thousands of low-paid Filipino household workers whose children live and are educated in the Philippines; in light of the extensive literature tying socioeconomic indicators to educational achievement, this cross-border employment surely affects both countries\u2019 scores. A similar situation obtains in other places with significant immigration and cross-border commerce, as Gerald Bracey points out in a 1997 article in the journal Educational Researcher<\/em>. \u201cEach morning thousands of Malaysians enter Singapore to sweep streets, pick up garbage, and do other low-level jobs. They return to their homes at night, relieving Singapore of having to educate the children of poor laborers,\u201d Bracey writes. \u201cIf one reads the [domestic] educational research literature, one is struck by the lengths \u2014 the extreme lengths \u2014 that researchers go to to ensure that samples in their studies are comparable….The research community would never accept test results in this country that simply compared scores in an inner-city slum and an affluent suburb as if they<\/em> were comparable,\u201d he writes. The opposite circumstance holds in the United States: Students from all socioeconomic rungs are educated and scored on these tests.<\/p>\n

Amid this deluge of confounding factors, the inference that the U.S. education system is going down the tubes is an unjustified logical leap. The United States is still pumping out tremendous numbers of new Ph.D.s in the sciences \u2014 more, in fact, than our economy can presently absorb, as there is a well-reported dearth of jobs for newly-minted science Ph.D.s. The same is true in engineering: According to a recent National Science Foundation report, the number of engineers graduating from U.S. schools will continue to grow into the foreseeable future, outstripping the number of available jobs. Of these new engineers and Ph.D.s, an increasing number are foreign-born \u2014 but increasing even faster is the percentage of those who decide to stay in the United States. Federal research funding for scientific research and development has consistently risen in absolute terms and as a fraction of discretionary spending \u2014 and industry research dollars have risen dramatically on top of that, to the tune of 7 percent per year in real terms \u2014 according to calculations by the Consortium for Science, Policy and Outcomes at Arizona State University. (Alarmist media reports often use GDP, against which research spending has fallen, as a comparative baseline.) And countries that have \u201coutperformed\u201d the United States in educational studies for many years \u2014 a number of European countries top this list \u2014 still fail to rival the U.S. in any measure of research productivity. When Bill Gates and others seem to appeal for school reform in the U.S., perhaps they are merely providing their companies with political cover and a post hoc<\/em> justification for employing foreign engineers who, while not better educated than U.S. workers, are often significantly cheaper.<\/p>\n

Nevertheless, there remains good reason to worry about what the global economy portends for those American students who really are badly educated. In only one other OECD country (New Zealand) are internal educational inequalities worse than in the United States, according to a recent analysis by researchers in England and Italy. Where these inequalities lie is no mystery. The gap in test scores between white and ethnically Asian students on the one hand and black and Hispanic students on the other is a well-known attribute of U.S. schools and is noted ruefully in nearly all cross-national studies. Two University of Pennsylvania researchers recently aggregated scores from a number of cross-national studies and found that white students in the United States, taken alone, consistently outperform the predominantly white student populations of several other leading industrial nations. \u201cThere is compelling evidence,\u201d they write,\u201d that the low scores of [black and Hispanic students] were major factors in reducing the comparative standing of the U.S. in international surveys of achievement. If these minority students were to perform at the same level as white students, the U.S….would lead the Western G5 nations in mathematics and science, though it would still trail Japan.\u201d In PISA, for instance, white students performed above most European countries, whereas black students performed on par with students in Thailand. So while the performance of minority groups in the U.S. does refute the alarmist assertion regarding an across-the-board decline in U.S. schools, it does so in a particularly unfortunate way \u2014 namely, it suggests that some American minority groups will be shut out of high-paying jobs as companies look for better-educated workers overseas. Although the most recent TIMSS saw the white-black score gap close slightly, it is almost certain to remain shockingly large in the near future.<\/p>\n

None of this is to say that other countries are not catching up technologically, nor that the United States is safe from competition in even a single technological sector. China is without doubt the most aggressive challenger. In the mid-twentieth century, Japan\u2019s economy grew 55-fold over the course of thirty years through stringent government control; observers of Japan\u2019s rise will remember the key role of its Ministry of International Trade and Industry, which employed many of the nation\u2019s brightest stars and guided the economy on a carefully directed path of technological growth. China\u2019s strategy has been similar, though its tremendous size has necessitated delegation of heavy-handed economic control to regional governments in what scholars have termed \u201clocal state corporatism.\u201d It has simultaneously harnessed the power of markets in a way Japan did not. Regional governments lavish tax breaks on high-tech industries (many of them funded from overseas) and pump millions into China\u2019s new universities \u2014 which are poised to graduate more Ph.D.s than the United States by 2010, according to some projections. Nearly all of China\u2019s top leaders are scientists and engineers by training: President Hu Jintao is a hydroelectric engineer, Premier Wen Jiabao is a geological engineer. Their predecessors, Jiang Zemin and Zhu Rongji, were both electrical engineers. The technocrats steering China\u2019s ship of state are working hard to modernize scientific education in their country.<\/p>\n

But the United States need not worry \u2014 not yet. The U.S. is by no means in technological decline, though China and India will inevitably pose challenges in years to come. Although not a crisis, this competition should motivate the U.S. to improve its science and math education, especially for poor and minority students who might lose out in a globalized, high-tech economy. If sensationalists must take up a cause, it should be the plight of those students and not a hyped-up \u201cthreat\u201d of China or the \u201cimpending decline\u201d of technological innovation here at home.<\/p>\n","protected":false},"excerpt":{"rendered":"

As if coordinated to provoke headlines, top executives at three of the nation\u2019s leading technology firms recently issued bleak appraisals of the American education system, criticizing especially how American students are taught science and mathematics. Microsoft Chairman Bill Gates minced no words at a summit of the nation\u2019s governors: until high schools are redesigned, he declared, \u201cwe will keep limiting, even ruining, the lives of millions of Americans every year.\u201d The chief executives of Intel and Cisco Systems shortly followed suit, suggesting that America\u2019s lackluster schools will increasingly force companies to look overseas for talent. Of course, these concerns are…<\/p>\n","protected":false},"author":1,"featured_media":0,"template":"","article_type":[4647],"noteworthy_people":[],"topics":[2267],"_links":{"self":[{"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/article\/9991"}],"collection":[{"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/article"}],"about":[{"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/types\/article"}],"author":[{"embeddable":true,"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":0,"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/article\/9991\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/media?parent=9991"}],"wp:term":[{"taxonomy":"article_type","embeddable":true,"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/article_type?post=9991"},{"taxonomy":"noteworthy_people","embeddable":true,"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/noteworthy_people?post=9991"},{"taxonomy":"topics","embeddable":true,"href":"https:\/\/www.thenewatlantis.com\/wp-json\/wp\/v2\/topics?post=9991"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}