News

The Arizona Test | Test Sample & Testing Procedure | Summary Part 1 | Summary Part 2

ACEE, in conjunction with its Offices for Economic Education at Arizona State University and the University of Arizona, is in the midst of conducting a much-expanded study of economic literacy and attitudes toward markets. This study will focus on the potential impact of the adoption of state standards in Economics. Additionally, a wider sample is being used and additional demographic information collected from students. This will enable the researchers to draw many more conclusions than in the last study.

Is Your School Interested in Participating?

If so, please contact the Council at: infoplease@azecon.org to join our effort!!!

Learning economics is important as it offers students a basis for principled reasoning about a wide range of personal, civic, and work-related problems. But many American students complete 12 years of schooling in which little or no attention is paid to economics. While 96 percent of Americans think economics should be taught in the schools, only 13 states require students to take an economics course in order to graduate; about three-fourths of the states have some standards for economics (Clow, 1999).

How do the young people of Arizona stand in respect to economic knowledge and attitudes? To find out, the Office for Economic Education at the University of Arizona administered a test developed by the Directors of the Offices for Economic Education at the University of Arizona and Arizona State University and the President of the Arizona Council on Economic Education to more than 1800 Arizona high school students in April 1999. Results overall showed poor performance. While they did outperform students participating in similar assessments nationwide, Arizona students on average registered low scores on measures of basic conceptual and factual knowledge about economics. In addition, responses to attitudinal questions showed that most students had little confidence in market solutions to economic problems.

But performance on the test was not uniformly poor. Students who had studied economics in high school outperformed those who had not, and students who had completed economics courses with rigorous standards did better still. Students with greater knowledge also had more favorable attitudes toward market-based solutions to economic problems. This pattern of results suggests that achievement in economics and acceptance of market solutions could be improved by a statewide plan to ensure a firm place for the study of economics throughout Arizona's K-12 curriculum, culminating in an 11th or 12th grade capstone high school course for every student.

The Arizona Test

The Arizona Test of Economic Knowledge and Attitudes was developed jointly by the Arizona Council on Economic Education and the Offices for Economic Education at Arizona State University and the University of Arizona. The knowledge portion of the test consists of 14 multiple-choice questions about concepts in economics. These concepts cover many of the 20 Voluntary National Content Standards in Economics (National Council on Economic Education, 1997) and the proposed Arizona standards (see Table 2). Of these 14 questions, six are taken from a test developed for the National Council on Economic Education (Harris & Associates, 1999); six are adapted from questions used originally by the Federal Reserve Bank of Minneapolis in its Economic Literacy Survey (The Region, 1998); and two are new questions written specifically for the Arizona test. In addition, the Arizona test includes five scenarios written specifically for this test, each describing a resource allocation problem followed by a list of four potential solutions. Students' responses to the scenarios provide a measure of attitudes toward various approaches to the allocation of scarce goods and resources. (A copy of the knowledge questions and scenarios is attached.)

The knowledge questions measure students' grasp of basic economic concepts including scarcity, incentives, the relationship between self - interest and the common good, marginal analysis, unintended consequences, the nature of inflation, and the dynamics of international trade. The scenarios probe the willingness of students to use prices to allocate resources. In particular, the scenarios ask students whether they would favor using prices to allocate resources in situations as mundane as a hot dog shortage and as profound as kidney transplantation.

back to top

Test Sample and Testing Procedure

A sample population of students was obtained by asking 70 teachers in high schools all around Arizona to volunteer their time to administer the test to two of their classes. Of the 70 teachers asked to participate, 41 were economics teachers; these 41 teachers were asked to administer the test to one economics class and to one class in another subject. In all, fifty-four teachers responded, and the result was that 1807 Arizona high school students in 37 schools across the state took the test.

School districts represented in the sample include the two largest districts in the state (Mesa Unified School District and Tucson Unified School District), districts from smaller metropolitan areas (such as Flagstaff and Yuma), and rural areas. The 1998 mean Stanford 9 scores for schools in the sample were 45.2 verbal and 52.8 quantitative. The mean scores for all Arizona students were 42 verbal and 49 quantitative. Seven hundred sixty students in the sample had never taken an economics course; 750 were concurrently enrolled in an economics course when they took the test; and 290 had previously enrolled in an economics course.

The students' tests were scored by the Instructional Assessment and Evaluation Services at the University of Arizona. Test-score results were compiled and analyzed by Michael K. Block, Director of the University of Arizona Office for Economic Education; William J. Boyes, Director of the Arizona State University Office for Economic Education; and John S. Morton, President of the Arizona Council on Economic Education. Research assistance was provided by Ding Yuan, a Ph.D. candidate at the University of Arizona.

back to top

Summary Part 1

Part 1. Summary of Results about Economic Knowledge

On average, Arizona high school students performed poorly on the 14 questions designed to measure economic knowledge. The major bright spot in an otherwise disappointing profile is that the Arizona students do appear to be more economically literate than the average high school student in the United States.

Arizona students scored low on questions designed to measure knowledge of economic concepts. Their average score on this portion of the test was 44 percent, which would ordinarily earn a grade of "F" in any high school class (See Figure A).

Students have a striking lack of knowledge about basic economic facts and concepts. For example, only 27 percent knew that competition channels self-interest into the general good in a market economy, only 29 percent could correctly identify the policies that government should follow in order to control inflation (See Figure A), and only 38 percent knew what the government budget deficit was.

Students scored very poorly on questions related to emotionally charged issues. Asked how U.S. workers are affected when U.S. firms establish low-wage branches in developing countries, only 20 percent responded correctly. Only about one-third (34 percent) of the students realized that economic efficiency in pollution control efforts requires a comparison of marginal benefits with marginal costs (See Figure B).

On the other hand, Arizona students performed well on questions related to international trade. Seventy-eight percent, on average, provided the correct answer to a question asking who gains from trade between countries, and 68 percent answered correctly in response to a question about who gains from trade policy that reduces automobile imports (See Figure B).

Moreover, Arizona students outperformed two other sample populations responding to similar assessment questions nationwide. Six questions on the Arizona test were used also in a nationwide test of high school students conducted by Harris and Associates for the National Council on Economic Education (Harris and Associates, 1999). On these six questions, Arizona students averaged 59 percent correct responses; students nationwide averaged 49 percent. A similar comparison, based on responses to six questions used in an assessment conducted by the Minneapolis Federal Reserve Bank (The Region, 1998), showed Arizona students outperforming a national sample of adults (34 percent versus 31 percent)(See Figure C).

Factors

Factors Associated with the Knowledge Results

The scores become more informative when they are broken down by reference to factors associated with higher and lower test-score levels.

1. Coursework in economics. High school coursework in economics correlates with Arizona students' knowledge of economics. While the average score for Arizona students who had not taken economics was 37 percent, the score for those who had taken an economics course or were enrolled in one concurrently at the time of the test was 50 percent (See Figure D). The score for those concurrently enrolled in an economics course was 55 percent.

2. Coursework taught to high standards. Currently the only curricular standards for economics courses in Arizona are the standards associated with the College Board's Advanced Placement Economics program (AP) and with the economics course designed for inclusion in the International Baccalaureate program (IB).

Arizona students enrolled in these two programs performed well on the test of economic knowledge. AP/IB students averaged 72 percent on the knowledge test as a whole (questions 1-14), 82 percent on the six questions from the NCEE test, and 70 percent on the six questions from the Minneapolis Fed test (See Figure E). On five questions, the AP/IB students averaged scores higher than 85 percent. Only on the emotionally supercharged question involving U.S. firms employing workers in developing countries at low wages did the AP/IB students fail to score higher than other students.

The positive effects of economics coursework taught to high standards become especially evident in light of students' responses to questions related to public policy issues. While only 39 percent of students who had not taken an economics course knew the actual effect of rent controls on the rental properties market, 78 percent of the AP/IB students knew that effect. On the topic of pollution control, only 26 percent of students who had not taken an economics course knew that efficiency requires a comparison of marginal benefits and costs, as compared with 66 percent of the AP/IB students. Even on the somewhat forgotten topic of inflation control, the high standards of the AP/IB courses made a difference. Only 27 percent of students who had not taken a course in economics could identify anti-inflationary strategies correctly, as opposed to 49 percent of the AP/IB students.

3. The impact of prior coursework versus current enrollment.

While those currently enrolled in economics courses averaged 55 percent, those who had previously passed an economics course averaged only 43 percent. While the 43 percent score is higher than the score earned by students who had never taken an economics course, it suggests that some of knowledge gained from a single course

back to top

Summary Part 2

Part 2. Summary of Attitudinal Results about Prices and Markets

One of the basic facts of life is that there is never enough in the world to satisfy everybody--never enough time, never enough goods and services, never enough resources. As a result, some get the goods and some do not. Who gets what depends, of course, on how the goods are allocated. For any given allocation problem, the government could decide how to distribute the goods, goods could be distributed by means of a lottery, distribution could be handled by a rule of first-come-first-served, or people could rely on prices and the market system. Economists argue that the market system provides the best approach to the allocation problem because it creates incentives that encourage people to make the best use of current and future resources. In particular, markets provide incentives for the supplies of goods and services to be increased and for standards of living to improve.

Arizona students were tested on five scenarios designed to assess their understanding of the allocation problem and their attitudes toward various approaches to it. For each scenario, students indicated whether they agreed or disagreed with the proposed use of certain allocative mechanisms including prices and markets, first-come-first-served, a lottery, and government decisions. Students who chose a price or market solution were considered to have faith or confidence in the market. Responses to the scenarios provide a measure of what might be called confidence in the market.

Overall, students' responses to the scenarios suggest that their confidence in the market is very limited. Only three percent of the students agreed "completely" or "with slight reservations" that the price mechanism should be used in every one of the five scenarios. Even in what seems to be the most innocuous of the scenarios (stadium vendors running out of hot dogs before the end of the game), rationing by price was not very popular. Only 20 percent of the students agreed completely with the price option in question 28: "Increase the selling price of hot dogs until sales drop enough so that the vendors stop running out of hot dogs before the end of the game." Adding in those who agreed with slight reservations, the total reaches only 30 percent. In fact, in this scenario the price mechanism was the students' least-favored option. Fewer students chose it than any of the other options, including simply not selling any hot dogs at all during the first three innings of a ball game (See Figures G and H).

Market solutions proved to be more popular for pollution control problems. Forty-seven percent of the Arizona high school students agreed completely or with slight reservations that charging polluters for their emissions would be a suitable approach to pollution control. This compares with 29 percent who agreed that prices should be used to allocate dormitory rooms, 30 percent who agreed that prices should be used to deal with a shortage of hot dogs at ball games, 34 percent who agreed that prices should be used to allocate kidneys for transplantation, and 38 percent who agreed that prices should be used to allocate gasoline during times of shortage (See Figure H).

Factors

Factors Associated with Students' Confidence in Markets

While students generally show low confidence in market solutions, the attitudinal results also become more informative as aggregate responses are broken down by reference to students' prior studies in economics.

1. Taking a course in economics. Taking a course in economics correlates positively with students' confidence in market solutions to allocation problems. Five percent of the Arizona students who had taken a course in economics agreed with the use of market solutions in all five scenarios, as opposed to two percent of those who had not taken an economics course. In four out of the five scenarios, the percentage of those who agreed with the proposed use of market solutions was higher among those who had taken an economics course than among those who had not. Only in the case of allocating kidneys for transplantation was the percentage in agreement with using prices the same among those who had and had not taken an economics course. And even in that case the percentage agreeing without reservation was slightly higher (ten percent versus eight percent) among those who had taken a course in economics.

Students enrolled in an Advanced Placement or an International Baccalaureate program showed the most confidence in market solutions. Twelve percent of the AP/IB students agreed with the proposed use of prices to allocate resources in all five scenarios. And in the respective scenarios, the percentage of AP/IB students agreeing with the proposed use of prices was higher than the percentage for any other group. In fact, for the gasoline and hot dog scenarios, the percentage of AP/IB students agreeing with the proposed use of prices was more than twice that of the students who had not taken an economics course (67 percent versus 30 percent in the gasoline scenario and 51 percent versus 23 percent in the hot dog scenario) (See Figure H).

2. Knowing more about economics. A particularly interesting finding in this survey is that students' knowledge about economics correlates positively with their preference for market solutions to allocation problems. Those who are more economically literate are more apt to prefer market solutions. In all five scenarios, students who scored in the top 15 percent on the knowledge test (questions 1-14) agreed with the proposed market solutions more often than those who scored lower on the test. For example, 61 percent of those scoring in the top tier agreed that charging polluters is an appropriate way to control pollution, while only 38 percent of the lower-scoring students agreed with that approach. Nine percent of the top scorers agreed with market allocation in all five scenarios, as compared with three percent among the lower-scoring students.

Conversely, students who know more economics are less likely than others to believe that government should solve allocation problems. Three scenarios (1, 2, and 5) suggest a government solution to an allocation problem. Of the Arizona students who scored in the top 15 percent on the economics knowledge test, only five percent agreed with the proposed government solution in all three cases; nine percent of the lower-scoring students agreed with the proposed

back to top

Summary and Discussion

The results show that Arizona students are not economically literate. In response to questions related to basic economic concepts including scarcity, incentives, self-interest and the common good, marginal analysis, and inflation, two-thirds to three-fourths of all students tested performed poorly.

Poor performance on the knowledge portion of the test is matched by the students' weak confidence in market solutions to allocation problems. Even in the case of allocation problems implying no highly charged issues of equity, most students favored non-market solutions.

When scores are broken down, however, four important points of interest stand out. The first point has to do with the impact of taking an economics course in high school. Students who had taken an economics course or were concurrently enrolled in one at the time of the test scored higher than students who had not taken an economics course.

The second point has to do with the impact of curricular standards. Students who had taken high-standards economics courses (in Advanced Placement or International Baccalaureate programs) scored higher, consistently, than all other groups of students.

The third point has to do with the impact of economic knowledge on students' confidence in market solutions to resource allocation problems. While most students preferred non-market solutions to problems posed in the five scenarios, the preference for market solutions was markedly higher both among students who performed well on the knowledge test and among students enrolled in an Advanced Placement or International Baccalaureate economics course.

Finally, students did not retain some of the information learned in a single high school course. Economic education needs reinforcement.

In light of this pattern of differentiated results, the implications for K-12 curricular policy seem clear. Improved rates of economic literacy in Arizona are within reach, provided that the state moves forward to ensure an opportunity for students to study economics throughout the K-12 curriculum. For optimal results, the statewide plan should provide for a sequenced, cumulative program of study, culminating in a required 11th or 12th grade capstone course. In students' early studies and the capstone course, instruction should be oriented to high academic standards, as exemplified in economics courses designed for use in the Advanced Placement and International Baccalaureate programs.

Finally, Arizona teachers must receive professional development in economics and personal finance education to implement Arizona's economics standards. For this reason, the Arizona Council on Economic Education and the Offices for Economic Education at the University of Arizona and Arizona State University are ready to offer a statewide plan to provide economic education for all teachers. Educating teachers should have a multiplier effect on all the public, charter, and private school students in Arizona. Arizona teachers also must receive the professional development to teach economics effectively.

References

Clow, John E. National Survey of Economic Education. New York: National Council on Economic Education, 1999.

Harris, Louis, and Associates. The Standards in Economics Survey. New York: National Council on Economic Education, 1999.

Meszaros, Bonnie, and Siegfried, John. Voluntary National Content Standards in Economics. New York: National Council on Economic Education, 1997.

"The Minneapolis Fed's National Economic Literacy Survey," The Region, Dec. 1998.

back to top