Education and Intelligence--Part 2

 The National Assessment of Educational Progress (NAEP)--the collection of exams at various grade levels that beganin the 1970s-- is usually cited as evidence of how well the nation’s students are doing over time. This involves comparing, say, the cohort of students in one year on the 8^th grade level exam with another cohort of students four years later on the same 8^th grade exam. (Hence the name “The Nation’s Report Card.) Beginning with the 1984 assessment, however, changes were made that enabled NAEP to assess the educational achievement of cohorts at different grade levels on a common scale (i.e., the 4^th, 8^th, and 12^th grade levels). This is a different but equally important measure of how well the schools are educating the nation’s students. There have also been similar studies of academic gains using the High School and Beyond (HSB) database.

While there is disagreement among experts about how to grade the nation’s schools on how well it is doing this job, there is no disagreement among the experts that schooling does make a contribution to cognitive and academic development. I give below some of the main findings:

• On NAEP’s reading assessments, 8^th grade students score about 42 points higher than 4^th grade students on the common scale. The value of the standard deviations for the individual mean scores is typically about 35. This amounts to a gain of about 1.2 standard deviations over a four-year period, or about .30 standard deviations of gain per year. (See the NAEP Data Explorer for other comparisons.)
• Alexander et al. (1985) analyzed the High School and Beyond (HSB) database as well as NAEP. Their best estimate of the improvement in academic growth from late elementary school through high school is about 0.22 of the standard deviation, or about 9 percentile points per year.
• By using data from the HSB database, which included data on dropouts, Alexander et al. (1985) were able to compare the achievement of high school students who completed their senior year with students from the same cohort who had dropped out after the sophomore year. Their estimate of the year-to-year academic growth that was attributable to school alone was about 0.10 SDU (standard deviation unit).
• A later study by the same group (Natriello et al. 1989) refined this result by comparing the academic achievement of students who dropped out before graduating with students who did graduate, broken down by academic, general, or vocational tracks. They found that students in all three tracks who remained in high school to graduate made achievement gains, but the students in the academic track—i.e., those who were tracked in a traditional, liberal arts high school curriculum—made the largest gains. This parallels the more recent findings from the Collegiate Learning Assessment that students concentrating in math and science, engineering, the social sciences, and the humanities outperform students concentrating in business, education, and social work.
• There is evidence that growth in cognitive and academic skills varies across socioeconomic and racial and ethnic groups, with minorities and low SES-background students being the biggest beneficiaries. (It would therefore appear to be especially perverse to deny these students the benefit of schooling at any level, and dropping out is particularly harmful to them.)
• Admittedly, the gains in academic and cognitive skills that are attributable to schooling are modest; nevertheless, they are real and important, particularly when viewed cumulatively. If we accept the Alexander et al. estimate—which might be low—that schools contribute 1/10 of a standard deviation to the growth of students' test scores, then over ten years the cumulative impact amounts to one full standard deviation (SD) of difference. Another way of assessing the importance of schooling to growth in academic and cognitive skills is to look at the same data using percentile point gains. In one-year comparisons, the growth looks modest; but the importance of schooling becomes apparent when longer time spans are considered. As John Ralph et al. (1994) have pointed out, 91% of the scores will match the scores of students one year later. This involves a considerable overlap; consequently, it would be difficult to distinguish between two groups of students one year apart on the basis of their test scores alone. However, the same analysis tells us that the overlap over three years will be 74%, and that the overlap will fall to 58% five years later. Growth in academic and cognitive skills does occur in school, and it is cumulative.
• As I pointed out previously, there is disagreement about how cognitive ability, academic ability, intelligence, and IQ—assuming they are not the same—are related to each other. This is not the place to enter into that controversy. I wish only to point out that some estimates of IQ gains from schooling look very similar to the Alexander et al. estimates of academic gains on the HSB and NAEP. Christopher Jencks has estimated that a student loses one IQ point for each year of missed school. (Cited by S. J. Ceci 1996: 80.) Keeping in mind that IQ tests are normed at a mean of 100 and a standard deviation of 15 points, it is of some interest to compare this estimate with the estimate of an increase of 1 standard deviation of improvement in academic achievement on NAEP and HSB over a 10 year period by Alexander et al. (and somewhat better than that for students on an academic track).

It is clearly a mistake to regard an individual’s IQ, academic aptitude, or cognitive skills as static. We all get smarter on all these measures as we get older, until our 30s, when all of them begin to decline. (That is why IQ tests have to be age-standardized.) At present, we know virtually nothing about how or why this happens. This is as true of the gains that occur outside of school as it is of the ones that occur in school. However, only a mystic could believe that we will never be able to understand why or how all this happens. As we get a better understanding of the phenomena, there is every reason to think that schools will be able to do a better job of producing gains in these abilities.

 

The similarity between NAEP and the CLA

I have not discussed the sample 8^th grade reading assignments from the NAEP that Murray covers in some detail in Real Education; I will discuss the 12^th grade level reading assignment instead. A look at the 8^th grade reading assessment shows its similarity to the Collegiate Learning Assessment (CLA ) performance task, but the 12^th grade test for the NAEP is even closer. It is a more advanced test, and therefore closer to the CLA assessment that is given to entering college freshmen. The performance tasks on the CLA and the 12^th grade NAEP reading tests also resemble each other more closely.

 

To see the similarities between NAEP and the CLA, and why one might expect them to yield fairly similar results, consider the performance tasks in the reading assignment for the 12^th grade NAEP with a sample performance task for the CLA. (The sample reading questions given below are taken from the 12^th grade NAEP reading assignment that is available here, beginning on p. 27):

 

Sample Reading Questions

Reading to Perform a Task

Grade 12 (p. 4)

 

The performance task begins with three pages from the "Bargain Basement" section of the classified ads in a daily newspaper. A wide range of items is given, classified by "$25 and under” and "$26 to 100.” Instructions for submitting an ad are not given in these three pages. They are given, instead, in the form that the student is required to fill out in Item 7. (This forces the student to return to earlier test items after he has seen the form that is given in a later item.)

 After the three pages of classified ads, the student is then given the following questions:

 1. The Bargain Basement ads are divided into sections identified by the headings "$25 and under" and $26 to $100." Suggest another way that this information could be organized and what the advantages would be.

 

2. Which three types of information are usually found in these classified ads?

 

A. Original cost, age of item, size of item.

B. Item description, home address, phone number

C. Phone number, item description, cost of the item

D. Item condition, seller's name, time of day to call

 

3. It is possible to place a free ad in the Bargain Basement section. If you want to place a free ad, your times must be:

 

A. sold within five days

B. priced at $25 or less

C. in good condition

D. inspected by the editor

 

4. Suppose you want to buy a bicycle. Look at the ads for bicycles listed in the Bargain Basement section. Tell which advertised bicycle interests you the most. Explain how you used the information in the ads to make your selection.

 

5. Abbreviations in the ads are useful because they:

 

A. Communicate information while saving space

B. Allow for different interpretations

C. Make each section more interesting

D. Make the items within the section appear to be smaller

 

6. What is an acceptable way to place a $1 Bargain Basement ad in this newspaper?

 

A. Phone in the ad, pay by credit card.

B. Phone in the ad, pay by money order.

C. Mail, the ad, pay by cash.

D. Mail the ad, pay by check.

 

 

8. Suggest a way to improve the ordering form for Bargain Basement classified ads.

 

9. What are three possible mistakes in writing an ad that could prevent it from being published?

 

Now compare this with a sample college-level performance task for the CLA given in the CLA brochure (p. 11):

 

SCENARIO

 

You advise Pat Williams, the president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTech’s sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers. Pat was about to approve the purchase when there was an accident involving a SwiftAir 235.      

 

DOCUMENT LIBRARY
 

• Newspaper article about the accident
• Federal Accident Report on in-flight breakups in single-engine planes
• Internal Correspondence (Pat’s e-mail to you & Sally’s e-mail to Pat)
• Charts relating to SwiftAir’s performance characteristics
• Excerpt from magazine article comparing SwiftAir 235 to similar planes
• Pictures and descriptions of SwiftAir Models 180 and 235

 

QUESTIONS

 

• Do the available data tend to support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups?
• What is the basis for your conclusion?
• What other factors might have contributed to the accident and should be taken into account?
• What is your preliminary recommendation about whether or not DynaTech should buy the plane and what is the basis for this recommendation?

 

The CLA’s performance task is considerably harder than the one in the sample exam from the 12^th grade NAEP reading assignment. This is appropriate, since the CLA is given to college students and the NAEP is given to a representative sample of the nation’s 12^th graders. The CLA requires the student to collate, organize, and analyze a number of different documents, and the analytical tasks involved in reading and analyzing the individual documents are more difficult. But both the NAEP and the CLA test the student for fundamental cognitive skills and for her ability to transfer skills learned in the classroom to new situations and contexts. Both tests involve testing whether academic skills have been developed in the classroom that can then transferred or used in new situations. Pre- and post-test comparisons using the NAEP and the CLA should therefore produce similar results, assuming that the secondary and higher education sectors in this country are doing equally well at developing students’ academic and cognitive skills at their respective levels.