Challenging Conventional Wisdom: Four Moments in the Research Career of James S. Coleman

Russell K. Nieli

This year marks the fiftieth anniversary of the publication of Equality of Educational Opportunity (EEO), the massive study of over 4,000 public schools and 600,000 public school students in America that many describe as the most important study of the U.S. public school system in the twentieth century.1 Popularly known as the “Coleman Report” after its lead writer, sociologist James S. Coleman, it was authorized as part of the 1964 Civil Rights Act and was intended to provide policy makers in Washington and elsewhere with solid empirical data on educational achievement within America’s vast system of public schools. The fate of black school children within racially integrated and all-black schools was a particularly important concern of the study.

The report was a landmark in its use of quantitative statistics and in the enormous scope of its data basis, and reflected the coming-of-age of modern computer-facilitated data storage and data analysis. But the Coleman Report was also a landmark in terms of the use to which highly quantitative social science could be put to influence a national debate on controversial issues of public policy. Many books before EEO had influenced the way public policy issues were debated in America—Gunnar Myrdal’s An American Dilemma (1944) and Michael Harrington’s The Other America (1962) being striking examples2—but the Coleman Report begins the period when sophisticated policy analysis, carried out by social scientists trained in the gathering and interpretation of quantitative data, begins to edge out in public influence works by sociologists and public intellectuals less tethered to databases and sophisticated survey research methodologies. Coleman’s study would set the pattern for future influence by prominent policy analysts, including Christopher Jencks, Charles Murray, William Julius Wilson, Eric Hanushek, and many more.

EEO, and the conclusions many drew from it, also represents one of four broad phases in James Coleman’s evolving thought on education in America. Some of the conclusions he and others culled from his EEO data, particularly the idea of enhancing black education by busing white children to black schools, would be greatly modified or abandoned in the light of Coleman’s future research and experience. As the University of Chicago Chronicle put it in his obituary: “Coleman never shied away from controversy. He was also known for having the courage to change his stance on issues in light of new data.”3 A look at each of Coleman’s four broad phases, spanning almost thirty years, follows.

Football Stars and Beauty Queens Alter the Learning Environment

Coleman was a product of Columbia University’s sociology department in the immediate post-WWII era, when it turned out many rising stars in American sociology. Other department alumni included such luminaries as Daniel Bell, Nathan Glazer, Seymour Martin Lipset, Alvin Gouldner, and Peter Rossi. Columbia professors Peter Lazarsfeld, who pioneered the use of data analysis to explain American voting patterns, and Robert K. Merton, a master of an older, less quantitatively-oriented style of sociology, were the two most important influences on Coleman’s thinking. Throughout his research career Coleman tried to combine the professional emphases of these mentors, gathering extensive quantitative data while maintaining a sense of the larger sociological picture.

Coleman’s first book, The Adolescent Society: The Social Life of the Teenager and Its Impact on Education (1961), was a study of ten public high schools in Illinois based on extensive interviews of and questionnaires completed by students, teachers, school administrators, and others during 1957 and 1958.4 A reading of the book today doesn’t suggest that Coleman discovered much about the values and atmosphere of the typical American high school that wasn’t already known to the general public, but that would be an anachronistic projection. Until World War I most public school children left school after eighth grade, usually to begin work. Only a small minority went on to high school, and high school attendance only became a mass phenomenon in America in the 1920s and 1930s. Until that time, the general image of the American public school was formed by the ubiquitous elementary school, despite the vast difference in the internal values and attitudes that began to distinguish primary from secondary education in America.

The Adolescent Society brought home like no previous work how a new type of “adolescent society” had developed in America, one centered around the public high school, and one whose values in many ways were independent of, and often antagonistic to, the values educators and school administrators sought to impart to students. Whereas the major culture-forming influences of the elementary school were largely those of teachers and school principals, those of the high school, Coleman discovered, were largely those of teenage peer groups within these schools. And these influences often took an anti-academic cast whereby academic values were clearly subordinate to the values of popularity, sports, social life, and teenage sex appeal.

Coleman found that outstanding students occupied a lowly place within the school status-hierarchy studied, while football stars, pretty women, and the socially most popular kids held the highest rank. The dominant values of the teenage peer culture were thus at odds with the aims of the school’s educators, a particularly troubling situation given the desire for peer acceptance and the power of social conformity pervading high school culture. While the situation was not as dire as more contemporary accounts of the “acting white” phenomenon in certain all-black schools, Coleman and many who read his study were alarmed at the situation described. It wasn’t clear, however, what could be done, since “the adolescent society” had taken on a life and independence that seemed impervious to the values and admonitions of the surrounding adult world. The American high school, Coleman reported, had become a largely self-contained and insular sociological unit. Coleman’s findings challenged conventional wisdom and identified a problem, but it was not clear what, if anything, to do to address it.

Coleman subsequently proposed school-versus-school academic competition to enhance the status of high school intellects, in the manner that athletic competition between schools enhanced the status of football stars, but he later acknowledged the idea as quixotic. The brains in high school just had to accept their status as greatly subordinate to that of the football stars and pretty cheerleaders.

Most Measurable School Inputs Don’t Matter but Racial Composition Does

While The Adolescent Society enhanced Coleman’s reputation among a small circle of educators, it was his choice in 1965 to lead the massive study of the nation’s public schools by the U.S. Office of Education that brought him national attention. Equality of Educational Opportunity involved a mammoth exercise in data gathering from an enormous number of American public schools, with extensive questionnaires completed by the major participants in each school—72 items for teachers, 100 for school principals, and 116 for high school students. No survey of the nation’s public schools, either before or since, has ever attained such a high level of thoroughness or specificity.

A primary goal of the study was to improve educational opportunities for struggling racial and ethnic minorities. It was these groups, the report indicated, who are at the greatest disadvantage in terms of “developing mental skills and imparting knowledge” (36). Five such groups were singled out for the report’s special attention: “Negro Americans, Puerto Rican Americans, Indian Americans, Mexican Americans, and Oriental Americans” (36). Among these groups, the Negro Americans (who had not yet come to prefer the designation “African American” or “black”) were singled out for particularly fine-grained analysis and data gathering.

Many surprises were in store for the Coleman researchers. One of the biggest: Once other factors were entered into the regression models, such things as the quality of a school’s physical plant, the average number of students in a class, the per-pupil student expenditures, the size of a school’s library, and the salary of the school’s principal didn’t seem to make much of a difference in student performance on standardized tests. These factors, which many had believed were the real movers and shakers in terms of a school’s educational output, were shown not to vary nearly as much as previously thought among schools attended by students of differing ethno-racial backgrounds, and were not, in any case, terribly important in explaining why some individuals and some groups did so much better than others.

Even in the rural South, where black education throughout the later nineteenth century had been practically ignored in terms of public funding, black/white differences in school factors were modest. While black elementary school students in the rural South still attended largely all-black schools with more students per classroom than their white counterparts—thirty-three per classroom for blacks versus twenty-six for whites—this difference was not enormous. On other important measures, too, there was very little difference. In some cases, such as the newness of the school building, the statistics favored rural Southern blacks. (This was no doubt due to the concerted effort of many Southern states, beginning in the 1940s, to avoid threatening litigation under the “separate but equal” doctrine by funding black education much more generously.)

Regarding how much students learned as measured by their grades on standardized tests, the most important factor by far the Coleman researchers found was the quality of the students in the school: their individual family background (socioeconomic, educational, racial) and the background of their classmates’ families. “Thus the large part of school-to-school variation in achievement appears to be not a consequence of effects of school variations at all, but of variations in family backgrounds of the entering student bodies,” the report noted (296). “The school appears unable to exert independent influences to make achievement levels less dependent on the child’s background—and this is true within each ethnic group, just as it is between groups” (297).

The pattern of ethno-racial group achievement that the Coleman researchers found has become all too familiar to us today: of the six ethno-racial groups studied, whites and Asians (“Orientals”) were far ahead of the other four groups on most measures of academic performance, with blacks usually occupying the bottom position. And the usual remedies proffered for improving the performance of lower performing students—smaller classes, higher teacher salaries, better curriculums, etc.—provided little hope for improvement. Improving the academic achievement of black and other minority students seemed a hopeless goal, at least under the American school system as it then currently operated. “[E]quality of educational opportunity through the schools must imply a strong effect of schools that is independent of the child’s immediate social environment,” the Coleman team wrote, but such a strong independent effect “is not present in American schools” (325). The “inequalities imposed on children by their home, neighborhood, and peer environment are carried along to become the inequalities with which they confront adult life at the end of school” (325).

Busing Leads to White Flight and Undermines Integration

While it is possible for pessimists to read the report as concluding that nothing will ever work, the Coleman team did show that minority children, especially blacks, perform noticeably better in schools with large numbers of white students. This was attributed to “the better educational background and higher educational aspirations that are, on the average, found among white students” (307). This was the one conclusion of the Coleman report seized upon by the mass media and much of the nation’s left-liberal elite, including many in the federal judiciary. It soon became the basis for a kind of crusade by the federal courts to bring white and black students together in the same schools, even if it required radically redrawing neighborhood district lines and transporting students long distances to achieve the desired racial mix. The Great Busing Wars of the late 1960s and early 1970s followed.

Initially Coleman seems to have been in favor of the drive to bring whites and blacks together even via the “forced busing” that opponents of these policies so passionately resisted. Later, he changed his mind completely in the face of new evidence. The catalyst for this was a study Coleman completed in 1975 on behalf of the Urban Institute on changes in the racial composition of big-city schools that had occurred between 1968 and 1973. Using a database gathered by the U.S. Civil Rights Commission, Coleman and two research assistants uncovered some truly alarming trends, which were discussed in 1975 in Trends in School Segregation, 1968–73.5 Efforts to desegregate urban schools through compulsory school reassignments—in particular requirements that whites send their children to schools in black neighborhoods—had completely backfired as they greatly accelerated the movement of whites to all-white suburban schools or private academies. The busing crusade spurred “white flight,” and the result was greater, not less, racial segregation in many of the nation’s large cities.

In merely documenting this high degree of white flight, Coleman overnight became a pariah to elements of the political Left that had previously embraced with great eagerness his earlier discoveries about the value of blacks attending schools with white majorities. So intense became the hatred of Coleman in certain quarters that a movement arose among fellow sociologists to expel him from the American Sociological Association, though the movement ultimately failed.6 Reflecting on the heated reaction to his white flight research, Coleman remarked years later:

The policies of school desegregation in central cities had acted to increase racial segregation between cities and suburbs….Most serious, analysis showed that loss of whites was more extensive where central-city desegregation had been most intensive….This, as one might imagine, was not welcome news, especially to those still working to extend those policies, and especially coming from a researcher who, since EEO, had been counted as an ally in the social movement to racially integrate America’s schools. The familiar tendency to “kill the messenger” led to another period of controversy, which made the earlier controversy over EEO pale by comparison.7

Catholic Schools Do It Better

The net effect of Coleman’s research on school desegregation and the performance in the public schools of ethno-racial minorities, especially blacks, seemed to encourage pessimism and despair. No generally valid formula for improving the performance of the lower performing groups within America’s public school system was found. Subsequent work by educational researchers in search of “effective schools,” where black and Hispanic youth progressed substantially beyond the national norms for their group, did locate several schools that met this criterion. But they were almost always the combined product of unusually forceful leadership at the top (often the academic equivalent of a General Patton), a cooperative school board, and cooperative and involved parents and parents organizations. Such combinations were extremely rare and not easily reproducible.

One little-researched area where such combinations were not rare existed, however, and in 1980 Coleman got the opportunity to extend his educational research beyond the public school arena to the realm of Catholic and other private schools. The results of his investigation were eye-opening to many and once again involved Coleman in controversy—although this time the controversy did not so much involve left-wing social warriors as the entrenched interests of public teachers unions and other defenders of state-dominated education.

Analyzing 70,000 high school sophomores and seniors, including those in Catholic schools and non-Catholic private schools as well as those in public high schools, Coleman and his research staff found that Catholic schools stood out in their ability to teach effectively students from lower-class black and Hispanic families and from single-parent homes. And contrary to widespread belief, the Catholic schools were not more racially segregated than the public schools. In Public and Private High Schools: The Impact of Communities (1987), Coleman and his research assistant Thomas Hoffer explained why the Catholic schools—and other private schools with similar features—were so much more effective than the public schools in teaching kids from lower-achieving racial and ethnic minority groups.8

Although this was not mentioned by Coleman and Hoffer, the Catholic schools had long experience with hard-to-educate children from troubled neighborhoods and homes, having extensively dealt with the challenge of educating the “Wild Irish” during the nineteenth and early twentieth centuries—the “problem minority” of an earlier era.9 The “Catholic school model” evolved largely as a creative adaptation to this earlier challenge, and much of what was learned applied with minor changes to the challenge of educating lower-class blacks and Hispanics in more recent times.

The secret of Catholic school success was explained by Coleman and Hoffer as a result of the school’s being embedded in a fairly homogenous community, with its various community members, including parents, teachers, students, and principals, all committed to the stated goals, values, and teaching style of the school. This enabled the school to attain a general coherence, along with superior student discipline, high level of student involvement, and agreed-upon sense of mission that was often lacking in the public schools. Coleman and Hoffer offer this pithy summation of the dilemma of the public schools:

A principal of a [public] school today in which attendance is based on residence has no set of dominant community values to uphold. Instead, there are a number of contending values, each claiming legitimacy, and at least some of them capable of being backed up by legal suits in court.…In the same school, half the parents might wish the school required their children to wear uniforms, while another parent might sue the school for disallowing her daughter’s miniskirt in school. Many parents might wish the school would require more homework, while others might demand that homework outside school hours be abolished.…A principal with strong personal force can make a particular set of values dominant within the school. But the potential for challenge to such values is always there….The principal no longer has the strength of a tightly knit functional community to support authoritative actions and may easily be defeated by such a challenge, with authority undermined from that time forward.10

As with his anti-busing conclusions, Coleman in his criticism of the public schools—and general praise of Catholic and other private schools—was often viciously attacked, some even condemning his decision to devote valuable time and energy to studying private schools rather than exploring ways to improve the nation’s public school system. In a 1981 New York Times opinion piece, Coleman answered such critics bluntly: “When there are many people in positions of power and influence giving verbal support to public schools but sending their own children to private schools, social research can be valuable in opening up an issue that hypocrisy has sealed.”11

Just as in Coleman’s other books, Public and Private High Schools contained no specific policy recommendations. But it was not lost on anyone that the policy implications of his newest study supported those calling for voucher and tuition tax credit plans that would foster the growth and expansion of private schools, Catholic or otherwise, and enable those from poorer families to have a greater choice in the schools they attended. Coleman’s admirers were now drawn from the ranks of religious traditionalists, social conservatives, ethnic and religious minorities, and economic libertarians, in contrast to the leftist social engineers and judicial activists who had previously so admired Equality of Educational Opportunity. Coleman himself, never the ideologue and never starting out with a preconceived agenda, always believed he was just trying to uncover the truth.

Conclusion

James S. Coleman was an honest man and an honest researcher in an arena strewn with ideological minefields and the overbearing presence of left-wing social warriors, self-interested teachers unions, public school bureaucrats, ed school ideologues, and various other purveyors of a stultifying conformity and mindless political correctness. Coleman had the courage to strike out on his own and discover just how often conventional wisdom is wrong. Some will criticize him for not addressing the now popular genetic-based theories of IQ and academic intelligence and how they may differ among racial and ethnic groups—i.e., the theories of such researchers as Arthur Jensen, Richard Lynn, J. Philippe Rushton, and the authors of The Bell Curve. But Coleman faced enough controversy and hostility for one lifetime. It would be churlish to fault him, a sociologist, for not delving into the complex and often bewildering field of psychometric studies, an area in which he had little professional training and expertise, and therefore less to contribute.

Nobel Laureate Gary Becker, who knew Coleman for many years when they were both on the University of Chicago faculty, should have the last word here:

Exceptional ability, fertile imagination and the courage to go against received opinion and to bear sometimes vicious attacks marked his distinguished career and explain his enormous contributions to social science.12


1 James S. Coleman, Equality of Educational Opportunity (Washington, DC: U.S. Department of Health, Education, and Welfare, Office of Education, 1966), http://files.eric.ed.gov/fulltext/ED012275.pdf. All further references to this work are cited parenthetically within the text.

2 Gunnar Myrdal, An American Dilemma: The Negro Problem and Modern Democracy (New York and London: Harper Brothers Publishers, 1944); Michael Harrington, The Other America: Poverty in the United States (New York: Macmillan, 1962).

3 “Obituary: James Coleman, Sociology,” University of Chicago Chronicle, March 30, 1995, http://chronicle.uchicago.edu/950330/coleman.shtml.

4 James S. Coleman, The Adolescent Society: The Social Life of the Teenager and Its Impact on Education (New York: Free Press of Glencoe, Illinois/Crowell-Collier Publishing Co., 1961; New York: Praeger, 1981).

5 James S. Coleman, Sara D. King, and John A. Moore, Trends in School Segregation, 1968–73 (Washington, DC: Urban Institute, 1975), http://eric.ed.gov/?id=ED117252.

6 To the great credit of that organization, Coleman in 1991 was elected president of the American Sociological Association in appreciation of his lifetime work in the sociology of education.

7 James S. Coleman, “Reflections on Schools and Adolescents,” in James S. Coleman, ed. Jon Clark (Washington, DC: Falmer Press, 1996), 21.

8 James S. Coleman and Thomas Hoffer, Public and Private High Schools: The Impact of Communities (New York: Basic Books, 1987).

9 On the Irish immigrants as the “problem minority” of an earlier era, see the incomparable essay by Daniel Moynihan on the Irish in Nathan Glazer and Daniel Patrick Moynihan, Beyond the Melting Pot, 2nd ed. (Cambridge, MA: M.I.T. Press, 1970), 217–87.

10 James S. Coleman and Thomas Hoffer, Public and Private High Schools: The Impact of Communities (New York: Basic Books, 1987), 11, 16.

11 James S. Coleman, “Schools of Thought,” Opinion, New York Times, June 30, 1981, http://www.nytimes.com/1981/06/20/opinion/schools-of-thought-by-james-s-coleman.html.

12 Quoted in “Obituary.”

  • Share
Most Commented

October 18, 2023

1.

Fleeing Tension by Fleeing Classics

Mark Adair avers that we may have reduced the role of the classics in university curricula simply because it is too hard....

Most Read

May 30, 2018

1.

The Case for Colonialism

From the summer issue of Academic Questions, we reprint the controversial article, "The Case for Colonialism." ...

July 2, 2020

2.

In Humans, Sex is Binary and Immutable

The idea that there are more than two sexes in human beings is a rejection of everything biological science has taught us. Unbelievably, this idea is coming directly from within the highest......

March 18, 2022

3.

The Case for Colonialism: A Response to My Critics

Political scientist and NAS board member Bruce Gilley’s article “The Case for Colonialism” (republished in Academic Questions in the summer of 2018), has been the subject o......