Section 1

How Reading Performance Is Measured and Reported

Beginning with the elementary-level tests operated under the rubric of the Pupil Evaluation Program, and continuing through high school with the Regents exams, New York State has one of the nation's oldest and most extensive standardized testing programs. In the wake of A Nation At Risk, the landmark 1983 report of the National Commission on Excellence in Education, New York was one of the first states to link test scores more directly to school improvement and accountability efforts.

For nearly 20 years, the reading component of New York's PEP system has been implemented through a test called Degrees of Reading Power (DRP), which measures the reading abilities of students at crucial stages in their schooling. Assisted by a grant from the Carnegie Foundation, the Education Department worked closely with local educators and reading experts to develop the DRP during the late 1970s. The test is now published by Touchstone Applied Science Associates (TASA) of Brewster, New York.

DRP reading tests are administered every spring to all third- and sixth-graders in the state, with the exception of a relatively small number of severely disabled children and recent immigrants who are classified as having Limited English Proficiency (LEP).⁽¹⁾

What the test measures

The DRP test, according to its developers, is designed to determine how well a student reads under "real-life" conditions. Each reading test consists of 300-word non-fiction prose passages on randomly selected topics, arranged in order of difficulty beginning with the easiest material. Seven words are deleted in each passage, as indicated by an underlined blank space. For each deletion, pupils must choose the correct answer from among five single-word response items.⁽²⁾

Most states administer standardized tests that report reading scores compared to a "grade level equivalent," or statistical norm. However, the New York State reading test is scored on a fixed scale of "readability," measured in Degrees of Reading Power units, that theoretically ranges from 0 to 100. In practice, the readability of most materials encountered at school or in the workplace ranges from 30 to 85, according to the test's developers.

Raw scores on the reading test always translate into a score of DRP units. The higher the DRP unit value, the more difficult the level of material the student is able to comprehend.

For the PEP test, the Board of Regents established a minimum standard of basic competency, also known as the "State Reference Point," equal to the estimated reading difficulty level of representative textbook samples for that grade. A pupil who passes the third-grade test is presumed capable of reading and understanding the average third-grade text in an "instructional" setting -- i.e., with the assistance of a teacher. Any pupil who scores below the minimum is required by the state to receive remedial reading instruction.

The state also has established a higher benchmark, the so-called mastery level, to "measure progress in achieving excellence."⁽³⁾ The mastery-level score in third grade is equivalent to the minimum competency level for sixth grade. In sixth grade, the mastery level score corresponds roughly to the reading difficulty level of a typical mass-circulation periodical or daily newspaper -- again, in an "instructional" context. Judged at an "independent" level, mastery in the sixth grade merely indicates an ability to fully comprehend a typical sixth-grade textbook without the assistance of a teacher.

The state reading benchmarks, including the mastery levels, do not appear especially high or demanding by most standards. They certainly are no more demanding than the informal standards in place at the last turn of the century, when New York was absorbing a previous wave of poor, often illiterate immigrants and their children.⁽⁴⁾

Students scoring just below the minimum on New York's third-grade test are able to read a passage like this:

"Bears are big. They need a lot of food. Bears eat meat. They eat bugs. They eat berries. They eat honey. They eat fish, too. Bears feed in the spring. They feed in the summer. They feed in the fall. Bears look for food then. They fish."

This, as Commissioner Mills points out, is "an example of low expectations."

"Almost everyone should be reading at that level or above in the third grade," he says.

But in hundreds of high-minority public schools, nearly half the pupils are not.

What the public knows

Although the Pupil Evaluation Program has been administered in varying forms for many years, results were not widely shared outside the education system until the Regents began requiring each district to file a public Comprehensive Assessment Report (CAR), beginning in 1988. Publication of the test results immediately underscored the separate-and-unequal duality of the state's school system.

On the one hand, more than 90 percent of pupils in most suburban schools easily surpassed the low minimum competency standard on both the third- and sixth-grade reading tests. (And until The Public Policy Institute and other critics documented in the early 1990s how ludicrously low this standard was, many of these districts cited this performance as evidence of their "excellence.")

Meanwhile, it became more obvious than ever that unacceptably high percentages of pupils in high-minority schools -- most located in New York City -- were failing to meet these low standards.

Low test scores formed the basis for the call to action in then-Education Commissioner Thomas Sobol's 1988 annual report to the Governor and the Legislature, which cited "a dramatic pattern of school failure among a substantial portion of our school population." "The students whom our system fails are disproportionately poor and minority," Sobol noted.

Beginning in 1989, Commissioner Sobol and the Regents stepped up their efforts to deal with school failure by inaugurating a "Registration Review" process focused on the lowest-performing schools.

Registration review is intended to be a form of intensive care for the system's educational basket cases. Once placed on the review list, a school is visited by a team of teachers, board of education members, curriculum specialists, SED staff and "other education experts."⁽⁵⁾ The visit results in a report to the school and the district including recommendations for improvements. The school district must then submit a corrective action plan, which is revised annually.

But of the 139 schools placed under registration review since 1989, only 40 have improved enough to be removed from the list. Another 13 schools first added to the list in 1989 have been "redesigned" by the New York City schools chancellor to avoid closure by the Education Department.

Our study's approach

To document the results in the schools serving minority children, The Public Policy Institute undertook the first independent, computer-aided analysis of the Education Department's statewide database of test scores for public and non-public schools in New York.

The analysis was made possible when Commissioner Mills ordered that the data be released to us in computer-readable format. Our study covers a seven-year period -- from 1989-90, the earliest school year for which data are available, through 1995-96, the latest school year for which statewide test scores had been reported as of September 1997.

1. Through 1996, LEP exemptions were granted only to children who had attended school in the United States for less than 20 months. Effective 1997, the LEP exemption from the reading test has been broadened to include all pupils who score below the 40th percentile on another measure.

2. DRP Handbook, TASA, p. 7. A "prototype" or sample test appears in Appendix II.

3. New York: The State of Learning, A Report to the Governor and the Legislature on the Educational Status of the State's Schools, February 1997, Vol. 1, p.109.

4. According to the DRP test developer, the reading difficulty level of a typical sixth-grade textbook today is roughly equivalent to that of the fourth-grade McGuffey's Eclectic Reader of 1909.

5. New York: The State of Learning, A Report to the Governor and the Legislature on the Educational Status of the State's Schools, February 1997, Vol. 1, p. 200.

			NEXT