Developing the National Reporting System

The Head Start Bureau has consulted with experts and Head Start leaders in developing and planning the NRS. by Tom Schultz

     SINCE THE ANNOUNCEMENT of the National Reporting System (NRS) in the President's Early Childhood Initiative— Good Start, Grow Smart— in April 2002, the Head Start Bureau has been working to develop the best possible design for this important new effort.

     The goal of this article is to explain the consultation, planning, and field test strategy which led to the NRS design for implementing the President's mandate and for strengthening the effectiveness and credibility of Head Start programs.

Consulting with Experts and Practitioners

     THE BUREAU CONVENED AND PARTICIPATED in an ongoing series of focus groups, workshops, and discussion sessions with leaders from local Head Start programs, early childhood researchers, and experts on assessment strategies for young children. These activities broadened our awareness of the strengths and limitations of currently available assessment tools, including those most commonly used in Head Start programs for ongoing assessment of children, for indicators of school readiness, and for research on key program components. Issues and concerns around development of the NRS were shared. The major events included:

     May 2-3, 2002. Focus Group on Child Outcomes with 17 Head Start managers, national assessment experts, and Head Start Bureau staff.

     June 17-18, 2002. Early Childhood Education and School Readiness Workshop: Conceptual Models, Constructs and Measures. This session was convened in collaboration with the National Institute of Child Health and Human Development (NICHD) and the Office of the Assistant Secretary for Planning and Evaluation (HHS). Forty-eight experts, including representatives from several Head Start grantees involved in research partnership efforts, attended.

     July 9, 2002. Discussion Session on the National Reporting System with 11 national experts and Federal staff from NICHD, ACF, and the Head Start Bureau.

     December 16, 2002 and February 12, 2003. Technical Work Group meetings of 16 experts, co-chaired by Dr. Craig Ramey, Georgetown University, and Dr. Clancy Blair, Pennsylvania State University. The Technical Work Group provides ongoing advice and support to the Bureau as we complete the development and implementation of the NRS.

     January 16, 2003. Associate Commissioner Windy M. Hill convened an invitational conference for Head Start and Early Head Start Directors to provide a comprehensive briefing on the status of planning and development of the NRS.

     January 16 and 23, 2003. Focus Groups with 49 Head Start Directors and Managers discussed the NRS and its relationship to their local system for ongoing assessment, staff training, and local computer capability.

     March 5, 2003. Language Diversity Group Meeting with six experts discussed strategies for assessment of non-English-speaking children in the NRS.

     May 2, 2003. The Technical Work Group reviewed reports of the field testing and other work related to the NRS.

Reviewing Measures and Assessment Tools

     THE HEAD START BUREAU is utilizing a variety of important criteria in selecting the child assessment measures for the NRS including:

• Focusing on skills and indicators that are important for pre-dicting school readiness and success.
• Determing the technical quality of the instruments in terms of accuracy, validity, and reliability for measuring the progress of 4-and 5-year-old Head Start children.
• Determining the feasibility of implementing the assessment with approximately 500,000 children in various program options.
• Limiting the total time required to administer the assessment so that children and staff are not overly burdened.

     Our first step in exploring child assessment options involved analyzing Program Information Report (PIR) data from 2001-2 on assessment tools currently in use. This effort revealed that programs are using 22 different assessment instruments and that more than 480 agencies are using locally developed tools or methods. Given the many assessment tools in use, it was determined that we could not create an NRS strategy based on data from current, ongoing assessments. Accordingly, a single uniform package of currently available assessments is being assembled to use in the NRS. This approach will allow the NRS information to be used effectively and for the intended purposes of designing and planning training and technical assistance efforts, augmenting existing assessment information used in decision-making about local curriculum and teaching strategies, and expanding the evidence of program effectiveness considered in Federal program monitoring reviews.

     The Head Start Bureau's commitment is to design and implement the NRS to benefit Head Start programs and the children and families we serve.

      Next, based on the President's charge to the Bureau in the Good Start, Grow Smart initiative and the bipartisan Congressional mandates in the 1998 reauthorization of the Head Start Act, the Bureau commissioned an analysis of the technical quality of existing assessment tools in the mandated areas of early literacy, language development and numeracy. This review included tools currently in use in major studies of Head Start children and programs (i. e., the Family and Child Experiences Survey (FACES), the Head Start Impact Study, and the Head Start Quality Research Center Consortium). In its review of assessment materials, the Bureau also considered input from the various consultation sessions and from the Technical Work Group.

     This process resulted in an initial set of measures to use in the NRS field test, including measures of the following five learning indicators as mandated by Congress in the Head Start Act in 1998:

• Understanding and using language to communicate for various purposes
• Using increasingly complex and varied vocabulary
• In the case of children whose native language is other than English, progressing toward acquisition of the English language
• Identifying at least 10 letters of the alphabet
• Numeracy awareness

      While we recognize the value of collecting and reporting on a more comprehensive set of domains and indicators, limiting the scope of the initial version of the NRS is a prudent and manageable way to begin.

Field Testing a Pilot NRS in Diverse Head Start Programs

     DURING APRIL AND MAY 2003, a sample NRS was implemented in a diverse group of 36 local Head Start programs, including two Migrant and two Indian Head Start agencies. This effort is expected to provide valuable information on how the NRS design, materials, and reporting system work in the programs. Representatives from each program participated in a training-of-trainers session on the child assessment and reporting system procedures. These participants, in turn, trained teachers and/ or other local staff to administer the child assessment and to record and enter the assessment information in the field test.

     Staff from our NRS support contractors, Westat, Inc. and Xtria, Inc., observed and gathered feedback on the implementation efforts in each program. For a limited sample of children, contract staff carried out a second assessment as a check on how well the training design worked to prepare staff to carry out the assessment procedures in an accurate, consistent way. Information from the field test and feedback from staff and managers in field test sites is a crucial source of input for final decisions on the NRS design, policy decisions, program guidance, and implementation strategies.

     In summary, we have successfully completed initial steps in developing the NRS, including several key decisions on the scope of the initial version of the system, the training procedures, and the reporting mechanisms. We appreciate the contributions from Head Start, early childhood, and research leaders in this unprecedented development of a system to collect and use child outcomes information as a new Head Start management and accountability tool.

     In closing, the National Reporting System is a work-in-progress. We continue to consult with experts, Head Start leaders, program staff, and parents as we design, analyze, understand, and use the NRS. We will evaluate pilot efforts to utilize the first year of NRS information along with other sources of information about the quality and outcomes of Head Start programs. As we build upon the initial experiences with the NRS efforts, we will broaden the measures of child outcomes and accomplishments and develop procedures in other languages spoken by Head Start children.

     The Head Start Bureau's commitment is to design and implement the NRS to benefit Head Start programs and the children and families we serve.

PROGRAMS PARTICIPATING IN THE FIELD TEST FOR TRAINING AND IMPLEMENTATION OF THE NATIONAL REPORTING SYSTEM (NRS) Thirty-six Head Start programs were field-test sites for the NRS; 72 centers and 144 classrooms participated. Approximately 1,440 children, 4 years and older, were assessed. In 17 of the programs, at least one-fifth (20%) of the children were English language learners.

CITY, STATE

Akron, OH
Albuquerque, NM
Aliquippa, PA
Allentown, PA
Altadena, CA
Anchorage, AK
Atlanta, GA
Berlin, NH
Bridgeport, CT
Caguas, PR
Chicago, IL
Coshocton, OH
Detroit, MI
El Centro, CA
Elizabethton, TN
Fargo, ND
Fort Washakie, WY
Ft. Lauderdale, FL
Gillespie, IL
Grand Junction, CO
Hagerstown, MD
Hughesville, MD
Laredo, TX
Lubbock, TX
Nacogdoches, TX
Norfolk, VA
Plaquemine, LA
Richardson, TX
Roma, TX
Russellville, AR
San Antonio, TX
San Juan, PR
Seattle, WA
Sharon, PA
Springfield, MO
Window Rock, AZ

---

Tom Schultz is Special Assistant to the Associate Commissioner, Head Start Bureau. T: 202-205-8323, E: tschultz@ acf. hhs. gov.