 |
Questions? |
|
Privacy |
|
Site Index |
|
Contact Us |
|
|
|
|
|
|
||||||||
|
|
|
|
||||||||||
|
Home | Services | Working with ACF | Policy/Planning | About ACF | ACF News | Search |
|||||||||||
|
|
|
|||||||||||
February 17, 2005 | October 27, 2004 | August 4, 2004
Head
Start Bureau February Updates
National Satellite Broadcast and Web Cast
Including a Report on Fall 2004 National Reporting System Results
Thursday, February 17, 2005 1:30-3:30 PM EST
The text from each segment of the February 17, 2005 Head Start Bureau Updates National Satellite Broadcast and Web Cast is available for viewing on-line. You may click on the segment number in order to view the text from that segment.
Windy M. Hill
Associate Commissioner
Head Start Bureau
Thank you, Ann.
It is my pleasure to welcome you to today’s broadcast. I’ll open today with a few updates related to the National Reporting System and follow that with some updates regarding other Head Start business and events.
You will notice in the broadcast today that one of the key staff is missing. Just a few weeks ago Dr. Tom Schultz resigned from the Head Start Bureau to accept a position at the Pew Charitable Trust working in the area of early childhood accountability. Dr. Michele Plutro, whom many of you know, will be taking on many of the roles that were Tom’s.
In this broadcast you will see that the rest of the team – both staff and contractors – remains in place and strong.
There are two other updates related to the NRS. First, a Report to Congress on this Head Start Assessment, approved by Assistant Secretary Dr. Wade Horn, is continuing to make its way through the ACF clearance process. I remain optimistic that this Report will reach the Congress this month.
Additionally, the General Accounting Office, GAO has conducted a study of the NRS development and implementation, including review of the assessment by members of the National Academy of Sciences. The GAO will be issuing its Report to Congress later this spring.
Now I’ll focus on some other areas of interest to the Head Start community.
You may have seen the press release in recent weeks announcing the 10-members of the Secretary’s Advisory Panel on Head Start Accountability and Educational Performance Measures. This group will be chaired by Dr. Susan Landry. Dr. Landry is a Michael Matthew Knight Professor and Chief of the Division of Developmental Pediatrics at the University of Texas Medical School in Houston, Texas.
The Committee will meet periodically over the next 16 to 24 months to review the existing sources of data in order to consider the status of Head Start Accountability and to make subsequent recommendations to the new Secretary of Health and Human Services, Michael Leavitt, on both the status and future considerations regarding various aspects of program accountability.
This includes data related to monitoring, the National Reporting System, the Impact Study, and FACES. You should look forward to hearing more about the work of this Committee in the next few months.
2005 represents the 40th Anniversary of the Head Start Program. You may be planning local events to celebrate and honor this historic occasion. The Head Start Bureau is currently engaged in planning a couple of events as well. You will hear more about these in the next several weeks.
Many of you in the viewing area today may also have been present two weeks ago for the first-ever Hispanic Institute that was held in Albuquerque, New Mexico. Over 2300 participants benefited from that experience and the various speakers and presentations that were part of the event, including 396 representatives from Migrant and Seasonal Head Start programs, and the states of Texas, Illinois, New York and California having the largest number of participants. We were happy to offer this opportunity to all grantees that serve Spanish-speaking children and families, and we are hearing a great deal of positive feedback about this event.
There will be a number of support efforts that will build on the experiences and the momentum created by this institute, and we look forward to hearing how the lessons learned there are translating into improved practices in Head Start programs.
Whether or not you participated in the Institute, I hope you will give some attention to the current issue of the Head Start Bulletin which was first disseminated at the Institute. This issue is focused on English Language Learners. It is packed with useful information.
The final subject I want to mention to day is Reauthorization. As you know, the reauthorization process was not completed last year. It has since resumed. You may want to follow the progress and think about the various proposals that are presented by the Congress.
We know that a number of ideas are already being discussed by various members of Congress and within committees.
The subjects under discussion include:
The final update I need to share with you involves a recently mandated cap on salaries in Head Start programs. I’m going to give a very brief introduction to this new regulation. You will soon receive additional written explanation and guidance regarding compliance.
The HHS appropriations bill that was passed in December 2004 contains a provision, Section 205, which requires that “none of the funds appropriated in this title for Head Start shall be used to pay the compensation of an individual, either as direct costs or any proration as an indirect cost, at a rate in excess of Executive Level II.” As of January, 2005, the rate of compensation for an Executive Level II employee of the federal government is $162,100 per year.
This requirement became effective on December 8, 2004, the day the President signed this bill into law. Regional Offices will be reviewing the proposals of any fiscal year 2005 refunding awards already made subsequent to this date and will, as appropriate, contact grantees for clarification.
Every Head Start program is responsible for assuring its compliance with this provision. This provision applies to the salary and fringe benefits of any staff employed by a Head Start grantee or delegate agency who is paid, in whole or in part, with Head Start funds.
Grantees may pay staff compensation rates which exceed $162,100, but no Head Start funds can be used for any portion of that compensation which is in excess of this statutorily mandated cap. Grantees are cautioned that failure to comply with this requirement could lead to a finding of disallowable costs.
That concludes my remarks for today’s broadcast. You can now look forward to hearing about the current status of the NRS efforts as well as a special preview of new mentor coach materials.
Next we’ll hear from Michele Plutro, Educational Affairs Specialist. Michele will provide updates on the first year NRS reports, and an update on the NRS Technical Work Group.
Michele...
Segment 1b: Update on NRS Year I Reports and Technical Work Group
Michele Plutro
Special Assistant to the Associate Commissioner
Head Start Bureau
Thank you, Windy.
It is my pleasure to also be with you for today’s NRS broadcast. My remarks today are very brief, which may surprise many of you who know me!
I have been a part of the development of the National Reporting System team since the idea was first released by President Bush in 2002. Sometimes it is still hard to believe how far we have come and to believe that in a couple of months we will already be celebrating two years into this NRS process.
Each time I review data, talk with programs about their experiences or talk with other federal or state employees about this process, the more I believe we are doing one more thing right on behalf of children.
We may not yet be doing everything as well as we can. We may not implementing every single procedure as well as possible, but it is truly important that we be able to report to the Congress and to the tax-paying public that children are indeed better prepared for school after their Head Start experiences, and that the preparation includes sustained attention to cognitive skills as well as health status, nutrition, social and emotional development, and parent education. Historically, we know that we have done a better job in reporting on those outcomes than we have on reporting children’s cumulative, curricular experiences.
As Ann Linehan said, you will receive many updates throughout today’s broadcast, so my messages are indeed few since the presentations today are quite rich.
I have two messages to share. First, I want to repeat the caution that we have been stating since the inception of the NRS – a message that has been clearly stated in each of the reports that you have received to date and a message that has been repeated by many throughout these NRS broadcasts.
That is, the NRS is still under development. The infrastructure, though in existence, still needs some strengthening. Therefore, the reports you receive, while very informative, should not be "overused”, prematurely used, or inappropriately used for local decision-making related to certain types of critical programmatic changes.
I encourage you to reexamine the narrative sections of your reports and to review some of the relevant questions and answers as they are archived on the NRS website.
My second message is an update on the partnerships necessary to continue the NRS development and implementation. As evidenced in today’s broadcast – we are continuing to work with the primary organizations you are familiar with: Xtria, Westat and Mathematica.
In addition, the Technical Workgroup – the TWG as we say – is continuing, even with the formation of the Secretary’s Advisory Committee on Head Start Accountability. The membership of the TWG has remained 100% intact since it was formed in the earliest days of discussion regarding the NRS.
That group, the TWG, has a full agenda for this coming year, beginning with their next meeting this spring. I think you will be interested in the subjects they are currently discussing and debating and look forward to the recommendations after their meeting. These items include, after OMB approval, a 2005 field test of a possible measure or measures in the social emotional domain and the examination of various approaches or options to annual sampling of four-year-old children for participation in the NRS.
Now we will be joined by Dr. Nick Zill, a project director at Westat, Dr. Ruth Hubbell-McKey, of Xtria, and Dr. Diane Paulsell from Mathematica. They will discuss the NRS Fall 2004 child assessment results, the CBRS, including summer migrant programs in 2004, and the lessons learned from year one and the baseline.
To you Nick…
Segment 1c: Head Start NRS Fall 2004 Child Assessment Results
Nick Zill
Project Director
Westat
Thank you Michele. It’s good to be with you again.
SLIDE 1
FALL ASSESSMENT RESULTS
I will be giving an overview of the results of the Fall 2004 National Reporting System Child Assessment.
SLIDE 2
BASELINE DATA COLLECTION
I am very happy to report that the fall 2004 data collection for the National Reporting System has been completed. Thanks to the continued efforts of Head Start program staff throughout the United States, baseline assessments were successfully carried out with nearly all the kindergarten-eligible children in Head Start this year.
Baseline program reports are being posted on the CBRS website right after this broadcast. You should receive your printed baseline report in the next two or three weeks.
SLIDE 3
NUMBER OF CHILDREN TESTED
Once again, Head Start staff around the country have done an outstanding job of completing the NRS child assessments in a timely and conscientious fashion. We have received, scanned and scored completed baseline answer forms for 427,473 children. Assessment data have been received from over 1,800 Head Start programs. When we break that down by language of assessment, the totals for the fall are that some 404,000 assessments, or 95 percent, were completed in English. Approximately 98,000, or 23 percent, were completed in Spanish. And some 75,000 children, or 18 percent, were assessed in both English and Spanish.
SLIDE 4
HIGH QUALITY DATA
It appears that Head Start local trainers and assessors have done an excellent job overall in terms of the quality and consistency of assessments. The internal-consistency reliability of the fall data show good reliability at the individual child level.
These good reliabilities at the individual level mean that reliabilities are excellent when child data are aggregated to the program level.
SLIDE 5
COMPARISON WITH FALL 2003
The fall assessment results are not only reliable; their findings were generally consistent with the results from the fall 2003 baseline data collection. This suggests that Head Start programs are serving children with similar early literacy and early numeracy skills as in previous years.
SLIDE 6
WHAT ASSESSMENTS SHOW
So what do the fall NRS assessments show? On average, in the fall most children scored near the middle of the range on the vocabulary and early math subtests but low on Letter Recognition in both the English and Spanish assessments. These findings are similar to the results of the Fall 2003 assessments.
SLIDE 7
MEAN PERCENT OF ITEMS CORRECT
If we look at the percent of items that children got correct, on average, we see that children got 80% of the items correct on the English language screener.
Those who passed the screener went on to the other English assessment components. On the vocabulary component, the average percent correct was 55%.
The average score on the letter recognition component was somewhat lower, with children recognizing 25 percent of the letters, or about 6 out of the 26 letters of the English alphabet.
On the early math component, the average percent correct was 50%, which was in the middle of the score range.
SLIDE 8
MEAN PERCENT OF ITEMS CORRECT FOR U.S. MAINLAND PROGRAMS
The national average scores for the Spanish assessment are presented by Head Start program location. Analyses of the first year data have shown that the spring Spanish assessment scores were significantly higher for children in programs in Puerto Rico, where the primary language of instruction is Spanish, than for children in programs on the U.S. mainland, where the primary language of instruction is typically English.
Therefore, the Spanish assessment results are presented for U.S. mainland Head Start programs and separately for those programs in Puerto Rico.
The average baseline results on the Spanish assessment, based on mainland programs, were similar, though generally somewhat lower than the results on the English assessment. Children who took the Spanish assessment got 63 percent of the items correct on the Spanish language screener.
On Spanish vocabulary, they got 54 percent of the items correct. On Spanish letter recognition, they named 10 percent of the letters of the Spanish alphabet correctly, or 3 out of 30 letters. On Spanish early math, they got 40 percent correct.
SLIDE 9
MEAN PERCENT OF ITEMS CORRECT FOR PUERTO RICO PROGRAMS
Turning to Puerto Rico, we see that the children there got 75 percent of the items correct on the Spanish language screener.
And in Spanish vocabulary, children in Puerto Rico got 62 percent of the items correct. On Spanish letter recognition, they named 7 percent of the letters of the Spanish alphabet, or about 2 out of 30 letters. On Spanish early math, they got 43 percent correct.
SLIDE 10
ENGLISH LANGUAGE SCREENER FOR ENGLISH LANGUAGE LEARNERS
Let us look now at how well children from non-English-speaking families did on the English screener.
25 percent of all English Language Learner children understood spoken directions fairly well or well. That is, they got 16 or more items correct out of the 20 on the English screener.
On the other hand, 36 percent of all English Language Learners had insufficient knowledge of English to receive the remainder of the English assessment. Whether or not they passed the English screener, children from Spanish-speaking families received the Spanish assessment. Those who came from other language backgrounds and did not pass the screener did not receive further assessment.
SLIDE 11
VOCABULARY
On the English vocabulary component, the typical Head Start child started the program with a vocabulary size consistent with skill level 3, or a vocabulary size equivalent to that of an average child 40 to 47 months old (just under 4-years old). Given that most of the children taking the NRS were 4 or 5 years old, this meant that the vocabulary development of the typical Head Start child was somewhat behind that of the average non-poor child of the same age.
However, there was considerable variation around this average level of development. Even early in the program year, 17 percent of the Head Start children had vocabularies equivalent to that of an average child in the U.S. of the same age or older. On the other hand, 50 percent of the children were a year or more behind in their vocabulary development.
SLIDE 12
LETTER RECOGNITION
As I mentioned earlier, the average number of letters known by Head Start children early in the year was between 6 and 7. But this average was boosted by the minority of children who know considerably more letters. In fact, the typical child – one in the middle of the distribution – entered Head Start identifying only one or two letters by name, and that was consistent with skill level 2.
The minority of children who knew more letters included 9 percent who knew all or nearly all of their letters. Three-quarters of the children, however, identified fewer than the congressionally-mandated 10 letters early in the Head Start program year.
SLIDE 13
EARLY MATH SKILLS
Turning to early math skills, the typical Head Start child counted up to 10 or 11 objects early in the program year, consistent with skill level 3.
14 percent of the children had the more advanced skill of being able to count 15 or more objects.
There was similar diversity among children assessed in Spanish.
SLIDE 14
SPANISH LANGUAGE SCREENER
Baseline skill level results for the Spanish assessment subtests are presented for the U.S. mainland programs.
The typical kindergarten-eligible children assessed in Spanish entered the Head Start program understanding directions moderately well, consistent with skill level 4.
23 percent understood directions well or fairly well in Spanish.
10 percent had insufficient knowledge of the language to receive the remainder of the Spanish assessment.
SLIDE 15
SPANISH VOCABULARY
On Spanish vocabulary, the typical Spanish-speaking Head Start child started the program with a vocabulary size equivalent to that of an average child 40 to 47 months old, or just under 4-years old. This was consistent with skill level 3.
At the beginning of the program year, 17 percent of the children assessed in Spanish had Spanish vocabularies equivalent to that of an average Spanish-speaking child of the same age or older. About one third of the children were a year or more behind in their Spanish vocabulary development.
SLIDE 16
SPANISH LETTER RECOGNITION
The average number of letters known by Head Start children early in the program year was 3 letters.
Approximately 6 percent of the Spanish-speaking children knew 17 or more letters. But over 90 percent identified fewer than 10 letters of the Spanish alphabet.
SLIDE 17
SPANISH EARLY MATH SKILLS
In math, the typical child assessed in Spanish could count fewer than 6 objects. 94 percent of the children could not count more than 11 objects. Only 2 percent could count 15 or more objects.
SLIDE 18
VARIATION ACROSS PROGRAMS
When we look at the fall 2004 NRS results across different Head Start programs within each region or across regions, we find that there are significant variations in average skill levels with differing demographic characteristics of programs. But these variations are not great and they are not all in the same direction.
SLIDE 19
WHICH DEMOGRAPHIC CHARACTERISTICS MATTER?
For example, programs who serve, on average, relatively older children are likely to score somewhat higher on the each of the assessment tasks than programs who serve relatively younger children. Similarly, children of parents with relatively higher levels of education tend to have higher assessment scores than children of less well-educated parents.
Children in programs with high concentrations of English language learners score lower on the English language screener and the English vocabulary assessment tasks than children in programs with low concentrations of language minority children, and this is as might be expected. These differences in English vocabulary exist despite the fact that it is given only to children who have passed the language screener.
On the other hand, programs with high concentrations of English Language Learners did not score lower than other programs on Letter Recognition, nor on Early Math Skills.
SLIDE 20
NARROWING THE GAP
Despite these associations between average baseline skill levels and demographic characteristics, analyses that we have done with first year NRS data show that Head Start seems to be making a difference in children’s early literacy and early numeracy skills. Over the course of the Head Start year, relationships between children’s demographic characteristics and average skill levels become weaker. The services provided by Head Start seem to be helping to narrow the gap between different groups of Head Start children.
SLIDE 21
BASELINE REFERENCE POINTS
As you review the data you will receive on the skill levels of children in your program, it is important to realize again that the scores represent children’s knowledge as they entered Head Start in the fall of last year.
They provide a baseline reference point for the progress children will make in these skill areas during the program year. In part because of children’s participation in Head Start, their knowledge in these areas is likely to increase noticeably by the end of the year, as they did in the first year implementation of the NRS. What is important is how much growth your children show by the end of the program year and the level of knowledge they have attained when they leave the program and go on to kindergarten.
SLIDE 22
DOCUMENTING PROGRESS
Based on scores from both the fall assessments and the spring assessments, we will calculate "gain scores" to document this progress in each skill area. Following the completion of the spring assessments, each of your programs will receive a customized report on the growth children in the program have shown and the levels of knowledge they have achieved.
SLIDE 23
TESTAMENT TO YOUR WORK
The high completion rates and the good quality of the fall assessment data are a testament to the hard work and care that Head Start staff throughout the United States have devoted to the National Reporting System enterprise.
But the effort is only half done. Similar diligence and care will be required to complete the spring assessments in a timely and high-quality manner. Keep up the great work in the spring.
Now I’m going to turn it over to Ruth.
Segment 1d: Head Start NRS Fall 2004 CBRS
Ruth Hubbell-McKey
Project Director
Xtria
Thanks you, Nick.
SLIDE 1
TITLE SLIDE
I will be reporting on data from the NRS Computer Based Reporting System – the CBRS. This is the web-based system that programs use to record their eligible children and provide descriptive data about the programs, centers, classrooms, teachers, assessors and children.
When the data collection period begins, programs first enter these data into the CBRS. The system assigns ID numbers to the children, and these numbers are then entered on the scoring sheets and used for the assessments. After the assessments are completed, Head Start staff go back and enter assessment completion data into the CBRS. This allows us to track how programs are doing and the numbers of assessments to expect from them. The system also provides valuable descriptive data for analysis.
SLIDE 2
FALL 2004 STATUS REPORT
We have had a very successful fall data collection period as far as CBRS data collection was concerned. Programs entered data successfully, including many that took advantage of the new copy-over and data import features to ease data entry.
For this fall, we had 1,841 or 95% of the programs log on and enter data. We closed the database in early December when all assessments should have been completed. We now have 477,915 children entered in the CBRS for the fall. Of these, 669 were not able to be assessed due to a severe disability, 718 had IEP’s that prevented assessment and 7,620 dropped out before being assessed. Therefore there were 468,908 children eligible for assessment.
SLIDE 3
Fall 2004 STATUS REPORT Continued
Of these, 6,320 children were not assessed due to parent refusal, 10,242 were not assessed due to prolonged absence and 3,470 children would not cooperate with the assessment. There were also 3,063 erroneous or missing ID’s and for 18,340 children their assessment status was unknown.
That gave us a total of 427,437 assessments that were scanned and scored. This is very good performance and shows the great effort that your programs and staff put forth.
SLIDE 4
MIGRANT STATUS REPORT (2004 DATA)
The Migrant and Seasonal programs participated in the NRS on a different schedule than the regional programs. Most of these programs begin operations in the spring and summer when the growing season starts and often operate only a few months. Only migrant programs in which children are enrolled more than 4 months were required to participate in the NRS. These programs completed their “baseline” or first set of assessments in the spring and their “growth” or second set of assessments at least four months later.
We had 40 of 68 migrant programs participate on this schedule. They entered 1,365 children in the CBRS, and we received 1,299 assessments in the baseline period and 742 in the growth period. We had 32 non-complete assessments and 37 children dropping out at the time of the baseline, and 43 non-completes and 242 children dropping out at the time of the growth assessment.
In 2004 we conducted a feasibility study of assessing children in migrant programs that operate less than 4 months. The study was conducted in 27 programs. The results of that study should be available later this spring.
Now Diane, over to you.
Segment 1e: Lessons Learned from Year 1 and Baseline Observations from Fall 2004
Diane Paulsell
Senior Researcher
Mathematica Policy Research
Thank you Ruth…
SLIDE 1
TITLE
Since the NRS was first implemented in fall 2003, the Head Start Bureau has contracted with Mathematica to conduct an independent, third-party assessment of how well the NRS is being implemented around the country. The purpose of this two-year study is to learn what is working well, and what can be improved in future years.
In the first year of implementation, we visited a nationally representative sample of 35 Head Start programs during the fall and spring assessment periods, and we visited 4 migrant Head Start programs participating in the NRS. We are visiting a different set of programs to learn about implementation during the current program year.
Our results are being reported to the Head Start Bureau, the NRS Technical Work Group, and Head Start programs.
SLIDE 2
OVERVIEW
Today I’m going to share our main findings on:
SLIDE 3
FALL OBSERVATION VISITS
During fall 2004 NRS implementation, our team visited a nationally representative sample of 34 Head Start programs. Each site visit lasted about 2 days. We observed Head Start staff conducting the NRS assessment with 354 children (295 assessed in English and 59 assessed in Spanish).
We observed a total of 126 local assessors; interviewed Head Start directors, NRS trainers, and data managers; and held focus groups with staff conducting the assessments to learn about their experiences.
SLIDE 4
LENGTH OF ASSESSMENTS
The average length of the assessments we observed was about 16 minutes on the English version and 18 minutes on the Spanish version. More than 90% of the children completed the assessment in 10-20 minutes. New assessors who had not assessed children in previous rounds took about 4 minutes longer, on average.
SLIDE 5
QUALITY OF ASSESSMENTS
Using the certification procedures as a guide, we coded the assessments we observed for errors and computed a certification score for each one. Nearly 80% of the English assessments we observed scored an 85 or higher, which is the minimum required for certification. Half scored in the 96-to-100 range. Overall, this indicates that programs did a good job in training staff to administer the assessments correctly and consistently.
We compared certification scores on assessments conducted by teachers who assessed children in their own classrooms with those of other assessors, as well as scores for experienced and new assessors. We found no significant differences in average scores for these groups.
The quality of Spanish assessments was similar. 80% scored an 85 or higher, and 65% scored in the 96-to-100 range.
We also analyzed our observations to understand more specific types of errors and problems in administering the assessment. The overall error rate was low. Compared to fall 2003, assessors made fewer errors in reading the script and scoring the items. However, they made more gesturing and coaching errors.
Gesturing errors involved omitting a gesture, making a wrong gesture, or adding a gesture where none was required. The gestures were among the more subtle and difficult aspects of NRS administration. Nevertheless, site visitors generally felt that most gesturing errors had little effect on children’s responses to the items.
Coaching errors occurred mostly in the Simon Says section, and usually involved giving nonverbal cues, such as looking down too soon for the item, “put your feet together.” Some of these actions may be unconscious and could be addressed by reinforcement of correct procedures during refresher training.
Assessors, on average, scored 97% of responses correctly. Errors occurred when assessors had difficulty seeing where the child was pointing, or when the child’s response was ambiguous. In addition, assessors had difficulty scoring the counting item correctly when children skipped marbles or counted the same marble twice.
AREAS FOR IMPROVEMENT
SLIDE 6
Although assessors made few errors overall, we identified some aspects of the assessment process in which staff may need more guidance and training. These include:
SLIDE 7
CHILDREN’S RESPONSES
Staff in two-thirds of the programs reported that most children reacted positively to the assessment. Many enjoyed the one-on-one time with the assessor.
At the same time, programs cited several child behaviors that made administering the assessment challenging. These included difficulty staying on task, being shy or afraid to go with the assessor, becoming bored or distracted, crying, and being unresponsive.
Assessors tried a variety of strategies to address these behaviors, such as redirection, taking breaks, allowing children to move around, and trying again on another day. Although refresher training provided some behavior management strategies, programs requested more guidance on handling children’s negative responses to the assessment.
Assessors found the vocabulary and letter naming sections to be the most difficult to administer. Many felt these sections were too long and reported difficulty sustaining children’s attention. As in the Year 1 study, some assessors noted that it was frustrating for children who did not know the letters to be asked repeatedly if they knew any more.
SLIDE 8
ENGLISH LANGUAGE LEARNERS
71% of the programs administered the NRS assessment to English-language learners.
Overall, assessors were pleased with the changes that allowed them to assess Spanish speakers in Spanish first and then in English. They wanted children to start with a positive experience.
However, some programs had difficulty determining whether children’s home language was English or Spanish, and they used a variety of methods to do so.
As in the Year 1 study, some programs also had concerns about word usage on the Spanish version, because of variation in Spanish word usage across regions. Some program staff also felt that the Simon Says task was confusing to English-language learners because they were not familiar with the game.
SLIDE 9
CHILDREN WITH DISABILITIES
All but 3 of the 34 programs we visited assessed at least one child with a disability. They made a number of accommodations for these children, including:
Two-thirds of these programs reported that the process went smoothly, and they felt comfortable deciding on the kinds of accommodations that should be made. At the same time, staff in about one-third of programs said they needed more guidance on when to administer the NRS to children with disabilities and on appropriate accommodations for these testing situations. Only a few children in the programs we visited could not complete the NRS due to a disability.
SLIDE 10
LOCAL IMPLEMENTATION APPROACHES
Most programs maintained the same basic approach to conducting the fall assessment that they used in during Year 1
More than half used at least some teachers as NRS assessors, but most of these did not allow teachers to assess children enrolled in their own classrooms.
In 26% of the sample programs, however, teachers did assess children in their own classrooms. In these programs, staff thought that children would feel more comfortable being assessed by their own teacher, and teachers would be better able to manage children’s behavioral issues.
Most programs kept their Policy Council and parents informed about NRS implementation. Overall, staff reported that most parents did not raise serious concerns or objections. In policy council meetings, the most frequently asked questions were about the purpose of the NRS, how the results would be used by the Head Start Bureau, and the appropriateness of testing 4-year-olds. Parents typically expressed interest in seeing their children’s scores on the NRS; some had questions about how the results would be used and by whom.
SLIDE 11
FALL REFRESHER TRAINING
Half of the sample programs trained at least 1 new assessor in fall 2004, and all programs provided refresher training to experienced assessors.
All but 1 of the programs followed the training agenda for new assessors, but less than half followed the guidelines for refresher training. A number of programs did not adhere to the training script, and 2 did not show the training video. Training for Spanish assessors was even less likely to follow the training protocol.
Overall, programs found the training materials helpful, especially sections of the video on handling challenging behaviors. However, they felt that the refresher training was too long and repetitive. Nearly half of the programs said they would prefer a shorter refresher training that focuses on changes to the assessment protocol and provides more time to practice with other assessors.
SLIDE 12
PROGRAM REACTIONS
During interviews, Head Start staff expressed four main concerns about the NRS. These concerns are very similar to those raised by programs that participated in the Year 1 study.
The concern expressed most often by local staff was about how the NRS results would be used at the national level. Many staff said they still did not have a clear understanding of the purpose of the NRS and its implications for local programs whose children did not perform well on the assessment.
Local staff also expressed concern about whether the NRS results would accurately reflect program performance, the amount of staff time and financial resources dedicated to the NRS, and whether the assessment process could adversely affect some Head Start programs and children by narrowing the program’s focus to the domains covered by the assessment.
SLIDE 13
VIEWS ON THE FALL NRS REPORT
In early spring 2004, each local Head Start program received a report on the results of the fall 2003 assessments. (At the time of the visits, programs had not yet received their report on the first full year of the NRS.)
Staff in about half of the programs found the report clear and easy to understand; staff in 30 percent of programs had some difficulty interpreting the information.
At the same time, staff in most programs said the results of their own local assessments were more useful to them than the NRS results, because they covered more domains, were available at the individual child and classroom levels, and were available soon after the assessments were completed. Most programs suggested that NRS results be reported by center or classroom.
Nevertheless, staff in most programs said they would use the NRS results for program improvement—to develop improvement plans, identify areas in which children did not perform well, create teacher training plans, or assess the curriculum.
In addition, staff in more than a third of the programs reported that they have already made changes in classroom practices as a result of the NRS, mostly by focusing more classroom instruction time on areas covered by the assessment. Letter recognition was a key area of increased focus.
In addition, one quarter of the programs reported using the NRS as a guide for purchasing classroom materials.
SLIDE 14
PROGRAM SUGGESTIONS
Programs made a number of suggestions for improving the NRS.
They suggested improving communication by providing more information about the purpose of the NRS and how results will be used; information about its development, validity, and reliability; and written materials for parents.
Programs also want to receive training materials and reports on NRS outcomes in a more timely fashion.
One quarter of the programs said they would like more guidance and support on using the NRS results for program improvement. Moreover, if children do not perform well in a particular domain, programs would like guidance and technical assistance on how to achieve the desired results.
SLIDE 15
SUGGESTIONS (CONT.)
Half of programs requested more training and guidance on administering the assessments. Specific topics included providing neutral encouragement, behavior management, assessing children with disabilities, and changes to the assessment and procedures.
Programs made a range of suggestions about the current assessment battery. These included combining it with the local assessment, using alternative assessment methods or materials, and suggestions for improving specific items.
Programs were split on whether new domains should be added.. Staff seemed to be torn between a desire to have their programs assessed on a broader set of outcomes—to better reflect the comprehensive nature of Head Start—and concern that a longer assessment would be more challenging for programs and children.
In summary, Mathematica’s implementation study found that programs have done a good job of administering the NRS assessment. The majority of assessments we observed met or exceeded the standards for quality established by the certification process.
We also identified some sections of the assessment that are still difficult for staff to administer or score, some areas in which staff may need more guidance and training, and some concerns that programs have about the NRS. Many of these issues were also identified in the Year 1 implementation study. We are in the process of reviewing our findings with the Head Start Bureau and the Technical Work Group, and they will use this information to plan for NRS implementation in spring 2005 and beyond.
Back to you Ann…
Segment 1f: Question and Answer Segment 1
Ann Linehan: Thank you Michelle, Nick, Ruth, and Diane. Our phone lines are now open. When you call in, please tell our operators your first name, title, affiliation and state. Please do not use your speaker phone and once you ask your question on the air, please hang up. We will attempt to get through as many calls as possible during the next 15 minutes. Remember, if you don’t get to ask your question live, we will pose representative questions and answers on the web. We will try to address the most frequently-asked questions. Every question is important to us. However, within the limited time allocated, we will provide answers to the questions that we hope will benefit the greatest number of viewers.
Ann Linehan: Thank you. Well, we need to bring this segment to a close and thanks again to all who phoned, faxed or E-mailed your questions or comments to our presenters. Following a short break, our presenters will return to discuss the spring 2005 NRS training assessment, the 2005 CBRS data collection and the sites to be visited in 2005. We’ll continue in two minutes.
Segment 2a: Spring 2005 NRS Training and Assessment
Nick Zill
Project Director
Westat
Thank you Ann.
SLIDE 1
NRS SPRING ASSESSMENT TRAINING
I will be giving an overview of the Spring 2005 Refresher training and the Spring 2005 Child Assessment.
SLIDE 2
GOALS OF TRAINING
Based on the results from the fall assessment, trainers and assessors have done an excellent job overall in terms of the quality and consistency of assessments. The goal of this refresher training is to equip you to continue that high standard of quality, so that your spring assessments show the most complete, fair and accurate picture of the progress of children in your program on the NRS measures.
Feedback from programs indicate that the refresher training materials, including the training video tapes and the role play scripts, were very helpful in preparing for last spring’s data collection.
SLIDE 3
SPRING TRAINING MATERIALS
HSNRS Spring 2005 training and child assessment materials will be automatically sent to you this spring. The materials for the spring assessments include:
Trainers should review and familiarize themselves with these new materials before conducting their local trainings.
SLIDE 4
SPRING 2005 ASSESSMENT
As in the fall, all kindergarten-eligible children will be given the same, standard direct assessment in the spring. The spring English assessment covers the same skill areas as the fall 2004 assessment, namely:
SLIDE 5
SPRING 2005 ASSESSMENT
Similarly, the spring Spanish assessment covers the same skill areas as the fall Spanish assessment, namely:
SLIDE 6
SPRING 2005 ASSESSMENT
In order to reduce the possibility of teaching to the test or practice effects, new items have been included in the Vocabulary and Early Math sections for both the English and Spanish assessments. These items have similar difficulty and cover the same content areas as the fall 2004 items. So even though new items are introduced, we will be able to compare scores from the fall and spring to measure children’s growth during the Head Start year.
The Simon Says, Art Show, and Letter Naming sections will remain the same as in the fall 2004 assessment.
SLIDE 7
SPRING 2005 ASSESSMENT
Some of the scripted instructions in the easels, such as the instructions for proper gesturing, have been simplified or clarified so that assessors and children will have a better understanding of what they are supposed to do in the task. Further, the updated assessor’s guide provides more guidance in how to score children’s responses.
SLIDE 8
REFRESHER TRAINING VIDEO
The Refresher Training Video features a demonstration of the entire Spring 2005 assessment conducted with a preschool-aged child.
The video demonstrates methods for handling various types of behaviors children may exhibit during an assessment, such as being rowdy, not paying attention, or being overly shy and reluctant to respond. Feedback from programs has indicated that this portion of the training video has been particularly helpful in assessing their children.
SLIDE 9
SPRING 2005 EASELS
New easels will be shipped to you for the Spring 2005 assessment. They contain the new items for the vocabulary and early math sections as well as the improved instructions. You should use these easels for your Spring 2005 assessor training and data collection. If you have not already done so, please collect and store the easels from previous assessments so that your assessors will not accidentally use them in your Spring 2005 assessments.
SLIDE 10
ADDENDUM TO ASSESSOR’S GUIDE
This addendum contains item-by-item instructions for the new Vocabulary and Early Math sections in the Spring 2005 assessment and answer sheets. Assessors can use these for guidance on changes to the assessment. It also includes guidance on how to handle any behavioral issues you may encounter during an assessment.
Please be sure each of your assessors has a copy of the guide with the new addendum.
SLIDE 11
BASIC FORMAT OF REFRESHER TRAINING
Instructions and a model agenda for the refresher training will be included with the training video. In the refresher training, the trainees should watch the Spring 2005 Refresher Training Video and make notes of important information for when they practice and conduct the assessments. They should also make notes of questions they want to discuss with their trainer during the question and answer period.
The trainer should take time to answer any questions that the trainees might have. This would be a good time to talk about the examples in the training video of techniques used to address behavioral challenges.
After the Q & A session, the trainees should familiarize themselves with the materials and role play the assessment. After a round of role plays, the trainer should provide feedback to the trainees on strengths and weaknesses observed in the role plays. The trainees should then go through another round of role plays.
SLIDE 12
TRAINING FOR CERTIFIED ASSESSORS
Assessors who were certified last fall must watch the refresher training video and role play with the spring assessment easel. The trainer should observe the trainees administering the assessment during the role play. These assessors do not need to be formally recertified.Once the training is completed, these assessors should use the same Assessor ID number they used in the Fall 2004 assessments.
SLIDE 13
PROCEDURES FOR PROGRAMS WITH NO SPANISH-SPEAKING CHILDREN
Programs without Spanish-speaking children (which therefore will not be training any Spanish-speaking assessors) do not have to watch the Spanish version of the video taped training.
SLIDE 14
MONOLINGUAL SPANISH PROGRAMS OR ASSESSORS
Monolingual Spanish speakers or programs with only Spanish-speaking children will only be trained on the Spanish version of the Spring assessment. As a result, these trainees would watch only the Spanish version of the refresher training video.
SLIDE 15
BILINGUAL ASSESSORS
Bilingual assessors, who will be administering both the English and the Spanish assessments, MUST watch both the English and the Spanish versions of the training. They also must participate in role plays and receive feedback for both the English and Spanish versions of the Spring assessment.
SLIDE 16
TRAINING FOR NEW ASSESSORS
New Assessors who were not certified for the fall data collection need to be certified. New Assessors should be given the one-day training in the fall 2004 assessment and be certified. As in the fall certification procedures, these new trainees must score 85% or more on both the Assessment Certification Form and the Quiz in order to become a Certified Assessor.
SLIDE 17
TRAINING FOR NEW ASSESSORS
Once they become certified, they must watch the Spring 2005 refresher training video, role play the spring 2005 assessment easel, and then be observed administering the spring 2005 assessment.
After the training, the names of these newly certified assessors should be entered into the Computer-Based Reporting System (CBRS) to obtain an Assessor ID number.
SLIDE 18
SPRING ASSESSMENTS
You will assess all 4 and 5 year old kindergarten eligible children who are in your program this spring, EVEN IF THEY WERE NOT ASSESSED IN THE FALL. If children have joined the program they need to be assessed now using only the Spring assessment. They will not be included in the change analysis from fall to spring, but they will be included in the spring only analysis, so be sure to assess them.
SLIDE 19
FOCUS ON QUALITY CONTROL
During the refresher trainings, trainers should emphasize the importance of reviewing the completed answer sheets for completeness and clarity. If Child ID numbers are missing from the answer sheet, or are inaccurately filled in, then the assessment is not usable. We want to have as complete a picture as possible of all the children in your program and we don’t want any of your hard work to be wasted because of a missing ID number. Please make certain that your assessors double-check the accuracy of Child ID numbers and birthdates.
SLIDE 20
QUESTIONS ON TRAINING OR SPRING ASSESSMENT
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com
SLIDE 21
QUESTIONS ABOUT MATERIALS
If you have any questions about the shipment of the spring materials, please call the Head Start Information and Publication Center (HSIPC) at 1-866-763-6481 or email questions to: puborder@headstartinfo.org
SLIDE 22
QUESTIONS ABOUT CBRS
If you have any questions about entering or updating data in the CBRS, please call the CBRS HOTLINE at 1-800-669-2170 or email questions to: NRSTechHELP@xtria.com
SLIDE 23
NRS DEPENDS ON YOU
The successful implementation of the Head Start National Reporting System depends on your continued hard work. You did a great job last fall. Let’s do an even better job with the spring assessments.
Thank you.
Ruth…
Segment 2b: Spring 2005 CBRS Data Collection
Ruth Hubbell-McKey
Project Director
Xtria
Thanks Nick…
SLIDE 1
NEW CBRS FEATURES
Now, I would like to briefly discuss the CBRS for this spring 2005. The CBRS will open for regional programs, on March 1, 2005. You will NOT be receiving new CBRS manuals or other materials this spring. The materials you received last fall have all the information you need.
I do want to highlight some of the newer features of the CBRS. Many of you have already used the copy over and data import features. These new features were designed to make data entry easier for you.
The copy over feature allows programs to transfer all of their data into the CBRS, except for child-level data. Thus, data on the program, centers, classes, teachers and assessors were copied from last year’s database into this year’s database. Then, programs updated any information that had changed and enter all of their kindergarten-eligible children.
For the Fall 2004 data collection period, 1,290 programs used the copy over feature. These programs then entered 348,544 children.
The Data Import Feature is intended for larger programs that currently are using an automated management information system, such as ChildPlus, Galileo, or HSFIS, to track program operations. Using the Import Feature, child-level data as well as classroom, center, and teacher data can be transferred to the CBRS.
For the Fall 2004 data collection period, 87 programs used the data import feature, with 32,147 children transferred.
For Spring 2005, Regional Head Start programs cannot use the copy over feature, but you can use data import FOR NEW CHILDREN ONLY! That is, if you have had new kindergarten-eligible children enter your program that were not assessed on the NRS in the fall, you can use data import to transfer their data. When you log-in to the CBRS for the first time this spring, you will be asked to choose data import or not. You only have one chance to make your choice, so think about it carefully. There is a description of this option after you log-in and in the CBRS manuals you received last fall with your other NRS materials. If you have any questions, please call our CBRS hotline at 1-800-669-2170 or send an email to NRStechhelp@xtria.com.
Migrant Head Start programs that are starting your second baseline data collection may want to use the copy over feature. Or you may want to use data import. You must choose one or the other. So make your decision and make your choice when you log in. Call the tech helpline if you need assistance.
SLIDE 2
NEW LOG ON PAGE
When you log on to the CBRS this spring, you will notice that the log-in screen has been modified. On the left of the screen you will now see selections to click, such as “Data Import Registration” and “Frequently Asked Questions”, as well as a section to go to related links. On the right of the screen you will see a “What’s New” section, with updates about the CBRS. The center of the screen is where you will enter your user name and password to log in. We think these changes will make it a little easier for you to navigate the HSNRS website.
As some Head Start staff are more comfortable working with the system in Spanish, all you need to do is click on the Spanish link on the upper left side of the log-in screen and the entire CBRS and its data entry fields and instructions will be shown to you in Spanish.
SLIDE 3
TIPS ON MANAGING ASSESSMENT MATERIALS
Now I am going to give you some tips on managing your assessment materials to ensure that the answer sheets are received in a timely manner and the data are able to be used for analysis. You have heard this before if you watched our other satellite broadcasts. However it continues to be a problem so we want to give you some reminders.
Based upon last year’s experience, many answer sheets were unable to be used because they were lost in the mail, incompletely or inaccurately filled out, or damaged. So much hard work has been put into the assessment process, that it is a shame not to be able to use all the data collected. So in order to prevent this from occurring again, I want to review some of the necessary steps that your program can take to ensure that the data arrive in good shape. These steps are also in your trainer’s manual.
First you should have a designated NRS manager who ensures that the following tasks are performed:
SLIDE 4
TIPS ON MANAGING ASSESSMENT MATERIALS
SLIDE 5
TIPS ON MANAGING ASSESSMENT MATERIALS
Continue to check each answer sheet to be sure the spring 2005 answer sheets are used. No other versions will be accepted. Be sure the bubbles are completely filled-in, that a number 2 pencil is used, that there are no stray marks or stains on the sheet, and the sheets are not wrinkled or creased. Do not separate the pages of the answer sheets! If any of these occur, the sheet may not be able to be read by the scanning machines. If there are stray marks, stains, or wrinkles, you should recopy all the data on a clean answer sheet. Programs may want to Xerox all your answer sheets and keep the copies, just in case your sheets are lost in shipping.
SLIDE 6
TIPS ON MANAGING ASSESSMENT MATERIALS
You again will receive your answer sheets in cardboard boxes. Save these boxes! You can use these boxes to return your completed answer sheets and you will need to follow the packing instructions that come with the boxes. Also, please do not ship your answer sheets in envelopes. They will get crumpled and the envelopes may break open.
Similar to last year, ship your sheets in large batches (at least 75% of your total in the first batch and the rest in one batch as soon as possible.) If you have fewer than 100 children, ship all of them in one batch. Don’t wait too long to ship your answer sheets. Once you have completed your assessments send them in. don’t wait for the last one or two! If you wait for months, the data scanning process will be closed and your sheets cannot be scanned. Please send them in within a month after the deadline.
Again, pack the answer sheets face up and turned in the same direction.
Be sure to include the Program ID and shipment forms in each box.
SLIDE 7
TIPS ON MANAGING ASSESSMENT MATERIALS
It is important that you ship your boxes of answer sheets using a shipping company that can trace your packages, for example, Federal Express, UPS or Airborne. While this may be more expensive than mailing them, it is critical that you be able to track your shipment. A few days after you have shipped your boxes, trace them using the tracing number to be sure they arrived and were signed for at Pearson. Then, keep your tracing number in your records so that if any question arises about whether they were received you will
have your tracing number.
I know these steps may sound tedious, but it is very important to get the shipments submitted correctly so that we can analyze your data. And the sooner you send it in, the sooner we can get the results back to you.Thank you for your continuing hard work.
Now I will turn it over to Diane, who will discuss site visits for 2005.
Segment 2c: Site Visits for 2005
Diane Paulsell
Senior Researcher
Mathematica Policy Research
Thanks Ruth,
In spring 2005, Mathematica staff will visit a new, nationally representative sample of 35 Head Start programs for the Quality Assurance Study. As in previous rounds of visits, we will observe a random sample of assessments, interview selected staff about NRS implementation, and conduct a focus group with staff conducting the assessments to learn about their experiences. Each visit will last about 2 days.
We will contact each of the selected programs within the next month to discuss their participation in the study and select the sample of children whose assessments we will observe. Staff from Mathematica will work closely with each program to schedule the visits, observations, and interviews at convenient times.
Thank you.
Back to you, Ann.
Segment 2d: Question and Answer Segment 2
Ann Linehan: Thank you, Nick, Ruth, and Diane. Our phone lines are open again. As a reminder, when you call please tell our operators your first name, title, affiliation and state. Please do not use your speaker phone and once you’ve asked a question on the air, please hang up.
Ann Linehan: Thank you. Again, thanks to all who phoned, faxed or E-mailed your questions or comments to Diane, Ruth and Nick. In the next segment, Dollie Wolverton and Karen McKinney will discuss the assessment module of the Steps to Success – Early Literacy Mentor-Coaches instructional design.
We’ll continue after this short break.
Segment 3a: Assessment Module of the Steps to Success - Early Literacy Mentor-Coaches Instructional Design
E. Dollie Wolverton
Chief, Education Services Branch
Head Start Bureau
Thank you, Ann.
Hello Everyone.
It’s a great pleasure to be back in touch with you via Satellite.
I’m particularly pleased to share this segment with my colleague Karen McKinney. Together, we welcome this opportunity to discuss a very important aspect of our responsibilities and to build upon the topic of child observation and ongoing assessment.
In Head Start, independent of the National Reporting System, the Head Start Program Performance Standards require us to intentionally observe and assess children on an ongoing basis so that we have the information we need to promote and support each child’s learning and developmental progress. This is to ensure that each child is progressing as a result of their Head Start experiences and services.
The reason we’re so diligent about observations and ongoing assessments is that Head Start and Early Head Start staff use the information to adapt an age-appropriate curriculum - whether it’s for infants, toddlers, or preschoolers – with individualized experiences for each child through staff and parents working together.
This afternoon, it’s my responsibility to do two things. First, I will set the context regarding why the Head Start Program Performance Standards, which are based on sound child development principles, require that we engage in ongoing child assessment. And second, I will have the privilege of introducing the new Instructional Framework for Early Literacy Mentor Coaches entitled Steps to Success.
Following that, Karen will give us a preview of Steps to Success – with a particular focus on Unit 4: Helping Staff Use Child Assessment Information to Guide Instruction.
SLIDE 1
Head Start Program Performance Standards
In the world of Head Start, “setting the context” – invariably begins with The Head Start Program Performance Standards, which are the “must dos” – that is, the experiences and services that each program must provide.
Because of how the Standards are presented for children birth to 5, we’re able to plan and implement programming that is developmentally appropriate on two levels –
First, at a general age-level for the group of children. This is referred to as an "age-appropriate curriculum".
And within that, we individualize for each child’s
SLIDE 2
Program Performance Standard 1304.21(c)(2)
Program Performance Standard 1304.21(c)(2), which appears on page 82 of your copy of the Program Performance Standards with Guidance is the one that specifically requires that we use a variety of strategies to promote and support children’s learning and developmental progress based on the observations and ongoing assessment of each child.
We need to know which skills children have mastered, which ones are emerging, and when children are not making expected progress.
Ongoing assessment helps us decide on the best approaches for individual children as small groups and classes as a whole.
This certainly makes a great deal of sense since observation and assessment go hand-in-hand with developmentally appropriate programming.
This year marks Head Start’s 40th year. Millions of children, birth to five, and their families, including pregnant women have participated.
As the nation’s largest and only comprehensive child development program, we’ve learned a great deal regarding research-based practices, which enhance young children’s learning and development.
However, unless we ensure that children are provided meaningful and challenging learning experiences, ongoing child observation and assessment has little value.
SLIDE 3
The Leaders Guide to Positive Child Outcomes
Once again, I’d like to draw your attention to the Head Start Leaders Guide to Positive Child Outcomes. This guide is based on both the Head Start Program Performance Standards and The Head Start Child Outcomes Framework, which is a comprehensive description of school readiness in 8 child development domains:
SLIDE 4
The Leaders Guide offers many proven strategies to strengthen curriculum and to foster children’s progress in each of these domains.
The strategies in this Guide can universally assist us in forwarding our planning and our curriculum implementation regardless of the specific curriculum used in our individual programs. The Leaders Guide provides clear examples of educational quality based on-
SLIDE 5
Ensuring developmentally appropriate programming provides a meaningful basis for observing and assessing children’s progress and promoting and individualizing learning and development.
To support the ongoing practice of child observation and assessment, I want to share some exciting news about our new mentor-coaching project to help support and sustain what we have already begun.
SLIDE 6
The Head Start Act
The Head Start Act as amended in 1998 required that the Head Start Bureau take steps to provide mentor-coaching, that is, on-the-job guidance and training to Head Start staff and volunteers in order to maintain high quality education services.
Mentor-coaching is also a corner stone of Good Start, Grow Smart – the President’s Early Childhood Initiative launched in April 2002.
You may recall that in November 2002, 3,000 Early Literacy Mentor Coaches, participated in the Head Start Mentor Coach Conference in Washington, D.C.
At that time, we received your suggestions regarding specific strategies to support Early Literacy Mentor Coaches working with classroom teams and home visitors.
There was an overwhelming consensus that a technology-based network be developed to facilitate communication and exchange of ideas among mentor-coaches.
In direct support of your recommendations, Step-Net was launched shortly thereafter.
SLIDE 7
Putting the Pro in Protégé
At that same November 2002 conference, a number of Mentor-Coaches also recommended that we build on the concepts included in Putting the Pro In Protege: A Guide to Mentoring in Head Start and Early Head Start to ensure a uniform understanding and approach regarding the role of the Mentor Coach in supporting protégés as members of classroom teams and as home visitors.
Your suggestions have culminated in the development of an Instructional Framework for Head Start and Mentor Coaches entitled “Steps to Success.”
Karen is now going to give us an overview of Steps to Success and then focus on some key strategies in unit 4: Helping Staff Use Child Assessment Information to Guide Instruction.
Karen McKinney
Program Specialist
Head Start Bureau
Thank you, Dollie
I’m very happy to announce the launch of Steps to Success: An Instructional Design for Early Literacy Mentor Coaches.
This project will offer all Head Start and Early Head Start programs a consistent framework to support mentor coaching with an innovative approach using various types of technology.
Let’s first take a tour of the Steps to Success design.
As you can see in this slide, Steps to Success covers four essential areas.
SLIDE 8
Today, I want to share with you some of the strategies from Unit 4—Using Child Assessment Information to Guide Instruction.
As you will see in these videotapes from Steps to Success, classroom teams use a variety of techniques, both with infant / toddler and preschool children.
VIDEO CLIP 1
Strategies
In the next video clip from Steps to Success, a classroom teacher uses questions to actively engage a child in a conversation which builds on the little girl’s keen interest in words and poems…and imagination.
VIDEO CLIP 2
Teacher and little girl
There are a number of approaches to collecting information about children in their program and home environments.
This next video clip illustrates a variety of strategies which together provide information collected over a period of time which documents the child’s progress and provides staff with the information needed to individualize the curriculum and adapt the environment to promote and support children’s learning.
VIDEO CLIP 3
Mentor Coach and Teacher with Portfolio
As we saw, staff used a camera, an audio tape, and samples of the child’s work to create a portfolio, which not only serves as a file, but also provides a concrete way in which to share this ongoing assessment with others.
Just as Mentor Coaches and protégés take time to build trust and a strong working relationship, protégés and parents build trusting relationships to support children’s development.
SLIDE 9
Involving Parents
Head Start fully recognizes that parental involvement is especially critical during the early years.
Let’s view a video clip that illustrates how a teacher may involve parents in a discussion on child observation and assessment.
VIDEO CLIP 4
Teacher and Parents and Portfolio
As we’ve seen in this brief overview of ongoing child assessment it is critical to plan and to be intentional in the use of observation and assessment strategies.
SLIDE 10
These strategies may include:
The Steps to Success instructional Design will be launched in Spring 2005 with full implementation in each region in Fall 2005. There are two ways you can continue to keep informed about mentor coaching activities.
SLIDE 11
STEP-Net Web site information and the toll-free 877 number
I encourage you to regularly check STEP-Net.org to keep up with the latest information on Mentor Coaching. There you will find our electronic newsletter which features Head Start programs and the strategies they are implementing to support mentor coaching in their program.
Another resource for mentor coaches is a hotline at 1-877-coach-04 where a Steps to Success staff member will be happy to discuss your questions or specific strategies. Callers who speak Spanish can leave a message and a Spanish speaker will return your call.
In this segment, Dollie discussed the Head Start Program Performance Standards’ requirement to use information from observations and ongoing assessment to promote and support each child’s learning and developmental progress.
I’ve offered you a glimpse into Steps to Success and how mentor coaching can support ongoing child assessment. By using the techniques we’ve discussed today your approach to individualizing for children will become a part of your daily activities.
It is this variety of enriching activities and experiences combined with ongoing child assessment and tracking of progress that leads our Head Start children on a journey to positive outcomes and many more steps to success!
Ann…
Segment 3b: Question and Answer Segment 3
Ann Linehan: Thank you, Karen. Our phone lines are again open. As a reminder when you call, please tell our operators your first name, title, affiliation and state. Please do not use your speaker phone and once you ask your question on the air, please hang up.
Again, I would like to thank everyone who participated in the studio and to all of you in the audience. We are certain that we did not get to every question during the last two hours; however, after this broadcast you will be able to continue sending your e-mails through 6:00 pm EST today.
Representative questions and answers will be posted on the Web site address you now see on the bottom of your screen.
Thank you for joining us today. Gracias por su participación hoy.
Head
Start Bureau October Updates
National Satellite Broadcast and Web Cast
Including a Review of the National Reporting System First Year Report
Wednesday,
October 27, 2004, 2:30 - 4:30 PM EDT
One of the agenda items for this broadcast was to discuss the new Spring 2004 Growth Reports. These reports are available under the “Report” menu on the CBRS (www.hsnrs.net) under link number 10: “Spring 2004 Growth Reports”. You must be a program level user to view the reports. Sample reports are also be available for download, and a short online video demonstration on how to access the reports is available in both English and Spanish at www.hsnrs.net under “What’s New”.
The text from each segment of the October 27, 2004 Head Start Bureau Updates National Satellite Broadcast and Web Cast is available for viewing on-line. You may click on the segment number in order to view the text from that segment. Where slides were included in the presentation during the broadcast, they are available to view on-line or to download as PowerPoint files. Just click on the presentation topic to access the slides.
Segment 1a: Introduction of the NRS 2003-2004 Program Report
Tom Schultz
Senior Advisor on Outcomes
Head Start Bureau
Thanks, Ann.
We welcome this opportunity to use technology to speak directly to so many local Head Start programs and people. We asked you to take time to view this broadcast because of the significance of this first progress report on our NRS assessment of more than 400,000 preschool Head Start children. Since your spring program report is longer and more complex than the reports on the fall assessment, we believe it will be help you to hear a detailed explanation of each section of the report.
SLIDE 1
We hope that each of you has a copy of the 3 parts of a first year NRS progress report, either for your local program or our sample version. Each agency will receive a cover letter from Assistant Secretary Wade Horn, a 10-page NRS Program Report of assessment information, and a 3-page document titled “How to Use Your Spring, 2004 NRS Report”.
These materials are accessible through the Computer-Based Reporting System website. Print copies will be mailed to programs in the next several weeks.
(SHOW SAMPLE COVER LETTER, PROGRAM REPORT, AND GUIDE)
The cover letter looks like this. The program report looks like this. And the guidance on using the reports looks like this.
I’ll begin by reviewing briefly the cover letter from Dr. Horn.
SLIDE 2
He expresses appreciation for your hard work and productivity in administering nearly 850,000 child assessments last year. As we’ve said before, Head Start has successfully managed the largest early childhood assessment effort in history. He also commends you for the high quality of completed assessments. As we reported in our last broadcast, last year’s national implementation study found that assessors were following the assessment script and procedures correctly and doing a good job of accurately recording children’s responses. We were also encouraged by evidence that the quality of assessments improved this spring, when compared to the first assessment effort last fall. Programs reported fewer problems, based on their greater experience, the spring refresher training and efforts to simplify the assessment instructions.
Finally, Dr. Horn also notes that programs did an excellent job of gaining cooperation from parents and children for this new standardized assessment initiative. Roughly 98% of all parents allowed their children to participate, and only 4700 assessments out of more than 840,000 weren’t completed due to children failing to cooperate.
Like everything we do in Head Start, these achievement come from a widespread and diverse set of community-based agencies. For what we call our "supergrantees", managing the NRS required mobilizing hundreds of assessors and tracking thousands of assessment forms. For example, our largest grantee in Los Angeles, CA submitted more than 15,000 spring assessments from 28 different delegate agencies; our grantee for the Commonwealth of Puerto Rico administered nearly 12,000 assessments across its 20 delegates. At the other end of the scale, several of our American Indian and Alaskan Native Head Start programs, such as the St. Regis Mohawk and the Coquille Indian Tribe programs assessed fewer than 10 children. However, regardless of their size, every program asked their children the same questions in the same way. This means we have, for the first time, consistent, comparable information on the progress of our children and their levels of competence as they leave Head Start and enter kindergarten.
Let me add one more thing: you deserve recognition for the overall levels of progress documented in the NRS assessments. As we will discuss in more detail later, at a national level, Head Start children made progress on all four of the NRS subtests. The typical Head Start child progresses in ability to understand language, and advances in knowledge in vocabulary, letter naming and early math skills. English language learners also progress in all four areas of the English language assessment and Spanish speaking children continue to enhance their home language competence.
Dr. Horn also highlights three areas of future work on the NRS:
SLIDE 3
#1 – We will work to support you in using NRS information. We will be providing technical assistance from our NRS support contractors to answer your questions about your program reports. In addition, we are asking our ACF Program Specialists and local Technical Assistance Specialists to be available to assist you as you study and discuss these materials. We will be convening a National Conference of the T&TA Network in mid-November to learn about your initial experiences in reviewing these reports and to provide additional training to TA and Regional Office staff on NRS issues
We also recognize the importance of helping improve existing screening and ongoing child assessment efforts in classrooms and home-based programs. Our Education Services Branch is developing a new package of training materials to assist mentor-coaches in working with teachers on child observation and the use of ongoing assessment information.
We will also be analyzing the first year NRS information to understand how it can help us in designing new program improvement initiatives.
#2 – We plan to field test additional measures this coming spring, including assessments of children’s social and emotional growth, for possible inclusion in the NRS in the future. Two weeks ago we met with our Technical Work Group to discuss these possible measures. Let me emphasize that our plan is only to try out these measures in a diverse set of Head Start programs. Any decision about adding to or changing the NRS assessment will come only after we review the results of this new field test effort.
#3 – We will work to improve the NRS for the future, in conjunction with a new Secretary’s Advisory Committee on Head Start Accountability and Education Performance Measures. These efforts will include new analyses of the quality of the NRS assessment – whether the tests do the best possible job of accurately measuring children’s knowledge and skills. We also recently convened a meeting of experts and local Head Start programs to discuss the challenges of assessing children who speak languages other than English and Spanish.
The letter concludes by urging you to complete the current fall, 2004 assessment effort in a high quality fashion. I want to commend you for your efforts so far. As of this morning, you have registered more than 442,000 children and reported completing nearly 256,000 assessments. I’ll say a few more things at the close of the broadcast on the importance of completing the remaining assessments as soon as possible.
Now we will turn to Nick Zill, who will walk through the first section of the program report. Nick…
Segment 1b: Head Start NRS Spring 2004 Child Assessment Report: Pages 1-4
Nick Zill
Project Director
Westat
Thank you Tom.
I will be giving an overview of the Spring 2004 Program Reports. The program reports are currently being posted on the CBRS website and you should be receiving hard copies of your report in the mail in a few weeks. We have already had more than 2,500 downloads of program reports from the website.
SLIDE 1
SPRING 2004 ASSESSMENTS: Covered same skill areas as last fall…
The Spring 2004 English assessment covered the same skill areas as last fall’s assessment, namely:
The Understanding Spoken English section, made up of the Simon Says and Art Show tasks, served as a language screener to assess the English proficiency of English-Language Learners for continuing with the other sections of the assessment.
Similarly, the spring Spanish assessment covered the same skill areas as the fall Spanish assessment, namely:
Likewise, the Understanding Spoken Spanish section served as a Spanish language screener.
The Simon Says, Art Show, and Letter Naming sections remained the same as in the fall assessment.
SLIDE 2
OVERVIEW OF REPORTS
Each program report is a ten-page document containing two sets of assessment charts. One set is on pages 3 and 4 of the report. The second set is on pages 7 and 8. The first set of charts describes the levels of achievement reached by children in the program at the end of the Head Start year, prior to their entering kindergarten. These charts are based on all children in the program who were assessed in the spring, whether they were also assessed in the fall or not. The second set of charts describes the growth in knowledge and skills that children in the program demonstrated from fall 2003 to spring 2004. These charts are based on children who were assessed in both fall and spring.
SLIDE 3
OVERVIEW OF REPORTS (2)
Each program report contains descriptive text explaining what is contained in the assessment charts and how to interpret them. The text is on pages 1 and 2 of the report, as well as on pages 5 and 6. The text provides definitions of the skill areas assessed and the skill levels attained by different children. At the end of each report, on pages 9 and 10, are two pages of program profile. The first part of the profile provides information about the numbers of children in the program who were eligible to be assessed, the number actually assessed, and the reasons for non-assessment. The second part of the profile provides information about the demographic and operational characteristics of the program.
SLIDE 4
WHAT SPRING ASSESSMENTS SHOW
Before describing the program reports in detail, a few words about what the spring NRS assessment results show. They show that most children leaving Head Start have basic academic skills that should enable them to do well as they enter kindergarten.
SLIDE 5
TYPICAL HEAD START CHILD CAN…
The typical Head Start “graduate” can easily understand and follow simple directions spoken in English. He or she can recognize and name between 10 and 16 letters of the alphabet.
SLIDE 6
TYPICAL HEAD START CHILD…
The typical Head Start “graduate” can do one-to-one counting of between 12 and 14 objects. And the typical Head Start child has an English-language vocabulary equivalent to that of an average four to four-and-a-half year old child in the United States.
At the same time, the NRS spring assessments show that there is considerable diversity in the skills Head Start children take with them into kindergarten. There are some Head Start graduates who leave the program with skills at or above national averages. And there are some graduates whose skills are substantially behind national averages as well as lagging the skills of the typical Head Start child. The program reports will enable you to see how many of the graduates of your program leave at these different skill levels.
And now my colleague Ruth Hubbell-McKey, who will also give a presentation in our next segment, will help illustrate the features of the charts as I describe the reporting format.
(SHOW CHART ON PAGE 3 OF SAMPLE PROGRAM REPORT)
SLIDE 7
SPRING 2004 CHILD ASSESSMENT REPORT TABLES
If you had a chance to review your program’s HSNRS report from last fall, the tables presenting the Spring 2004 assessments results will be familiar to you. The format of these tables is similar to that of the Fall 2003 report.
Each row on the table presents the results for each of the four skill areas: Understanding Spoken English, Vocabulary, Letter Recognition, and Early Math Skills.
For each skill area, the first score is the mean score for that program. This shows the average percentage of items the children in your program answered correctly on the task. In this example report, the mean score for the Understanding Spoken English task is 93%.
Right below, the mean score is also presented in terms of the average number of items the children in your program answered correctly on the task. In this example report, the mean score is 18.7 out of the 20 items in the Understanding Spoken English task.
The second score is the median skill level for the assessed children in your program. Children’s performance on the spring assessment is reported according to a set of six skill levels, Levels 1 through 6. The median skill level is the skill level of a “typical child” in your program. Half of the children in your program will have skill levels at or below this level, and half will have skill levels above this level. In this example report, the typical child has achieved the highest skill level of “6” on the Understanding Spoken English task by the end of the program year.
For each skill area, the percentage of assessed children who are at each skill level is also shown. From this information, your program can get a sense of the range of skills that children have when they leave the program, as well as how your children are performing on average at the end of the program year.
For each skill area, anchor points for each of the 6 skill levels are included in the table. Anchor points describe specific accomplishments of children at each skill level. For instance, anchor points for the Letter Recognition task describe the number of letters children at each skill level can recognize. Children at Level 1 cannot identify any letters. Children at Level 2 can identify 1-2 letters, children at level 3 can identify 3-9 letters, and children at Level 4 have reached the Congressionally-mandated goal of 10 or more letters.
The skill levels for a given task are slightly different in these spring tables than those in the fall report. The skill levels have been revised from the fall definitions to better reflect the progress that children make across the program year. These revised definitions better account for the entire range of skill represented by the children in your program in the spring. If you are interested in the distribution of skill levels the children in your program had last fall based on these spring definitions, you will be able to find revised fall tables on the CBRS website.
(BACK TO CHART ON PAGE 3 OF SAMPLE PROGRAM REPORT)
The last two columns on the table present the U.S. national mean score (in terms of percent correct) and national median skill level for all kindergarten-eligible Head Start children who participated in the HSNRS spring assessment. This will provide you with some contextual information to help interpret your spring results.
For programs that assessed Spanish-speaking children, a table (similar to the table for the English assessment) will show children’s scores on the Spanish assessment. Programs that did not assess any Spanish-speaking children will not get this table. This Spanish assessment table presents the assessment scores for the children who were assessed in Spanish at the end of the program year, regardless of whether they were assessed last fall.
SLIDE 8
REPORTED SCORES
Scores on the Understanding Spoken English task are for all assessed children. However, scores for the Vocabulary, Letter Recognition, and Early Math tasks on both the English and Spanish assessment are only for children who passed the respective language screener.
SLIDE 9
AVERAGES FOR SPANISH ASSESSMENT
The averages reported for the Spanish assessment tables depend on whether the program is located on the U.S. mainland or in Puerto Rico. Analyses of the HSNRS data show that the spring Spanish assessment scores are significantly higher for children in programs in Puerto Rico, where the primary language of instruction is Spanish, than for children in programs on the U.S. mainland, where the primary language of instruction is typically English.
Therefore, to make comparisons fairer and more valid, Spanish-language reports for Head Start programs in Puerto Rico will include average scores and skill levels for all children who participated in the Spanish assessment in Puerto Rico.
For non-Puerto Rico programs, average scores and skill levels for all children who participated in the Spanish assessment in mainland U.S. programs are provided.
Back to you Ann…
Segment 2a: Head Start NRS Spring 2004 Child Assessment Report: Pages 5-8
Nick Zill
Project Director
Westat
Thanks Ann.
SLIDE 1
FALL 2003-SPRING 2004 GROWTH REPORT
The second section of the report describes the progress that the children in your program have made across the program year.
As you may recall, new items were included in the Spring 2004 Vocabulary and Early Math sections for both the English and Spanish assessments to reduce the possibility of teaching to the test or practice effects. These new items have similar difficulty and cover the same content areas as the items in the Fall 2003 assessment. So even though new items are introduced, we are able to compare scores from the fall and spring to measure children’s growth this past year.
The average fall and spring scores reported in the growth charts may not exactly match the average scores reported in the fall baseline reports or in the spring report. This is because the numbers in the growth charts are based on children who were assessed in both fall and spring. In the spring report, the average spring scores are based on all the children in your program who were assessed in the spring.
Children’s growth in each of the four skills areas is reported on the bar charts that Tom Schultz is showing you.
(SHOW TOP-LEFT CHART ON PAGE 7 OF SAMPLE PROGRAM REPORT)
This example chart shows the results for the children who were assessed in English in both fall and spring. For each skill area, bars show the average scores from fall 2003 and spring 2004. Average growth from fall to spring can be calculated by comparing these two scores.
“T-bars” indicate the national percentages for each skill area. This allows you to see how the fall to spring growth of the children in your program compares with national averages.
As you can see in this example chart, children in this program improved in all four skill areas from fall to spring. For instance, in Vocabulary the children improved from 55% correct in the fall to 67% correct in the spring. In Letter Recognition, they improved from being able to recognize 28% of the letters in the fall to 62% of the letters in the spring.
This example chart shows the fall-spring growth in the English assessment tasks for all children in the program assessed in English. Please note that the scores for the vocabulary, letter recognition, and early math tasks on the English assessment are only for children who passed the English-language screener.
(SHOW TOP-RIGHT CHART ON PAGE 7 OF SAMPLE PROGRAM REPORT)
If the program conducted Spanish assessments this past year, their report will include a bar chart presenting the fall-spring growth on the Spanish assessment for all children assessed in Spanish. On the Spanish assessment, scores for the Spanish vocabulary, Spanish letter recognition, and Spanish early math tasks are only for children who passed the Spanish-language screener.
Programs in Puerto Rico will have the Spanish assessment chart only.
Again, national averages for the Spanish skill areas will depend on location of the program. T-bars for reports to programs in Puerto Rico represent national percentages for all children who were assessed in Puerto Rico. For non-Puerto Rico programs, t-bars represent national percentages for all children who were assessed in mainland U.S. programs.
If a program did not conduct any Spanish assessments, this area will just say “NA.”
SLIDE 2
FALL-SPRING GROWTH CHARTS: Growth charts for English assessments…
Fall-Spring Growth charts for the English assessments are also reported for two subgroups. The first subgroup is native English speakers in your program. These are children who primarily speak English in their home. For this chart, the T-bars indicate the national averages for all assessed native English speakers. The second subgroup is English Language Learners. These are children who primarily speak a language other than English at home. For this chart, the T-bars indicate the national averages for all assessed English Language Learners.
Because they did not conduct any English assessments, programs in Puerto Rico will not have these charts; instead this area will just say “NA.”
Spanish assessment scores are not broken down by English Language Learning status.
(SHOW BOTTOM-LEFT CHART ON PAGE 8 OF SAMPLE PROGRAM REPORT)
Children’s fall to spring growth is also presented in terms of the percentage of children who achieve higher skill levels by the end pf the program year compared with the percentage of children who are at these levels in the fall. For instance, for Letter Recognition you can see the percentage of assessed children who have achieved the relatively high skill of identifying 17 or more letters in the fall and in the spring.
In addition, the percentage of children who are performing at relatively low skill levels in the fall is presented with the percentage in the spring. For Letter Recognition you can see the percentage of assessed children who can identify only 9 or fewer letters in the fall and in the spring.
From these skill level growth charts, you can see how well your assessed children are doing in terms of the increase in the percentage of children who are performing at higher skill levels by the end of the program year and the percentage of children who are no longer performing at lower levels of skill by the end of the year.
The T-bars indicate the national percentages for each skill area. Program staff can use these to compare their percentages with those of all Head Start programs in the country. It is important to note that for the low skill levels, it is desirable to be below the national average. This indicates that compared to the national average, the program has a smaller percentage of children who are performing at relatively low levels. This also means that progress is seen when the size of the spring bar is smaller than the fall bar.
Tom Schultz:
You know Nick, as you’re talking through this particular chart, it seems to me that it would be worthwhile reviewing the different formats that we’re providing in terms of the progress reports and then to talk through this particular chart one more time, given that it’s maybe a little bit complex. As you mentioned, we’re providing feedback in a number of ways on each of the English language skill areas. The initial chart that was presented showed the results for all children, in this case looking at letter recognition, while the chart isn’t still up there, for that program, they began at a typical level of 28% in the fall and then improved to 62% in the spring--which is roughly progress from about 7 letters of the alphabet in the fall to 16 letters in the spring.
What we’re trying to do in these subsequent charts is to provide more of a breakdown of how that progress works for different types of children. One form of that breakdown that we didn’t display, but that Nick described, is to show the results separately for English language learners and for children whose native language is English. Then, in this example, what we’re doing is breaking down that information in two different ways by skill level, as Nick mentioned.
And again, to just work it through one more time, to possibly help the audience to follow and to clarify questions you would want to ask. What we’re showing on the left-hand side of this chart is measured against the standard of the Congressionally-mandated ten letters of the alphabet goal. What we’re showing is the proportion of children in the fall and in the spring that were at or above that level. So for this program, they had a very large proportion of their children--71%--that had not achieved knowledge of ten or more letters of the alphabet in the fall. But by the spring that percentage had been dramatically reduced by more than half.
Another way you could look at this chart is to say--which could be the more positive way of saying it--at the beginning of the year, only 29% of the children knew ten or more letters of the alphabet. By the end of the year, two-thirds of the children in the program were at or above that level. So I think this is a way of getting underneath the typical growth level and seeing how children perform based on this other method.
Further, as Nick mentioned, the desired way for these charts to look, is for the chart, for the spring results to be smaller than the height of the bar for the fall. Because what we’re trying to show is movement of children out of the lower skill levels to a higher skill level.
Finally, by comparing these results to the T-bar results, what we’re able to see is how this program stacks up to assessment results for Head Start programs nationally. In this instance, if you’re a program that is above average in terms of your children’s knowledge of the letters of the alphabet, your results are going to be below the T-bar which means you have a smaller proportion of your kids at this at the lower skill level. To explain the same format that’s defined according to the criteria of a high level of skill, a relatively high level of skill, over on the right-hand side of this chart.
What we’re looking at is what percentage of these children in this program could identify 17 or more letters of the alphabet. What we see is almost one out of five of them early on in the Head Start year could demonstrate that level of success based on the NRS assessment. By the end of the year, a majority of their children could identify 17 or more letters of the alphabet. And again they are showing those results in comparison to other Head Start programs at a slightly higher level in both the fall and the spring. So the kids that they’re taking into their Head Start program know a little more about the alphabet than the typical Head Start child, but they’re continuing to help that group of kids progress at the about same rate as other Head Start programs.
So with that, I think we’re going to move along to another example. I thought it would be helpful to pause and work it through one more time verbally with folks.
Nick Zill:
Thanks, Tom.
The second example Skill Level Growth Chart shows the progress made by a sample program’s assessed children in Vocabulary.
(SHOW TOP-RIGHT CHART ON PAGE 8 OF SAMPLE PROGRAM REPORT)
The skill levels for the vocabulary test are defined in terms of the average vocabulary size of typical U.S. children at various ages. The high skill level represents having a vocabulary size that is equal to the average vocabulary size of U.S. children (not just Head Start children) who are of the same age as the assessed child or older. In this example, you see that in the fall, 15% of the program’s assessed children have a vocabulary that is equal to that of typical child at the same age or older. By the spring, 25% of the assessed children have such a vocabulary.
The low skill level for Vocabulary represents a vocabulary size that is equal to the average vocabulary size of U.S. children who are one-year or more younger than the assessed child. The vocabulary of children scoring at this level is significantly less than what would be expected of children of that age. In this example, over half (or 52%) of the program’s assessed children have a vocabulary size equal to that of a child one year or younger than they are. However, by the spring, this percentage drops to 38%. Tom…
Tom Schultz:
Sure, let me just mention, without going through the explanation of the relative size issue which is the same for this chart as for the prior one, that we have used a different approach in reporting the vocabulary skills, as Nick described, using the number of words that children answered correctly and didn’t answer correctly, and based on the individual age of children, converting those scores to show their relationship to national norms in a sense.
And then in this chart, what we’ve reported out is the progress of the program in moving children closer to a typical level of vocabulary for any four or five year old and adjust them for the individual ages of children. Again, the desired direction to see in this kind of chart is to have a lower level in the spring than in the fall of children at this lower level of vocabulary capacity and to put an increasing number of children into this essentially standard of meeting national norms.
One other point just to help people as you’re absorbing these results for your program is that you might have noticed that in the example of the letters of the alphabet, this sample program was at higher levels in both the fall and the spring. In the case of English language vocabulary, the pattern is reversed. There, children are entering the program with a higher proportion of kids that are a year or more behind in age by vocabulary and that is continuing to be true, again, comparing to other Head Start programs against this metric of national norms.
It is still the case that in a typical Head Start program, there were fewer children at this low level in the spring than in the fall. It is also clear, if you look carefully, that they are narrowing the gap here, which is that they are getting closer to the typical proportion for other Head Start programs.
And with that one, we turn back to Nick to continue to talk through the remainder of the growth charts.
Nick Zill:
Thanks again, Tom.
Please note that these charts are meant to give you reference points to help in evaluating the progress that children in your program are making. They are not meant to convey a message about what the Head Start Bureau expects as a minimum rate of progress or level of achievement by the time of the spring assessment.
SLIDE 3a
SKILL LEVEL GROWTH CHARTS: Chart for Understanding Spoken English…
It is important to note that the skill level growth charts for Understanding Spoken English is reported only for English Language Learners who were assessed in both fall and spring. This is because Understanding Spoken English task serves primarily as a screener to determine which English Language Learners have sufficient English proficiency to continue with the rest of the English assessment.
SLIDE 3b
SKILL LEVEL GROWTH CHARTS: Chart for Understanding Spoken English…
The skill level growth charts for Vocabulary, Letter Recognition, and Early Math Skills are for all children who passed the English-language screener.
Now Dr. Ruth Hubbell-McKey of Xtria will tell you about information in the report derived from the Computer-Based Reporting System. Ruth…
Segment 2b: Head Start NRS Spring 2004 Child Assessment Report: Pages 9-10
Ruth Hubbell-McKey
Vice President, Research and Program Services
Xtria
Thanks Nick…
SLIDE 1
PROGRAM PROFILE INFORMATION
The Program Profile at the end of each report contains two sections: Spring 2004 assessment numbers for each program, and demographic and operational characteristics of the program.
SLIDE 2
COUNT OF SPRING 2004 ASSESSMENTS
Spring 2004 Assessment numbers are on page 9 of the report. This page shows the number of kindergarten-eligible children entered into the CBRS, the number of children who dropped out of the program, and then the numbers for each type of “non-complete” or “NC” statuses that were entered such as assessment prohibited by IEP. Child had a severe disability, no Spanish assessor available or prolonged child absence during the data collection period. Then it shows the number of valid assessments that we were able to receive and scan by English, Spanish and both languages. These numbers are sometimes different than the number of eligible children minus non-completes because the assessment scoring sheets were sometimes missing key data such as an ID number or were damaged or lost. Finally it shows the number of valid assessments for children who were assessed in both the fall and spring.
SLIDE 3
PROGRAM PROFILE
On page 10 of the report, the program profile provides a summary of the descriptive information entered into the CBRS for your program. Under Program Characteristics this includes child characteristics, such as average age; classroom characteristics such as percentage of teachers with a Bachelor’s degree or higher; and program characteristics such as Auspice. On several of the items, your program is categorized into High, Moderate and Low subgroups. These subgroups are defined in the footnotes and correspond to reference tables that you can download from the CBRS.
SLIDE 4a
REFERENCE TABLES/CHARTS
These reference tables, which will be posted on the CBRS tomorrow, can be used to compare Spring and Growth results with those of programs with similar characteristics or in your same region or state. The reference tables include subgroups based on average child age, class day option, percentage of English Language Learners, percentage of racial or ethnic minority children, percentage of children in their first year of Head Start, program auspices, and program location.
The reference tables contain Spring child assessment scores, Fall-Spring Growth Charts, and Skill Level Growth Charts of programs with certain characteristics. You can compare your own program profile, as well as your own program’s baseline child assessment scores with those of other programs with characteristics similar to your program. These reference tables will help you interpret your data and put them into context.
SLIDE 4b
REFERENCE TABLES/CHARTS
To access the reference tables, program users can click on the Report menu of the CBRS and select report #5, “Spring 2004 Reference Tables”. Detailed instructions and a video demonstration on how to access the new Spring Growth reports are available on the CBRS under the “What’s New” section.
SLIDE 5
QUESTIONS ON SPRING 2004 REPORTS
If you have any questions about the scores on your Spring report, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to:
SLIDE 6
QUESTIONS ABOUT CBRS
If you have any questions about the Computer-Based Reporting System, please call the CBRS HOTLINE AT 1- 800-669-2170 or email questions to:
The successful implementation of the Head Start National Reporting System depends on your continued hard work. You did a great job last year. Let’s do an even better job this new program year.
Thank you.
Back to you Ann…
Segment 3: How to Use Your Spring 2004 NRS Report; Fall 2004 Data Collection
Tom Schultz
Senior Advisor on Outcomes
Head Start Bureau
Thanks, Ann. In this last presentation, I will talk through our guidance on how to understand and use the spring 2004 NRS Reports. Second I will highlight the importance of completing the current fall assessment effort.
You may remember that our report to you on last fall’s assessment included a single page of do’s and don’ts on how to understand and use the information. Among the points of emphasis were to view NRS information in context with your local ongoing child assessment results, to view the feedback as a baseline from which children will be expected to progress from, and to not view the report as evidence of your program’s effectiveness.
Our guidance on using the results of spring assessments and fall-spring progress builds on the points and outlines a 5-step process for working with your new program report:
SLIDE 1
CREATE A POSITIVE CLIMATE
Step 1: set up a positive process for discussing your report. We recognize that people are curious about their program’s NRS information, given the hard work they invested in implementing this new initiative. They may be anxious about seeing a report on children’s progress that compares results from your agency to results from other Head Start programs. We suggest some simple points of perspective to bear in mind as you share and discuss your program report, including:
SLIDE 1a
CREATE A POSITIVE CLIMATE
- CONT.
Step 2, work step-by-step through your report to understand which each chart tells you about your assessments of last year’s 4- and 5-year-olds. Our suggestions in this area follow the explanations provided by Nick Zill and Ruth Hubbell-McKey in this broadcast.
SLIDE 2
REVIEWING ASSESSMENT
We urge you to recognize that each part of the report provides additional information to understand and think about in terms of a picture of last year’s preschoolers.
SLIDE 2a
REVIEWING ASSESSMENT
We encourage you to read the explanations of the spring skill level charts on pages 1& 2 and of the fall-spring progress charts on pages 5 & 6.
SLIDE 2b
REVIEWING ASSESSMENT
Take advantage of the information on national averages for all Head Start children and the reference table information for programs with similar characteristics to your program.
SLIDE 2c
REVIEWING ASSESSMENT
If you still have questions about how to understand any part of your report, don’t hesitate to ask for help.
SLIDE 3
REVIEWING ONGOING CHILD ASSESSMENTS
Step 3: review ongoing child assessment information on last year’s preschools alongside your NRS report. We expect that programs have on hand information from last year’s local assessment of children, including how they progressed on different indicators through the year. We also assume your local assessment includes information on the Congressionally-mandated measures covered by NRS. Our recommendation is that you review this information alongside your NRS report. You can begin by comparing patterns of progress from your NRS and local assessment information. Beyond that you can use your local assessment information to explore additional questions. Since NRS reports are limited to program-level at present, you can’t use them to understand whether rates of progress were similar or different across classrooms and centers in your agency. However, we can explore that question by studying your local assessment information.
Step 4: consider implications for program improvement.
Before reviewing our suggestions in this area, let’s step back. One way to frame your perspective on what this information means for your program is to think for a minute about what these reports tell us and what they don’t tell us.
The NRS is designed to provide three types of information on preschool children, for each grantee and delegate agency:
SLIDE 4
WHAT NRS REPORTS TELL US
SLIDE 4a
WHAT NRS REPORTS TELL US
SLIDE 4b
WHAT NRS REPORTS TELL US
However, your NRS report does not answer the most central questions in planning ways to enhance your program’s effectiveness:
SLIDE 5
WHAT NRS REPORTS DON’T TELL US
Developing and testing answers to these questions calls on all of the experience and knowledge of your staff, management team, parents and community partners. The Bureau is dedicated to helping you in this process. We are continuing to analyze our research and evaluation studies and our new NRS assessment data base to document relationships between program strategies and child outcomes. And our training and technical assistance network is dedicated to sharing promising and proven program improvement strategies across the Head Start community.
With this backdrop, we have some suggestions to offer on ways to make use of your initial NRS program report:
SLIDE 6
IMPLICATIONS FOR PROGRAM IMPROVEMENT
We recognize that your program is already underway for this year’s children. Given that reality, keep your NRS results in mind as you work with teachers and other program staff to implement your curriculum, individualize learning opportunities, and provide ongoing professional development for staff. As we have mentioned in previous broadcasts, if you are looking for specific teaching strategies and age-appropriate strategies to enhance progress on specific learning indicators,
SLIDE 6a
IMPLICATIONS FOR PROGRAM IMPROVEMENT
Check out the Head Start Leader’s Guide to Positive Child Outcomes, available through the Head Start Publications Center.
SLIDE 6b
IMPLICATIONS FOR PROGRAM IMPROVEMENT
Two additional suggestions are to provide strong support to your ongoing child assessment efforts. As an example, make sure that child development staffs have the time and assistance to review and discuss assessments and their implications for working with children. Finally, plan to incorporate NRS information into your program’s self-assessment process, whenever it occurs during the next year.
SLIDE 7
COMMUNICATING NRS INFORMATION
Step 5: communicating NRS information. We recommend that you share information from your report with your management team, staff, governing board, and Policy Council. Sharing information is appropriate based on the investment of program resources in the NRS and its intended purpose as a management tool.
SLIDE 7a
COMMUNICATING NRS INFORMATION
We recommend that you present NRS results along with other assessment and program effectiveness information.
SLIDE 7b
COMMUNICATING NRS INFORMATION
We also suggest that you describe how this information will be incorporated into you local program self-assessment process.
We hope these suggestions are helpful to you. We continue to view the NRS as a work-in-progress. We welcome your feedback from you on this version of the NRS report. We would like to know how you are able to use it and your suggestions for improving it in the future.
Before we open our final question and answer session, let me urge programs to finish up the current fall assessment effort. As I said earlier, programs have registered over 442,000 children in the CBRS and reported on the status of assessments for nearly 256,000 children. This reflects a huge effort and a faster pace of progress than last fall. However, we still have a big job to complete. We have nearly 200,000 children that have not been assessed. We have about 90 grantees that have not yet registered this year’s children on the CBRS. And we have another 400 grantees that have not yet reported completing any fall assessments.
As you should know, we established October 31st as our deadline for completing all fall, 2004 assessments. Last year our goal was to complete the initial assessments by the end of November. We set an earlier deadline this year in order to have assessment information for children at a point closer to the beginning of their Head Start experience. If we can achieve this goal, next year’s NRS program reports will more fully document the progress children make over the course of this year of Head Start experiences.
So, I want to wrap up with reminding programs about the six steps you should be completing for this fall’s NRS assessment effort:
SLIDE 8
COMPLETING FALL NRS ASSESSMENTS
SLIDE 8a
COMPLETING FALL NRS ASSESSMENTS
One final priority I will share is to urge you to continue to work with integrity in managing the NRS. At the national level, the quality of the first year assessments was excellent. However we have heard some stories of programs making copies of answer sheets for individual children and teaching to the test in inappropriate ways. We believe you want to do the right thing for the children that you work with every day. We urge you to resist any sense of pressure to teach children specific answers from NRS assessments or to coach them on test-taking. Instead, spend your time with them in engaging, challenging and age-appropriate learning experiences. This is the way to maximize their benefits from Head Start and their readiness to success in school.
Back to you Ann…
Transmisión en Internet
La transmisión por internet de Octubre 27, 2004 “Head Start Bureau Updates”, incluyendo una versión con subtítulos en español, estará disponible en vivo en Octubre 27 y por demanda o solicitud en el archivo hasta Enero 27, 2005 en:
www.flyingcolorsinteractive.com/headstart - Cada buscador de información, media player y presentadores seran utilizados combinadamente para presentar éste importante programa. Desafortunadamente, no todas las combinaciones de éstos recursos permiten ver la transmisión. Recomendamos que usted utilice Microsoft Internet Explorer browser (versión 5 o 6). Si usted tiene una versión anterior puede mejorarla visitando:
www.microsoft.com/windows/ie/downloads/critical/ie6sp1/default.asp - Cada persona necesita una versión de Microsoft Windows Media Player (versión 6.4, 8 o 9) para ver la transmisión. Si usted no tiene Windows Media en su computadora, puede instalarla grátis visitando:
www.microsoft.com/windows/windowsmedia/download/default.asp - Para probar la capacidad de su computadora para acceder a ésta transmisión antes del evento favor visitar www.flyingcolorsinteractive.com/headstart. La prueba por éste medio será el programa “Head Start Bureau Updates” en Agosto 4. Si no puede ver ésta prueba y no puede resolver el problema, puede obtener apoyo técnico escribiendo al support@flyingcolorsinteractive.com.
Head
Start Bureau August Updates
National Satellite Broadcast and Web Cast
August 4, 2004, 1:00 - 3:00 PM EDT
Presentations
The text from each segment of the August 4, 2004 Head Start Bureau Updates National Satellite Broadcast and Web Cast is available for viewing on-line. You may click on the segment number in order to view the text from that segment. Where slides were included in the presentation during the broadcast, they are available to view on-line or to download as PowerPoint files. Just click on the presentation topic to access the slides.
Segment 1a: NRS Spring 2004 Assessment Results
Nick Zill
Project Director
Westat
Thank you Ann.
SLIDE 1
HSNRS SPRING 2004 ASSESSMENT RESULTS
I will be giving an overview of the results of the Spring 2004 National Reporting System Child Assessment.
SLIDE 2
CHILD ASSESSMENT RESULTS
I will present information on the number of children assessed this spring and the number for whom matched assessments were obtained in both fall 2003 and spring 2004. I will also present information on the accuracy and reliability of the spring assessments. Then I will share some preliminary findings from our analyses of the growth in skills that Head Start children showed from fall to spring of the program year. I will also outline our plans for reporting of the spring results to Head Start programs and for analysis of the fall-spring growth data.
SLIDE 3
NUMBER OF SPRING ASSESSMENTS (1)
Once again, Head Start staff around the country have done an outstanding job of completing the NRS child assessments in a timely and conscientious fashion. To date, we have received and scanned completed spring answer forms for nearly 407,000 children. Assessment data have been received from 1,766 Head Start programs. Although these numbers are slightly lower than the totals for last fall, they still represent a monumental and highly gratifying accomplishment.
SLIDE 4
NUMER OF SPRING ASSESSMENTS (2)
Information entered into the Computer-Based Reporting System indicates that an additional 10,000 assessments were completed this spring for which we have not yet obtained answer forms. There are some 132 Head Start programs that have reported carrying out significant numbers of assessments this spring, but have apparently not sent any assessment answer forms to Pearson, the scanning contractor. NRS representatives and regional office staff have been contacting these programs to find out what why the delay in receiving answer sheets has occurred. If you have completed answer sheets at your program that have not been sent in, please do so immediately. There is still time to get your data entered into the final spring analysis and reporting process.
SLIDE 5
SPRING 2004 ASSESSMENTS
Broken down by language of assessment, the totals for the spring are that some 369,000 assessments, or 91 percent, were completed in English. Approximately 87,000, or 21 percent, were completed in Spanish. And some 52,000 children, or 13 percent, were assessed in both English and Spanish.
SLIDE 6
FALL-SPRING GROWTH DATA
How many of the children assessed in the spring were also assessed in the fall, enabling us to track their growth from fall to spring? Thus far, we have matching fall and spring assessments for some 333,000 children, representing 82 percent of the total number of children assessed in the spring. The primary reasons for not obtaining matched fall-spring assessments were that children dropped out of Head Start between the fall and spring, or enrolled in Head Start after the fall assessments were completed.
Let us turn now to information on the accuracy and reliability of the spring assessments. Based on the data we have analyzed to date, it appears that trainers and assessors have done an excellent job overall in terms of the quality and consistency of assessments. It also appears that the changes in procedure introduced to improve accuracy and reliability have had their intended effects.
SLIDE 7
ACCURACY IN ENTERING IDs
The accuracy with which assessors entered child ID numbers - and the addition of information on the child’s birth date - enabled us to do a better job of matching up child ID numbers across the Computer-Based Reporting System and the scanned answer sheets, compared to what was achieved in the fall. Only one percent of the total number of answer sheets, or 4,074 answer forms, could not be properly matched with data entered by the same program in the CBRS. We believe that we will be able to use supplementary procedures to match a substantial fraction of these unmatched assessment forms, thus reducing the unmatched percentage still further. And we hope that programs will do even better this fall in double-checking their assessment IDs for accuracy.
SLIDE 8
RELIABILITY OF ASSESSMENT (1)
Our preliminary look at the internal-consistency reliability of the spring assessment data show good reliability at the individual child level. For the English assessment components, the reliability coefficients ranged from 93 percent for the Early Math subtest to 76 percent for the Understanding English screener. The reason why the reliability of the screener was a bit lower than that of the other English components is that the mean scores were so high. The other reliabilities were just where we anticipated they would be based on the field test and fall assessment results.
SLIDE 9
RELIABILITY OF ASSESSMENT (2)
Another indication of improved administration is that the internal consistency reliabilities of the Spanish assessment components in the spring were as good as the those of the English components. In the fall, the reliabilities of the Spanish components had been somewhat lower than the English reliabilities. The spring Spanish reliability coefficients at the individual child level ranged from 93 percent for Spanish Letter Naming to 81 percent for Understanding Spoken Spanish.
SLIDE 10
RELIABILITY OF ASSESSMENT (3)
These good reliabilities at the individual child level mean that reliabilities are excellent when child data are aggregated to the program level. At the program level, the mean scores for all English and Spanish assessment components have reliability coefficients at or above 90 percent.
Let us turn now to a preliminary look at the growth in cognitive skills that Head Start children showed from the fall to the spring NRS assessments. These analyses are based on the 333,000 matched fall and spring assessments.
SLIDE 11
PRELIMINARY GROWTH ANALYSIS RESULTS
The preliminary analyses show that Head Start children showed significant growth in all English-language skill areas. For example, in the subtest assessing children’s Understanding of Spoken English, Head Start children went from a mean of 15.9 items out of 20 correct in the fall to a mean of 17.8 items correct in the spring, or from 79 percent correct to 89 percent correct.
SLIDE 12
GRAPH: Growth in Understanding Spoken English
This subtest was used as the English language screener. The proportion of Head Start children who showed a basic understanding of spoken English sufficient to pass the language screener increased from 90 percent in the fall to 96 percent in the spring.
SLIDE 13
GRAPH: Basic Understanding of Spoken English Increased…
This means that 60 percent of those children who could not pass the English screener in the fall acquired sufficient language skills by the spring to be able to pass the screener.
Head Start children showed growth in their English vocabulary knowledge.
SLIDE 14
GRAPH: Growth in Vocabulary
In the Peabody Picture Vocabulary subtest, Head Start children went from a mean of 53 percent of the items correct in the fall to a mean of 67 percent correct in the spring, or from 12.7 items correct to 16.1 items correct out of 24. The gap between where Head Start children were in the fall and national age norms for vocabulary knowledge was significantly reduced by the spring.
Head Start children also showed significant growth in their ability to recognize and name letters of the English alphabet.
SLIDE 15
GRAPH: Growth in Letter Recognition
In the fall, Head Start children could identify an average of 6 letters or 25 percent of the letters of the alphabet. By the spring, they could identify an average of 14 letters, or 54 percent of the letters of the alphabet.
SLIDE 16
GRAPH: Majority At or Above Congressional Benchmark
In the fall, three-quarters of Head Start children knew less than 10 letters of the alphabet, and many could only recognize one or two letters. By spring, the proportion who knew less than 10 letters fell to 42 percent. A 58 percent majority of Head Start children were at or above the Congressional benchmark for letter recognition, knowing 10 or more letters of the alphabet.
Indeed, by the spring a substantial minority of Head Start children could recognize most or all of the letters of the alphabet. The proportion knowing 17 or more letters grew from 16 percent in the fall to 43 percent in the spring.
Head Start children also showed growth in Early Math Skills.
SLIDE 17
GRAPH: Growth in Early Math Skills
They went from a mean of 46 percent of items correct on the Early Math subtest in the fall to a mean of 68 percent of the items correct in the spring, or from 9.6 out of 21 items to 14.3 out of 21 items correct.
SLIDE 18
PRELIMINARY SPANISH GROWTH ANALYSIS RESULTS
Head Start children assessed in Spanish in the fall and spring also showed growth in all skill areas. However, we should realize that many of the children in Head Start programs on the U.S. mainland did not receive instructional support in Spanish. They did receive such support in programs in Puerto Rico and in some bilingual programs on the mainland. Therefore, it should not be surprising that Head Start children in Puerto Rico showed more substantial growth in their Spanish-language skills than did Spanish-speaking children in mainland Head Start programs.
Overall, children assessed in Spanish showed growth in Understanding Spoken Spanish. Their mean scores on the Spanish language screener went from 67 percent correct in the fall to 77 percent in the spring.
SLIDE 19
GRAPH: Growth in Understanding Spoken Spanish
The proportion of children who passed the Spanish screener went from 93 percent in the fall to 96 percent in the spring.
Head Start children assessed in Spanish showed growth in their Spanish vocabulary knowledge.
SLIDE 20
GRAPH: Growth in Spanish Vocabulary
They went from a mean of 12.8 items correct out of 24 in the fall to a mean of 15.3 correct in the spring, or from 53 percent to 64 percent correct.
Compared to children on the mainland, children in Head Start programs in Puerto Rico started out with greater Spanish vocabulary knowledge in the fall and showed greater vocabulary growth by the spring.
SLIDE 21
GRAPH: Children in Puerto Rico Show Greater Spanish Vocabulary Growth
Head Start children in Puerto Rico went from 59 percent of items correct on the Spanish vocabulary test in the fall to 78 percent correct in the spring. By comparison, Spanish-speaking Head Start children in mainland programs went from 52 percent correct in the fall to 60 percent correct in the spring.
Spanish-speaking Head Start children showed growth in Spanish letter naming from fall to spring, but their letter recognition skills were behind those of children assessed in English.
SLIDE 22
GRAPH: Growth in Spanish Letter Naming
In the fall, children assessed in Spanish identified a mean of 3 letters out of the 30 letters of the Spanish alphabet, or 10 percent. By spring, they identified a mean of 9.6 letters, or 30 percent.
Children assessed in Spanish also showed growth in their Spanish Early Math Skills.
SLIDE 23
GRAPH: Growth in Spanish Early Math Skills
They went from a mean of 8 out of 21 items correct in the fall, or 38 percent, to a mean of 12.7 items correct in the spring, or 61 percent.
Let me turn now to our plans for reporting results from the first full year of National Reporting System implementation.
SLIDE 24
NRS Reporting Plans (1)
Each Head Start program that submits sufficient data will receive an individualized report on its assessment results in September. This report will describe the mean achievement levels achieved by children in the program in the spring, and the gains achieved by children in the program who were assessed in both the fall and spring.
SLIDE 25
NRS Reporting Plans (2)
In the reports each program receives in September, the program will be able to see how its average achievement levels and gains compare with those of other programs.
SLIDE 26
NRS Reporting Plans (3)
Analyses available in the reports and on the CBRS will enable programs to compare themselves not only with national averages, but with achievement levels and gains of programs similar to themselves.
We are in the process of carrying out analyses at the program level that will facilitate this process.
SLIDE 27
NRS Analysis Plans (1)
Average achievement levels and gains shown by Head Start programs are being related to demographic and operational characteristics of programs. Separate analyses are being carried out for each of the assessment components, and for English and Spanish-language assessments.
SLIDE 28
NRS Analysis Plans (2)
These analyses will examine such things as the difference in growth shown by children in full-day versus part-day Head Start programs. The results may show which operational characteristics are associated with greater or lesser gains in early achievement. The analyses will also enable programs to compare how their children are doing compared with the levels and gains that might be expected, given the social and economic disadvantages of the families they serve.
By any measure, the fall and spring implementations of the Head Start National Reporting System have been an outstanding success. That success has been built on the hard work of staff in the overwhelming majority of programs that cooperated fully with the NRS implementation. We look forward to working with programs to harvest the guidance that can be gained from the treasure trove of comparative data that has been generated. And we hope that the small number of programs that did not participate will see how much can be learned from NRS data and will participate fully in the coming year.
Thank you, Ruth….
Segment 1b: NRS Spring 2004 CBRS Data
Ruth Hubbell-McKey
Vice President, Research and Program Services
Xtria
SLIDE 1
TITLE SLIDE
Thank you Nick….
SLIDE 2
OVERVIEW
I will highlight data from the Computer Based Reporting System (CBRS) about the assessments completed and not completed and I will discuss some of the characteristics of children and assessors. I also will provide a brief overview of some of the new and improved features of the CBRS that will make the data entry for your programs easier in the Fall and I will give you some tips on how to manage your assessment materials so that they are received in a timely manner and able to be used for analysis.
SLIDE 3
NON-COMPLETE ASSESSMENTS
The vast majority of children participating in the NRS last year were able to be assessed, however, some were not. This slide shows the total number of non-complete assessments and the top five reasons why children were not assessed for the fall and spring. Specifically, in fall 2003, a total of 29,726 children were not assessed. In spring 2004, this number increased to 87,314, mainly due to the fact that children dropped out of the program. As you can see, in the fall, 10,400 children dropped from Head Start, but in the spring this number increased to 58,409.
The next most frequent reason for non-completion of assessments was prolonged absence. 7,745 children in the fall and 11,974 children in the spring were not assessed because they were not in attendance during the assessment period. Parent refusals also were a reason for non-complete assessments, accounting for 5,816 of the non-complete assessments in fall and 7,921 in spring. Further analysis indicates that the majority of the non-complete assessments due to parent refusals were generated by a small number of programs. Technical assistance will be provided to these programs in order to assist them in their efforts to gain parent support for the NRS.
In terms of child cooperation with the assessment, there were considerably fewer children in the spring who would not cooperate, only 1,406 down from 3,349 in the fall. While this may reflect child maturation, it may also be due to increased assessor skill in persuading children to participate. Lastly, having a severe disability accounted for 936 non-complete assessments in the fall and 779 non-complete assessments in the spring.
SLIDE 4
PERCENT OF CHILDREN WITH IEPs
Based on data from the CBRS, in fall 2003, 7.5% of kindergarten-eligible
children had a disability and an Individualized Education Plan (IEP). By spring
2004, this percentage increased to nearly 9%. This increase probably reflects
the fact that time is needed to detect and identify disabilities and develop
an IEP. However, this is less than data from the Program Information Report
(PIR), which indicates that 11.5% of all Head Start children have a disability
and an IEP.
Although the PIR data include children between 3 and 5 years old instead of
just 4 and 5 year olds as the CBRS does,
SLIDE 5
TIME INTERVAL BETWEEN ASSESSMENTS
This slide shows the length of time between the fall and spring assessments for those children who were assessed twice. They average length of time was 7 months for both English and Spanish children. And although the time was shorter or longer for some children, 98% of fall and spring assessments were separated by 6 to 8 months. For this fall’s assessment, I want to emphasize the importance of assessing children as soon as possible. We want to capture a true baseline measurement of children’s performance at the beginning of their Head Start experience. So we encourage you to schedule your local assessment training and begin assessments as soon as possible after the children start this Head Start year.
SLIDE 6
TEACHERS’ LANGUAGE: FLUENCY & INSTRUCTION
While we aren’t presenting data on all the CBRS elements today, this slide provides some data on teacher language fluency and language instruction that hasn’t been presented before. As this table shows, there is considerable diversity in the languages teachers speak fluently and the languages in which they provide instruction. Of the 45,612 registered teachers, approximately 95% are fluent and provide instruction in English. This includes teachers in programs in such areas as Puerto Rico, where instruction is primarily in another language. Teachers are also fluent and provide instruction in a number of different languages including Spanish and Native American languages, but also include languages that are grouped here under Other, but include Far Eastern, Pacific Island, Middle eastern, Central American, European and Slavic, Caribbean and African languages. Teachers can be identified as being fluent in more than one language in the CBRS.
We will also be able to use this information for analyses about English language learners and the language of instruction.
SLIDE 7
ASSESSOR’S ROLE IN HEAD START
Now moving to who conducted the child assessments for the NRS for both the fall and spring assessments, approximately 35% of assessments were performed by Head Start teachers and 40% by other Head Start professionals, such as education coordinators. A much smaller percentage of assessments were conducted by other Head Start staff, consultants, and graduate students. There wasn’t much change from fall to spring, except that the percentage of teachers increased slightly and the number of other Head Start professionals decreased slightly.
Thank you… Back to you Ann….
Segment 2a: Quality Assurance Study Findings
Diane Paulsell
Senior Researcher
Mathematica Policy Research
SLIDE 1
TITLE
Thank you Ann. The Head Start Bureau contracted with Mathematica to conduct an independent, third-party assessment of how well the NRS is being implemented around the country. The purpose of the study is to learn what is working well, and what can be improved in future years.
SLIDE 2
OVERVIEW
Today I’m going to share our main findings on:
SLIDE 3
SPRING OBSERVATION VISITS
During fall NRS implementation, our team visited a nationally representative sample of 35 Head Start programs, and we visited the same programs again this spring. Each site visit lasted about 2 days. In the Spring, we observed Head Start staff conducting the NRS assessment with 349 children (300 children assessed in English and 49 assessed in Spanish). We observed a total of 119 local assessors; interviewed Head Start directors, NRS trainers, and data managers; and held focus groups with staff conducting the assessments to learn about their experiences. We are currently in the process of visiting 4 Migrant and Seasonal Head Start programs to observe assessments and talk with their staff.
SLIDE 4
LENGTH OF ASSESSMENTS
The average length of the assessments we observed was 15 minutes on the English version and 16.5 minutes on the Spanish version. Most of the children completed the assessment in 10-20 minutes.
SLIDE 5
QUALITY OF ENGLISH ASSESSMENTS
Using the certification procedures as a guide, we coded the assessments we observed for errors and computed a certification score for each one. 87% scored an 85 or higher, which is the minimum required for certification. 60% scored in the 96-to-100 range. Overall, this indicates that programs did a very good job in training staff to administer the assessment correctly and consistently.
We compared certification scores on assessments conducted by teachers who assessed children in their own classrooms with those of other assessors, as well as scores for experienced and new assessors. We found no significant differences in mean certification scores for these groups.
We also analyzed our observations to understand more specific types of errors and problems in administering the assessment. The overall error rate was low, with assessors making fewer errors, on average, than they did in the fall.
On average, assessors scored 97% of responses correctly. Errors occurred when assessors had difficulty seeing where the child was pointing, or when the child’s response was ambiguous. In addition, assessors had difficulty scoring the counting item correctly when children skipped marbles or counted the same marble twice. New assessors made more scoring errors, on average, than those who conducted assessments in both fall and spring.
Script errors, in particular, were lower in the spring than in the fall. In the fall, errors in following the script typically occurred on items with more complex or lengthy instructions, or when children did not wait for instructions to be given before responding. This was less of a problem in the spring, perhaps because instructions in some sections had been shortened or reworded.
AREAS FOR IMPROVEMENT
SLIDE 6
Although assessors made fewer errors overall, we identified some aspects of the assessment process in which staff may need more guidance and training. These include:
SLIDE 7
SPANISH ASSESSMENTS
We observed 49 assessments in 9 programs administered using the Spanish version of the NRS. 93% of the Spanish assessments met or exceeded the certification standard. In general, errors were similar to those on the English version.
Some staff administering the Spanish version questioned some of the Spanish words used in the instructions or accepted as correct. Spanish-speaking families in their programs come from a variety of regions of Latin America and the United States. Because of variation in Spanish word usage across regions, children use a range of words for the same object or action. In addition, staff in some programs said that parents’ report of home language was sometimes not an accurate indicator of children’s language ability, and felt they should be able to use their own judgment to determine which version of the assessment to use.
SLIDE 8
CHILDREN’S RESPONSES
Overall, program staff reported that the spring assessment process went more smoothly than the fall process; and that most children responded positively. Program staff attributed this improvement to 3 main factors:
About half of the sample programs reported that problems with difficult behaviors during the assessment persisted from the fall, but overall the problems were less severe.
SLIDE 9
CHILDREN WITH DISABILITIES
All but 2 of the 35 the programs we visited assessed at least one child with a disability in the spring. They made a number of accommodations for these children, including:
Some staff wanted more flexibility to make additional accommodations, such as paraphrasing instructions to help children understand them.
SLIDE 10
LOCAL IMPLEMENTATION APPROACHES
Most programs maintained the same basic approach to conducting the spring assessment that they used in the fall. When programs did make changes, most were due to staff turnover, rather than a rethinking of the program’s approach to the NRS.
On average, the sample programs trained 15 assessors, and 40 percent allowed teachers to assess children in their own classrooms.
Most programs kept their Policy Council and parents informed about NRS implementation throughout the year; and had shared or planned to share the fall assessment results with them. Overall, staff reported that parents continued to have few concerns or questions about the NRS, although in five of the sample programs, some parents refused consent for the assessment. The question asked most frequently by parents was whether they could receive their child’s individual NRS assessment results.
SLIDE 11
SPRING REFRESHER TRAINING
Overall, staff reported that they felt well prepared to administer the NRS assessment in the spring. Staff at most programs we visited found the Spring training materials helpful, especially the training video and guidance on managing children’s behavior during the assessment.
The format for spring refresher training included 3 components: reviewing the spring materials, watching the refresher video, and role playing. About half of the 35 programs covered all of these components during spring training, and 30 of them covered at least 2 of the 3. Only 6 of the 22 programs with Spanish assessors followed this format, however; most Spanish assessors watched the training video on their own.
SLIDE 12
VIEWS ON THE FALL NRS REPORT
In early spring 2004, each local Head Start program received a report on the results of the fall NRS assessment; results were reported both for the individual program and for the Head Start program nationally. During site visit interviews, staff at two-thirds of the programs said they found the reports clear and easy to understand.
At the same time, staff in many programs said that while the fall report was interesting, it was of limited use to them because it was not sufficiently detailed, because it arrived too late in the program year, or because it was only baseline data.
Many program staff we talked to, especially teachers, evaluated the usefulness of the report based on whether they could use the results to individualize program services for addressing skill areas in which particular children did not perform well.
Because local assessments conducted by program staff typically yield such data on individual children and classrooms, cover more domains the NRS, and provide results to teachers early in the school year, staff in many programs felt the local assessments were more useful.
Nevertheless, more than three quarters of the programs reported some plans for using the NRS results for program improvement.
To improve the NRS report, many programs suggested breaking the results out by center or classroom. A quarter of the programs said they would like to receive more assistance in interpreting the report, and more guidance on how they can use the results for program improvement.
SLIDE 13
PROGRAM REACTIONS
During interviews, Head Start staff expressed four main concerns about the NRS:
First, as in the fall, programs worried about how the assessment results would be used. Many staff said they did not have a clear sense of the purpose of the NRS or of what the implications would be for their programs if children didn’t perform well.
Second, programs expressed concern about how much time and resources were required to implement the NRS, especially in the context of other assessments and activities that occur at the beginning and end of the school year.
Third, some programs questioned whether the results would accurately reflect, first, the children’s abilities, but also, the program’s performance.
Finally, some staff were concerned that the NRS might adversely affect Head Start programs, and thus, by extension, the children, by shifting the focus of Head Start too much in the direction of "academics." Staff also worried that a desire to perform well on the NRS might lead teachers to "teach to the test."
SLIDE 14
PROGRAM SUGGESTIONS
Programs made a number of suggestions for improving the NRS. The most common were to:
In summary, Mathematica’s implementation study found that programs have done a good job of administering the NRS assessment. The majority of assessments we observed met or exceeded the standards for quality established by the certification process. And overall, the quality of observed assessments improved from fall to spring.
We also identified some sections of the assessment that are still difficult for staff to administer or score, some areas in which staff may need more guidance and training, and some concerns that programs have about the NRS. Programs also provided useful feedback on the report on fall NRS results. We are in the process of reviewing our findings with the Head Start Bureau and the Technical Work Group, and they will use this information to plan for NRS implementation in fall 2004 and beyond.
Tom…
Segment 2b: Head Start Bureau Next Steps
Tom Schultz
Senior Advisor on Outcomes
Head Start Bureau
Thanks, Diane, for that preliminary look at the quality assurance study findings.
Before we turn to another Q&A opportunity, I want to highlight the significance of what you’ve heard so far this afternoon about your accomplishments in implementing the first year of the National Reporting System. Then I will share the next steps the Bureau will be working on regarding the NRS over the coming year:
In terms of your accomplishments as Head Start managers and staff members:
We know these results weren’t guaranteed in advance. They only came about due to your hard work. And of course, we recognize several areas for improvement as we begin the second year of the NRS. For example, based on the feedback we received from the implementation study, and other concerns from the field, we are committed to working to improve several areas, such as more timely shipping of materials, and stronger efforts to communicate with programs about the purpose of the NRS and how this new information will be used by the Bureau and ACF Regional Offices. We are also well aware of numbers of articles containing critical comments on the design of the NRS. We hope to do a better job of responding publicly to these concerns in the future.
Accordingly, in terms of next steps, our key priorities over the next year look like this:
Let me share some specifics in particular about our third priority area of work. We will be working with the NRS Technical Work Group and the new departmental Advisory Committee on Head Start Accountability and Education Performance Measures to continue to review the quality, reliability and validity of the NRS assessment. For example, we have commissioned an analysis to compare the results of NRS and parallel standardized assessments in programs involved in our FACES research project. We will also track linkages between NRS results and performance of children in the FACES study as they progress into kindergarten. This analysis will provide important new evidence on the validity of the NRS assessments.
In addition, we are pleased that Mathematica Policy Research, Inc. will be continuing its study of NRS implementation for another year, visiting a diverse sample of Head Start programs to provide objective feedback on what aspects of the NRS are working well and areas where we need to improve. Finally, based on guidance from the Technical Work Group we will be field testing additional forms of assessment next year, including measures of children’s social development. We are committed to pilot testing of any new measures in a diverse group of head Start sites before considering any expansion of the NRS.
Back to you Ann…
Segment 3a: Shipment of Materials
Ken Burns
Director
Head Start Information and Publication Center
SLIDE 1
TITLE SLIDE
Thank you Ann, Good afternoon, everybody, I’m Ken Burns, Director of the Head Start Information and Publication Center and I’d like to give you a brief update on our shipping operations in support of the upcoming Fall NRS assessment process.
SLIDE 2
STATUS
The good news is that we have all needed NRS materials in our warehouse. We began shipping to programs this past Monday and some of you will begin receiving your materials before the end of the week. As we have done in the past, we are focusing our efforts in the distribution center, almost exclusively, on shipping NRS materials. We expect to have completed shipping all materials by August 27th.
SLIDE 3
DO I HAVE TO ORDER?
To make things easier for you, and to expedite getting your NRS materials to you, the HSIPC established a web-based, proactive order fulfillment system. Nearly 1,100 programs took advantage of this system to update the quantities needed and to update their contact and shipping information. If you were unable to update your data before the deadline, your program will receive the same number of easels and answer sheets as you received in the Spring. If you need additional answer sheets for refresher training please feel free to make as many copies locally as you need.
SLIDE 4
WHAT IF I NEED MORE?
If for some reason you need more easels or answer sheets because your needs have changed you will find an NRS Order Form included in your order. The order form will include complete information on how to order more materials from us via email, fax, phone or the post office.
SLIDE 5
WHAT AM I GOING TO GET?
In the package or packages we send to you, you will be receiving
SLIDE 6
And…
Also CBRS materials consisting of the following items:
SLIDE 7
How Soon Will I Get My Materials?
We prioritized our shipping plan so that programs opening first will have their orders shipped first. We based our plan on the update information you provided or on PIR data if you were unable to update your data before the deadline. We have already completed shipping materials to all programs outside the U.S. mainland, except for Puerto Rico. We will ship to the Puerto Rico programs on the week of August 23rd. As I mentioned before we will complete all shipping no later than August 27th.
SLIDE 8
Shipping
We will ship your order to you using the United Parcel Service whenever possible. In order to do this we must have a valid street address entered into our system since UPS is unable to deliver packages to Post Office Boxes. When we ship your NRS materials to you using UPS you will receive an automatic email notification the day that your order is shipped informing you of the number of packages to expect, their estimated arrival date and a hot link to the UPS web site allowing you to track your order from our distribution center directly to your program.
SLIDE 9
Why is My Grant Number Important?
Your Grant Number is very important to all phases of the NRS process. The Grant Number provides for continuity in ordering, tracking and reporting throughout the entire NRS process. HSIPC also uses the Grant Number to aggregate, or roll up, individual orders from programs into a single customer record to be used in our proactive, automatic order fulfillment system.
SLIDE 10
What do I do with Last Year’s Materials?
Many of you have contacted the HSIPC with questions about what to do with assessment materials left over from last year’s initial NRS cycle. You may dispose of these materials since they will not be used again for assessing children.
SLIDE 11
Contact HSIPC
If you need to contact the HSIPC Staff for any reason please do not hesitate. You can stay abreast of current NRS news by visiting the HSIPC Web site at www.headstartinfo.org and then by clicking on the link to the NRS Information and Resources page. You can order additional NRS materials or ask us questions by emailing us at puborder@headstartinfo.org.
SLIDE 12
Contact HSIPC
You can also fax orders to our distribution center at 703-683-5769 or phone them directly at 703-683-2878. Our toll-free number is 1-866-763-6481. Calling that number will connect you with our Information Services staff who will be happy to answer your questions or transfer your call to the warehouse if you want to place an order.
The HSIPC staff is ready to assist you in making the Fall Assessment process run smoothly. Please do not hesitate to contact us.
Thank you and now back to you Ann.
Segment 3b: NRS Year Two: Expectations and Changes
Nick Zill
Project Director
Westat
Thank you Ruth.
SLIDE 1
NRS FALL ASSESSMENT TRAINING
I will be giving an overview of the training for the Fall 2004 Child Assessment.
SLIDE 2
GOALS OF TRAINING: To equip you to continue…
Based on the results from last year’s assessment, trainers and assessors have done an excellent job overall in terms of the quality and consistency of assessments. The goal of this training on the Fall 2004 assessment is to equip you to continue that high standard of quality.
SLIDE 3
GOALS OF TRAINING: To have fall assessments give…
Having your assessors participate in your locally conducted training is critical so that your baseline assessments show the most complete, fair and accurate picture of the skills children have when entering your program. There are some new elements to the Fall 2004 child assessment that were not in the previous versions of the assessment, so it is important that all assessors be trained so that the assessment scores adequately reflect the skills of the children in your program.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 4
FALL 2004 ASSESSMENT
The Fall 2004 English assessment covers the same skill areas as last year’s assessment, namely:
The Understanding Spoken English section, made up of the Simon Says and Art Show tasks, will again serve as a language screener to assess the English proficiency of English-Language Learners for continuing with the other sections of the assessment.
SLIDE 5
FALL 2004 ASSESSMENT
In order to reduce the possibility of teaching to the test or practice effects, new items have been included in the Vocabulary and Early Math sections for both the English and Spanish assessments. These items have similar difficulty and cover the same content areas as the items in last year’s assessment. So even though new items are introduced, we will be able to compare scores from the last year’s assessment to this year’s assessment to get a picture of the skills different classes of children bring to your program from year to year.
The Fall 2004 assessment easels and answer sheets include the new items for the Vocabulary and Early Math sections. Please do not use any leftover answer sheets from last year’s assessment because the items on old answer sheets will not match up with those in the Fall 2004 assessment.
SLIDE 6
EXAMPLES OF NEW EARLY MATH ITEMS
Here are examples of some of the new Early Math items in the Fall 2004 child assessment.
[SHOW VIDEO CLIP]
SLIDE 7
FALL 2004 ASSESSMENT
The Simon Says, Art Show, and Letter Naming sections will remain the same as in last year’s assessment.
SLIDE 8
FALL 2004 ASSESSMENT
Like the English-language version of the assessment, the Fall 2004 Spanish assessment covers the same skill areas as last year’s Spanish assessment, namely:
Likewise, the Understanding Spoken Spanish section will serve as a Spanish language screener.
SLIDE 9
CHANGES IN THE FALL 2004 PROCEDURE
Last year, Spanish-speaking children were assessed in English first and then assessed in Spanish. This fall, Spanish-speaking children will be assessed in Spanish first and then assessed in English. This procedure was followed for Migrant and Seasonal Head Start programs and worked well in those programs.
SLIDE 10
FALL TRAINING MATERIALS
As Ken Burns mentioned, HSNRS Fall 2004 training and child assessment materials will be automatically sent to you based on the number of materials you received last spring. The materials for the fall assessments include:
Trainers should review and familiarize themselves with these new materials before conducting their local trainings.
You should use these materials for your Fall 2004 assessor training and data collection. Please do not use the easels or any other materials from last year’s assessment (such as Fall 2003 or Spring 2004 materials) for this fall’s data collection. These materials should be discarded
SLIDE 11
FALL 2004 TRAINER’S MANUALS
The Fall 2004 Trainer’s manual contains item-by-item instructions for the new Vocabulary and Early Math sections and new procedures for assessing Spanish-speaking children. Further, the updated assessor’s guide provides more guidance in how to score children’s responses and how to handle behavioral issues you may encounter during an assessment.
SLIDE 12
FALL 2004 TRAINER’S MANUALS
The Fall 2004 Trainer’s manual contains guidance on how to prepare your staff to implement the National Reporting System in your program. It contains the training script used in the one-day training of your local assessors, the hard copy originals of the assessment certification form, and the quiz so you can certify your local assessors on the Fall 2004 assessment. It also contains role play scripts for practice with the new assessment.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 13
FALL 2004 EASELS
New easels will be shipped to you for the Fall 2004 assessment. They contain the new items for the vocabulary and early math sections. Further, some of the scripted instructions in the easels have been simplified or clarified from last year so that assessors and children will have a better understanding of what they are supposed to do in the task.
SLIDE 14
ASSESSMENT DEMONSTRATION VIDEO
The Assessment Demonstration Video features a demonstration of the entire Fall 2004 assessment conducted with a preschool-aged child.
The video also demonstrates methods for handling various types of behaviors children may exhibit during an assessment, such as being rowdy, not paying attention, or being overly shy and reluctant to respond.
SLIDE 15
CLIP FROM NRS ASSESSMENT DEMONSTRATION VIDEO
Here are some of the helpful tips on handling behavior problems presented in the video.
[SHOW VIDEO CLIP]
SLIDE 16
BASIC FORMAT OF REFRESHER TRAINING
As this is the second year of implementing the National Reporting System, there are two paths for training your local assessors in the Fall 2004 child assessment. The only difference is whether or not assessors need to be certified. Those assessors who were certified last year won’t need to be certified again this fall. New assessors will need to be certified. Regardless of whether they need to be certified or not, all assessors have to participate in the one-day training of the new child assessment.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 17
TRAINING FOR CERTIFIED ASSESSORS
Assessors who were certified last fall must participate in the one-day training on the Fall 2004 assessment easel and role play with the new assessment easel. The trainer should observe the trainees administering the assessment during the role play. While these assessors do not need to be formally recertified, the Assessment Certification Form can be used to provide feedback on their strengths and weaknesses. If the trainer observes any problems, the trainer should provide feedback and plan for another observed role play.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
Once the training is completed, the names of these assessors should be entered into the Computer-Based Reporting System (CBRS) to obtain an Assessor ID number for the new program year.
SLIDE 18
TRAINING FOR NEW ASSESSORS
New Assessors need to be certified. New Assessors must participate in the one day training on the Fall 2004 assessment easel and role play with the new assessment easel. The trainer should observe the trainees administering the assessment during the role play and provide feedback on their strengths and weaknesses.
Please contact the Westat NRS assessment and training hotline at 1-888-847-1176 from 9 a.m.-7 p.m. Eastern Time if you have questions. Questions may also be submitted via e-mail to hsnrshelp@westat.com.
SLIDE 19
TRAINING FOR NEW ASSESSORS
As was true from last year, new assessors must take the quiz and be observed administering an assessment. The certification assessment should be done with an actual child if at all possible. The trainer should use the Assessment Certification Form to evaluate how well the trainee follows the standardized assessment procedures. The trainer can also use the Assessment Certification From to provide feedback on strengths and weaknesses.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 20
TRAINING FOR NEW ASSESSORS
New trainees must score 85% or more on both the Quiz and the Assessment Certification Form in order to become a Certified Assessor.
After administering the quiz, trainers should review the quiz answers with all trainees. If a quiz score is under 85%, the trainer should provide feedback to the trainee and ask the trainee to take the quiz again.
After trainers have observed the assessment, they should review the assessment with the trainee, discussing any problems. If the certification score is less than 85%, the trainer should provide feedback and plan for another observed assessment.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
After being certified, the names of these newly certified assessors should be entered into the Computer-Based Reporting System (CBRS) to obtain an Assessor ID number.
SLIDE 21
PROCEDURES FOR PROGRAMS WITH NO SPANISH-SPEAKING CHILDREN
Sites without Spanish-speaking children (which therefore will not be training any Spanish-speaking assessors) will train their assessors only on the English version of the Fall 2004 assessment. Since they are only training on the English assessment, these sites will not use any materials for the Spanish assessment.
SLIDE 22
BILINGUAL ASSESSORS
Bilingual assessors, who will be administering both the English and the Spanish assessments, MUST receive both the English and the Spanish versions of the training. They also must participate in role plays, receive feedback, and be certified on both the English and Spanish versions of the Fall 2004 assessment.
SLIDE 23
FALL 2004 ASSESSMENTS
Since these are baseline assessments, the goal is to assess the children at the beginning of their Head Start year, so complete the assessments as soon as possible. The deadline for completing all assessments is October 29, 2004.
SLIDE 24
QUESTIONS ON TRAINING OR FALL ASSESSMENT
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 25
NRS DEPENDS ON YOU
The successful implementation of the Head Start National Reporting System depends on your continued hard work. You did a great job last year. Let’s do an even better job this new program year.
Thank you…. Ruth….
Segment 3c: CBRS Changes and Tips
Ruth Hubbell-McKey
Vice President, Research and Program Services
Xtria
SLIDE 1
TITLE SLIDE
Thanks Nick… Now, I would like to briefly highlight some of the new features of the CBRS. We know that some of you have already starting logging on and entering data for the Fall assessment even though the system has only been open for two days! That’s great!
SLIDE 2
NEW CBRS FEATURES
After responding to your phone calls for technical assistance on the CBRS and listening to your suggestions for improvements, we have added several new features to the CBRS. These new features have been designed to make data entry easier for you. First, we have arranged for programs to transfer data into the CBRS in one of two ways. Programs may use either the Copy Over Feature or the Data Import Feature. But you cannot use both features. You have to choose one or the other. The copy over feature will allow programs to transfer all of your 2003-2004 data into the CBRS, except for child-level data. Thus, data on the program, centers, classes, teachers and assessors can be copied from last year’s database into this year’s database and all related IDs will remain the same. Then you must update any of this information that has changed (for example if a teacher resigned over the summer or you added a new center). Then you enter all your kindergarten-eligible children. Even if a child is returning to Head Start who was kindergarten eligible last year, you will need to reenter him or her. The copy over feature is ideal for smaller programs.
The Data Import Feature is intended for larger programs that currently are using a commercial management information system, such as ChildPlus, Galileo, or HSFIS, to track program operations. We have provided those vendors with the specifications and requirements for the import of data into the CBRS from their systems. You will need to contact your vendor directly for additional information and to determine whether the feature is currently available on their system. If your program chooses to use the Import Feature, then child-level data as well as classroom, center, and teacher data can be transferred to the CBRS. Programs will be required, however, to review the data and enter any missing information, such as data on assessors. Large programs that have a custom management information system may be able to use this feature as well. These programs should contact our technical assistance hotline for more information.
If you are a new program or do not want to use copy over or data import, you can refuse these and enter your data yourself. When you log-in to the CBRS for the first time this fall, you will be asked to choose copy over or data import or say no to both. You only have one chance to make your choice, so think about it carefully. There are descriptions of both options after you log-in and in the new CBRS manuals you are receiving now with your other NRS fall materials. If you have any questions, please call our CBRS hotline at 1-800-669-2170 or send an email to NRStechhelp@xtria.com.
SLIDE 3
NEW CBRS FEATURES
When you log-in to the CBRS, you will notice that the log-in screen has been modified. The screen now contains 4 additional tabs: What’s New, Contacts, and Frequently Asked Questions (FAQs). The "What’s New" tab outlines the new features of the CBRS; the Contacts tab provides unique phone numbers and email addresses for assistance with the CBRS, the assessment, and materials; and the FAQ tab highlights answers to the most commonly asked questions.
SLIDE 4
NEW CBRS FEATURES
We recognize that some Head Start staff, particularly those in Puerto Rico, may be more comfortable in working with the system in Spanish. As a result, the entire CBRS and its data entry fields and instructions have been translated into Spanish. In order to convert the system into Spanish, all you need to do is click on the link "Cambiar para ver en español" at the log-in screen.
SLIDE 5
NEW CBRS FEATURES
To facilitate movement through the system, a search function has been added to the program menu to help you find individual centers, classes or children by ID or name. You just enter the type of data and the ID or name of what you are looking for.
SLIDE 6
NEW CBRS FEATURES
To help grantees monitor the completion of assessments by their delegates, grantees have been given the ability to view their delegate agencies’ data. In addition, several reports have been added to enhance the ability to both monitor data collection and view program data. All reports are now printer-friendly. These are on the reports menu at each user level.
SLIDE 7
NEW CBRS FEATURES
The CBRS manual has been modified to incorporate each of the new features. The CBRS manual will be shipped to you along with your programs’ assessment materials. The manual also is accessible online from the "Technical Assistance" link on the top right corner of most CBRS screens. In addition, an online training presentation on the CBRS is available at http://webtraining.hsnrs.net.
SLIDE 8
TIPS ON MANAGING ASSESSMENT MATERIALS
Now I am going to give you some tips on managing your assessment materials to ensure that the answer sheets are received in a timely manner and the data are able to be used for analysis. Based upon last year’s experience, many answer sheets were unable to be used because they were lost in the mail, incompletely or inaccurately filled out, or damaged. So much hard work has been put into the assessment process, that it is a shame not to be able to use all the data collected. So in order to prevent this from occurring again, I want to review some of the necessary steps that your program can take to ensure that the data arrive in good shape. These steps are also in your trainer’s manual.
First you should have a designated NRS manager who ensures that the following tasks are performed:
As assessments are completed, check each answer sheet to be sure it is complete, and that the entire assessment was done appropriately
Be sure the correct child ID and birthdate are entered on the sheet
Make sure the assessment date is accurately recorded; and
Verify that the assessor identification number is noted and the assessor’s relationship to the child, whether they are the teacher, is coded either yes or no.
SLIDE 9
TIPS ON MANAGING ASSESSMENT MATERIALS
Continue to check each answer sheet to be sure the Fall 2004 answer sheets are used, the blue ones. No other versions will be accepted. Be sure the bubbles are completely filled-in, that a number 2 pencil is used, that there are no stray marks or stains on the sheet, and the sheets are not wrinkled or creased. Do not separate the pages of the answer sheets! If any of these occur, the sheet may not be able to be read by the scanning machines. If there are stray marks, stains, or wrinkles, you should recopy all the data on a clean answer sheet. Programs may want to Xerox all your answer sheets and keep the copies, just in case your sheets are lost in transit.
SLIDE 10
TIPS ON MANAGING ASSESSMENT MATERIALS
You again will receive your answer sheets in cardboard boxes. Save these boxes! You can use these boxes to return your completed answer sheets and you will need to follow the packing instructions that come with the boxes. Also, please do not ship your answer sheets in envelopes. They will get crumpled and the envelopes may break open.
Similar to last year, ship your sheets in large batches (at least 75% of your total in the first batch and the rest in one batch as soon as possible.) If you have fewer than 100 children, ship all of them in one batch.
Pack the answer sheets face up and turned in the same direction.
Be sure to include the Program ID and shipment forms in each box.
SLIDE 11
TIPS ON MANAGING ASSESSMENT MATERIALS
It is important that you ship your boxes of answer sheets using a shipping company that can trace your packages, for example, Federal Express, UPS or Airborne. While this may be more expensive than mailing them, it is critical that you be able to track your shipment. A few days after you have shipped your boxes, trace them using the tracing number to be sure they arrived and were signed for at Pearson. Then, keep your tracing number in your records so that if any question arises about whether they were received you will have your tracing number.
I know these steps may sound tedious, but it is very important to get the shipments submitted correctly so that we can analyze your data. And the sooner you send it in, the sooner we can get the results back to you.
Thank you for your continuing hard work.
Now I will turn it over to Tom……
Segment 3d: NRS Ten Step Program
Tom Schultz
Senior Advisor on Outcomes
Head Start Bureau
Thanks, Ruth. We have provided a solid overview of information you need to know about the second year of the NRS. You have heard about what is new, what will stay the same, what we are expecting from you, and how you can obtain help in the areas of materials, the child assessment process, training of assessors and the CBRS. To wrapup this segment, I will walk through what we call our 10-step program for the fall, 2004 NRS effort.
SLIDE 1
In the next few minutes I will walk your through a sequence of priority tasks that you can begin work on immediately and continue over the next 3 months through to shipping your completed child assessment forms. Completing these steps will assure that your program’s NRS assessments will fairly represent the capabilities of your children at the beginning of this coming year.
SLIDE 2
Step 1, Materials. As Ken Burns explained, if you have not already updated information on the quantity and types of materials you need for this fall’s assessment, contact the HSIPC.
SLIDE 3
Step 2, Planning. Make sure you have people in place to carry out all the aspects of your fall NRS activities. We suggest designating an overall coordinator for this effort to make sure that all the pieces fit together and to monitor progress. In addition you will need people to carry out the CBRS data entry, a training team, and a sufficient number of staff or managers administer assessments. If you are assessing both English and Spanish-speaking children, remember the importance of having certified trainers for both the English and Spanish language versions of the assessment.
We also suggest that you develop an overall timeline for beginning and completing key tasks. This may remind you to address other needed preparation such as arranging for substitutes to cover staff assigned to administer assessments, or arranging appropriate space for the assessments.
SLIDE 4
Step 3, Communicate. Brief key groups including your governing board and policy council on this coming year’s NRS effort, address their questions and incorporate their suggestions. Remember to include the NRS in materials you create to orient parents of incoming children. If you program obtains written permission from parents for other screening and assessment efforts, incorporate the NRS into that process.
SLIDE 5
Step 4, CBRS Preparation. Take advantage of resources such as the updated online training presentation on the 2004-5 CBRS to prepare to use new features and options such as the Data Import and Copy Over elements.
SLIDE 6
Step 5, CBRS Data Entry. If you encounter problems, contact the Xtria technical assistance website or hotline for help.
SLIDE 7
Step 6, Training Prep. If you are adding new people to your team of assessors, make arrangements for the certification process, including recruiting preschool children, if possible so that participants will have the opportunity to be observed administering a practice assessment to a child.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 8
Step 7, Training & Certification of Assessors.
If you have any questions on the training procedures, please call the HSNRS ASSESSMENT HOTLINE AT 1-888-847-1176 or email questions to: HSNRShelp@westat.com.
SLIDE 9
Step 8, Assessing Children. We suggest that you follow up your fall training with suitable quality assurance efforts. We suggest spot-checking as assessments are administered and providing extra support to individuals who are carrying out assessments for the first time. Make use of the Westat helpline and web-site to obtain help if you encounter problems or unusual situations that aren’t addressed in the NRS materials.
SLIDE 10
Step 9, Review Answer Sheets to make sure they are complete and in good shape for the scanning process, and..
SLIDE 11
Last but not least, Step 10, Shipping Assessment Forms. Make sure that your program receives credit for all of your hard work by following the suggestions regarding shipping of assessments forms to our scanning contractor.
Before I turn it back to Ann for your questions, I want to make an appeal to everyone in our audience about the importance of the integrity of this year’s assessment. It is crucial to support consistency in the assessment process, to avoid coaching children towards the right answers to questions, and to resist the temptation to teach to the test. We do want every program to provide rich learning opportunities for children to progress in vocabulary, language comprehension, letters of the alphabet, English language acquisition and early mathematics. We don’t want teachers to teach children specific questions and answers from the NRS assessment.
From everything we know, you did at excellent job in managing a high quality assessment effort in year I of the NRS. Let’s continue on that high road, and strengthen Head Start’s reputation for integrity and high quality implementation of all our policies and standards.
Segment 4: Head Start Bureau Updates and Announcements
Ann Linehan
Director, Program Support
Head Start Bureau
Thanks again to all who phoned, faxed or emailed your questions or comments to Tom, Ruth, Nick and Ken.
On behalf of our Associate Commissioner, Windy Hill, I’d like to give some important updates and announcements from the Head Start Bureau.
It’s that time of year again. A new group of more than 900,000 young children is getting ready to begin Head Start. I know they are excited, and their parents are looking forward to their opportunity to partner with you in helping their child grow and learn. I know that many programs are beginning services this month and staff are busy with preservice training and preparing their classrooms.
I hope that this broadcast has helped inform you about how to prepare for our second year of the National Reporting System. Year One was a learning experience for all of us. I believe we are all approaching Year Two with a greater sense of confidence and less anxiety.
As we wrap up this broadcast, I want to share information with you about three additional new Head Start initiatives. We expect each of these efforts to strengthen Head Start’s capacity to serve our Nation’s most disadvantaged young children and their families.
First, we are in the midst of our 2004 Head Start Parent-Mentor Training Program, designed to develop a nationwide cadre of parents who can help other parents support their children in learning. Hundreds of parents are attending 2 ½ day intensive training conferences in nine of our regional office cities. This training will equip parents with specific strategies to foster children’s progress in key areas of language development and early literacy skills. We are hearing wonderful feedback from training participants. Please support parents from your program as they return ready to work with other families in the next year.
Second, all Head Start grantees recently received a letter announcing the availability of funding to provide training to strengthen local governance. We know from our decades of program monitoring reviews that sound and effective governance is the foundation for a quality Head Start program. Therefore we are providing a new funding opportunity for locally-developed training and technical assistance to strengthen program governance systems. Programs are invited to develop strategies to improve communication and strengthen working relationships among governing boards, Policy Councils and program managers. We encourage all agencies to take advantage of this opportunity by submitting a proposal to their Regional Office.
Third and finally, programs will be receiving an announcement soon of the availability of funding to serve an additional 4000 preschool children. Eligibility for these funds for program expansion will be limited to current Head Start grantees. We view this funding as a significant opportunity to expand Head Start services to underserved and unserved populations in a variety of communities across the country.
Please be in touch with your Regional Office Program Specialist for more information on these activities and funding opportunities.
Again, on behalf of Windy Hill, our Associate Commissioner, and the Head Start
Bureau, our best wishes for the coming year, and thanks for all the work that
you do for children and families.
Presentaciones
Los textos de cada segemento de la transmissión simultaneas via satellite y Web de Head Start de Febrero 17, 2005 sobre información del Head Start Bureau está disponible para verse en linea. Puede hacer click en el número del segmento para ver el texto de ese segmento. Si se incluyeron imagines durante la transmisón, se pueden ver en linea. Solo haga click en el tema de la presentación para ver las imagines.
Segmento 1a: Comentarios de apertura
Windy Hill
Comisionada Asociada
Head Start Bureau
Gracias, Ann.
Es un placer para mi darles la bienvenida a la transmisión de hoy. Comenzaré con algunos datos relacionados con
el Sistema Nacional de Información (NRS) y continuarémos con información nueva sobre otros asuntos y
eventos de Head Start.
Tal vez se han dado cuenta que uno de los miembros de Head Start no está con nosotros hoy. Hace unas semanas el Dr. Tom Schultz renunció del Head Start Bureau para aceptar una posición con el Pew Charitable Trust, trabajando en el área de responsabilidades en el aspecto infantil. La Dra. Michele Plutro, quienes muchos de ustedes conocen, estará a cargo en varias de las responsabilidades que Tom tenía.
Durante ésta transmisión ustedes se darán cuenta que el resto del equipo, tanto empleados como contratistas permanecen en sus puestos y con mas energia.
Hay dos nuevos informes relacionados con el NRS. Primero, un reporte al Congreso en evaluaciones de Head Start, aprobado por el secretario adjunto, Dr. Wade Horn, continúa su trayectoria para obtener su aprobación por parte del ACF. Continuamos optimistas en que éste reporte llegará al Congreso éste mes.
Adicionalmente, la oficina general contabilidad (GAO) ha realizado un estudio del desarrollo e implementación del NRS, incluyendo una revisión de la evaluación por parte de miembros de la Academia Nacional de Ciencia. La GAO estará entregando su reporte al Congreso en la primavera.
Ahora me concentraré en otras áreas de interés de la comunidad de Head Start.
Tal vez habrán visto el informe de prensa de las últimas semanas anunciando el comité de 10 miembros de la oficina de responsabilidades de la secretaría del comité de consejería y de medidas de desempeño de educación. Este comité estará presidido por la Dra. Susan Landry. La Dra Landry es profesora de Michael Matthew Knight y jefe de la división de desarrollo pediátrico en la escuela de medicina de la Universidad en Houston, Texas.
El comité se reunirá periodicamente durante los próximos 16 a 24 meses para revisar los recursos existentes de información para considerar el estado de responsabilidad de Head Start y hacer las recomendaciones necesarias al nuevo Secretario de salud y servicios humanos, Michael Leavitt, en ambas áreas, tanto en el estado como en consideraciones futuras de varios aspectos de la responsabilidad del programa.
Esto incluye información relacionada con el monitoréo, el Sistema Nacional de Información (NRS), el Impace Study, y FACES. Deben estar pendientes sobre el trabajo de éste comité en los próximos meses.
El 2005 representa el aniversario 40 del programa de Head Start. Ustedes estarán planeando eventos locales para celebrar y honrar ésta ocasión histórica. El Head Start Bureau actualmente tambien está involucrado en planear un par de eventos. Les informarémos más al respecto en las próximas semanas.
Muchos de los que nos están viendo tal vez estuviéron presentes hace dos semanas en el primer Conferencia Nacional de Head Start sobre Asuntos Hispanos que se realizó en Albuquerque, Nuevo Méjico. Mas de 2,300 participantes se beneficiaron de ésta experiencia y de los varios conferenciantes y actividades de ése evento, incluyendo 396 representantes de los programas de Head Start de Migrantes y temporales, los estados de Texas, Illinois, New York y California con el mayor número de participantes. Nos sentimos contentos de haber ofrecido ésta oportunidad a todos los beneficiarios quienes sirven a los niños de habla hispana y sus familias, y hemos recibido información positiva muy buena de ese evento.
Habrá un número de esfuerzos de apoyo que se fortalecerán en las experiencias y las oportunidades creadas por el instituto, y estámos ansiosos de conocer cómo las experiencias aprendidas allí se transforman en prácticas innovadoras en los programas de Head Start.
Asi usted haya o no participado en el instituto, espero que le preste atención al actual boletín de Head Start cuyo primer ejemplar se distribuyó en el instituto. Este primer ejemplar se centra en aprendizaje en el idioma inglés. Tiene bastante información muy útil.
El punto final que quiero mencionar hoy es la reautorización. Como ustedes saben, la reautorización no se completó el año pasado. Ahora se ha reiniciado. Ustedes querrán hacerle un seguimiento y pensar sobre las varias propuestas que se han presentado por el Congreso.
Sabémos que actualmente se discuten un número de ideas por parte de varios miembros del Congreso dentro de sus comités.
Los puntos en discusión incluyen:
La última información que quiero compartirles incluye un mandato reciente en el máximo en los salarios en los programas de Head Start. Daré una corta presentación a ésta nueva regulación. Pronto recibirán por escrito explicaciones adicionales y guias sobre cumplimiento.
Las apropiaciones de HHS que pasó en diciembre 2004 contiene un punto, sección 205, que requiere que "ninguno de los fondos apropiados en éste artículo para Head Start debe ser utilizado para pagar el salario de un individuo, tanto en costos directos o cualquier otro gasto en costos indirectos, en el nivel de ejecutivo nivel II". En enero 2005, el nivel de compensación para un ejecutivo a nivel II en el gobierno federal era $162,100 por año.
El requerimiento se hizo efectivo en diciembre 8, 2004, el dia que el presidente firmó la ley. Las oficinas regionales revisarán las propuestas de cualquier reembolso de becas del año fiscal 2005 que ya se hayan hecho a la fecha y si fuere necesario, contactará los concesionarios para clarificación.
Cada programa de Head Start es responsable para el aseguramiento de ésta provisión Esta provisión aplica al salario y una parte dee beneficios de cualquier empleado de un concesionario de Head Start o agencia delegada quien es pagado, en su totalidad o parcialmente, con los fondos de Head Start.
Los concesionarios pagarán compensación a sus empleados por salarios que excedan $162,100, pero no se podrán utilizar fondos de Head Start para una porción de esa compensación que exceda el mandato de éste límite instituido. A los concesionarios se les ha advertido que el no cumplir con éste requisito puede provocar el rechazo de fondos por costos.
Esto concluye mi presentación para la transmisión de hoy. Ahora disponganse a escuchar el estado actual de los esfuerzos de NRS lo mismo que una presentación especial de los nuevos materiales de asesores-mentores.
Ahora escucharémos de Michele Plutro, especialista de asuntos educativos. Michele nos proveerá información nueva de los reportes del primer año de NRS, e información nueva del grupo de trabajo técnico en NRS.
Michele
Regresar al Indice de Presentación
Segmento 1b: Información nueva en el reporte del primer año de NRS y el grupo de trabajo técnico.
Michele Plutro
Asistente especial al comisionado asociado
Head Start Bureau
Gracias, Windy
Me place estar con ustedes hoy para ésta transmisión de NRS. Mis comentarios hoy son muy cortos, que sorprenderá a muchos que ya me conocen!
He sido parte de desarrollo del Sistema Nacional de Información (NRS) desde que se presentó la idea por primera vez por el Presidente Bush en 2002. Algunas veces es aun dificil de entender qué tan lejos hemos llegado y pensar que en un par de meses estarémos celebrando dos años en el proceso de NRS.
Cada vez que reviso la información, hablo con los programas sobre sus experiencias o hablo con otros empleados federales o estatales sobre éste proceso, creo aun más que estamos haciendo algo bueno a favor de los niños.
Tal vez no estarémos haciendo todo lo que podamos hacer. Tal vez no estamos implementando cada procedimiento tan bien como sea posible, pero es muy importante que podamos reportar al congreso y al público que paga impuestos que de verdad los niños estan mejor preparados para la escuela luego de su experiencia en Head Start, y que su preparación incluye una atención apoyada en habilidades lo mismo que estado de salud, nutrición, desarrollo emocional y social, y educación de los padres. Historicamente, sabemos que hemos hecho un mejor trabajo en los reportes de éstos resultados que reportar en las experiencias cumulativas y del curriculum de los niños.
Como lo dijo Ann Linehan, ustedes recibirán nueva información durante ésta transmisión, por eso mis mensajes son de verdad muy pocos pues las presentaciones de hoy son muy enriquecedoras.
Tengo dos mensajes para compartir. Primero, quiero repetir la advertencia que hemos dicho desde el comienzo del NRS – un mensaje que se ha presentado claramente en cada uno de los reportes que ustedes han recibido hasta la fecha y un mensaje que ha sido repetido por muchos a traves de las transmisiones de NRS.
Esto es, que el NRS aun está en el proceso de desarrollo. La infraestructura, aunque en existencia, aun necesita fortalecimiento. Por lo tanto, los reportes que ustedes reciben, aunque muy informativos, no deben ser "sobreutilizados", utilizados prematuramente, o utilizados inapropiadamente para hacer decisiones locales en relación a ciertos tipos de cambios de programática escencial.
Les animo a reexaminar las secciones narrativas de sus reportes y a revisar algunas de las preguntas y respuestas relevantes y que están archivadas en la página web del NRS.
Mi segundo mensaje es sobre una información nueva en las sociedades necesarias para continuar el desarrollo y la implementación de NRS. Como evidencia, en la transmisión de hoy, continuamos trabajando con las organizaciones primarias con las que ustedes ya estan familiarizados: Xtria, Westat y Mathematica.
Ademas, el grupo técnico – TWG como le conocemos – continua, aun con la formación del comité de consulta en la responsabilidad de Head Start. La membresía de TWG se ha mantenido intacta al 100% desde su formación en los primeros dias de discusión sobre el NRS.
Ese grupo, el TWG tiene una agenda ocupada para el año entrante, comenzando con su próxima reunion en la primavera. Creo que debe intersarles los puntos que actualmente se discuten y debaten y esperamos sus recomendaciones luego de la reunión. Estos puntos incluyen, luego de la aprobación de OMB, una prueba de campo del 2005 de una posible medida en el área social emocional y la examinación de varias aproximaciones u opciones del ejemplo anual de niños de cuatro años de edad para participación en el NRS.
Ahora nos acompañaraacute; el Dr. Nick Zill, director de proyecto de Westat, Dra. Ruth Hubbell-McKey, de Xtria y la Dra. Diane Paulsell de Mathematica. Ellos discutirán los resultados de la evalaución del NRS del otoño 2004, el CBRS, incluyendo los programas de verano de migrantes en el 2004, y las lecciones aprendidas desde el primer año y la información básica.
Volvémos a ti, Nick….
Regresar al Indice de Presentación
Segmento 1c: Resultados de evaluación de niños del NRS de Head Start Otoño 2004
Nick Zill
Director de Proyecto
Westat
Gracias Michele. Me agrada volver a estar con ustedes.
IMAGEN 1
RESULTADOS DE EVALUACIONES DE OTOÑO
Estaré dando un vistazo general a los resultados del Sistema Nacional de Información (NRS) otoño
2004.
IMAGEN 2
RECOLECCION DE INFORMACION BASICA
Me agrada informar que la recolección de información del otoño 2004 para el Sistema Nacional de
Información (NRS) se ha completado. Gracias al continuo esfuerzo de todo el personal de Head Start en los Estados
Unidos, las evaluaciones básicas se llevaron a cabo con éxito en casi todos los niños elegibles para el
kindergarten en Head Start éste año.
Los reportes básicos de los prográmas se están colocando en el sitio de la red de CBRS luego de
ésta transmisión. Su copia impresa la estarán recibiendo en dos o tres semanas.
IMAGEN 3
NUMERO DE NIÑOS EXAMINADOS
De nuevo, el personal de Head Start en todo el pais ha hecho un trabajo excelente al completar a tiempo y a conciencia las
evaluaciones NRS de los niños. Hemos recibído, copiado y tomado el puntaje de 427,473 formas completas de
niños. Se ha recibido información de evaluación de mas de 1,800 programas de Head Start. Al separarlas
por idioma, el total de evaluaciones de otoño son aproximadamente 404,000 o el 95% se completaron en inglés.
Aproximadamente 98,000, o el 23%, se completaron en español. Mas o menos 75,000 niños, o el 18%, se evaluaron
en ambos idiomas, inglés y español.
IMAGEN 4
INFORMACION DE ALTA CALIDAD
Aparenteménte los instructores y asesores locales de Head Start han realizado un excelente trabajo en la calidad y
consistencia de las evaluaciones. La confiabilidad de la dependencia en la información interna muestra una gran
confiabilidad a nivel de cada niño.
Esta confiabilidad a nivel individual muestra que la confiabilidad es excelente cuando la información de los
niños se agrega a nivel de programa.
IMAGEN 5
COMPARACION CON EL OTOÑO 2003
Los resultados de la evaluación del reporte de otoño no son solamente confiables, los resultados fueron
generalmente consistentes con los resultados de información básica recolectados en el otoño 2003. Esto
sugiere que los programas de Head Start están sirviendo a niños con habilidades similares en idioma y
matemáticas que en los años anteriores.
IMAGEN 6
QUE MUESTRAN LAS EVALUACIONES
Qué nos muestran las evaluaciones NRS del otono? En promedio, en el otoño el puntaje de la mayoría de
los niños fue el promedio en los exámenes de vocabulario y matemáticas, pero bajos en reconocimiento de
letras tanto en inglés como en español. Estos resultados son similares a los obtenidos en las evaluaciones del
otoño 2003.
IMAGEN 7
PORCENTAJE DE PUNTOS CORRECTOS
Si miramos el porcentaje de los puntos que los niños respondieron correctamente, en promedio, podemos ver que los
niños contestaron el 80% de los puntos correctamente en los filtros en inglés.
Aquellos que pasaron el filtro pasaron a los otros puntos de la evaluacion en inglés. En el vocabulario, el porcentaje
acertado fue del 55%.
El puntaje promedio en reconocimiento de letras fue un poco mas bajo, con niños reconociendo el 25% de las letras, o 6
de las 26 letras del alfabeto en inglés.
En matemáticas elemental, el promedio del porcentaje correcto fue de 50%, que es la mitad de la tabla del puntaje.
IMAGEN 8
PORCENTAJE DE PUNTOS CORRECTOS PARA LOS PROGRAMAS EN ESTADOS UNIDOS CONTINENTAL
Los puntajes nacionales para las evaluaciones en espanol se presentan por localidad del programa de Head Start. Los
análisis de la información del primer aáo muestran que los puntajes en español de la primavera
son mucho mas altos para los niños en programas en Puerto Rico, donde el idioma primario de instrucción es el
español, comparado con niños de Estados Unidos continental, donde regularmente el idioma primario de
instrucción es el inglés.
Por lo tanto, los resultados de las evaluaciones se presentan separadamente para los programas de Head Start en Estados
Unidos continental y los programas en Puerto Rico.
Los resultados básicos promedio en las evaluaciones en español, basados en programas de Estados Unidos
continental, fueron similares, aunque generalmente un poco mas bajos que los resultados de las evaluaciones en inglés.
Niños que tomaron la evaluación en español acertaron el 63% en el filtro en español.
En vocabulario en español, ellos acertaron el 54% de los puntos. En reconocimiento de las letras en español,
acertaron el 10% de las letras del alfabeto en español, o 3 de 30 letras. En matemáticas en español,
ellos acertaron el 40%.
IMAGEN 9
PORCENTAJE CORRECTO PARA LOS PROGRAMAS DE PUERTO RICO
En Puerto Rico vemos que los niños allí acertaron 75 % de los puntos en el filtro en el idioma
español.
En vocabulario en español, los niños en Puerto Rico acertaron el 62%. En reconocimiento de las letras en
español, acertaron el 7% de las letras del alfabeto en español, o 2 de las 30 letras. En matemáticas en
español, ellos acertaron el 43%.
IMAGEN 10
FILTRO EN INGLES PARA ESTUDIANTES APRENDIENDO INGLES
Veamos ahora los resultados de los niños de familias que no hablan inglés en el filtro en inglés.
El 25% de todos los niños aprendiendo en inglés entendieron bien o relativamente bien las indicaciones
habladas. Esto es, que acertaron 16 o mas puntos de los 20 del filtro en inglés.
Por otro lado, el 36% de los niños aprendiendo el idioma inglés no tenian los conocimientos suficientes del
inglés para continuar con la evaluación en inglés. Asi no hayan pasado el filtro en inglés, a los
niños de familias de habla hispana se les administró la evaluación. Aquellos que hablan otros idiomas y
no pasaron el filtro no recibieron mas evaluación.
IMAGEN 11
VOCABULARIO
En la sección de vocabulario en inglés, el típico niño del programa de Head Start comenzó
con un vocabulario consistente al nivel 3, o un vocabulario equivalente al de un niño promedio de entre 40 a 47 meses
de edad (un poco menor de 4 años de edad). Debido a que la mayoria de los niños a quien se les
administró la evaluación tenian entre 4 y 5 años de edad, ésto significa que el desarrollo del
vocabulario de un típico niño de Head Start estaba un poco mas atrasado que el promedio de los niños menos
pobres de su misma edad.
Sin embargo, una considerable variación en éste nivel de desarrollo. Aun temprano en el año del
programa, el 17% de los niños de Head Start tenian un vocabulario equivalente al de un niño promedio en los
Estados Unidos de su misma edad o mayor. Por otro lado, el 50% de los niños estaban un año o mas, atrasados en
el desarrollo de su vocabulario.
IMAGEN 12
NOMBRE DE LAS LETRAS
Como lo mencioné anteriormente, el promedio de letras conocidas por un niño de Head Start al comenzar el
programa era entre 6 y 7. Pero éste promedio aumentó por la minoria de los niños que reconocen mas
letras. De hecho, el típico niño, uno hacia mediados de la distribución, comenzó en Head Start
identificando solo dos o tres letras, consistente con el nivel 2 de habilidades.
La minoría de los niños que conocian mas letras incluyen el 9 por ciento quienes conocian todas o casi todas
las letras. Las tres cuartas partes de los niños, sin embargo, identificaron menos de las 10 letras ordenadas por el
Congreso, temprano en el programa del año de Head Start.
IMAGEN 13
CONOCIMINETOS BÁSICOS DE MATEMÁTICAS
En conocimientos básicos de matemáticas, el tipico niño de Head Start cuenta hasta 10 u 11 objetos al
comenzar el programa anual, consistente con el nivel 3 de habilidades.
El 14% de los niños tenian habilidades más avanzadas al poder contar 15 o más objetos.
Hubo una diversidad similar entre los niños evaluados en español.
IMAGEN 15
VOCABULARIO EN ESPAÑOL
En el vocabulario en español, el tipico niño de Head Start de habla hispana, comenzó el programa con un
vocabulario equivalente a un niño promedio de entre 40 y 47 meses de edad o un poco menor de 4 años de edad.
Consistente con un nivel 3 de habilidades.
Al comienzo del año del programa, el 17% de los niños evaluados en español tenían un vocabulario
equivalente al de un niño promedio de habla hispana de la misma edad o un poco mayor. Cerca de una tercera parte de
los niños estaban un año o más, atrasados en el desarrollo de su vocabulario en español.
IMAGEN 16
NOMBRE DE LAS LETRAS EN ESPAÑOL
El promedio de letras conocidas por niños de Head Start al comenzar el programa fué de tres letras.
Aproximadamente el 6% de los niños de habla hispana sabian 17 o más letras. Pero más del 90%
identificó menos de 10 letras del alfabeto en español.
IMAGEN 17
CONOCIMIENTOS BÁSICOS DE MATEMÁTICAS EN ESPAÑOL
En matemáticas, el típico niño evaluado en español pudo identificar menos de 6 objetos. El 94% de
los niños evaluados en español no pudo contar mas de 11 objetos. Solo el 2% pudo contar 15 objetos o
más.
IMAGEN 18
VARIACIONES EN LOS PROGRAMAS
Cuando observamos los resultados del NRS del otoño 2004 en los diferentes programas de Head Start dentro de cada
región y entre regiones, encontramos que hay variaciones significativas en el promedio de nivel de habilidades con
caracteristicas demográficas diferentes de programas. Pero éstas variaciones no son mayores y no todas van en
la misma dirección.
IMAGEN 19
CUALES CARACTERISTICAS DEMOGRAFICAS INTERESAN?
Por ejemplo, programas que sirven, en promedio, a niños un poco mayores tienen la tendencia a un puntaje más
alto en cada una de las evaluaciones que los programas que sirven a niños menores. Similarmente, niños con
padres con un nivel de educación relativamente mas alto, tienen la tendencia a tener mayor puntaje en las evaluaciones
que los niños de padres menos educados.
Niños en programas con una alta concentracion aprendiendo el idioma inglés tuvieron un puntaje menor en las
evaluaciones del filtro del vocabulario del idioma inglés que niños en programas con una baja
concentración de niños que hablan otros idiomas, tal como se esperaba. Las diferencias en el vocabulario en
inglés existen a pesar de que se les dá solamente a los niños que han pasado el filtro del lenguaje.
Por otro lado, programas con alta concentracion de niños aprendiendo en el idioma inglés no tuvieron un puntaje
mas bajo que otros programas en reconocimiento de letras o habilidades en matemáticas básicas.
IMAGEN 20
ACORTANDO LA DISTANCIA
A pesar de éstas asociaciones entre los niveles de habilidades básicas promedio y las caracteristicas
demográficas, el análisis que hemos realizado con la información del primer año de NRS demuestra
que Head Start parece estar haciendo la diferencia en los niños en sus habilidades del idioma y numéricas.
Durante el año de Head Start, la relación entre las características demográficas de los
niños y las habilidades promedio se hacen más debiles. Los servicios prestados por Head Start parecen estar
ayudando a acortar la distancia entre los diferentes grupos de los niños de Head Start.
IMAGEN 21
PUNTOS BASICOS DE REFERENCIA
Al revisar la información que se recibe sobre el nivel de habilidades de los niños en su programa, es
importante reconocer que los puntajes representan el conocimiento de los niños al entrar al Head Start en el
otoño del año pasado.
Ellos proveen un punto de referencia básico para el progreso que los niños harán en éstas
áreas de habilidades durante el año del programa. En parte por la participación de los niños en
Head Start, su conocimiento en éstas áreas está supuesta a crecer notablemente al finalizar el
año, como lo hicieron en la implementación del primer año del NRS. Lo importante es ver cuánto
progreso muestran sus niños al final del año y el nivel de conocimiento que ellos han alcanzado al salir del
programa y entran al kindergarten.
IMAGEN 22
DOCUMENTANDO EL PROGRESO
Con base en los puntajes de las evaluaciones del otoño y la primavera, calcularémos los "puntajes
ganados" para documentar éste progreso en cada área de habilidades. Al completar la evaluación de
la primavera, cada programa recibirá un reporte sobre el crecimiento que los niños han mostrado en el programa
y los niveles de conocimiento que han alcanzado.
IMAGEN 23
TESTIMONIO A SU TRABAJO
El alto puntaje de conclusión y la alta calidad de la información del otoño son un testimonio al duro
trabajo y cuidados que los empleados de Head Start a traves de los Estados Unidos han dedicado a la empresa del Sistema
Nacional de Información. (NRS)
Pero el esfuerzo va solo por la mitad. La misma diligencia y cuidados se necesitarán para completar las evaluaciones
de primavera a tiempo y con la mas alta calidad. Mantengan ese buen trabajo en la primavera
Ahora voy a entregarle a Ruth.
Regresar al Indice de Presentación
Segmento 1d: Head Start NRS Otoño 2004 CBRS
Ruth Hubbell-McKey
Director de Proyecto
Xtria
Gracias Nick.
IMAGEN 1
TITULO
Estaré informando sobre datos del sistema computarizado de informacion NRS- el CBRS. Este es un sistema computarizado
que utilizan los programas para mantener información de sus niños elegíbles y proveer información
descriptiva sobre los programas, centros, aulas de clase, maestros, asesores y los niños.
Cuando comienza el período de la recolección de información, los programas primero entran ésta
información en el CBRS. El programa asigna un número de identificación a los niños, los cuales
despues son incluídos en las páginas de puntajes y utilizados para las evaluaciones. Luego de que se completan
las evaluaciones, el personal de Head Start entra las evaluaciones completas en el CBRS. Esto nos permite hacer un
seguimiento de cómo los programas están progresando y el número de evaluaciones que esperámos de
ellos. El sistema tambien provee información descriptiva muy valiosa para análisis.
IMAGEN 2
ESTADO DEL REPORTE OTOÑO 2004
La recolección de información del CBRS ha sido un periodo muy productivo. Los programas entraron la informacion
con éxito, incluyendo a muchos que tomaron ventaja de las opciones como copiar e importación de
información para facilitar el proceso.
Para éste otoño, tuvimos 1,841 o sea el 95% de la información de los programas en el sistema. Cerramos a
comienzos de diciembre cuando se suponía toda la información debíía estar completa. Ahora tenemos
477,915 niños en el sistema del CBRS para el otoño. De éstos, 669 no pudieron ser evaluados debido a
severa discapacidad, 718 tuvieron IEP que no permitieron la evaluación y 7,620 que se retiraron antes de ser
evaluados. De todas maneras hubo 468,908 nivos elegibles para ser evaluados.
IMAGEN 3
ESTADO DEL REPORTE OTOÑO 2004 continuación
De éstos, 6,230 niños no tuvieron evaluación debido a que sus padres no lo permitieron, 10,242 no se
evaluaron por ausencia prolongada y 3,470 niños no cooperaron con la evaluación. Tambien hubo 3,063
identificaciones equivocadas o que se perdieron y para 18,340 niños el estado de su evaluación no se conocia.
Esto nos dió un total de 427,437 evaluaciones que fueron copiadas y archivadas. Este es un buen número y
muestra el gran esfuerzo que sus programas y su personal hicieron.
IMAGEN 4
ESTADO DEL REPORTE DE MIGRANTES (INFORMACION 2004)
Los programas de migrantes y trabajadores temporales participaron en el NRS en un calendario diferente al de los programas
regionales. La mayoria de éstos programas comienza sus operaciones en la primavera y el verano cuando comienza la
época de las cosechas y regularmente opera solo por unos meses. Solo a los programas migrantes en donde los
niños participan por mas de cuatro meses se les requirió participar en el NRS. Estos programas completaron sus
evaluaciones "básicas" o sea el primer paso en las evaluaciones de primavera y su "crecimiento" o
segundo paso de evaluaciones al menos cuatro meses despues.
Tuvimos 40 de 68 programas que participaron en este calendario. Ellos registraron 1,365 niños en el CBRS y recibimos
1,299 evaluaciones en el periodo de las evaluaciones básicas y 742 en el periodo de crecimiento. Hubo 32 evaluaciones
no completadas y 37 niños que se retiraron al momento de evaluaciones basicas y 43 no completadas y 242 niños
retirandose al momento de las evaluaciones de crecimiento.
En 2004 realizamos un estudio de probabilidades al evaluar niños en los programas migrantes que operan al menos cuatro
meses. El estudio se realizó en 27 programas. El resultado de ese estudio debe estar disponible para la primavera.
Diana, volvemos a ti.
Regresar al Indice de Presentación
Segmento 1e: Lecciones aprendidas del primer año y las observaciones básicas del
otoño 2004.
Diane Paulsell
Investigador Principal
Mathematica Policy Research
Gracias Ruth.
IMAGEN 1
TITULO
Desde que se implementó el primer reporte de NRS en el otoño del 2003, el Head Start Bureau ha hecho un
contrato con Mathematica para llevar a cabo una evaluación independiente, un énte tercero, sobre cómo se
está implementando el NRS en todo el pais. El propósito de éste estudio de dos años es saber lo
que está funcionando bien y lo que se puede mejorar en el futuro.
En el primer año de implementación, visitámos 35 sitios de Head Start, ejemplos representativos a nivel
nacional durante los periodos de evaluación de la primavera y el otoño, y también visitámos 4
programas migrantes de Head Start participando en el NRS. Estamos visitando un grupo diferente de programas para conocer
sobre la implementación durante el año actual del programa.
Nuestros resultados están siendo reportados al Head Start Bureau, al grupo técnico de NRS y los programas de
Head Start.
IMAGEN 2
VISTAZO GENERAL
Hoy voy a compartir nuestros principales resultados sobre:
IMAGEN 3
OBSERVACIONES SOBRE VISITA DEL OTOÑO
Durante la implementacion del NRS en el otono, nuestro equipo visito un grupo de 34 sitios representativos a nivel nacional
de los programas de Head Start. La visita a cada sitio duró cerca de dos dias. Observámos cómo el
personal de Head Start llevaba a cabo las evaluaciones NRS con 354 niños (295 en inglés y 59 en español)
Observámos un total de 126 asesores locales; entrevistamos directores de Head Start, facilitadores de NRS,
administradores de información; y tuvimos grupos de análisis locales con personal haciendo las evaluaciones
para conocer sobre sus experiencias.
IMAGEN 4
DURACION DE LAS EVALUACIONES
El tiempo promedio de evaluaciones que observamos fue de mas o menos 16 minutos en la versión en inglés y 18
minutos en la versión en español. Mas del 90% de los niños completaron la evaluación entre 10 y
20 minutos. Los nuevos evaluadores, quienes no lo habian hecho antes, se tomaron un promedio de 4 minutos mas.
IMAGEN 5
CALIDAD DE LAS EVALUACIONES
Utilizando los procedimientos de certificación como guía, marcamos las evaluaciones, observámos por
errores y sumamos un puntaje de certificación para cada uno. Cerca del 80% de las evaluaciones en inglés
observamos que tuvieron un puntaje del 85 o más, lo cual es el mínimo requerido para la certificación.
La mitad tuvo un puntaje entre el 96 y el 100. En general, ésto indíca que los programas hicieron muy buen
trabajo al entrenar el personal para que administrara las evaluaciones correcta y consistentemente.
Comparámos los puntajes de certificación con aquellos realizados por los maestros quienes evaluaron alumnos en
sus propias aulas de clase con aquellos de otros evaluadores, lo mismo que los puntajes entre los evaluadores con experiencia
y los nuevos. No encontramos diferencias mayores entre éstos grupos.
La calidad de las evaluaciones en español fue similar. El 80% obtuvo un puntaje mayor del 85, y el 65% obtuvo puntaje
entre el 96 y 100.
Tambien analizámos nuestras observaciones para entender tipos mas específicos de errores y problemas al administrar
las evaluaciones. El promedio de error fue bajo. Comparado con el otoño del 2003, los evaluadores cometieron menos errores al
leer las instrucciones y sumar el puntaje. Sin embargo, ellos tuvieron mas errores en gestos y dirección.
Errores de gestos consisten en omitir un gesto, haciendo el gesto equivocado, o añadiendo un gesto donde no se
requería. Los gestos fueron uno de los aspectos mas delicados en la administración del NRS. Sin embargo,
algunos visitantes a los locales de evaluación creen que los errores en los gestos tuvieron muy poco efecto en las
respuestas de los niños.
La mayoria de los errores en los gestos sucedió en la sección Tío Simón dice, y regularmente fue
con dar pistas no habladas, tales como mirar hacia abajo muy rápido en el punto "coloque sus pies juntos".
Algunas de éstas acciones pueden ser inconcientes y se pueden mejorar al hacer énfasis en los procedimientos
correctos durante el ejercicio de re entrenamiento.
Los evaluadores, en promedio, obtuvieron el 97% de respuestas correctamente. Los errores ocurrían cuando los
evaluadores tuvieron dificultad al ver hacia donde estaba señalando el niño, o cuando su respuesta era confusa.
Adicionalmente, los asesores tuvieron dificultad sumando el puntaje de contéo cuando los niños se saltearon las bolas
de cristal, o las contaron dos veces.
IMAGEN 6
AREAS PARA MEJORAR
Aunque los evaluadores en general cometieron pocos errores, hemos identificado algunos procedimientos de la evaluación
donde el personal necesitaría más guia y entrenamiento. Incluyendo:
IMAGEN 7
RESPUESTAS DE LOS NIÑOS
El personal en dos terceras partes de los programas reportaron que la mayoría de los niños reaccionaron
positivamente a las evaluaciones. Muchos disfrutaron el tiempo que pasaron con sus asesores uno a uno.
Al mismo tiempo, los programas citaron varios comportamientos de los niños lo cual hicieron el administrar la
evaluación aún más desafiante. Estas incluían dificultad en permanecer en el objetivo, ser
tímidos o temerosos de ir con el evaluador, aburrirse o distraerse, llorar y no cooperar.
Los evaluadores trataron varias estratégias para enfrentar éstos comportamientos, tales como orientarlos de
nuevo, tomar descansos, permitir que los niños se movieran en el salón y tratar de nuevo otro dia. A pesar de
que el re entrenamiento proveyó algunas estratégias de comportamiento, los programas solicitaron mas
orientación en niños que respondian negativamente a la evaluación.
Los evaluadores descubrieron que las secciones de vocabulario y nombrar las letras fueron las más dificiles de
administrar. Algunos sienten que éstas secciones son muy largas y reportaron dificultad en mantener la atención
de los niños. Tal como en el estudio del primer año, algunos evaluadores se dieron cuenta que era frustrante
para los niños que no conocian las letras preguntarseles en repetidas ocasines si conocian mas.
IMAGEN 8
ESTUDIANTES APRENDIENDO INGLÉS
El 71% de los programas administraron la evaluación del NRS a niños aprendiendo inglés.
En general, los evaluadores estaban complacidos de los cambios que les permitieron evaluar a los niños de habla
hispana, primero en español y luego en inglés. Ellos quisieron que los niños comenzaran con una experiencia
positiva.
Sin embargo, algunos programas tuvieron dificultad en determinar si el idioma primario en casa era el inglés o el
español, y ellos utilizaron varios metodos para determinarlo.
Tal como en el primer año del estudio, algunos programas tuvieron preocupación acerca de algunas palabras
utilizadas en el idioma español, por la variación del uso de palabras entre regiones. Empleados de algunos
programas creen que el punto Tío Simón dice fue confuso para algunos niños en el idioma inglés ya
que no están familiarizados con el juego.
IMAGEN 9
NIÑOS CON DISCAPACIDADES
Solo 3 de los 34 programas que visitamos evaluaron al menos un niño con discapacidad. Ellos hicieron algunos arreglos
para éstos niños, tales como:
IMAGEN 10
PLANTEAMIENTO LOCAL DE IMPLEMENTACION
La mayoria de los programas mantuvieron el mismo sistema básico en las evaluaciones del otoño que se
utilizaron durante el primer año.
Mas de la mitad utilizaron al menos algunos maestros como evaluadores del NRS, pero la mayoria no permitieron que los
maestros evaluaran a niños en sus propias aulas de clase.
En el 26% de los programas ejemplos, sin embargo, los maestros evaluaron los alumnos de sus propias aulas de clase. En
éstos programas, el personal creyó que los niños se sentirían mas tranquilos al ser evaluados por
sus propios maestros, y los maestros podrían manejar el comportamiento de sus niños.
La mayoría de los programas mantuvieron a sus Consejos de Politica y los padres informados sobre la
implementación del NRS . En general, el personal reportó que la mayoría de los padres no tenían
preocupaciones u objeciones serias. En las reuniones de Consejo de Política, las preguntas mas frecuentes eran sobre
el propósito del NRS. Cómo se utilizarían los resultados en el Head Start Bureau, y la forma correcta de
evaluar los niños de 4 años de edad. Los padres expresaron interés en querer ver los resultados del
puntaje del NRS; algunos tenian preguntas sobre cómo y por quién serían utilizados los resultados.
IMAGEN 11
RE-ENTRENAMIENTO DE OTOÑO
La mitad de los programas ejemplo entrenaron al menos un nuevo evaluador en el otoño del 2004, y todos los programas
ofrecieron re entrenamiento a los evaluadores experimentados.
Solo uno de los programas no siguió la agenda de entrenamiento para los asesores nuevos, pero menos de la mitad
siguieron las indicaciones para el re entrenamiento. Un número de prográmas no se adhirieron al texto del
entrenamiento, y dos no se presentaron al entrenamiento en video. Era menos probable que los asesores en español
siguieran el protocólo de entrenamiento.
En general, los programas encontraron favorable el material de entrenamiento, especialmente el video de cómo manejar
comportamientos desafiantes. Sin embargo, ellos creen que el re entrenamiento era muy largo y repetitivo. Cerca de la mitad
de los programas dicen que ellos prefieren un re entrenamiento mas corto que se centre en los cambios al protocólo de
evaluaciones y provéa más tiempo a prácticar con otros evaluadores.
IMAGEN 12
REACCIONES AL PROGRAMA
Durante las entrevistas, el personal de Head Start expresó cuatro preocupaciones principales sobre el NRS. Estas
preocupaciones son muy similares a aquellas presentadas por los programas que participaron en el estudio del primer
año.
Las precoupaciones mas communes del personal local es sobre cómo la información de los resultados del NRS sería
utilizada a nivel nacional. Varios empleados dicen que aun no les es claro el propósito del NRS y sus implicaciones
para los programas locales en los cuales los niños no tuvieron un buen desempeño en las evaluaciones.
Personal local tambien expresó preocupación de que los resultados del NRS no reflejaría adecuadamente la
actuación, el número de personal y los recursos financieros dedicados al NRS, y si los resultados negativos de
las evaluaciones afectarían algunos programas de Head Start y sus niños al disminuirles el enfoque del programa
a los puntos cubiertos en las evaluaciones.
IMAGEN 13
REVISION DEL REPORTE DE OTOÑO DEL NRS
Muy temprano en la primavera del 2004, todos los programas locales de Head Start recibieron un reporte con los resultados de
las evaluaciones del otoño 2003. (Al momento de las visitas, los programas aún no habian recibido su reporte
del primer año completo del NRS)
El personal en mas o menos la mitad de los programas encontraron que el reporte es claro y fácil de entender; el
personal en el 30% de los programas tuvieron cierta dificultad en interpretar la información.
Al mismo tiempo, en la mayoría de los programas dijeron que los resultados de sus propias evaluaciones fueron
más útiles que los resultados del NRS, pues cubren mas temas, están disponibles a nivel de cada
niño y aula de clase, y estuvieron disponibles tan pronto se completaron las evaluaciones. La mayoría de los
programas sugirieron que los resultados fueran entregados por centro o aula de clase.
Sin embargo, el personal en la mayoría de los programas dijeron que utilizarán los resultados del NRS para
mejorar los programas, para desarrollar planes de mejoras, identificar áreas donde los niños no obtuvieron
buenos resultados, crear un plan de entrenamiento para maestros o evaluar el plan de trabajo.
Además, el personal en más de la tercera parte de los programas reportaron que ya han implementado cambios en
sus prácticas en el aula de clase como resultado del NRS, especificamente concentrándose en más tiempo de
instrucción en áreas cubiertas durante la evaluación. El reconocimiento de letras es una área importante
de atención.
Además, una cuarta parte de los prográmas reportaron el uso del NRS como guía para comprar material para
las aulas de clase.
IMAGEN 14
SUGERENCIAS DE LOS PROGRAMAS
Los programas han hecho varias sugerencias para mejorar el NRS.
Ellos han sugerido en mejorar la comunicación, proveyendo mas información sobre el propósito del NRS y
cómo serán usados los resultados; información sobre su desarrollo, validéz, y confiabilidad; y
material por escrito para los padres.
Los programas tambien quieren recibir más a tiempo materiales para su entrenamiento y los reportes de NRS.
Un cuarto de los programas dicen que les gustaría mas guianza y apoyo al utilizar los resultados del NRS para la
mejoría de los programas. Ademas, si los a niños no les va bien en un área en particular, a los
programas les gustaría recibir mas asistencia técnica en cómo alcanzar los resultados deseados.
IMAGEN 15
SUGERENCIAS (CONT)
La mitad de los programas solicitaron mas guia en la administración de las evaluaciones. Puntos especificos incluyeron
proveer apoyo neutral, manejo del comportamiento, evaluación de niños con discapacidades, y cambios a las
evaluaciones y procedimientos.
Los programas hicieron una variedad de sugerencias sobre el sistema de las actuales evaluaciones. Estas incluian combinarlas
con evaluaciones locales, utilizar metodos o materials alternos, y sugerencias para mejorar puntos específicos.
Los programas estaban divididos en si se deberian añadir nuevos puntos. Los empleados estaban debatiendo entre el
deseo de tener sus programas evaluados en un área mas amplia de resultados, para reflejar mejor la naturaleza del
programa de Head Start, y preocupados de que una evaluación mas larga sería un mayor reto para los programas y los
niños.
En resumen, el estudio de implementación de Mathematica encontró que los programas han hecho un muy buen
trabajo al hacer las evaluaciones del NRS. La gran mayoría de las evaluaciones que observamos cumplieron o excedieron
las expectativas establecidas por el proceso de certificación.
Tambien observamos algunas secciones de la evaluación que aun son dificiles para que el personal administre o
califique, algunas áreas donde el personal necesita mas guia y entrenamiento, y algunas preocupaciones que los
programas tienen sobre el NRS. Muchos de éstos asuntos tambien se identificaron en el estudio de implementación
del primer año. Estamos en el proceso de revisar nuestras conclusions con el Head Start Bureau y el grupo de apoyo
técnico, y ellos utilizarán ésta información para planear la implementación del NRS en la
primavera del 2005 y el futuro.
De regeso a ti Ann…
Regresar al Indice de Presentación
Segmento 1f: Preguntas y Respuestas segmento 1
Ann Linehan: Gracias Michele, Ruth y Diane. Nuestras lineas estan disponibles. Cuando llame por favor digale a nuestras operadoras su nombre, titulo, afiliación y el estado de donde llama. No utilice el altoparlante en su teléfono, y una vez que haga su pregunta al aire, por favor cuelgue. Tratarémos de tomar la mayor cantidad de llamadas en los próximos 15 minutos. Recuerde, si no puede hacer su pregunta en vivo las colocarémos en la página del internet junto con las respuestas. Tratarémos de contestar las preguntas más populares. Toda pregunta es importante para nosotros. Sin embargo, dentro del tiempo permitido, darémos respuesta a las preguntas que esperamos beneficien a la mayoria de los televidentes.
Ann Linehan: Gracias. Necesitamos cerrar éste segmento y una vez mas, gracias a todos los que nos han llamado, han enviado sus preguntas o comentarios via fax o via correo electrónico a nuestros presentadores. Luego de una corta pausa, nuestros presentadores. regresarán para discutir sobre las evaluaciones de entrenamiento del NRS 2005, la colección de información CBRS 2005 y los lugares que serán visitados en el 2005. Continuarémos en dos minutos.
Regresar al Indice de Presentación
Segmento 2a: Entrenamiento y Evaluaciones primavera 2005 NRS
Nick Zill
Director de Proyecto
Westat
Gracias Ann.
IMAGEN 1
ENTRENAMIENTO PARA EVALUACION PRIMAVERA 2005 NRS
Daré un vistazo al re entrenamiento de primavera 2005 y la evaluación de niños primavera 2005.
IMAGEN 2
METAS DEL ENTRENAMIENTO
Con base en los resultados de las evaluaciones del otoño, instructores y evaluadores han realizado un trabajo
excelente en cuanto a la calidad y consistencia de las evaluaciones. La meta de éste re entrenamiento es equiparlos
para que continuen con la mas alta calidad, para que las evaluaciones de primavera muestren la imágen mas completa,
justa y correcta del progreso de los niños en su programa bajo las medidas del NRS.
Comentarios de los programas indican que el re entrenamiento, incluyendo los videos de entrenamiento y los textos, fueron de
mucha ayuda al preparar la recolección de información de la primavera pasada.
IMAGEN 3
MATERIALES DE ENTRENAMIENTO DE LA PRIMAVERA
Los materials de entrenamiento de HSNRS de la primavera le serán enviados automaticamente en ésta primavera.
Los materiales para la evaluación de primavera incluyen:
IMAGEN 4
EVALUACIONES PRIMAVERA 2005
Tal como en el otoño, a todos los niños seleccionados para el kindergarten se les dará la misma,
evaluación estandard en la primavera. Las evaluaciones en inglés cubren las mismas áreas que las de las
evaluaciones del otoño 2004, tales como:
IMAGEN 5
EVALUACIONES PRIMAVERA 2005
Igualmente, las evaluaciones de primavera 2005 en español cubren las mismas áreas de habilidades que las
evaluaciones en español del otoño, tales como:
IMAGEN 6
EVALUACIONES DE PRIMAVERA 2005
Para reducir la posibilidad de enseñar el exámen o de efectos por práctica, se han incluído
nuevos puntos en las secciones de vocabulario y conocimientos básicos de matemáticas para las evaluaciones en
inglés y en español. Estos puntos tienen dificultades similares y cubren las mismas áreas de contenido
que las del otoño del 2004. Pero a pesar de que se han añadido nuevos puntos, podrémos comparar puntajes
del otoño y la primavera para medir el crecimiento de los niños durante su año en Head Start.
Las secciones Tío Simón dice, exposición de arte, y nobre de las letras permanecerán igual que en
las evaluaciones del otoño 2004.
IMAGEN 7
EVALUACIONES PRIMAVERA 2005
Algunas de las instrucciones escritas en los caballetes, tales como instrucciones para los gestos adecuados, han sido
simplificadas o aclaradas para que los evaluadores y los niños entiendan mejor lo que se supone que hagan en la
prueba. Mas adelante, la guia revisada para los asesores provee más direcciones en cómo sumar el puntaje en las
respuestas de los niños.
IMAGEN 8
VIDEO DE RE ENTRENAMIENTO
El video de re entrenamiento da una demostración completa de la evaluación de primavera 2005 con un niño
en edad pre escolar.
El video muestra metodos de cómo manejar varios tipos de comportamiento que los niños pueden mostrar durante
una evaluación, tales como ser rudo, no prestar atención, o ser demasiado tímido y no querer responder.
Información de los programas indica que ésta sección del video de entrenamiento ha sido de mucha
utilidad al evaluar sus niños.
IMAGEN 9
CABALLETES PRIMAVERA 2005
Nuevos caballetes serán enviados para su evaluación de primavera 2005. Ellos contienen los nuevos puntos para
las secciones de vocabulario y conocimientos básicos de matemáticas lo mismo que las nuevas instrucciones.
Ustedes deben usar éstos caballetes para sus sesiones de entrenamiento de evaluadores de la primaveras 2005 y
colección de información. Si aun no lo ha hecho, por favor recoja y guarde los caballetes de evaluaciones
anteriores para que sus evaluadores no los usen accidentalmente en sus evaluaciones de primavera del 2005.
IMAGEN 10
ANEXO A LA GUIA DE EVALUADORES
Este anéxo contiene instrucciones, punto por punto para las secciones nuevas de vocabulario y conocimientos
básicos de matemáticas en las evaluaciones de primavera lo mismo que las páginas con respuestas. Los
evaluadores pueden utilizarlas como una guia sobre cambios en las evaluaciones. Tambien incluye instrucciones en cómo
manejar asuntos de comportamiento que podría encontrar durante una evaluación.
Por favor asegurese que cada evaluador tenga una copia de la guia con el nuevo anexo.
IMAGEN 11
FORMATO BASICO DEL CURSO DE RE ENTRENAMIENTO
Las instrucciones y una agenda modelo para el curso de re entrenamiento se incluirán con el video de entrenamiento. En
el re entrenamiento, los evaluadores deberán ver el video de re entrenamiento de la primavera 2005 y tomar nota de
información importante para cuando practiquen y se lleven a cabo las evaluaciones. Tambien deben anotar las preguntas
que se quieran discutir con su instructor durante el periodo de preguntas.
El instructor debe tomarse el tiempo para contestar cualquier pregunta que sus pupilos tengan. Este debe ser un buen tiempo
para hablar sobre los ejemplos del video de entrenamiento o técnicas utilizadas para enfrentar retos de
comportamiento.
Luego del periodo de preguntas y respuestas, los entrenados deben familiarizarse con los materiales y su papel en las
evaluaciones. Luego de una sesión sobre su papel en las evaluaciones, el instructor debe proveer información a los
entrenados sobre sus debilidades o puntos fuertes observados durante el ejercicio. Luego los entrenados deben tener una nueva
ronda de ejercicios sobre su papel en las evaluaciones.
IMAGEN 12
ENTRENAMIENTO PARA EVALUADORES CERTIFICADOS
Los evaluadores que fueron certificados el otoño pasado deben ver el video de re entrenamiento y su papel en las
evaluaciones con el caballete de evaluaciones de la primavera. El instructor debe observar los entrenados administrando las
evaluaciones durante el ejercicio. Estos evaluadores no necesitan ser certificados formalmente.
Una vez que se ha completado el entrenamiento, éstos evaluadores deben utilizar el mismo número de
identificaci´n que utilizaron en las evaluaciones del otoño del 2004.
IMAGEN 13
PROCEDIMIENTOS PARA LOS PROGRAMAS CON NIÑOS QUE NO HABLAN ESPAÑOL
Los programas con niños que no hablan español (quienes por lo tanto no entrenarán evaluadores en
español) no tienen que ver la version en español del video de entrenamiento.
IMAGEN 14
PROGRAMAS O EVALUADORES EN UN IDIOMA - ESPAÑOL
Personas que solo hablan español o programas con niños que solo hablan español solo serán
entrenados en la version en español de la evaluación de la primavera. Como resultado, éstos pupilos
verán solo la version en español del video de re entrenamiento.
IMAGEN 15
EVALUADORES BILINGUES
Evaluadores bilingues, quienes estarán dando las evaluaciones en ingles y en español, DEBEN ver las dos
versiones del entrenamiento, en inglés y español. Tambien deben participar en el ejercicio y recibir
información de las versiones en inglés y español de las evaluaciones de la primavera.
IMAGEN 16
ENTRENAMIENTO PARA EVALUADORES NUEVOS
Los nuevos evaluadores que no fueron certificados para la recolección de información del otoño necesitan
ser certificados. Los nuevos evaluadores deben recibir el entrenamiento de un dia en las evaluaciones del otoño del 2004 y
recibir su certificación. Tal como en el proceso de certificación del otoño, ellos deben recibir un
puntaje mayor del 85% tanto en la forma de certificación de evaluaciones y el cuestionario para llegar a ser un
evaluador certificado.
IMAGEN 17
ENTRENAMIENTO PARA EVALUADORES NUEVOS
Una vez que adquieren su certificación, deben mirar el video de re entrenamiento de la primavera 2005, el caballete de
ejercicio de participación primavera 2005 y luego ser observados administrando la evaluación de primavera 2005.
Luego del entrenamiento, los nombres de los nuevos evaluadores certificados deben ser registrados en el sistema computarizado
de información (CBRS) para obtener un número de identificación de evaluador.
IMAGEN 18
EVALUACIONES DE LA PRIMAVERA
Ustedes evaluarán todos los niños de 4 y 5 años de edad, listos para el kindergarten, quienes
están en sus programas ésta primavera, AUN SI NO FUERON EVALUADOS EN EL OTOÑO. Si los niños
acaban de entrar al programa deben ser evaluados ahora utilizando solo la evaluación de la primavera. Ellos no deben
ser incluidos en los análisis de cambios del otoño a la primavera, pero serán incluídos solo en
el análisis de la primavera, por favor haga seguro que sean evaluados.
IMAGEN 19
CONCENTRESE EN CONTROL DE CALIDAD
Durante éstos re entrenamientos, los instructores deben hacer énfasis en la importancia de revisar todas las
respuestas dadas para que estén completas y claras. Si los números de identificación de los niños
no están en la información, o no estan correctos, entonces la evaluación no se puede usar.
Querémos tener la información de los niños en su programa tan completa como sea posible y no
querémos que su gran esfuerzo se desperdicie solo porque no hay un número de identificación. Por favor
asegúrese que los evaluadores revisen la veracidad de la identificación del niño y sus fechas de
nacimiento.
IMAGEN 20
PREGUNTAS SOBRE ENTRENAMIENTO O EVALUACIONES DE PRIMAVERA
Si usted tiene preguntas sobre los procedimientos de entrenamiento, por favor llame a la LINEA GRATIS DE HSNRS sobre
evaluaciones al 1-888-847-1176 o envíe sus preguntas via correo electrónico a: HSNRShelp@westat.com
IMAGEN 21
PREGUNTAS SOBRE MATERIALES
Si tiene alguna pregunta sobre el envío del material de la primavera, por favor llame al centro de publicaciones e
información de Head Start (HSIPC) al 1-866-763-6481 o envíe sus preguntas via correo electrónico a:
puborder@headstartinfo.org
IMAGEN 22
PREGUNTAS SOBRE CBRS
Si tiene preguntas sobre cómo registrar o poner al dia la información en el CBRS, por favor llame a la LINEA
GRATIS DE CBRS al 1-800-669-2170 o envíe sus preguntas via correo electrónico a: NRSTechHELP@xtria.com
IMAGEN 23
NRS DEPENDE DE USTED
El éxito en la implementación del Sistema Nacional de Información (NRS) de Head Start depende de su
continuo y duro trabajo. Ustedes hicieron una gran labor el otoño pasado. Hagamos un trabajo aun mucho mejor en las
evaluaciones de ésta primavera.
Gracias.
Ruth…
Regresar al Indice de Presentación
Segmento 2b: Recolección de información CBRS Primavera 2005
Ruth Hubbell-McKey
Directora de proyecto
Xtria
Gracias Nick…
IMAGEN I
NUEVOS PROCEDIMIENTOS DEL CBRS
Quisiera discutir brevemente el CBRS para la primavera del 2005. El CBRS abrirá para sus programas regionales en marzo
1, 2005. Ustedes NO estarán recibiendo nuevos manuales del CBRS u otros materiales en la primavera. Los materiales que
recibieron el otoño pasado tienen toda la información que necesitan.
Quiero resaltar algunas de los nuevos procedimientos del CBRS. Muchos de ustedes ya han utilizado cómo registrar y
copiar la información. Estos nuevos procedimientos se diseñaron para que fuera más fácil para
ustedes registrar la información.
El procedimiento de copia de información le permite a los programas transferir toda su información en el CBRS,
excepto la información del niño. Además, la información del programa, centros, aula de clases,
maestros y evaluadores fueron copiados de la información del año pasado a la información de éste
año. Luego, los programas pusieron al dia cualquier información que hubiera cambiado y la registraron en todos
los niños elegibles para el kindergarten.
Para la recolección de información del otoño 2004, 1,290 programas utilizaron el procedimiento de copia.
Estos programas registraron un total de 384,544 niños.
El procedimiento de importar la información está dirigida a programas más extensos que actualmente
están usando un sistema automatizado de manejo de información, tal como el ChildPlus, Galileo o el HSFIS, para
monitorear las operaciones del programa. Utilizando el procedimiento de importación de información,
información del niño lo mismo que el aula de clase, centro, y la información del maestro puede ser
transferida al CBRS.
Para el período de la recolección de información del otoño 2004, 87 programas utilizaron el
procedimiento de importación de información, con un total de 32,147 niños transferidos.
Para la primavera 2005, los programas regionales de Head Start no pueden usar el procedimiento de copiar información,
pero puede usar el procedimiento de importación de información, SOLO PARA NIÑOS NUEVOS ! Esto es, si usted
tiene niños que son elegibles para el kindergarten que entraron al programa pero que no fueron evaluados en el NRS del
otoño, usted puede usar la importación de información para transferir su información. Cuando
usted entra por primera vez a la pagina CBRS en ésta primavera, se le preguntará si quiere o no utilizar el
procedimiento de importación de información. Solo tiene una oportunidad para escoger, por eso debe pensarlo
cuidadosamente. Hay una descripción sobre ésta opción luego de entrar a la página de CBRS y en
los manuales de instrucción que recibió el otoño pasado con los otros materiales del NRS. Si tiene
preguntas, por favor llame a la línea gratis del CBRS al 1-800-669-2170 o envíe su correo electrónico a
NRStechhelp@xtria.com
Los programas de Head Start para migrantes que están comenzando su segunda recolección de información
básica tal vez quieran usar el procedimiento de copiar la información. O tal vez quiera usar importación
de información. Debe escoger solamente uno. Haga su decisión y escoja una vez que haya entrado al sistema.
Llame a la línea de asistencia técnica si necesita ayuda.
IMAGEN 2
NUEVA PAGINA DE INGRESO
Cuando usted ingrese a la página de CBRS en la primavera, se dará cuenta que ha sido modificada. En la parte
izquierda de la pantalla ahora verá selecciones para escoger, tales como "registro de importación de
información" y "preguntas frecuentes" lo mismo que una sección para entrar a contactos
relacionados. A la derecha en la pantalla, encontrará la sección "Qué es Nuevo", con
información nueva del CBRS. Al centro de la pantalla es donde usted colocará su identificación y
contraseña para ingresar. Creemos que éstos cambios harán un poco más fácil para usted
poder ver la información en la pálgina de internet del HSNRS.
Debido a que algunos empleados de Head Start se siente más cómodos trabajando con el sistema en español,
todo lo que tiene que hacer es presionar en el contacto de español en la parte alta izquierda de la pantalla y toda la
página en información del CBRS lo mismo que sus instrucciones podra verlas en español.
IMAGEN 3
CONSEJOS PARA EL MANEJO DE MATERIALES DE EVALUACION
Ahora les daré algunos consejos para el manejo de su material de evaluación para asegurar que las
páginas de las respuestas se reciban a tiempo y su información pueda ser utilizada para el análisis. Tal
vez lo ha escuchado antes si ha visto nuestras transmisiones anteriores. Sin embargo continúa siendo un problema por
lo tanto querémos seguirlo recordando.
Basados en la experiencia del año anterior, muchas de las páginas con las respuestas no se pudieron utilizar
porque se perdieron en el correo, estaban incompletas, incorrectas o dañadas. Se ha puesto tanto trabajo en el proceso
de evaluación, que sería una lástima el no poder utilizar la información recolectada. Por eso,
para prevenir que ocurra de nuevo, quiero revisar algunos de los pasos necesarios que su programa debe tomar para asegurarse
que toda la información llegue correctamente. Estos pasos tambien se comentan en su manual de entrenamiento.
Primero usted debió asignar un administrador del NRS para asegurar que los siguientes pasos se cumplan:
IMAGEN 4
CONSEJOS PARA EL MANEJO DE MATERIALES DE EVALUACION
IMAGEN 5
CONSEJOS PARA EL MANEJO DE MATERIALES DE EVALUACION
Continúe revisando cada página para asegurarse que las páginas de respuestas de la primavera del 2005 se
utilizaron. Ninguna otra versión será aceptada. Asegurese que cada sección esté completa, que se
utilice un lápiz #2, que no hay rayas o manchas en la hoja, y que las hojas no estén arrugadas o cortadas. No
separe las hojas con las respuestas! Si algunas de éstas cosas ocurren muy posiblemente no podrán ser leidas por las
máquinas procesadoras. Si hay manchas, rayas, páginas arrugadas o cortadas, debe hacerlas de nuevo copiando
toda la información en una página nueva. Los programas tal vez quieran hacer fotocopias de todas las páginas y
guardarlas, por si se pierden en el correo.
IMAGEN 6
CONSEJOS PARA EL MANEJO DE MATERIALES DE EVALUACION
De nuevo recibirán su páginas con respuestas en cajas de cartón. Guarde las cajas!. Usted puede usarlas
para enviar éstas páginas con las respuestas y necesita seguir las intrucciones de empaque que van en las
cajas. Tampoco envíe sus páginas con respuestas en sobres. Los sobres se pueden arrugan o abrir.
Igual que el año pasado, envíe sus páginas con respuestas en cantidad (al menos el 75% del total en su
primer envío y el resto en otro envío lo más pronto posible). Si tiene menos de 100 niños, envíelos
todos en un solo paquete. No espere demasiado para enviar sus respuestas. Una vez que haya terminado sus evaluaciones,
envíelas. No espere por una o dos! Si espera por meses, el proceso se cerrará y no se podrá archivar. Por favor
envíelas dentro del mes siguiente luego de la fecha límite de cierre.
Una vez más, coloque las respuestas hacia arriba y en la misma dirección.
Asegurese de incluír la identificación del programa y las formas de envío en cada caja.
IMAGEN 7
CONSEJOS PARA EL MANEJO DE MATERIALES DE EVALUACION
Es importante que envíe sus cajas con sus respuestas por una compañia de envíos que pueda rastrear sus
paquetes, por ejemplo, Federal Express, UPS o Airborne. Aunque sea un poco más costoso que enviarlos por correo
regular, es importante que los pueda rastrear. Unos dias despues de haber enviado sus cajas, rastréelas usando el
número de rastréo para asegurarse que llegaron y alguien firmó en Pearson. Mantenga el número de
rastréo en sus archivos en caso de que haya preguntas sobre si llegaron, y usted tendrá el número de
rastréo.
Se que éste procedimiento puede sonar molesto, pero es importante que los envíos se hagan correctamente para
poder analizar su información. Entre más rápido los envíe, más rápido
podrémos envíarles los resultados.
Gracias por su duro trabajo.
Ahora les dejamos con Diane, quien discutirá los sitios de visita para el 2005.
Regresar al Indice de Presentación
Segmento 2c: Sitios de visita para el 2005
Diane Paulsell
Investigador Principal
Mathematica Policy Research
Gracias Ruth.
En la primavera del 2005, el personal de Mathematica visitará representantes a nivel nacionales de 35 programas ejemplo de Head Start para el estudio de control de calidad. Como en la ronda de visitas anteriores, observarémos una variedad de evaluaciones, entrevistas a personal seleccionado sobre la implementación del NRS, y llevar a cabo un grupo de enfoque con el personal que hace las evaluaciones para aprender de sus experiencias. Cada visita durará cerca de 2 dias.
Contactarémos a cada uno de los programas seleccionados en el próximo mes para discutir su participación en el estudio y seleccionar los niños que nos servirán como ejemplo en las evlauaciones que observarémos. El personal de Mathematica trabajará muy de cerca con cada programa para organizar las fechas de las visitas, observaciones y entrevistas en un tiempo conveniente.
Gracias.
Regresamos a ti, Ann.
Regresar al Indice de Presentación
Segmento 2d: Preguntas y Respuestas Segmento 2
Ann Linehan: Gracias Nick, Ruth, y Diane. Nuestras líneas telefónicas están disponibles una vez más. Les recordamos, cuando llame, por favor digale a nuestras operadoras su nombre, título, afiliación y el estado. Por favor no utilice el parlante de su teléfono, y una vez que haga su pregunta en vivo, por favor cuelgue.
21. Ann Linehan: Vamos a ti Diane, y éste es un e-mail que llegó hace unos minutos. "Cómo seleccionarán por azar cuáles sitios de Head Start visitarán?”
22. Ann Linehan: Gracias. Vamos a un e-mail de Kristen, y tiene que ver con la información que los programas recibirán - nuevos materiales. "Estarémos recibiendo nuevas páginas de puntaje?" Se que habló de nuevos materiales pero se estarán recibiendo nuevas páginas de puntaje?
23. Ann Linehan: Gracias. Vamos a otro e-mail de Faye. "Nosotros compartimos los resultados de nuestros exámenes con los cuerpos directivos locales en el área de servicios de nuestro programa. Es posible que los resultados finales de los exámenes puedan ser formualdos por éstas áreas de servicio - por áreas - a cambio de un reporte cumulativo para cada programa?"
24. Ann Linehan: Muy bien. Gracias. Vamos a un e-mail de Collen. "Nosotros tenemos un número de niños de habla hispana. Sin embargo, no tenemos un evaluador certificado en español en nuestro programa. Dónde podríamos conseguir un evaluador certificado en español para servirle mejor a nuestros niños?"
25. Ann Linehan: Gracias. Vamos a otro e-mail de Elizabeth. Este es para Ruth. "Cómo podríamos saber cuántas de nuestras evaluaciones no fueron contadas debido a información incompleta o dañada?"
26. Ann Linehan: Bien. e-mail de Sandy quien es una coordinadora de educación. "Por qué se deben re entrenar los evaluadores?"
27. Ann Linehan: Tenémos tres e-mails mas. Este es de Pedro. "El hecho de que éstos exámenes de evaluadores, los maestros, quieren ver mejoras, qué tan confiables pueden ser éstos resultados? Tal vez es para Diane o Nick.
28. Ann Linehan: Gracais. Un e-mail de Sandy, creo que ya lo ha contestado, pero déjeme preguntarle de nuevo. "Preocupaciones en programas fueron identificados por el grupo Mathematica. Nuestro programa tiene varias de las mismas preocupaciones. Por quién y cuándo serán atendidos? Y creo, Nick, que usted habló sobre algunos de los cambios y creo que Diane, usted mencionó algunos de los cambios tambien. Podrían hablar al respecto un poco mas?
29. Ann Linehan: Gracias. Voy a terminar con el primer e-mail que recibimos durante ésta transmission y es de Irma. “Qué hago con los folderes y evaluaciones viejas?” Ahora, yo se Nick, usted nos ha dicho que los escondamos. Pero yo creo que ésto indica a los maestros de que probablemente pueden ser un buen ejercicio de práctica para los niños durante el año. Se que no quiere que los confundamos con las evaluaciones de la primavera, pero cuál es el mejor consejo con relación al material viejo?
Regresar al Indice de Presentación
Segmento 3a: Modulo de Evaluación sobre Pasos Para el Éxito- Diseño Instructivo para el Asesor-Mentor de Alfabetización Temprana
Dollie Wolverton
Directora, Servicios de Educación
Head Start Bureau
Gracias Ann.
Hola todos.
Es un gran placer volver a estar con ustedes a traves de ésta transmisión via satelite.
Me agrada particularmente compartir éste segemnto con mi colega Karen McKinney. Juntas aprovechamos la oportunidad para
discutir un aspecto muy importante de nuestras responsabilidades y fortalecer el tema de la observación de los niños y la
evaluación continuada.
En Head Start, independientemente del Sistema Nacional de Información (NRS), las Normas de Desempeño del Programa de Head
Start requiere que intencionalmente observémos y evaluémos a los niños de manera continuada para obtener la información que
necesitamos para promover y apoyar el progreso de desarrollo y aprendizaje de los niños. Esto es para asegurarnos que cada
niño está progresando como resultado de las experiencias y servicios de Head Start.
La razón por lo cual somos tan cuidadosos y diligentes en nuestras evaluaciones continuadas es que el personal de Head Start
y Early Head Start usan la información para adaptarla a un curriculum de acuerdo a su edad, asi sea para infantes, niños que
comienzan a caminar o en edad pre escolar - con experiencias individualizadas para cada niño a traves del trabajo conjunto
de padres y empleados.
Esta tarde, tengo dos responsabilidades. Primero concretar el contexto de por qué las Normas de Desempeño del Programa de
Head Start, lo cual se basa en principios sólidos de desempeño de los niños, requiere que llevémos una evaluación continuada
de los niños. En segundo lugar, tendré el privilegio de presentar el nuevo Diseño Instructivo para el Asesor-Mentor de la
Alfabetización Temprana de aprendizaje titulado, Pasos Hacia el Éxito.
En seguida, Karen nos dará un vistazo de Pasos Hacia el Éxito, con un énfasis muy especial en la unidad 4: Ayudando a los
empleados a utilizar la información de evaluación para guiar la instrucción.
IMAGEN I
NORMAS DE DESEMEPEÑO DEL PROGRAMA DE HEAD START
En el mundo de Head Start, "estableciendo el contexto" – invariablemente comienza con las Normas de Desempeño del
Programa de Head Start, que son obligatorios - ésto es, las experiencias y servicios que cada programa debe proveer.
De la forma que las Normas son presentados a los niños desde su nacimiento hasta los 5 años, podemos planear e implementar
programas que son apropiados para el desarrollo en dos niveles –
Primero, a un nivel general de edades para el grupo de niños. Aqui se menciona como "curriculum para edad
apropiada".
Y con eso, lo individualizamos para cada niño
IMAGEN 2
NORMAS DE DESEMPEÑO DE PROGRAMA Visual – 1304.21(c)(2)
Las Normas de Desempeño del Programa 1304.21(c)(2), que aparece en la página 82 de su copia del estándares de desempeño del
programa con sus guias es el que especificamente requiere que usemos una variedad de estrategias para promover y apoyar el
progreso de desarrollo y el aprendizaje de los niños con base en las observaciones y las evaluaciones continuadas de cada
niño
Necesitamos saber cuáles habilidades manejan bien los niños, cuáles están surgiendo, y cuándo los niños no progresan de
acuerdo a lo esperado.
Las evaluaciones continuadas nos ayudan a decidir la mejor manera de acercarnos a los niños individualmente como grupos
pequeños y tambien como una clase en general.
Esto realmente tiene mucho sentido debido a que observación y evaluación van de la mano con programación de desarrollo
apropiada.
Este año se celebran 40 años de Head Start. Millones de niños, desde el nacimiento hasta los cinco años, y sus familias,
incluyendo mujeres embarazadas, han participado.
Como el único y mas extenso y mas completo programa de desarrollo del niño en la nación, hemos aprendido mucho sobre las
practicas de investigaciones en su base, lo cual aumenta el desarrollo y el aprendizaje de los niños.
Sin embargo, a menos que nos asegurémos que a los niños se les presentan experiencias de aprendizaje significativas y
desafiantes, una observación y evaluación continuada de los niños tiene muy poco valor.
IMAGEN 3
Guía de los Resultados Positivos del Niño para los que Dirigen Head Start
Una vez mas quiero llamar su atención a laGuía de los Resultados Positivos del Niño para los que Dirigen Head Start. Esta
guia se basa tanto en las Normas de Desempeño del Programa de Head Start y el Marco de Resultados de los niños de Head Start,
lo cual es una descripción comprensiva del alistamiento para la escuela en 8 puntos de desarrollo del niño:
IMAGEN 4
IMAGEN 5
IMAGEN 6
Ley de Head Start
La Ley de Head Start como enmienda en 1998 requiere que el Head Start Bureau de pasos para proveer enseñanzaz
Asesoria-Mentoria, ésto es, en la guia de trabajo y entrenamiento para el personal y voluntarios de Head Start para poder
mantener los servicios de educación de la mas alta calidad.
Asesoria-Mentoria tambien es un punto clave de Good Start, Grow Smart – la iniciativa presidencial de Early Childhood
iniciada en abril del 2002.
Tal vez recordará que en Noviembre 2002, 3000 Asesores-Mentores de la Alfabetización Temprana, participaron en la Conferencia
de Asesores-Mentores de Head Start en Washington, D.C.
Para entonces, recibimos sus sugerencias en estrategias específicas para apoyar Asesores-Mentores de la Alfabetización
Temprana trabajando con equipos de aulas de clases y visitantes a los hogares.
Hubo un gran consenso que se desarrollara una red de tecnología para facilitar la comunicación e intercambiar ideas entre
Asesores-Mentores.
En apoyo directo a sus recomendaciones, Step-Net se inició un poco despues de ésto.
IMAGEN 7
Putting the Pro in Protegé
En esa misma conferencia de noviembre del 2002, un buen número de Asesores-Mentores recomendaron que desarrolláramos los
conceptos incluidos en Putting the Pro in Protegé: Una guía para enseñar en Head Start y Early Head Start para asegurar
un entendimiento y acercamiento unificado sobre el papel de Asesor-Mentor de aprendizaje apoyando los protégés como miembros
de equipos de salones de clase y como visitantes de hogares.
Sus sugerencias han culminado con el desarrollo de un Diseño Instructivo para los Asesores-Mentores de Head Start y Early
Head Start titulado “ Pasos Hacia el Éxito”
Karen ahora dará un vistazo general sobre Pasos Hacia el Éxito y luego se concentrará en algunas estratégias claves en la
unidad 4: Ayudando al personal a utilizar la información de evaluación del niño para guiar la enseñanza.
Karen McKinney
Especialista del Programa
Head Start Bureau
Gracias, Dollie.
Me agrada mucho anunciar el lanzamiento de Pasos Hacia el Éxito: Un Diseño Instructivo para el Asesor-Mentor de la
Alfabetización Temprana.
Este proyecto ofrecerá a todos los programas de Head Start y Early Head Start un marco consistente de trabajo para apoyar
asesor-mentor con un nuevo acercamiento usando varios tipos de tecnología.
Démos un vistázo al diseño de Pasos Hacia el Éxito.
Tal como lo ven en la imágen, Pasos Hacia el Éxito, cubre cuatro areas escenciales
IMAGEN 8
1. Construyendo relaciones
2. Observación y análisis del personal
3. Reflexiones de practicas, y
4. Usando la información de evaluación del niño para guía de instrucción
Hoy, quiero compartir con ustedes algunas de las estratégias de la unidad 4 – Usando la información de la evaluación del niño
para guia de instrucción.
Tal como se ve en éstos videos de Pasos Hacias el Éxito, los equipos en las aulas de clase usan varias técnicas, tanto en los
infantes, niños que comienzan a cominar y niños en edad pre-escolar.
VIDEO 1
Estratégias
En el próximo video de Pasos Hacia el Éxito, un maestro en el aula de clase utiliza preguntas para involucrar a los niños en
conversaciones que comienza con el interés de la pequeña niña en palabras y poesias …e imaginación.
VIDEO 2
Video de maestro y niña
Hay una variedad de formas para colectar información sobre los niños en sus programas y su ambiente de familia.
El próximo video ilustra una variedad de estratégias que al ponerlas juntas provee información recogida en un período de tiempo que documenta el progreso del niño y provée al personal con la información necesaria para individualizar el curriculum y adaptar el ambiente para promover y apoyar el aprendizaje del niño.
VIDEO 3
Asesor-Mentor y maestro con portafolio
Tal como lo vimos, el personal utitlizó una cámara, una grabadora de audio, y ejemplos del trabajo del niño para crear un portafolio, que no solo sirve como archivo, pero provée una manera concreta para compartir ésta evaluación continuada con otros.
De la misma manera que los asesores-mentores de aprendizaje y sus protegidos se toman el tiempo para crear confianza y una relación fuerte de trabajo, los protegidos y los padres construyen relaciones confiables para apoyar el desarrollo de los niños.
IMAGEN 9
Involucrando los padres
Head Start reconoce plenamente que el involucramiento de los padres es crítico durante los primeros años.
VIDEO 4
Maestros y padres y portafolio
Como lo hemos visto en éste pequeño video sobre evaluaciones continuadas de los niños, es crítico planificar y ser
intencional en el uso de las observaciones y estratégias de evaluaciones.
IMAGEN 10 Estas estratégias pueden incluir:
El Diseño Instructivo de Pasos Hacia el Éxito se inaugurará en la primavera 2005 con su implentación total en cada region en el otoño del 2005. Hay dos formas como usted puede continuar informándose sobre las actividades de asesor-mentor.
IMAGEN 11
Información de sitio web de STEP- NET y
número grátis 877
Les sugiero revisar regularmente STEP-Net.org para que se actualice con lo último en la información sobre asesores-mentores. Allí encontrará nuestro panfleto electrónico que presenta programas de Head Start y las estratégias que se están implementando para apoyar mentor-maestro en su programa.
Otro recurso de Asesores-Mentores es la línea gratis 1-877-coach-04 donde un miembro del personal de Pasos Hacia el Éxitocon mucho gusto discutirá sus preguntas o estratégias específicas. Los que nos llaman en español pueden dejar un mensaje y alguien les responderá su llamada en su idioma.
En éste segmento, Dollie discutirá los requerimientos de Normas de Desempeño del Programa de Head Start para utilizar la información de las observaciones y las evaluaciones continuadas para promover y apoyar el progreso de aprendizaje y desarrollo de cada niño.
Les he ofrecido un vistazo a Pasos Hacia el Éxito y cómo Asesor-Mentores puede apoyarle en las evaluaciones continuadas de los niños. Al usar las técnicas que hemos discutido hoy, su acercamiento a la individualidad para los niños será parte de sus actividades diarias.
Es ésta variedad de actividades enriquecedoras y experiencias combinadas con evaluaciones continuadas y el seguimiento del progreso que lidera nuestros niños del programa de Head Start en un viaje con resultados positivos y muchos mas pasos hacia el exito!
Ann…
Regresar al Indice de Presentación
Segmento 3b: Preguntas y Respuestas segmento 3
Ann Linehan: Gracias Karen. Nuestras líneas estarán nuevamente disponibles, para recordarles, cuando llame por favor diga su nombre a la operadora, título,afiliación y el estado de donde nos llama. Por favor no use el parlante de su teléfono, y una vez que haya hecho su pregunta al aire, por favor cuelgue.
30. Ann Linehan: Tendrémos una corta sección de preguntas y respuestas y comencémos con un e-mail de Denise. “Estará disponible ésta transmisión en su totalidad para verla en el futuro?” Por supuesto me encanta dar buenas noticias, por eso la contestaré yo. Me agrada informarles que estará disponible a partir de la 1 de la tarde mañana. Y estará en la página web por un período aproximado de 90 dias.
31. El siguiente e-mail es de un consultor local de TA. “Karen, qué papel, o Dollie, tendrá el nuevo sistema de TA para apoyar el Diseño Instructivo, su implementación?”
32. Ann Linehan: Gracias, Karen. Este es de Juanita, una supervisora. “Karen podría hablar sobre el uso de la tecnología en su Diseño e implementación?
33. Ann Linehan: Tengo un e-mail de Debra, quien es una nueva coordinadora de educación. “En vista de ésto, quisiera saber en cuántos meses se implementará el Diseño Instructivo de Asesor-Mentor. Con quién le gustaría comenzar?
34. Ann Linehan: Gracias, Karen. Tengo un e-mail de Renee, que creo es un gran comentario para el cierrre de hoy y ella es de early Head Start. "Me alegra ver que se concentra en todas las areas del aprendizaje y todos los niños, desde su nacimiento hasta los 5 años" Muchas gracias.
De nuevo, quiero agradecer a todos los participantes en el estudio y a todos ustedes en la audiencia. Estamos seguros que no alcanzamos a contestar todas sus preguntas en éstas dos horas; sin embargo luego de ésta transmisión pueden seguir enviando sus e-mails hasta las 6:00 pm EST del dia de hoy.
Las preguntas de los representantes y sus respuestas serán colocadas en la página web que ahora aparece en la parte de abajo de la pantalla.
Gracias por acompañarnos. Gracias por su participación hoy
Regresar al Indice de Presentación
| Home | Publications | Partnership/Collaboration | Recruitment/Awareness | Information Center | What's New? |
|
For information requests contact AskUs
We welcome your comments and suggestions, contact webmistress@headstartinfo.org For website technical assistance contact technical@headstartinfo.org To order publications contact puborder@headstartinfo.org |
 The Head Start Bureau |
Copyright © 1998-2006 Trans-Management Systems Corporation. All rights reserved.
Please Note: Links on this site are verified monthly.
While links are evaluated before being included on this site, HSIPC is not responsible for the information presented on external sites.
Last Modified: 08/09/05