By Antoinette Kellaher and Jeff Maher

Journal of Staff Development, Fall 2003 (Vol. 24, No. 4)

Copyright, National Staff Development Council, 2003. All rights reserved.

"The mentor teacher has acted like a lifeline to me this year. ...This was a very difficult year and an even more difficult assignment. ... Although she is unaware of this, I could not have made it without her support this year."
- A first-year teacher, Prince George's County Public Schools

While teachers can tell us wonderful stories of success, anecdotes and other informal assessments are not enough in an age of accountability. In Prince George's County Public Schools (Maryland), our evaluation processes helped us blend soft data with hard data--and helped us learn key lessons in the process, both about building a mentoring program and refining it.

Our beginning

Prince George's County Public Schools, the nation's 17th-largest school system, began a Mentor Teacher Program in 1997 after the state required the district to provide new teachers in low-performing schools and schools with high teacher turnover with master teachers to guide their professional growth.

The district regularly hires more than 1,200 newcomers a year in a system of more than 8,500 teachers, and the number of provisionally certified teachers has increased as more fully credentialed veterans retire. But more importantly, the district noticed a disturbing trend--51% of new teachers resigned by their second year, and many teachers left midyear because they were unprepared for the challenges of teaching in an urban setting. Schools with high teacher turnover were at the bottom of student achievement on state tests. The mentor program focused on reducing teacher attrition at these schools and increasing student achievement.

The district created a program in just a few months, beginning with a one-week professional development session and following up with monthly meetings and workshops that concentrated on reflection, observation, instruction, and assessment.

The district set clear standards and goals for improving teacher retention and student performance. But while an operational manual is the foundation, the heart of the program occurs at the school level. The district systematically examined data that mentors collected throughout the program and through mentor-led action research to develop professional development opportunities, refine the program, and guide school-based instructional decision making.

Setting goals

The Mentor Teacher Program focuses on three specific goals:

• Reducing the attrition rate of teachers in their first three years;
• Reducing the number of provisionally certified teachers; and
• Increasing student achievement.

Mentors work with a cohort of up to 15 new teachers (10 new teachers for part-time mentors) to meet mentees' individual needs, which may include gaining certification, classroom management, or instructional challenges. Mentors provide support through coaching, modeling demonstration lessons, and assisting with planning. Monthly workshops focus on team-building and assisting mentees with instructional needs, guided by the school improvement plan.

Each mentor also is part of a mentor study group focused on instructional issues that face mentees. The mentors coach each other, and program coordinators coach mentors. Mentors have online discussions with one another and program coordinators to solve problems and share ideas. Mentors also meet regularly with principals.

The results of the program are encouraging. In schools where mentors were assigned for at least two years, evaluations show both student achievement and teacher retention improved and met district targets. Meeting goals and demonstrating results means evaluation must be a part of the program.

Collecting data

Evaluation has been at the core of the program's procedures since its inception. Mentor teachers in the program understand the need for evaluation and reflection. The first two program goals (reducing the attrition rate and reducing the number of provisionally certified teachers) are clearly quantifiable through data from human resources and through exit surveys.

Mentors work to identify specific interventions that contribute to these goals by systemically tracking data and through semiannual benchmarking using a variety of instruments:

• Exit surveys to determine why teachers resign;
• Checklists to help mentors analyze new teachers' classroom performance;
• Weekly schedules and logs to examine mentors' use of time and the focus of their work;
• Surveys for mentees about what types of support they received from their mentors;
• Workshop data collection forms asking what certification, experience, and assignment participating teachers have;
• Workshop evaluation forms designed using the National Staff Development Council's Standards for Staff Development as a framework for analyzing the workshops' content, context, and process;
• Focus groups of new teachers, mentors, and administrators in which an outside evaluator gathers qualitative information about the quality of service delivery; and
• Monthly journals mentors use to reflect on areas of growth and need in their individual situations.

Current journal questions include:

1. Identify and briefly evaluate the effectiveness of the strategies (e.g., demonstration lessons, workshops, videotape review, guided observations of other teachers, etc.) you use to help mentees become better teachers.
2. What would it take for your mentees to be more effective?
3. What recommendations do you have to improve the Mentor Teacher Program for yourself, for your mentees, and for operations within your building?

These instruments for data collection and evaluation are due at different times of the school year, and mentors and program coordinators review the data in monthly sessions.

Additionally, the state of Maryland this year required that mentors work with mentees weekly for a specific amount of time. This rule required that the district document mentors' time. The district went further to analyze mentors' interactions with mentees (see chart at right). Interactions can be analyzed both by the focus (e.g., instructional strategies, assessment, special needs, etc.) and by the method of interaction (e.g., analysis of student work, coaching conversations, lesson planning, etc.). This process helped us identify both mentees' and mentors' needs.

Measuring the impact of professional development on student achievement (the program's third goal) is more difficult as there are multiple school interventions that contribute to improved teacher performance and student achievement.

To help understand how mentors fit into this equation, each mentor led an action research process with select members of their cohort examining student work and teacher practice around a theme or guiding question based on their school's needs. In the collaborative action research, the mentor can identify the effect specific strategies have on student achievement with his or her mentee using formative assessment data. The district also uses schoolwide assessment data to measure the program's impact.

Evaluating the data

Collecting data, analyzing it, and making adjustments takes time. The program's comprehensive evaluation process requires mentors to plan extensively, both in time and on task, to ensure that mentees' needs are met and that the mentors still can make time for collaborative action research. The mentors' monthly meetings serve two functions--both as training time and time for data analysis. By having mentors examine and reflect on progress indicators, e.g., mentees' growth on teaching standards, benchmark progress in formative assessments, or progress towards full certification of mentees, mentors can adjust their service delivery. For example, after reading performance declined, elementary mentors focused on students' reading achievement by modeling and co-teaching small-group reading rotations every day.

In another instance, mentors at the high school level noticed low student achievement and that teachers didn't understand content standards. They saw a need for further professional development --specific, content-based workshops for the mentees--and the district then addressed the need. The monthly mentor training sessions provide an opportunity to refine the district's staff development and tailor it to suit the needs of mentors and mentees. Mentors' end-of-year portfolios allow both mentors and program coordinators to reflect on changes that occurred and needed changes to be implemented for the next program cycle.

Analyzing exit surveys of each mentee who resigns during the school year or at the end of the school year helped program coordinators gather information about why new teachers leave their school and the school system. During the 2000-01 school year, 17% of new teachers indicated "lack of principal's support" as a key reason for leaving. After sharing this with principals and mentors, the district included more school-based activities in the mentoring program and improved communication between school administrators and mentors. The next year, only 4% of new teachers stated that lack of principal support was a key reason for their leaving.

Refining the program

This process of continual improvement and reflection has allowed us to refine our evaluation practices. Just as mentors conduct action research at their schools to help them improve, the program follows a similar cycle.

Since program coordinators recognize personal involvement in the program may affect their objectivity, the district budgets each year for an outside evaluation. An evaluator from the University of Maryland at College Park has analyzed mentor journals, conducted focus groups of mentors and mentees, and reviewed other mentoring activities. She helped develop key questions and refine questions for journal responses.

The outside evaluation has resulted in some recommendations to refine the program, such as: "...it appears mentor teachers would benefit from increased opportunities to interact with each other around challenges/issues they face in their mentoring role. In fact, mentors are likely to benefit from an occasional problem-solving session, where small groups of mentor teachers (experienced and inexperienced) are tasked with solving an issue they commonly face (e.g., time management, receptivity of mentees, finding one's place within the school community, meeting the varied professional development needs of mentees, certification issues, etc.)..."

After that recommendation, program coordinators built time into monthly meetings for study group meetings and problem-solving sessions. The group also began using an internal web-based network for discussion and to share resources.

In addition, coordinators analyzed the program using critical questions and found the following results:

To what extent are the school and school administration supporting the mentor?
Mentors cannot work alone. All supporting specialists in the building must work toward new teachers' success.

How have additional duties and responsibilities within the school building affected the mentor's performance and/or work with his/her cohort of mentees?
Mentors' primary purpose is assisting new teachers. If additional school duties, such as serving on committees or working on the school improvement plan, take too much time, program coordinators must intervene.

What human or material support does the mentor need to assist new teachers?
Analyzing journals and weekly schedules tells coordinators a great deal without the mentors directly saying it. For example, when mentors were frustrated with questions around new teachers' certification, coordinators provided those answers or resources to the mentors by inviting certification specialists to monthly meetings to share information and answer questions.

How are mentors using their time?
Through their weekly schedules, mentors detail hour by hour what they do with mentees. Reviewing these schedules weekly, coordinators determine common areas of need for both mentors and mentees, as well as see areas of growth. For example, if the time spent with mentees on classroom management decreases, coordinators would assume mentees' needs in this area also have decreased.

At the conclusion of each yearly program cycle, the district has a symposium to make public recommended improvements. After the first year, 11 recommendations were made and the program changed accordingly.

Conclusion

Some cynics at first questioned the need to evaluate: If mentors' jobs are to work with new teachers, why did they have to spend time on evaluation? We realized that everyone must be involved, as a matter of accountability and consistency, for the plan to work. We found that when mentors did not complete some part of the process, they tended to become involved in other activities not supportive of the program goals. For example, if the mentor didn't plan out a weekly schedule showing visits to new teachers, school administrators saw open blocks as "free time" and assigned the mentor other duties.

Data collection and analysis lead us in program and process improvement. Because of conscious, focused attention to our goals and ongoing evaluation of our progress to refine what we do, our program has helped teachers and students. The district program has now grown to include 42 full-time mentors, eight itinerant mentors, 33 part-time mentors, and job-alike mentors working with teachers of special populations (e.g., limited English proficient). After five years, we've learned some of what works and what doesn't--and we're still learning.

About the Authors

Antoinette Kellaher is coordinating supervisor of staff development for Prince George's County Public Schools and has coordinated the Mentor Teacher Program for five years.

Jeff Maher is a staff development specialist for Prince George's County Public Schools and co-coordinator of the Mentor Teacher Program.

You can contact them at the Department of Staff Development, 6111 Ager Road, Hyattsville, MD 20782, (301) 408-4680, fax (301) 408-4698, e-mail Kellaher: akellahe@pgcps.org and Maher: jmaher@pgcps.org.