Skill shop: Beyond pencil and paper

Today's needs assessment must be customized, informal, and repetitive

By Robby Champion

Journal of Staff Development, Winter 2000 (Vol. 21, No. 1)

It’s not uncommon at mid-year to pause and ask, "What in the world were we thinking when we set up this professional development program?" If your planning committee is concerned about its needs assessment practices, you’re not alone.

Needs assessment strategies that were considered "standard procedure" several years ago are proving to be less than adequate in today’s environment. Your planning committee may have been scorched by an error in one of its program proposals or has an "itch" for some other reason. Now may be the right time for you to consider updating how you are doing your needs assessment.

In most places, "needs assessment" is still synonymous with "pencil-paper survey" but the ubiquitous survey is more and more often inadequate. The environment changes unexpectedly and quickly with the increase in state mandates and with increasing shifts in top leadership in many schools and districts. Not surprisingly, "learning priorities" also shift without much warning. A well-intentioned, laborious survey done in April may yield useful data but that information can look outdated and incorrect by December.

Let’s begin with a new definition of needs assessment: It’s the process of scanning the environment using a variety of information to sort out problems, priorities, opportunities, and learning needs so decisions can be made about future professional development programs. It involves making judgments about how to deal with the discrepancy between what we know and do and what we need to know and do to achieve the organization’s expectations for performance.

One hurdle keeping us from improving needs assessment practices is that many people still hold onto an old concept of needs assessment. It connotes a process that is formal, standardized, and linear. The attributes of good needs assessment practice today are actually the opposite of that. Good need assessment in today’s environment must of necessity be a) informal, b) customized to fit each situation, and c) repetitive.

As the new definition implies, the ways in which we "sort out priorities" can vary significantly from one situation to another. Resources (time, energy, expertise, and money) for professional development may vary from one organization to another, but there are rarely ever enough resources to respond to every imaginable "need." Needs assessment ultimately always involves making judgments, suggesting choices, and setting priorities.

Here are four questions to guide your discussions as your planning committee analyzes its needs assessment practices:

1. Did we plan the needs assessment activities collaboratively?

One of the commandments of professional development is: Thou shalt not plan programs in a vacuum. That holds true for needs assessment. Planning and doing needs assessment tasks alone may seem more efficient but it is a sure path to failure. Adult learners want to be involved in making decisions about their learning and their involvement has been proven over and over to enrich the results.

Include as many voices in the conversations as you can manage. Consider these examples. If archival material would be useful, ask a variety of people to independently analyze the same material and submit their analyses to your committee. Parents who cannot attend meetings may be willing to be interviewed briefly over the telephone. Office secretaries, guidance counselors, and attendance clerks may be willing to keep logs for a specific purpose. Students might be willing to submit their work samples and discuss their school experiences in a focus group. Principals, curriculum coordinators, state department of public instruction staffers, and student assessment experts should be considered stakeholders who need to be heard along the way.

In earlier decades, seeking wide involvement meant investing lots of time and wasting lots of paper. The return-on-investment was questionable. But new technology makes wide involvement much easier and faster. Possibly because of the novelty of electronic networks, you’ll likely get more responses using technology than with pencil-paper surveys. That novelty may wear thin soon and people may hit the "delete" button when they start to get too many requests for their responses, but at the present time we can take advantage of technology’s version of the ‘’Hawthorn Effect," the phenomenon of individuals putting forth extra effort and feeling good about it because they are involved in something new.

2. Did we use more than one source of information to make decisions?

While not all data are equally valuable, good needs assessments invariably use a combination of subjective and objective information to inform decision making. We need to use a combination of tools and to involve a variety of people in order to make the best decisions possible.

Feelings, preferences, beliefs, opinions, and attitudes are considered soft or subjective data. To capture them requires tools that help elicit information that is not necessarily available publicly. Interviews done face-to-face or by telephone, focus groups, pencil-paper or e-mail surveys, and observations in the workplace are ways to gather feelings, preferences, beliefs, opinions, and attitudes.

Objective or hard data can be collected through systematic analysis of work samples or portfolios, test scores, archival records of student enrollments in particular programs, attendance, detentions, retentions, promotions, honor roll, and student grades. These are often "found data," collected for another purpose by the organization but available for program planning.

Student performance data, of course, usually provide the most valuable data and hold more weight than other data when you are designing professional development for school improvement. That doesn’t mean student performance data are easy to use in needs assessment. No matter how closely one scrutinizes student performance data, it is always perplexing to extrapolate exactly what staff might need to know and how they might need to learn in order to improve those student data.

3. Did we use available data well?

In gathering information, you’ll uncover all sorts of information about the organization. Not all problems that you uncover can be solved through staff development. Poor morale, unhealthy norms, sacred cows, rivalries, dirty laundry, and disagreements over salary and work conditions may surface but will likely not help you to target staff learning interests or needs.

Even some of your primary information sources, such as student performance data, can and often do cause erroneous extrapolation about a staff’s capabilities. In other words, improving student performance data may take a major intervention that is not a professional development program. For example: Changing the daily schedule and the school calendar to provide teachers with time to meet and analyze student performance may be more appropriate ways to address student performance than providing even the best professional development program.

To appropriately use available information, one kind of information needs to be examined in conjunction with other kinds of information. Use information as soon as possible to inform the next step in your needs assessment process. For example, if you do an e-mail survey of teachers, tabulate and analyze it within two weeks. Then involve some of those teachers in a discussion of the results, perhaps in a focus group or in random stairwell interviews to help capture the meaning of the numbers and the words on the survey.

4. Did we gather needs information over time?

When you gather information influences its quality and its value as much as how you gather it and from whom you gather it.

If your committee gathered information during a period of stress or work overload, you may have uncovered seasonal needs that disappear in a month. Equally problematic is gathering all of the data at the same time. To be most effective, the needs assessment should be a continuous process so that emerging information can be inserted throughout the year. Once you begin implementing the professional development program you have designed, needs assessment should continue. Programs nearly always need to be adjusted as they’re being implemented so that they continue to meet participants’ shifting interests and growing skills. Canned professional development programs are no longer acceptable because organizations expect results from the investment in professional development.

Conclusion

Making judgments about a professional development program will never be a painless, easy part of a planning committee’s work. But the time may be right for planning committees to learn more about alternative ways to do needs assessment well. Now is also a good time to reflect on how we do needs assessments, including the kinds of information we use, how and when we gather information, and who is involved in the process.

 

A short, related article follows here.

A word to workshop leaders

Check the accuracy of the needs information you were given on-the-spot as you start every workshop. It need not be an elaborate or time consuming segment but it can save you from proceeding with an agenda based on inadequate information or stale information collected months ago. You can use a simple, interactive discussion exercise or a pencil-paper task, such as a problem or task to complete, pre-test, crossword puzzle, or 10 minutes of open-ended question and answer. Also, nothing beats the adhesive colored dots on mural paper on the wall to visually display participants preferences and opinions at the beginning of the day.

When I lead workshops, I favor opening with strategies which enable me to listen to participants within the first 15 minutes of contact. If I give the group a small group or paired task, I quickly circulate and take mental notes of what I see and hear. Equally important, I have found that it sets a positive tone for me to invite adjustments to the agenda and then negotiate changes with the group spontaneously.

 

To learn more

 

About the author

Robby Champion is president of Champion Training & Consulting. She can be reached at 9712 Riverside Circle, Ellicott City, MD 21042, (410) 750-1920, fax (410) 750-8599, e-mail: champion@erols.com

Click for NSDC Home Page