Identifying Situations and Instruments

With objectives defined, the next step is to find places where the desired behaviour can actually be seen. You cannot appraise a behaviour the learner never gets a chance to show, so evaluation depends on identifying the right situations and then having the instruments to capture them.

Identifying situations

The aim of step two is to give learners a chance to express the behaviour implied by the objectives, an opportunity to show that behaviour. This means finding situations that do two things: permit the expression of the desired behaviour, and evoke it. Only then can the degree to which the objective is realised be observed.

Different behaviours need different situations. Questioning can serve as a stimulus to look for evidence of interests. Free play and work, where there is genuine opportunity for free choice of activity, can reveal personal-social adjustments. The principle behind all of them is simple: any evaluation situation is the kind of situation that gives learners an opportunity to express the type of behaviour one is trying to appraise.

Flashcard

What must an evaluation situation do?

Tap to reveal

Answer

Both permit and evoke the behavior an objective implies

Only then can the realization of the objective be observed. Questioning can evoke interests; free play and work can reveal personal-social adjustment. Each behavior needs a fitting situation.

When a situation is hard to use

Not all situations are under control and accessible to evaluators. Some behaviours show themselves only in conditions that are hard to set up or observe. When a situation that directly evokes the desired behaviour is difficult to handle, the task is to find another, simpler situation that has a high correlation with the results obtained from the difficult one.

The logic is practical. If the ideal situation is unworkable, a simpler stand-in that reliably produces the same results will do. The correlation is what justifies the substitution: the easy situation must track the hard one closely, or the evidence is not trustworthy.

Selecting and developing instruments

Once objectives are defined and situations identified, the next step is to examine the evaluation instruments to see how far they can serve the purpose. A developer checks each proposed instrument against the objectives, asking whether it uses situations that evoke the behaviour the objectives call for. This check has three possible outcomes:

1. The instrument is quite satisfactory for certain objectives.
2. The instrument can be modified to be appropriate for certain other objectives.
3. The instrument cannot properly be used for the existing objectives.

In the third case, it becomes necessary to develop a new method for gathering evidence about those objectives. Developing an instrument means trying out some of the situations suggested for expressing the desired behaviour, to see whether they serve as convenient ways of gathering evidence. For an objective about the ability to analyse problems, one situation is to present a set of problems in written form and ask learners to analyse them, then try it out to see how far the responses give an adequate basis for checking the ability. For an objective about interests, one situation is a questionnaire listing a variety of activities for learners to mark as interesting or not, again tried out to see how satisfactorily it works. Trying situations out is how possible evaluation tools are developed into forms that can be used with confidence.

Flashcard

How is a new evaluation instrument developed?

Tap to reveal

Answer

By trying out situations that evoke the behavior, to see if they gather good evidence

For an analysis objective, present problems in writing and check the responses; for interests, try an activity questionnaire. The try-out turns a possible tool into one that can be used with confidence.