Experimentation in Teaching
Experimentation in three steps
| Step | What happens |
|---|---|
| Design | Plan the experiment, including the abstraction needed |
| Action | Run the experiment in the classroom |
| Evaluation | Check the result against the goal |
Why experimentation matters
Most technological progress until the seventeenth century came from experimentation, not from theory. The same is true for much teaching. Trying things, watching the result, and adjusting is how craft develops.
Speculation
Going beyond current knowledge by formal “if A then B” reasoning. The Greeks formalised this with the theorem. Speculation generates hypotheses that experimentation can then test.
The classroom move
A small, structured experiment with one variable changed at a time produces useful evidence. A grand redesign produces noise.
A teacher tries a new way of starting the lesson. They keep everything else the same. They watch what happens for two weeks. Then they decide whether to keep the new opening or revert. This is experimentation, and it is one of the most useful methods a teacher has.
Most teachers experiment without naming it. Naming the method makes the work sharper.
What experimentation is
The main purpose of experimentation is to generate positive results: to make things work. There may be abstraction in the design step, but the abstraction is in service of the effort to design something that works in practice.
Most technological advancement until the seventeenth century was achieved through experimentation. Bridges were built before there was a theory of bridges. Medicines worked before there was a theory of medicine. The pattern repeats in teaching: most useful classroom techniques were discovered by trying things, not by deriving them from a theory.
This does not mean theory is useless. It means experimentation is a respectable method in its own right.
The three steps of experimentation
Experimentation has three steps. Design, action, evaluation.
Design
The design step is where the teacher plans what to try. The plan includes:
- What is being changed. One thing, ideally. A small change is testable. A large change is not.
- What is being kept the same. Everything else. The point is to isolate the effect of the change.
- What evidence will count as success. Defined before the experiment, not after.
- How long the experiment will run. Long enough to see real effects. Short enough to abandon if it is going badly.
The design step often involves abstraction. The teacher has to think about what they are testing in general terms. “I am testing whether opening with a question rather than a statement increases the number of students who participate in the first ten minutes.” The abstraction makes the experiment portable to other lessons.
Action
The action step is running the experiment in the classroom. The teacher does what was designed and watches what happens.
The discipline of this step is to actually run the planned experiment. Many teacher experiments fail at this step because, in the moment, the teacher reverts to old habits. A note on the lesson plan reminding the teacher of the experiment helps.
Evaluation of result
The evaluation step asks whether the experiment worked. Did the change produce the predicted effect? Did anything else happen?
A clear evaluation is honest about three things:
- The intended effect. Did it happen?
- Side effects. What else changed?
- The teacher’s judgement. Should the change be kept, modified, or dropped?
A good evaluation produces a decision and a record. The record matters because it lets the teacher run a series of experiments over a term and see patterns.
Speculation
When we try to think beyond our current knowledge frontier, we are speculating.
The theorem is a formal method of speculation developed by the Greeks. The core of the theorem is: if A then B; A is a hypothesis and B should be proved. Speculation can be very important for generating hypotheses that experimentation then tests.
For a teacher, speculation looks like this: “If I open with a question instead of a statement, then more students will participate in the first ten minutes.” The “if A then B” structure is the speculation. Whether it is true is a matter for experimentation.
Speculation without experimentation is just thinking. Experimentation without speculation is just trying things at random. The two methods together are what real classroom inquiry looks like.
A worked example
A teacher wants to find out whether group work in a Class 8 social studies class produces better understanding than lecture.
Speculation. “If I run a unit on the Indus civilisation as group work instead of lecture, then students will produce more interconnected ideas in the end-of-unit essay.”
Design. The teacher will run the unit as group work for one section and as lecture for another. Same content, same time, same final essay. The essays will be marked using a rubric that scores interconnection of ideas.
Action. The teacher runs both versions over four weeks. The teacher does not change the design partway through, even when the group-work section feels harder.
Evaluation. The essays are scored. The group-work section produces a higher average score on interconnection. The lecture section produces a higher average on factual recall. The teacher decides to keep group work for conceptual units and to use lecture for fact-heavy units.
The experiment produced a useful conclusion. It also produced a hypothesis for the next experiment: which kinds of unit benefit most from group work, and which from lecture?
When experimentation is the wrong tool
Experimentation works for problems that can be isolated. It struggles with problems where the variable cannot be cleanly separated from the rest of the situation.
Three signs that experimentation is the wrong tool:
- The variable cannot be controlled. (E.g. weather, school events, student attendance.)
- The effect is too long-term to observe in the available time.
- There are ethical reasons not to run the experiment. (E.g. depriving one section of a method that is likely to help, just to test it.)
For these problems, other methods are needed: scientific method with statistical analysis on observational data, the SECI cycle for tacit knowledge, design and modeling. Each is a different tool for a different kind of question.
Design, action, evaluation
Design plans the experiment with one variable changed and clear success criteria. Action runs the experiment as planned. Evaluation checks the result against the goal and decides whether to keep, modify, or drop the change. The discipline is in not changing the design partway through.
How experimentation builds craft
Each experiment a teacher runs adds to a small library of evidence about what works in their particular context. Over a term, the library has a few useful entries. Over a career, it has hundreds.
The library is one reason an experienced teacher who experiments is so much better than an experienced teacher who does not. They are working from evidence built up over years. The teacher who never experiments is working from intuition and habit, both of which can be wrong without showing it.
Experimentation is also the method that connects best with inductive action planning. Each cycle of inductive action planning is, in effect, a planned experiment with a longer time horizon. The two methods reinforce each other.