When impact entered the educational lexicon it became a real game changer. Professor John Hattie took as his mantra ‘Know thy Impact.’
What he meant by this was to become an evaluator of your practice. Only by doing this could you focus on the quality of learning happening. Previously, the focus had been on teacher performance through the likes of lesson observations.
To unpack that a little further below, we unpick three of the most important dimensions in Impact.
Model of Impact
Impact is more complex than achievement and therefore this is best seen as a model.
Design. Inputs. Process. Soft Outcomes. Impact outcomes
By seeing impact as a model, it is much easier to see all the parts moving simultaneously. This way, we can see easily if something is working from the outset during an implementation process.
Design refers to our theory of action, what we intend to get done. Each one requires its own plan ready for execution and these needs to be designed with the ‘end’ in mind. An easy judgement of potential for impact is to share that plan with mentors or experienced colleagues. Feedback allows fast iteration ahead of action.
Inputs are the resources and time we are going to commit to defined project. This could include training time, supporting materials, launch event, raising awareness and knowledge uplift. This is a necessary stage, but should not be confused with the main event. INSET days and conferences are inputs, they plant ideas, raise awareness but don’t necessarily lead to any change.
Processes are the actions we take. As a profession, we are naturally doers, moving quickly to this stage. It accounts for busyness. The design should have defined the actions we are to undertake. A better design will contain contingencies and best processes have checks in place to know that they have been taken. Additional measures let us know they are holding and working. Think not only what have I got to do but also what will be happening differently if it is working?
Soft outcomes are the immediate changes we want to see. By embarking on the change cycle, we may have anticipated higher levels of attendance, greater motivation, positive feedback and greater urgency. Such evidence can be gathered through learning walks, book scrutiny and feedback routines. They offer a proxy to improved measurable outcomes that is faster to attain. They should allow us to know that planned change is working.
Be aware that they are not evidence of true impact. Ofsted’s recent move away from even looking at internal data is one example of false readings that soft outcomes can bring due to their subjective nature.
Impact outcomes are the objectively measurable improvements of the intended actions. Due to growing complexity and the ‘messy’ nature of school improvement process these can take a lot longer to become seen. A change of culture or a focus on learning can take two to three years to pass into improved examination results or higher school roles. This is known as sink time. The earlier described impacts can give us some reassurance that we are on course to reach the intended impact. It is vital that such hard measures are well defined and base-lined ahead of any change project.
Causal vs Correlational
It is not enough to gather evidence of impact. We also need to ensure that what we planned to do improves as a result of what we did rather than could have happened anyway.
Causal improvement is where a direct chain of events can be linked back to what we set out to achieve. Rather like a chain of dominos dropping in sequence to the nudge before.
Correlational can show the same outcome but the chain sequence may have been the result of wind, an extra nudge or luck.
There is nothing wrong with getting lucky. One school we worked with wanted to improve learning and increase numbers on role. The former involved an investment over time in Visible Learning. The result was measured and case studied. The hard work paid off and through every measure that could be applied they improved and their inspection grade went to the top ranking. Clear causal link.
The latter improved with a new marketing manager. However this would only account for the input side. What transpired was that a local school was hit by a scandal leading to transfers of pupils. At best this impact could only be correlational.
Behavioural change is by far the easiest to enact. There is no need to understand current situation. Humans are pack animals and tend, on the whole, to follow new codes rather than fight them. Especially if a system of rewards and punishments are used to reinforce the planned changes.
Take away the codes and sanctions and rewards and all too often we revert to previous behaviours. As any school leader and teacher will know, it is necessary to keep modelling and enforcing behaviours we want to see. So they are only sustainable with our continued intervention.
For improvements to be sustainable in the longer term, it is necessary to go through a series of stages. The acronym KASH is one we use behind the very successful OTI programme (Outstanding Teaching Intervention). Here 95% of teachers in the programme develop sustainable better Expertise.
K = knowledge uplift. Without this people are unsure on clarity and specificity of actions
A = Attitude. In this case belief and intentionality to carry out intended changes, to take risks
S = Skills. Dedication and repetition to building new routines and the nuances required in execution
H = Habits. The change in the default settings to ensure new ways of doing things stick
With any improvement project, it is vital that we get to habit formation. Often this is as much about unlearning poor old habits to make way for finding new ways to do things better. This is so often why initiatives fail to bring any impact. New research needs to be chewed upon, integrated into existing routines and practiced under expert guidance.
It takes 3 weeks daily practice to build the skill. A new habit takes an average of 66 days to acquire. It becomes wired only after 9 months.
If you want Impact, think ‘Model, causation and sustainability’. This is not so easy, but is worth the investment.