Checklists, getting better at getting better, and ‘visible learning inside’ : 2015-08-25

Checklists, getting better at getting better, and ‘visible learning inside’

The acclaimed surgeon and writer, Atul Gawande, highlights the need for a different strategy; one that is ‘ridiculous in its simplicity’. This strategy, he says, builds on experience and takes advantage of the knowledge people have but also makes up for our ‘inevitable human inadequacies’. It is a checklist!

In his brilliant work, The Checklist Manifesto: How to Get Things Right, Gawande details the power of such checklists, charting their use in the airline industry, their translation into the medical domain, and beyond medicine into other fields, from disaster response to investment banking, sky scraper construction and businesses of all kinds. Gawande gives a fascinating account of his innovative work with the World Health Organisation and a global program aimed at reducing avoidable deaths and harm from surgery. Gawande says that:

It’s ludicrous… to suppose that checklists are going to do away with the need for courage, wits, and improvisation. The work of medicine is too intricate and individual for that: good clinicians will not be able to dispense with expert audacity. Yet we should also be ready to accept the virtues of regimentation.

I was particularly intrigued by Gawande’s findings so far as the building industry is concerned. He found that the use of checklists redistributed power, as opposed to what checklists are usually about – dictating instructions to the workers below to ensure they do things the way we want. Gawande describes how, on the particular large-scale building project he was exploring, that when confronted with complex, non-routine problems, the philosophy was to push the power of decision making out to the periphery and away from the centre. You give people the room to adapt, based on their experience and expertise. All you ask is that they talk to one another and take responsibility. That is what works. A democratic strategy.

After reading and thinking on Atul Gawande’s brilliant work I returned to the piece of work that has held my attention for several months now, something that keeps drawing me back: Bryk et al’s Learning to Improve: How America’s Schools can Get Better at Getting Better. It was, indeed, this work that had signposted me to Gawande’s in the first place. It was this particular citation that compelled me.

Making systems work is the great task of my generation of physicians and scientists. But I would go further and say that making systems work – whether in health care, education, climate change, making a pathway out of poverty – is the great task of our generation as a whole.

Bryk and colleagues use ideas from improvement science to show how ‘a process of disciplined inquiry can be combined with the use of networks to identify, adapt, and successfully scale up promising interventions in education’. Interestingly, Bryk, highlights Gawande’s warning that the use of great components, even evidence-based components, does not assure that quality outcomes will result. ‘Making all of the pieces fit together is a formidable and quite distinctive improvement task.’
I like the fact that the discipline of improvement science directs us to step back before we move forward; before looking to improve the current system, seek to fully understand it first.

Further, Bryk and colleagues are adamant that rather than “implementing fast and learning slow,” educators should adopt a more rigorous approach to improvement that allows the field to “learn fast to implement well”. The authors describe themselves as “analogical scavangers”. That, because they are constantly looking to other fields that share a concern about improving practices and that have made some significant progress. I love the idea of being an “analogical scavenger” and ‘getting better at getting better’!

An improvement culture puts learning to improve at its centre and challenges everyone to put failure to work toward valued goals. This kind of culture encourages people to report when things don’t go according to plan. It makes people feel safe because they know that failure means that their team is testing its theories and bettering the work of the organisation. Part of the genius in the quality improvement logic of starting small and aiming to learn fast is that the costs of early failure are reduced and the dangers of confirmation bias – seeing only what we want to see – also lessened. In this regard starting small eases the path toward subsequent success.

In summary, the authors concisely recount their six improvement principles:

1. Make the work problem-specific and user-centered. Quality improvement starts with a single question: “What specifically is the problem we are trying to solve?” It enlivens a co-development orientation. Engage key participants as problem definers and problem solvers from the earliest phases of development through large-scale implementation.

2. Focus on variation in performance. A networked improvement community aims to advance efficacy reliably at scale. Identifying and addressing the sources of variability in outcomes is essential. Rather than documenting simply “what works,” as in estimating an on-average effect, aim to learn “what works, for whom, and under what set of conditions.” Develop the know-how to make innovations succeed for different students across varied educational contexts.

3. See the system that produces the current outcomes. It is hard to improve a system if you do not fully understand how it currently operates to produce its results. Seek to better understand how local conditions shape work processes and resulting outcomes. Use this analysis to explicate a working theory of improvement that can be tested against evidence and further developed from what is learned as you go.

4. We cannot improve at scale what we cannot measure. Measure outcomes, key drivers, and change ideas so you can continuously test the working theory and learn whether specific changes actually represent an improvement. Constantly ask: “Are the intended changes actually occurring? Do they link to changes in related drivers and to desired system outcomes?” Anticipate and measure for unintended consequences too.

5. Use disciplined inquiry to drive improvement. Common inquiry protocols and evidentiary standards guide the diverse efforts of NICs. Engage in systematic tests of change to learn fast, fail fast, and improve fast. Remember that failure is not a problem; not learning from failure is. Accumulate the practical knowledge that grows out of failure, and build on it systematically over time.

6. Accelerate learning through networked communities. NICs aim to break down silos of practice and research. They enliven a belief that we can accomplish more together than even the best of us can accomplish alone. A shared working theory, common measures, and communication mechanisms anchor collective problem solving. Organise as a NIC to innovate, test, and spread effective practices sooner and faster.

Adopting such an approach makes good sense to me. Through this reading, and more, I found that I was constantly able to make direct reference to the Visible Learning Framework that we, as a team at Bader Primary School, have embraced and are working with as a school improvement driver. In a previous post I have detailed how we have adopted Sir Michael Barber’s Deliverology model as we look to deliver on our shared aims and aspirations, namely:

All members of our school community are assessment-capable learners
• I know what to do when I don’t know what to do
• I Know where I am, how I am doing and where I am going next

All members of our school community are self-evaluators
• I know how I am doing, where I am going next and how I am going to get there
• I see assessment as feedback to me

All members of our school community are effective collaborators and communicators

All members of our school community have a growth mindset

This week, I returned to John Hattie’s Visible Learning for Teachers: Maximising Impact on Learning. Lo and behold – missed on earlier readings – there he is again… Atul Gawande, mentioned by John Hattie himself. Hattie points out how Gawande shows how checklists help to achieve the balance between specialised ability and group collaboration. Hattie moves on to Michael Scriven’s distinction between types of checklists, from the laundry list, the sequential list, flow charts, and the merit checklist. And then… hey presto!! Hattie directs us to the fact that contained within each chapter of Visible Learning for Teachers (and collated in Appendix A: Checklist for ‘visible learning inside’) is a set of criteria that can each be considered, with those reviewing the evidence making a decision about merit and worth. Such an approach, he says, offers flexibility in providing evidence and acting to ensure that a school is working towards making learning visible.

We have a ‘checklist’ to guide us in moving forward on our Visible Learning journey at Bader, and we shall adopt it on 1st September!! Now, sometimes it takes a while for the penny to drop with me, but I have thoroughly enjoyed indulging myself in the work of these great people. And how often do you find yourself going full circle when immersing yourself in great literature?!

A personal aspiration for me, now, is to get better at getting better and make it my task to make systems work at school in such a way that we may say that Bader Primary has ‘visible learning inside’.

What am I thinking on now? Based on the above, how might we make collaborative inquiry work at Bader, and incorporate individual engagement in that process in our performance management cycle? Watch this space!

Leave a Reply