There are many types of systems. Social, mechanical, biological, and ecological systems are examples. This post explores the key features of social systems and their implications for leaders of organisations of all types.
Systems
Let’s start with a definition.
A system is a network of interdependent components that work together to try to accomplish the aim of the system.
Edwards Deming, 1994, The New Economics: For industry, government and education, MIT, Massachusetts, p. 5
A key feature of systems is interdependence among the elements comprising the system. Another key feature is that systems can nest and be contained within other systems.
To illustrate, the human body is a system of interdependent components. Within the containing system of the human body there are systems such as the nervous, respiratory and circulatory systems. These systems are clearly interdependent.
Figure 1. Systems nest, and are contained, within other systems
Social Systems
Social systems are a specific type of system that exhibit four unique characteristics:
A social system can choose its purpose.
A social system can choose the methods by which it will seek to achieve its purpose.
Elements within a social system can also choose their purpose.
Elements within a social system can choose the methods by which they seek to achieve their purpose.
These characteristics explain why organisations are such dynamic and complex systems to manage and lead.
Consider a school, for example. A school is a social system. Within the context of the local education system and its community, a school chooses both its purpose and how it proposes to achieve its purpose. Figure 2 shows the purpose statement for Wanniassa Hills Primary School in the ACT. The school consulted extensively with parents, students and staff over the past few months to discover and document this purpose statement.
Figure 2. Wanniassa Hills Primary School Purpose Statement
Within a school, the various faculties and teams also choose their purposes and methods to achieve them. For the school to optimise its efforts, the aims and methods of the individuals and teams need to align to those of the school as a whole.
Many teachers and school leaders can share stories of individuals and groups within a school not being aligned with the aspirations of the school community as a whole. It can be very frustrating for everyone.
Similarly a class is a social system. The class can choose its purpose, like that shown in Figure 3. The class can also choose how it will seek to fulfil its purpose. Within a class, individual students and groups of students choose their purpose and how they wish to go about it. This is evident from moment to moment as students may choose to ‘tune in’ or engage in disruptive behaviour.
Figure 3. A class clarifies its purpose with a mission statement
In summary:
The system as a whole chooses purpose and methods, as do elements within the system.To optimise the efforts of a school, the purpose and methods of the individuals and groups within the school need to align to that of the school as a whole. The same applies to classrooms.
Interaction effects
A key reason organisations are such dynamic and complex entities is the impact of interactions among the elements. The interactions among members of a class and the interactions among teachers and groups of teachers are two examples of these interaction effects.
As the number of people in an organisation increases, the number of potential interactions increases exponentially. In fact, the impact of interaction effects becomes way more dominant than the impact of individuals within the organisation.
The interactions among individuals and groups can be positive and add to the performance of the whole. They can also be negative and detract from performance.
Interactions can lead to harmony, and they can lead to discord.
This is self evident to most teachers and school leaders who have worked in a variety of schools, from those with a tired and cynical culture to the vibrant and collaborative team.
Interaction may reinforce efforts, or it may nullify efforts.
Edwards Deming, 2012, The Essential Deming: Leadership Principles from the Father of Quality, edited by Joyce Orsini, McGraw-Hill, New York, p. 78.
Interactions dominate the behaviour and performance of social systems. Consequently, the behaviour and performance of a classroom, workgroup or organisation is much more heavily influenced by the interaction among people than by the direct impact of each individual. This is why creating a school climate that promotes caring and productive relationships is so important.
Conclusion
Schools are complex, dynamic and highly relational social systems. The interdependence among the individuals and groups within schools and the interactions among them, are key determinants of the culture, behaviour and performance of the school. The same applies to classrooms.
It is incumbent on leaders, therefore, to ensure that system and processes promote positive interactions with purposeful and harmonious relationships.
In 1993, Myron Tribus proclaimed: The job of the manager has changed.
People work in a system. The job of a manager is to work on the system, to improve it, continuously, with their help. Myron Tribus, 1993, “Quality Management in Education”, Journal for Quality and Participation, Jan–Feb, p. 5. Available at http://www.qla.com.au/Papers/5.
What did Tribus mean?
System
Firstly, we need to understand what he meant by system. Dr Deming defined a system to be:
A system is a network of interdependent components that work together to try to accomplish the aim of the system. Edwards Deming, 1994, The New Economics: For industry, government and education, MIT, Massachusetts, p. 50.
Because Tribus is referring to managers, we understand him to be speaking of organisations. Organisations are systems comprising interdependent components working together towards some aim. A school is a system. A classroom is a system. A school district or region is a system.
A way of thinking about systems, in this context, is to think about how all the elements work together, as a whole, to get things done. How do school policies, procedures, facilities, committees, teams, classrooms, parents, leaders, teachers and students, for example, all work together to achieve the purpose and vision for the school?
Manager
Secondly, we need to understand whom Tribus is referring to in saying the job of the manager has changed.
Management is the ability to organise resources and coordinate the execution of tasks necessary to reach a goal in a timely and cost effective manner. Kovacs and King, 2015, Improving Learning: A how-to guide for school improvement, QLA, Canberra, p387
Managers therefore are those seeking to reach goals, by working with tasks, resources, systems and processes. Under this definition, it’s hard to identify individuals who are not managers. Everybody in a school is organising resources and coordinating tasks to achieve goals, even students! For this conversation, however, let us limit our discussion to adults. Principals, teachers and support staff are all working with their colleagues and students to achieve the goals of the school and classroom.
Working in and on
Thirdly, Tribus makes the distinction between working in the system and working on the system.
Working in the system is doing the daily work of the system.
For a teacher, this usually means managing the daily routines of learning and teaching in the classroom: planning, programming, instruction, assessment, reporting and so on. For school leaders this includes: meeting with parents, providing support to school staff, attending meetings, managing the budget, responding to emails and phone calls, and so on. This is all the daily work – working in the system.
Working on the system is improvement work.
Working on the system comprises two types of activities: improvement projects and innovation projects. Both involve making changes to the existing system.
Improvement projects focus on making the existing system more efficient and/or effective.
This is achieved by improving how the elements of the system work together, usually by making changes to the processes and methods by which the work is done. Refining the enrolment or reporting process in a school would be examples of improvement projects. Improvement projects build on existing approaches to make the existing system work better.
Innovation projects are about creation of new systems, processes, products and services by the organisation.
In a school context, innovation projects are about new technologies, new programs and system reforms. Replacing parent-teacher interviews with student-led conferences would be an example of an innovation project. Innovation projects are about new approaches that prepare or position the organisation for the future.
Given this, Tribus is telling us that all mangers within an organisation have an obligation to contribute to improvement efforts. But there is a subtle twist in the last three words of his proclamation: with their help.
…with their help
Finally, Tribus is explicit that managers should not unilaterally impose changes upon those working in the system. All managers need to be involved in projects that work on the system, but these projects need to engage those working within the system. After all, it is those doing the daily work of the system that know most about how it is done and could be improved.
Students know best the barriers to their learning; teachers know best what gets in the way of their teaching.
Within a school context, Tribus is saying that all adults need to be engaged working on the system to bring about improvement. They need to be participating in improvement and innovation projects, as project leaders in their own areas or team members with others’ projects. Students also need play an active role, contributing to improving their school and classroom.
The five whys tool was developed within the Toyota Motor Corporation as a means to identify the underlying causes of problems. When root causes are identified and addressed, the problem can be fixed and stay fixed.
Five whys can also be used to explore personal motivations.
The process is very simple: the issue under investigation is identified and noted. Examples: “Why are students disengaged from their learning”, “Why are we proposing to hold the meeting?”, “Why are enrolments dropping?”, “Why do we come to school?”, “Why are teachers not listening?”.
“Why?” is then asked five times (the number of repetitions is not immutable, but in most cases five repetitions have been found to be sufficient).
In this example (Figure 1), year 8 students consider why they study mathematics.
Figure 1. Five Whys: Why do we study mathematics?
Some years ago a teacher from a secondary school in Victoria told us the following story.
A class was constantly disrupted by the inappropriate behaviour of a student. Instead of responding in the usual manner by removing the child from the classroom, the teacher took the student to one side and applied the five whys tool to investigate the cause of the behaviour.
The student revealed that he found it difficult to make friends with others in the classroom, and that the behaviour was a means of getting attention and connecting with others.
The teacher worked to help the student learn strategies to develop relationships with others. This was a far more productive and longlasting solution than would have been achieved by reacting to the symptom and removing the student from the classroom.
We have previously discussed the power of the Plan-Do-Study-Act (PDSA) cycle in bringing about collaborative, sustainable improvement. We have also emphasised the importance of allowing students to play a key role – giving students a ‘real voice’ – in improving their school and classroom. In this blog, we share another example. This time, how the PDSA process was used by a teacher and students to improve learning and engagement in their physical education (PE) classroom. (You can also view this as a QLA case study video.)
Teacher, Chris, with her PE class PDSA storyboard
Chris is a leading specialist teacher at a Victorian primary school. She observed the school’s Year 6 students becoming increasingly disengaged during their weekly PE lessons. PE teachers were stressed and student behaviour was worsening. No one was enjoying PE!
Chris decided it was time to set students and teachers to work to improve PE by applying the PDSA cycle.
As we have seen previously:
PDSA is a highly effective improvement approach, based upon a cycle of theory, prediction, observation, and reflection.
It involves applying a structured process to achieve sustainable improvement.
A nine step PDSA process
This includes:
defining the opportunity for improvement by agreeing the purpose and establishing a shared vision of excellence
focusing improvement efforts on a system or process (rather than blaming individuals)
identifying root causes not symptoms
developing and applying a theory for improvement
reflecting on the outcomes achieved to agree a new ‘best method’ or further improvement needed.
Here’s how…
Chris applied the PDSA process with her students. They documented a comprehensive storyboard to capture their agreements, the data collected, and to reflect their progress in applying the PDSA process.
Here’s what they did:
Students and teachers discussed to agree the opportunity for improvement – to improve their PE classes.
They studied the current situation – what did PE currently look like, feel like, and what was going on? They agreed: students were disengaged, disinterested and not putting in their best efforts; some students were disrupting the class, preventing others from enjoying PE; and teachers were frustrated.
PDSA storyboard extract: brainstorm of the current situation in PE
They collected data to measure the extent of the dissatisfaction with PE. A correlation chart was used to measure student
PDSA storyboard extract: collecting data using a correlation chart – how much are students enjoying and learning in PE?
perception. The data revealed low levels of student enjoyment
(fun) and learning in the PE classroom.
Students then brainstormed and themed the drivers and barriers associated with motivation and participation in their PE classroom. They used sticky notes and an affinity diagram to facilitate this. The major barriers they identified were: ‘inappropriate behaviour’, ‘boring classes’, ‘lack of student choice’, ‘the weather’ and ‘wasting time’.
PDSA storyboard extract: affinity diagram of the barriers to student motivation and participation in PE
These barriers were analysed to agree the root causes using an interrelationship digraph. (They knew that by working on the root causes of their problem that they would realise the greatest return on their improvement efforts.) For the PE students this revealed ‘lack of choice’ as the major or root cause. A lack of choice by students in their PE lessons was seen as a major barrier to participation and motivation. It was impacting upon the other causes and driving the observed problems with behaviour and performance in their classroom.
PDSA storyboard extract: interrelationship digraph analysing the root causes of a lack of student motivation and participation in PE
A bone diagram was used with students to further explore the current situation, and to agree a vision of excellence for PE – what they wanted PE to be like. The resulting student vision showed students believed: student choice, a clear purpose and process for each session, appropriate behaviour, more minor games, a mix of skills, effective use of time, student’s understanding what was expected, and knowing whether they were improving; were the key characteristics students believed were essential for a great PE lesson.
PDSA storyboard extract: bone diagram agreeing a vision of excellence for PE
They brainstormed possible solutions which included: ‘kids teaching kids’, students ‘choosing activities’ and ‘writing their own report’, agreeing a student ‘code of behaviour’, clarifying expectations (quality criteria: ‘know what a good throw, jump looks like’), and students ‘making up games’.
PDSA storyboard extract: brainstorm of possible solutions to improve PE
These solutions helped them to develop a ‘theory for improvement’ comprising the following key strategies:
multi-voting to agree the focus of each lesson
agreeing the lesson format – flowcharting the teaching and learning process
appointing student skill coaches and documenting skill cards to help the coaches do their job
students undertaking peer evaluation together with their teacher/coach. They developed capacity matrices for key areas of learning to help them to do this. They also documented quality criteria describing how to execute essential skills with a high degree of excellence (e.g. how to do an overhand throw). Students used the capacity matrices and quality criteria as the basis for reflection and evaluating their progress in PE
agreeing a code of behaviour
everyone reflecting and giving feedback after each lesson.
PDSA storyboard extract: agreed strategies to improve PEPE – capacity matrix for gymnasticsPE – quality criteria for an overhand throw
The outcome?
The PE classes applied the agreed strategies and new processes, and a few weeks later reflected on the effectiveness of the improvements they had made (the ‘study’ phase of the PDSA
process).
Behaviour and engagement improved. Students were motivated and learning
Students ‘owned’ and were running the PE lessons with minimal guidance from PE teachers! They were responsible for their learning
PE lessons had a productive ’buzz’! Students were excited. Teachers were happy.
The processes they had developed together were adopted as the new way for PE lessons.
Chris described the PDSA based collaborative process as having an amazing impact.
Applying the PDSA process, working ‘with the kids’ and not ‘doing to the kids’, brought about significant positive change to PE lessons – improving the way teachers were teaching and students were learning – to great effect!
What is your school’s theory of teaching and learning?
Some schools waste time focusing their efforts on trying to control and manage the actions and behaviours of individuals. They would do better examining the underpinning theory, systems and processes driving the action and behaviour. Reflecting deeply on, and defining (making explicit), the beliefs upon which current approaches to learning and teaching are based, can lead to great focus, alignment and return on efforts to improve.
Fundamental to improving learning is to agree (define) the theory guiding our teaching and learning.
The following anthropological model adapted from the work of Martin Weisbord can help us understand why this is so. It describes a hierarchy of influences on organisational behaviour. The model is consistent with Deming’s teachings on how systems drive performance and behaviour, and the need to develop theory to drive improvement.
Weisbord’s anthropological model illustrating an organisational heirachy of theory driving action and behaviour
Weisbord’s model illustrates the relationship between beliefs, philosophy (theory), systems, processes, choices and action. An organisation’s systems and processes reflect and reinforce its values, beliefs and philosophy. These systems and structures dictate the processes and methods, and shape the dilemmas and choices faced by individuals of the organisation. The choices made by individuals, in turn, produce the actions and behaviours we observe.
Let’s look at an example to illustrate. Say we believe students are inherently lazy, that they have little desire to improve, and need to be motivated to learn. We will then develop systems and processes in our school and classrooms in an attempt to extrinsically motivate them. Our systems and processes will usually be based upon incentives and rewards, fear and punishment. If, however we believe we are born with an innate desire to learn and to better ourselves, and that the motivation to learn comes from within, then we will design very different systems of learning in our classrooms. These systems usually focus upon building ownership of learning, and working with students to identify and remove the barriers to their intrinsic motivation and learning.
Defining a theory and designing systems and processes can be a deliberate and thoughtful action or it can occur through natural evolution – the choice is ours.
We can make a conscious choice to define and make explicit our values and beliefs regarding teaching and learning. An operational definition is used to achieve and document a shared understanding of the precise meaning of concept/s. Operational definitions provide clarity to groups of individuals for the purposes of discussion and action.
It follows that once we have defined our theory of teaching and learning, we can design structures, systems, processes and methods that are aligned to it and naturally promote the actions and behaviours we desire.
Of course, we draw upon evidence-based research to craft our theory. We can then work together over time testing, reinforming and reaffirming this theory, and improving systems and processes to produce the performance and behavioural outcomes we wish to see.
How to…
Our work with schools in defining a learning and teaching philosophy has typically followed the process summarised in the flowchart below. All staff are invited to be involved in agreeing the philosophy which takes place through one or more workshops.
Flowchart of a process to create a school learning theory
Step 1. Agree a research or evidence-base to inform the philosophy
The first step is to agree and draw upon a research or evidence-base to inform the philosophy. Education systems in Australia have, over time, adopted different pedagogical models. Schools have adopted many different models, all purporting to reflect the latest research and providing the theory necessary to guide excellent teaching practice. The Quality Teaching model, the National School Improvement Tool, the e5 Instructional Model, and the International Baccalaureate are examples of pedagogical models currently in use. Explore the preferred model/s with all staff before defining your philosophy to agree which one or more resonate and align with the needs of your learning community. Of course, if there is a model that adequately describes the philosophy to teaching and learning that your school community wishes to adopt, the job is made easier. Job done – just agree to use it!
Step 2. Brainstorm ideas
Something we tend to overlook is to recognise the ‘prior knowledge’ of our teachers. Every educator will have developed a theory – based upon their understanding and experience – as to the greatest influences on learning in their classroom. Ask staff also to reflect upon their own teaching and learning values and beliefs. We have found it helpful to express the learning and teaching philosophy as a set of (documented) principles.
To define the philosophy, ask staff to brainstorm their key learning and teaching beliefs, concepts and principles. This can be achieved by every staff member providing their input to the process by writing down their individual ideas as statements on sticky notes – one statement per sticky note.
Step 3. Collate the ideas using an Affinity Diagram
The staff input can then be collated by creating an Affinity Diagram with the sticky notes. Headings are applied to the Affinity Diagram reflecting the agreed major themes (as in the figure below).
Affinity Diagram – theming ideas for a learning theory
Step 4. Agree theory statements
These themes can be documented as a set of agreed statements (principles). For example, the following are the principles of learning and teaching agreed to by Knox Primary School in Melbourne, Victoria.
Knox Park Primary School, Victoria Learning and Teaching Philosophy
Here is another example of an agreed learning and teaching philosophy. It is the Learning Model developed by the Leander Independent Schools District in Texas, USA.
Leander Independent Schools District, Texas, USA Learning Model
The theory as a foundation for continual improvement
The school’s theory of learning and teaching, or principles, are then used as an ongoing reference to develop, review and continually improve consistency in policy and practice across the school. Each principle is subject to ongoing exploration through reflection and dialogue to develop deeper and shared understanding, and to inform the development of agreed learning systems and processes – the school’s pedagogical framework.
Naturally, the philosophy is dynamic. Like any theory or hypothesis, to be relevant and effective in an ongoing way, it will need to be regularly reviewed, reaffirmed or reinformed by further research and our experiences of applying it over time.
A final note
John Hattie’s research (Teachers Make a Difference: What is the research evidence? Australian Council for Educational Research, October 2003) revealed greater variation between the classrooms in an Australian school than between Australian schools. Defining the theory that will guide teaching and learning across your school is a way to reduce this variation.
Continual improvement is derived, in large measure, from the efforts of individuals and teams working together to bring about improvement. For example, many schools have introduced professional learning teams (PLTs). PLTs usually involve teams of teachers working together on agreed improvement projects aimed at improving classroom learning and teaching practice.
Sadly, ‘how’ we work on these improvement efforts is frequently left to chance. The result is valuable time and effort wasted as sub-optimal solutions are derived. So how can we make the most of these rich opportunities to improve?
The answer lies in applying a scientific approach to our improvement efforts – a structured learning and improvement process. Many know this as action learning or action research. We call it PDSA: the Plan-Do-Study-Act improvement cycle.
The history of PDSA
The PDSA cycle is attributed to the work of Walter Shewhart, a statistician working with the Bell Telephone Laboratories in New York during the 1930s (although it can be traced back further to John Dewey’s profound writings on education in the 1800’s).
Shewhart was the first to conceptualise the three steps of manufacturing — specification, production, and inspection – as a circle, rather than a straight line. He observed that when seeking to control or improve quality, there must be reflection upon the outcomes achieved (inspection) and adjustments made to the specifications and production process.
He proposed the move from this:
Figure 1. The linear process of specification, production and inspection
To this:
Figure 2. The cycle of specification, production and inspection
You may notice similarities with the traditional teaching methods of plan, teach, and assess.
Figure 3. The linear approach to Plan, Teach, Assess
In recent times there has been a focus in schools on “assessment for learning” (in contrast to “assessment of learning”). It parallels Shewhart’s observation of the need to close the loop in manufacturing.
Shewhart went on to identify the three steps of manufacturing as corresponding to the three steps of the dynamic scientific process of acquiring knowledge: making a hypothesis (or theory), carrying out an experiment, and testing the hypothesis (see Figure 4).
Figure 4. The three steps of acquiring knowledge
Source: Adapted from Walter Shewhart, 1986, Statistical Method from the Viewpoint of Quality Control, Dover, New York, p. 45.
With these thoughts, Shewhart planted the seeds for W. Edwards Deming to develop the Plan-Do-Check-Act cycle, which was published as the Shewhart cycle in 1982. Deming taught the Shewhart cycle to the Japanese from 1950 who picked it up and renamed it the Deming Cycle.
The PDSA Cycle
Deming published the cycle in The New Economics in 1993, as the Plan–Do–Study–Act (PDSA) cycle. He changed “check” to “study” in order to more accurately describe the action taken during this step. PDSA is the name by which the cycle has become widely known in recent times. (Figure 5.)
Figure 5. The Deming Cycle
Source: W. Edwards Deming, 1993, The New Economics: For industry, government, education, MIT, Cambridge.
The Plan-Do-Study-Act cycle is a structured process for improvement based on a cycle of theory, prediction, observation, and reflection.
There are, of course, many variants of the improvement process, with many and varied names. In overview, the concepts are the same.
There is a strong tendency for people to want to race through the “plan” stage and get straight into the “do” stage. Schools in particular find it difficult to make time for the reflective step of “study”. Many individuals and teams just want to get into the action and be seen to be making changes, rather than reflecting on whether the change has been an improvement, or just a change.
A detailed and structured process
Where an improvement opportunity is of a significantly complex nature, a comprehensive application of the PDSA process is necessary.
Our work in industry, government and education over the past two decades has shown the nine step PDSA process, illustrated in Figure 6, to be particularly effective. This nine step process has been compared with dozens of alternate models of PDSA and refined over the past two decades.
Figure 6. A nine step PDSA process
In developing such a process, there is a balance to be struck between the technical considerations of having a robust process that will deal with diverse contexts and issues, and the simplicity that makes the improvement process accessible and practical for busy people. Over the years, we have continually sought to simplify the model to make it more accessible. For nearly a decade, the nine steps have remained constant, but the specific actions and tools comprising each step have been progressively refined.
The process has beed designed to ensure it meets the criteria necessary to achieve sustainable improvement, namely:
Be clear about mutually agreed purpose
Establish a shared vision of excellence
Focus upon improving systems, processes and methods (rather than blaming individuals or just doing things)
Identify the root causes of dissatisfaction, not the symptoms
Carefully consider the systemic factors driving and restraining improvement, including interaction effects within the system and with containing systems
Identify strengths to build upon as well as deficiencies to be addressed
Identify the clients of the improvement efforts and understand their needs and expectations
Achieve a balance in addressing the competing, and sometimes contradictory, needs and expectations of stakeholders in improvement efforts
Be clear about the theory for improvement, and use this to predict outcomes
Reflect on the outcomes of improvement efforts, in the context of the selected theory for improvement, in order to refine the theory for improvement
Use operational definitions to ensure clarity of understanding and measurement
Not copy others’ practices without adequate reflection about their proper implementation in a new context — adapt not adopt.
These requirements have been reflected in the nine step PDSA improvement process shown in Figure 6.
To provide clear guidance, we have developed a comprehensive PDSA chart (Figure 7). The PDSA improvement process is framed as a series of questions to be answered by the improvement team (or individual). These questions address the considerations necessary to achieve sustainable improvement as detailed above. The process also refers the user to specific quality learning tools that can be used to address the questions, promoting collaboration and evidence-based decision-making.
Future 7. A detailed nine-step PDSA cycle
This is not a perfect process for improvement — there is no such thing. It is a process for improvement that can be adapted (not adopted), applied, studied, and improved. It can be used as a starting point for others, like you, who may wish to create a process of their own.
There are enormous benefits to applying a standard improvement process: an agreed improvement process that everybody follows. This can be standard across the school or whole district. Everyone can use the same approach, from students to superintendent. The benefits, apart from maximising the return on effort, time and resources, include having a common and widely used model, language, set of concepts, and agreed tools. It also establishes an agreed process that can itself be reviewed and improved, with the contribution of everybody in the organisation.
The assessment dimensions of Approach-Deployment-Results-Improvement (ADRI) can be very helpful for self-assessment. ADRI provides a structure under which the activities and results of an organisation can be broken down to identify, quite specifically, where the strengths and opportunities for improvement may lie.
ADRI provides a structured approach to organisational self-assessment. The ADRI dimensions help you to analyse how your organisation goes about: thinking and planning (Approach); implementing and doing (Deployment); monitoring and evaluating (Results); and reviewing and improving (Improvement).
It is a feature of the Australian Business Excellence framework and underpins many other performance excellence frameworks around the world.
ADRI Review Process Poster for Self-assessment
Source: Adapted by QLA from the Australian Business Excellence Framework, the Malcolm Baldrige National Quality Awards Criteria and the European Foundation for Quality Management Excellence Model.
The ADRI assessment dimensions
Approach
Approach relates to the thinking and planning behind the area of endeavour – how it has been designed.
Considering an Approach leads to an examination of:
clarity of purpose
clients, key stakeholders and their respective needs
desired outcomes – the vision of excellence
design of strategies, structures and processes to meet the desired outcomes
identification of measures of success.
In most organisations the senior leaders, sometimes with the support of content specialists, determine the approach.
For example, senior leaders of a school, frequently with the help of a specialist curriculum committee, usually lead the approach to curriculum. It is the responsibility of these leaders to identify and make clear the purpose of curriculum in the school, the desired curriculum outcomes for the school, and to understand the needs of key stakeholders (including teachers, families and curriculum regulatory bodies). Armed with this understanding, the structures and processes (including documentation) necessary to meet the intentions of the school can be designed. It is during the design stage that the measures of success are also determined from the desired outcomes (the vision of excellence): what data will be used to monitor progress over time? Senior leaders do the thinking and planning – the design.
Deployment
Deployment relates to implementing and doing – how the design is put into effect.
Considering Deployment leads to an examination of:
the degree to which the designed strategies, structures and processes have been implemented across the organisation and down through the organisation
the extent to which staff understand and have embraced the organisation’s approach
how well the strategies, structures and processes have been integrated into the day-to-day operation of the organisation.
Those doing the daily work know most about how the daily work is done. Those that are expected to implement an organisation’s approach know most about it has been deployed. The school curriculum committee may have designed an excellent approach, but it is up to each classroom teacher to implement it. If classroom teachers are not adhering to the agreed school curriculum approach, it has not been deployed well.
Results
Results relates to monitoring and evaluating – how success is gauged.
Considering the Results dimension leads to an examination of:
how performance is monitored
how the data relating to the measures of success (determined as part of the Approach) are collected, collated and reported
the degree to which trends of improvement are evident in these data.
Monitoring and evaluating is a management responsibility. School leaders are responsible for monitoring and evaluating the data used as measures of success for their approach to curriculum. Unless these data are collected, collated and reported, the effectiveness of the approach and its deployment will be unknown.
Improvement
Improvement relates to the processes of reviewing and improving the approach and its deployment.
Considering the Improvement dimension leads to an examination of:
the process by which the appropriateness and effectiveness of the approach and its deployment are reviewed
how these reviews have led to improvement
how the lessons learned are captured and shared.
Improvement is a management responsibility. Continuing the school curriculum example, a school’s senior leaders are expected to regularly review and refine the school’s curriculum. This assessment dimension examines the process by which that is undertaken, the improvements that have resulted and how these improvements are documented and shared with staff and other key stakeholders.
Why ADRI is useful
The assessment dimensions are useful for two purposes: diagnosis and design.
Diagnosis
When something is not working well in an organisation, ADRI provides a lens for examining activities and results to determine why it isn’t working and then to determine what to do about it.
When things aren’t going well, it could be because:
the Approach is weak
the Deployment is poor.
If the approach is weak, attention must be paid to reviewing and improving the design. Deploying a poor approach will not deliver good results.
A sound approach, poorly deployed, will not deliver good results either. If the approach is well thought through but is not being applied, then attention needs to be paid to ensuring people know about and implement the agreed approach.
Note that these two causes – a weak approach and poor deployment – have the same effect: disappointing results. Yet the actions required to address the disappointing results are quite different. ADRI can assist in determining which cause is more significant.
For example, a school may identify parent dissatisfaction with student reports. Firstly, knowledge of ADRI would lead the school leadership team to seek clarity and reflect upon the school’s design for assessment and reporting. Which assessments are to be undertaken? What is the schedule? What is the agreed process for reporting? Is the approach appropriate? These are questions regarding the approach. Secondly, they would explore the extent to which the approach is being applied in practice. Do staff understand and follow the agreed procedures? Are timelines being met? These are questions regarding deployment. Actions required to address parent dissatisfaction will be quite different depending where the opportunities for improvement lie: in the approach, or more to do with deployment.
Design
ADRI is also useful when designing organisation’s systems, structures and processes. In thinking about how to pursue any area of endeavour, ADRI provides useful guidance to ensure key considerations are not overlooked. If you look back over the considerations associated with each of the dimensions, you can easily identify key questions to be answered when determining how to design processes that will achieve an organisation’s goals. These questions could include:
Have we clearly articulated our purpose, desired outcomes and a vision of excellence?
What are the needs of our clients and key stakeholders?
What strategies, structures and processes are required to achieve our aspirations?
What data do we need to measure effectiveness and track progress over time? How will these data be collected, collated and regularly reported?
How will we document, train and coach people to adopt the new approach?
How will we monitor the acceptance and application of the new approach?
How will ongoing performance data be monitored and evaluated?
What is the cycle of review and improvement for this approach and its deployment?
How to use ADRI
The assessment dimensions of ADRI can be used in many ways. Typically, they are used as:
a checklist for reflecting upon the activities and results of an organisation
a framework for describing the activities and results of an organisation
an assessment model to evaluate the activities and results of an organisation.
ADRI is commonly used as a checklist to think about what is happening in an organisation. For example, thinking about whether observed difficulties are due to a deficient approach or poor deployment is a common application.
ADRI can provide a structure for describing how an organisation goes about its business. This use of ADRI is common in performance excellence awards processes such as those based on the Australian Business Excellence Awards, the Malcolm Baldrige National Quality Awards in the USA, and those of the European Foundation for Quality Management. Organisations describe explicitly how they go about each assessment dimension for each area of endeavour. The areas described are usually the categories or items of these specific frameworks (for example, Leadership, Strategy and Planning, Customers and Other Stakeholders). The use of ADRI in this way is applicable to any area of organisational activity.
A school (or district) could use ADRI as a structure to describe any program, initiative, project or other area of endeavour. For example; a district could document its thinking and planning (approach) to community engagement, how that has been implemented across the district (deployment), how data demonstrate effectiveness in community engagement (results), and the process by which the district reviews and improves community engagement (improvement). The act of documenting the activities and results in this structure usually leads to the identification of strengths and areas of opportunity for improvement.
ADRI can also be used to evaluate or rate the organisation’s activities and results. The figureabove, ADRI Review Process Poster for Self-assessment, which you can download free of charge from our website, provides a structure for such an evaluation. Each of the four dimensions, ADRI, is evaluated and given a rating, which leads to identification of strengths and opportunities for improvement.
How does ADRI relate to PDSA?
The Plan-Do-Study-Act (PDSA) cycle, provides a step-by-step process to bring about improvement over time. ADRI provides specific dimensions to reflect upon the activities and results of an organisation at a specific point in time.
Both PDSA and ADRI are based on the learning and improvement cycle, also known as scientific method and action research. PDSA provides a method for realising improvement. ADRI provides a structure for identifying where improvement may be required.
We have worked with many schools over the last 15 years, each one committed and working really hard to continually improve. Yet our experience shows few have established an effective process to capture the many improvements they make. Organisational improvement is quickly lost where there is not a process in place to hold the gains made.
So what is your approach to capture organisational learning?
Ours is system documentation.
System documentation
System documentation is the term we use to describe a structured and disciplined approach to capture organisational knowledge, learning and improvement. It relates to a collection of key documents that reflect the way in which an organisation conducts its business.
Excellent system documentation:
provides a central repository of documents critical to the running of the organisation that are readily accessed and understood by everyone
has an easy-to-use format and structure that facilitates documentation, stakeholder involvement, and the capturing of organisation innovation and improvement
is regularly reviewed for effectiveness and efficiency
is easily maintained, updated and distributed
has one person assigned to oversee its ongoing review, consistency and improvement.
System documentation: ‘the chuck under the continuous improvement wheel’
How to…
We have found the following structure for system documentation to be effective and efficient.
Suggested components of system documentation
Usually in electronic form, the documentation comprises:
school directional and planning documents
processes (procedures) describing methods, sequencing and responsibilities
policies describing what the organisation will do with respect to a particular endeavour and why (policies relate to processes and supporting documents)
supporting documents – standard documents pertaining to a specific process or policy, including: templates, letters, forms, presentations, etc.
records – documents containing data, facts, information and/or evidence relating to the organisation’s operations
document control which facilitates the identification and locating of documents and ensures people have the latest version of the right document.
We believe there are several key steps to getting started with system documentation:
Assign one person to oversee the design, implementation and improvement of the system documentation process.
Agree a structure and index and establish folders.
Place all existing processes, policies and supporting documents into the folders (they will quickly accumulate and can reviewed later on).
Agree a format for processes, policies and supporting documents
Agree and document a document control policy and associated processes (how will documents be uniquely identified?).
Begin by documenting a new process, related policy and supporting document that will immediately add value or reduce risk. Involve key stakeholders to build ownership and understanding.
Encourage people to document processes, policies and supporting documents as they engage with them to build the system over time.
The benefits
There are many benefits to having effective system documentation. These include:
Increased accessibility to important documents. (Often in schools, these documents are scattered across different people’s computers, or worse still; are to be found only in the heads of those who left the school last year!)
Openness, transparency and accountability
Effective communication
The basis for inducting, training and mentoring staff and other stakeholders
Consistency of approach (agreement as to ‘our best known way’)
Providing the foundation necessary for ongoing reflection, review, continual improvement and innovation.
Over the past 15 years the conversations about school improvement have been changing.
Key observations about school improvement, which were not discussed at the turn of the millennium, are now clearly on the agenda.
However, there are still critical elements that are not yet part of this dialogue. This post reflects upon these important conversations.
What is being talked about
Items now on the agenda include:
The systemic nature of school systems. There is growing awareness of the interconnectedness that exists among the component parts of these complex social systems. These components need to work together if the system is to be optimised towards its aim.
Students’ learning as the aim, rather than teachers’ teaching. This may seem a pedantic distinction, yet it profoundly changes the emphasis in dialogue, and thereby the focus of improvement efforts: from educational policy right through to the classroom.
Our systems of school education are not meeting the needs of significant proportion of students. Not only is the system failing our more disadvantaged students, dissatisfaction is widespread and evident in the perceptions of students, teachers and families. Furthermore, in Australia at least, the performance of the education system has largely flat-lined over the past decade, in spite of significantly increased resources. There are pockets of excellence and dramatic improvement, but across the system improvement progress is glacial.
The use of data to inform improvement. With the development of national testing in Australia, and elsewhere, there is now systemic learning data that stimulates and informs conversations and debate. Like it or not, these data are here to stay, and they continue to inform the debate about school improvement.
The organisational improvement theory, derived primarily from the work of Dr. W Edwards Deming, leads directly to each of these observations. They have been part of our conversations with friends, clients and colleagues for over a decade. We are encouraged that the significance of these issues and opportunities is now recognised.
What is yet to be talked about
Improvement theory also points to other key opportunities that are not yet common in dialogue about school improvement.
Student contribution to school improvement
Students have a lot to contribute
It is only students who truly know what helps and hinders their learning.
Unless their insight is accessed, acknowledged and acted upon, attempts to improve learning are likely to be misguided. Furthermore, if all students are to benefit, this action needs to be taken regularly at the local level, in each and every classroom. Simple tools, such as the Parking Lot and Force-field Analysis, provide mechanisms that enable this from pre-school to senior high school levels.
Students can contribute significantly to improving the operation of the school.
Some schools emphasise ‘student voice’ through mechanisms such as Student Representative Councils and Student Leaders. These process, while important, are in no way sufficient. A much greater opportunity lies in having students skilled and experienced in leading and participating in improvement project teams. These teams can apply the Plan-Do-Study-Act learning cycle to directly address opportunities to improve the school. Not only does this approach build the capability of students to personally contribute to improvement (a capability that can be of great benefit to them in their future), it also builds student ownership of the school and classroom while relieving the burden on school leaders, administrators and teachers.
Too much what, not enough how
There is far too much discussion of what excellence in schools looks like, at the expense of how schools might achieve it.
Knowing what to improve is not sufficient. We must also know how to improve it. Copying others’ examples of good practice rarely delivers sustainable improvement. Schools can learn from others’ good practice, but each school must develop and test its own theory for improvement. The Plan-Do-Study-Act improvement cycle and the associated Quality Learning tools provide the ‘how to’ for developing and testing a theory for improvement.
These two observations, which remain to be discovered by leaders of the school improvement movement, hold the most significant promise for delivering demonstrable and continual school improvement.
Note: Our thanks to Hallett Cove R-12 School, South Australia, for the images, which were taken during QLA facilitated Student Improvement Team workshops in 2014.
Discussion
We would love to hear your views on this topic. What do you see missing from the dialogue about school improvement?
