Joseph Juran, a highly influential American quality specialist, defined the term “breakthrough” (which was very popular in the late 1980s and early 1990s) as “an improvement to unprecedented levels of performance”. He pointed out:
All breakthrough is achieved project by project, and in no other way.
Joseph Juran, 1988, Juran’s Quality Control Handbook, fourth edition, McGraw-Hill, New York, p. 22.
To achieve significant improvements in capability and performance, in line with the priorities in a school plan, improvement projects are needed.
Improvement plans need to be broken down into finite, definable projects that can be managed over the life of a plan.
A project is a temporary endeavour undertaken to meet specific goals and objectives with a defined beginning and end.
Projects and processes
Projects contrast strongly with daily routines. Daily routines involve the ongoing enactment of an organisation’s processes.
Projects are temporary endeavours to improve an organisation’s processes, to create new products, services or processes, or to build infrastructure.
In short, working in the system is accomplished by process; working on the system is accomplished by project.
Project teams, not committees
This presents a challenge for schools, which are accustomed to establishing committees rather than project teams.
Committees are a common feature of schools. They usually carry responsibilities associated with management and improvement in specific areas of school endeavour but are problematic in that they have an ongoing role and can easily be distracted from improvement efforts.
Project teams are formed for specific, defined timeframes and purposes. Guided by a precise purpose and structured processes, such as the Plan-Do-Study-Act cycle, project teams usually realise greater success. They stay focused and can maintain the energy necessary to see through the improvement, due to a tight, defined timeframe and an effective progress reporting approach.
Our study of schools, in Australia at least, reveals that schools in general, have neither well-developed project management methodologies, nor the management structures and disciplines to execute their improvement plans in this manner. This is a significant capability gap. Until these structures and disciplines are more strongly established, school improvement efforts are likely to continue providing disappointing results.This is not a criticism of schools or those that work within them, rather it is an observation of a systemic failure, which needs to be addressed by senior administrators and policy makers.
An Improvement Project Brief can be used to agree and document the arrangements for each project team.
Collectively, the annual set of improvement projects can be documented and monitored through the annual improvement plan.
In this post, we consider how the concepts of accountability, responsibility and authority are being applied in the name of school improvement. We explain why a strong focus on tightening accountability is unlikely to result in improvement in schools (or any other organisations for that matter).
We defined accountability:
Accountability: the collection of outcomes that an individual is charged to produce and for which the individual can be held to account
The drive to hold educators to account for improving performance has become stronger in recent years. Considerable effort has been expended clarifying the accountabilities and responsibilities of school leaders across many jurisdictions.
Sadly, tightening accountabilities is unlikely to lead to improved performance.
Here’s why.
Intent
There is one benefit to be derived from clarifying accountabilities and responsibilities. Doing so makes clear what is agreed to be important and how performance will be measured. This focuses attention, which can be beneficial. This is particularly beneficial when the process of agreeing and accepting accountabilities is open and collaborative. For example, team members can agree to take on specific tasks between meetings; each agrees to be accountable to the team for their actions. This can be affirming and effective.
Where organisations take a more formal approach to assigning accountability, it becomes problematic.
Assumptions
The accountability approach to school improvement is based on many questionable assumptions.
Held to account?
What does it mean to be held to account? It can be a requirement to explain what happened, or didn’t happen. It can also mean criticism, blame and punishment. Whatever the meaning, being judged is implicit in the definition.
Routinely passing judgements upon one another is not a feature of highly trusting or collaborative relationships and can be toxic.
Sufficiently comprehensive?
Establishing the specific outcomes for which one will be held to account does focus attention, but this can be at the expense of other areas requiring attention. The approach assumes that all the important outcomes have been identified and codified into accountabilities. This is rarely achieved. The current attempts to hold educators to account for student performance based on standardised testing, for example, is leading some to focus attention on the content to be tested; this can be at the expense of other areas of learning.
It is very difficult to establish an accountability system to address all stakeholders needs.
Numerical goals and targets.
Frequently, accountabilities are expressed as numerical goals or targets, which, it is assumed, can be measured accurately. We discussed in chapter five of our book Improving Learning that numerical goals and targets can lead to distortions of the data and/or the system. Each year the newspapers report cases of teachers and schools ‘cheating’ on high-stakes tests. Unachievable numerical goals may be at the heart of the clean diesel scandal at VW.
As Dr. Deming said, “Fear invites wrong figures.”
Locus of control.
It is assumed that an individual has sufficient control over the activities and results for which they are accountable to ensure the outcomes are met. This is not always the case. A teacher, for example, can have almost no control over the home life of his or her students, which has a very significant impact on the student’s learning.
It hardly seems reasonable to be held accountable for things outside one’s control.
Stable and capable processes?
The processes in which the individuals work are assumed to be stable and capable. In other words, it is assumed that the processes are predictable and producing desirable results, making achievement of the accountabilities possible. As we highlighted in chapter two of Improving Learning:
Many processes in school education are not capable.
Motivation?
The approach is based in the assumption that individuals require extrinsic motivation. Motivation has been discussed at length in an earlier series of posts.
Extrinsic motivation factors focus attention on obtaining the rewards and avoiding the punishments; not on the intrinsic value of the tasks themselves.
Negotiated?
While there is, in theory, scope for negotiation of accountabilities, in practice many are established historically and imposed – top-down. It is the people doing the daily work of the system that best know what needs improving. In particular, they know the barriers to improvement, which can frequently only be addressed by individuals more senior to them.
Top-down imposition of accountabilities may address management’s priorities, but are likely to neglect root causes of waste, frustration and poor performance.
Interdependence.
Each individual, being held to account for specific outcomes, is based on an assumption of independent relationships within the organisation: each party acts with autonomy towards their own goals.
Relationships in organisations are far more interdependent than autonomous.
Optimisation of the whole.
A system of accountabilities across an organisation may seek to optimise the performance of the organisation as a whole; there is an assumption that optimising each part of the system will optimise the whole. As we discussed in chapter three of Improving Learning the opposite is true.
By optimising the parts, the whole will be sub-optimised.
On balance, it would appear that focussing on systems to tighten accountabilities holds little promise for delivering improvement. Not only is it difficult to develop and sustain accountability systems within organisations, doing so is based on questionable assumptions. Furthermore, to date, it has demonstrated little systemic improvement.
An alternative
If accountability is not a viable route to improved performance, what should be done instead?
The answer is as simple as it is complex:
Equip everybody so they can work with others to improve the systems of learning for which they are responsible.
In this post, we seek to clarify the concepts of accountability, responsibility and authority. These terms have specific meanings; the concepts are frequently confused. Lack of clarity can result in micro-management.
In the next post, we will explore challenges related to making school leaders accountable for school performance.
Accountability
Let’s begin with a definition.
Accountability: the collection of outcomes that an individual is charged to produce and for which the individual can be held to account
Outcome based
Most individuals in organisations are charged with achieving certain outcomes, their accountabilities.
Negotiated
The outcomes for which an individual is held to account are usually determined through a process of negotiation. The head of a government agency negotiates with their respective government Minister. A chief executive negotiates with their Board. A principal negotiates with the district superintendent, or equivalent.
Outcomes are monitored and reported
Once agreed, progress towards the agreed outcomes is monitored and reported. The most senior people in organisations are held to account for the outcomes achieved by their organisation. This does not mean they are expected to achieve these outcomes on their own; they must work with others in the organisation to achieve their accountabilities.
Cannot be delegated
In a corporate context, a chief executive can negotiate with the chief financial officer, for example, the range of outcomes for which the chief financial officer agrees to be held to account. This is likely to include an accountability that the financial accounts are kept in accordance with relevant laws and accounting standards. The chief executive officer is not absolved from her accountability to ensure that the organisation complies with all relevant laws. The most senior executives remain accountable for the organisation’s performance. This is true even when failure to meet agreed outcomes is the result of someone further down the organisation failing to meet their accountabilities.
In a school context, principals are frequently held to account for student learning outcomes. This accountability is negotiated with their line manager and cannot be delegated. The principal must work with teaching staff to achieve this accountability.
In practice, the negotiation of accountabilities is frequently lost to the history of an organisation. The accountabilities associated with particular roles in the organisation were negotiated long ago and are now accepted as part of that job description. Under these circumstances, there is no fresh negotiation with a new incumbent for a position, rather, the accountabilities are accepted with the job.
Can be accepted
Achievement of outcomes, and thus achievement of accountabilities, is contingent upon the quality of the organisation’s systems and process, as was described in detail in chapter three of our book Improving Learning. The senior executive of the organisation is thus accountable for the performance of all the organisation’s key systems and processes. To manage this accountability in a practical manner, she negotiates for others to accept accountability for specific organisational processes. For example, a principal may negotiate with a deputy for the deputy to accept accountability for the student discipline and welfare processes. Similarly, the school leaders negotiate with classroom teachers the outcomes for which the teachers will be held accountable.
Accountability defines who is to be held to account for the achievement of outcomes.
Responsibility
Accountabilities are achieved through meeting responsibilities.
Responsibility: the work activities and outputs an individual is charged to complete.
Process based
The outcomes for which an organisation strives are achieved through enacting processes. Individuals within the organisation complete their work activities, which, in turn, link together as the organisation’s processes. These processes may be documented as deployment flowcharts that make explicit the responsibilities of those charged with enacting the process steps.
Can be delegated
Responsibilities can be delegated. The principal may, for example, ask the deputy principal to run a staff meeting. A teacher may ask an aide to prepare learning materials. The office manager may delegate responsibility for stationery supplies to an assistant.
When a responsibility is delegated, the accountability for the outcome is not.
Distinct from accountability
At senior levels of an organisation, accountabilities and responsibilities may be significantly different. Senior executives remain accountable for many things upon which they take no action on a day-to-day basis. A school principal remains accountable for the safety and wellbeing of all students in the school, yet has little day-to-day responsibility for sickbay, for example.
At more junior levels, the division between accountabilities and responsibilities becomes less distinct. A classroom teacher is typically responsible for the learning and teaching programs of her classes; the teacher is also accountable for the outcomes of those programs.
Responsibility defines who will undertake specific processes and actions.
Documenting Accountability and Responsibility
A Process Accountability Matrix may be used to document agreements regarding accountabilities and responsibilities. Key processes are listed in the rows of the matrix and roles identified in the columns. Within each cell of the matrix, the role may be listed as:
Accountable for outcomes of the process.
Responsible for performing actions within the process.
Consulted or informed during execution of the process.
A Process Accountability Matrix can be used to ensure there are no gaps or overlaps in accountability, i.e. each process has one and only one role Accountable for the process. The matrix can also identify roles that have little or too much responsibility and accountability.
Figure 1. Process Accountability Matrix example
Authority
Any discussion of accountability and responsibility is incomplete if it does not also discuss authority.
Authority: the delegated right to make decisions
Decision based
It is important to be clear who is accountable for outcomes and who is responsible for actions. It is equally important to ensure that appropriate authority is delegated; who is authorised to make decisions?
Governments define the structures by which decisions will be made and disputes settled. These delegations are detailed in legislation and regulations. Law enforcement agencies, such as the police, are established to enforce the determinations. Courts are also established to adjudicate disputes.
Delegated
Governments also delegate specific rights to manage and regulate public institutions, including schools. These rights can be delegated to government agencies, and they can also be delegated to non-government agencies such as religious authorities. These agencies and authorities, in turn, delegate specific rights to officials, including school leaders. Through this process of delegation, school leaders have rights to make defined determinations for the school. The specific rights delegated to school leaders vary by jurisdiction. In some cases, school leaders have the right to hire and fire; some have the authority to manage the whole-of-school budget. In other cases, the school principal has significantly less authority.
Authority can be delegated.
Supports responsibility and accountability
Responsibility must be accompanied by authority to make decisions and take action. The school principal can authorise the bursar or business manager to keep financial records and to pay accounts, which is consistent with the bursar’s responsibilities. The office manager may have the responsibility and authority to enrol students. Teachers have the authority and responsibility to report on students’ progress.
Similarly, the negotiation and acceptance of accountability needs to be accompanied by the agreement to delegate the necessary authority to meet those accountabilities. For example, if a deputy principal accepts the accountability of ensuring that school programs comply with the requirements of a national curriculum, they will also need the authority to establish school policies and procedures to ensure this accountability is met.
Avoids micro-mangement
Where authority is not aligned with accountabilities and responsibilities there will be frustration and wasted effort.
Where there is a failure to clearly delegate authority, organisations can become paralysed. If an officer is unsure if they have the authority to make a decision, they will push the decision ‘up the line’. This results in delays and frustration. It also frequently results in more senior leaders’ time being taken up with decisions that could and should have been made at more appropriate levels in the organisation.
Micro-management is a failure to effectively delegate authority.
As is the case with accountabilities and responsibilities, the establishment of delegated authorities is also frequently lost to the history of the organisation, and simply accepted as inherent in the job description.
Authority defines who has the right to make decisions.
See an example of a school Process Accountability Matrix.
The work of every organisation is accomplished through processes.
Processes Defined
Process: a sequence of actions that are enacted to achieve a purpose.
A process consists of actions that are completed in sequence. The sequence of actions comprising a process can be documented, as illustrated in Figure 1.
Figure 1. A Guided Reading Flowchart, from Theodore Primary School
The actions are activities or tasks: things that are done by people, computers and machines.
Getting up in the morning and going to school is a process. It includes actions such as waking up, having breakfast, taking a shower, getting dressed, packing a school bag, and walking to school. These actions are completed in sequence.
A process is, then, a way of doing something: a method.
Key Characteristics of Processes
Purposeful
Processes are means by which the work of an organisation is accomplished. Nothing is achieved without taking action. A sequence of actions is enacted in order to meet some purpose.
Consciously designed… or not
Processes may by consciously designed, defined and documented, or they may be informal and made as needed. Either way, a sequence of actions comprises the process.
Regularly repeated… or not
Most processes in everyday life tend to be repeated on a regular basis.
Processes are usually repeated, in sequence, time after time.
Schools routinely report to parents in a formal manner twice per year, for example. Home learning (homework) process cycles around most days after school. The morning process, such as illustrated in Figure 2, occurs every school day.
Figure 2. A classroom morning process, from Plenty Parklands Primary School.
A few processes may occur only once. A special centenary celebration of a school’s birthday will come around only once. Even so, the preparation, implementation and evaluation of the celebration still comprise a process: it has a sequence of actions that are enacted for a purpose.
Processes are central to improvement
Processes create outputs and deliver outcomes.
If we wish to improve an outcome, we must turn our attention to improving the processes that produced the outcome.
It is important to work on the process that produced the fault, not on him that delivered it.
Edwards Deming, 1994, The New Economics: For industry, government and education, MIT, Massachusetts, p. 39.
This leads to a change of questioning when things don’t go as desired. Instead of asking “whose fault is this?” we can ask “how did our systems and processes allow this to happen?”
If things aren’t going well, fix the process not the blame.
Watch a year two student from Sherbourne Primary School describes the Deployment Flowchart for the spelling process.
There are many types of systems. Social, mechanical, biological, and ecological systems are examples. This post explores the key features of social systems and their implications for leaders of organisations of all types.
Systems
Let’s start with a definition.
A system is a network of interdependent components that work together to try to accomplish the aim of the system.
Edwards Deming, 1994, The New Economics: For industry, government and education, MIT, Massachusetts, p. 5
A key feature of systems is interdependence among the elements comprising the system. Another key feature is that systems can nest and be contained within other systems.
To illustrate, the human body is a system of interdependent components. Within the containing system of the human body there are systems such as the nervous, respiratory and circulatory systems. These systems are clearly interdependent.
Figure 1. Systems nest, and are contained, within other systems
Social Systems
Social systems are a specific type of system that exhibit four unique characteristics:
A social system can choose its purpose.
A social system can choose the methods by which it will seek to achieve its purpose.
Elements within a social system can also choose their purpose.
Elements within a social system can choose the methods by which they seek to achieve their purpose.
These characteristics explain why organisations are such dynamic and complex systems to manage and lead.
Consider a school, for example. A school is a social system. Within the context of the local education system and its community, a school chooses both its purpose and how it proposes to achieve its purpose. Figure 2 shows the purpose statement for Wanniassa Hills Primary School in the ACT. The school consulted extensively with parents, students and staff over the past few months to discover and document this purpose statement.
Figure 2. Wanniassa Hills Primary School Purpose Statement
Within a school, the various faculties and teams also choose their purposes and methods to achieve them. For the school to optimise its efforts, the aims and methods of the individuals and teams need to align to those of the school as a whole.
Many teachers and school leaders can share stories of individuals and groups within a school not being aligned with the aspirations of the school community as a whole. It can be very frustrating for everyone.
Similarly a class is a social system. The class can choose its purpose, like that shown in Figure 3. The class can also choose how it will seek to fulfil its purpose. Within a class, individual students and groups of students choose their purpose and how they wish to go about it. This is evident from moment to moment as students may choose to ‘tune in’ or engage in disruptive behaviour.
Figure 3. A class clarifies its purpose with a mission statement
In summary:
The system as a whole chooses purpose and methods, as do elements within the system.To optimise the efforts of a school, the purpose and methods of the individuals and groups within the school need to align to that of the school as a whole. The same applies to classrooms.
Interaction effects
A key reason organisations are such dynamic and complex entities is the impact of interactions among the elements. The interactions among members of a class and the interactions among teachers and groups of teachers are two examples of these interaction effects.
As the number of people in an organisation increases, the number of potential interactions increases exponentially. In fact, the impact of interaction effects becomes way more dominant than the impact of individuals within the organisation.
The interactions among individuals and groups can be positive and add to the performance of the whole. They can also be negative and detract from performance.
Interactions can lead to harmony, and they can lead to discord.
This is self evident to most teachers and school leaders who have worked in a variety of schools, from those with a tired and cynical culture to the vibrant and collaborative team.
Interaction may reinforce efforts, or it may nullify efforts.
Edwards Deming, 2012, The Essential Deming: Leadership Principles from the Father of Quality, edited by Joyce Orsini, McGraw-Hill, New York, p. 78.
Interactions dominate the behaviour and performance of social systems. Consequently, the behaviour and performance of a classroom, workgroup or organisation is much more heavily influenced by the interaction among people than by the direct impact of each individual. This is why creating a school climate that promotes caring and productive relationships is so important.
Conclusion
Schools are complex, dynamic and highly relational social systems. The interdependence among the individuals and groups within schools and the interactions among them, are key determinants of the culture, behaviour and performance of the school. The same applies to classrooms.
It is incumbent on leaders, therefore, to ensure that system and processes promote positive interactions with purposeful and harmonious relationships.
The assessment dimensions of Approach-Deployment-Results-Improvement (ADRI) can be very helpful for self-assessment. ADRI provides a structure under which the activities and results of an organisation can be broken down to identify, quite specifically, where the strengths and opportunities for improvement may lie.
ADRI provides a structured approach to organisational self-assessment. The ADRI dimensions help you to analyse how your organisation goes about: thinking and planning (Approach); implementing and doing (Deployment); monitoring and evaluating (Results); and reviewing and improving (Improvement).
It is a feature of the Australian Business Excellence framework and underpins many other performance excellence frameworks around the world.
ADRI Review Process Poster for Self-assessment
Source: Adapted by QLA from the Australian Business Excellence Framework, the Malcolm Baldrige National Quality Awards Criteria and the European Foundation for Quality Management Excellence Model.
The ADRI assessment dimensions
Approach
Approach relates to the thinking and planning behind the area of endeavour – how it has been designed.
Considering an Approach leads to an examination of:
clarity of purpose
clients, key stakeholders and their respective needs
desired outcomes – the vision of excellence
design of strategies, structures and processes to meet the desired outcomes
identification of measures of success.
In most organisations the senior leaders, sometimes with the support of content specialists, determine the approach.
For example, senior leaders of a school, frequently with the help of a specialist curriculum committee, usually lead the approach to curriculum. It is the responsibility of these leaders to identify and make clear the purpose of curriculum in the school, the desired curriculum outcomes for the school, and to understand the needs of key stakeholders (including teachers, families and curriculum regulatory bodies). Armed with this understanding, the structures and processes (including documentation) necessary to meet the intentions of the school can be designed. It is during the design stage that the measures of success are also determined from the desired outcomes (the vision of excellence): what data will be used to monitor progress over time? Senior leaders do the thinking and planning – the design.
Deployment
Deployment relates to implementing and doing – how the design is put into effect.
Considering Deployment leads to an examination of:
the degree to which the designed strategies, structures and processes have been implemented across the organisation and down through the organisation
the extent to which staff understand and have embraced the organisation’s approach
how well the strategies, structures and processes have been integrated into the day-to-day operation of the organisation.
Those doing the daily work know most about how the daily work is done. Those that are expected to implement an organisation’s approach know most about it has been deployed. The school curriculum committee may have designed an excellent approach, but it is up to each classroom teacher to implement it. If classroom teachers are not adhering to the agreed school curriculum approach, it has not been deployed well.
Results
Results relates to monitoring and evaluating – how success is gauged.
Considering the Results dimension leads to an examination of:
how performance is monitored
how the data relating to the measures of success (determined as part of the Approach) are collected, collated and reported
the degree to which trends of improvement are evident in these data.
Monitoring and evaluating is a management responsibility. School leaders are responsible for monitoring and evaluating the data used as measures of success for their approach to curriculum. Unless these data are collected, collated and reported, the effectiveness of the approach and its deployment will be unknown.
Improvement
Improvement relates to the processes of reviewing and improving the approach and its deployment.
Considering the Improvement dimension leads to an examination of:
the process by which the appropriateness and effectiveness of the approach and its deployment are reviewed
how these reviews have led to improvement
how the lessons learned are captured and shared.
Improvement is a management responsibility. Continuing the school curriculum example, a school’s senior leaders are expected to regularly review and refine the school’s curriculum. This assessment dimension examines the process by which that is undertaken, the improvements that have resulted and how these improvements are documented and shared with staff and other key stakeholders.
Why ADRI is useful
The assessment dimensions are useful for two purposes: diagnosis and design.
Diagnosis
When something is not working well in an organisation, ADRI provides a lens for examining activities and results to determine why it isn’t working and then to determine what to do about it.
When things aren’t going well, it could be because:
the Approach is weak
the Deployment is poor.
If the approach is weak, attention must be paid to reviewing and improving the design. Deploying a poor approach will not deliver good results.
A sound approach, poorly deployed, will not deliver good results either. If the approach is well thought through but is not being applied, then attention needs to be paid to ensuring people know about and implement the agreed approach.
Note that these two causes – a weak approach and poor deployment – have the same effect: disappointing results. Yet the actions required to address the disappointing results are quite different. ADRI can assist in determining which cause is more significant.
For example, a school may identify parent dissatisfaction with student reports. Firstly, knowledge of ADRI would lead the school leadership team to seek clarity and reflect upon the school’s design for assessment and reporting. Which assessments are to be undertaken? What is the schedule? What is the agreed process for reporting? Is the approach appropriate? These are questions regarding the approach. Secondly, they would explore the extent to which the approach is being applied in practice. Do staff understand and follow the agreed procedures? Are timelines being met? These are questions regarding deployment. Actions required to address parent dissatisfaction will be quite different depending where the opportunities for improvement lie: in the approach, or more to do with deployment.
Design
ADRI is also useful when designing organisation’s systems, structures and processes. In thinking about how to pursue any area of endeavour, ADRI provides useful guidance to ensure key considerations are not overlooked. If you look back over the considerations associated with each of the dimensions, you can easily identify key questions to be answered when determining how to design processes that will achieve an organisation’s goals. These questions could include:
Have we clearly articulated our purpose, desired outcomes and a vision of excellence?
What are the needs of our clients and key stakeholders?
What strategies, structures and processes are required to achieve our aspirations?
What data do we need to measure effectiveness and track progress over time? How will these data be collected, collated and regularly reported?
How will we document, train and coach people to adopt the new approach?
How will we monitor the acceptance and application of the new approach?
How will ongoing performance data be monitored and evaluated?
What is the cycle of review and improvement for this approach and its deployment?
How to use ADRI
The assessment dimensions of ADRI can be used in many ways. Typically, they are used as:
a checklist for reflecting upon the activities and results of an organisation
a framework for describing the activities and results of an organisation
an assessment model to evaluate the activities and results of an organisation.
ADRI is commonly used as a checklist to think about what is happening in an organisation. For example, thinking about whether observed difficulties are due to a deficient approach or poor deployment is a common application.
ADRI can provide a structure for describing how an organisation goes about its business. This use of ADRI is common in performance excellence awards processes such as those based on the Australian Business Excellence Awards, the Malcolm Baldrige National Quality Awards in the USA, and those of the European Foundation for Quality Management. Organisations describe explicitly how they go about each assessment dimension for each area of endeavour. The areas described are usually the categories or items of these specific frameworks (for example, Leadership, Strategy and Planning, Customers and Other Stakeholders). The use of ADRI in this way is applicable to any area of organisational activity.
A school (or district) could use ADRI as a structure to describe any program, initiative, project or other area of endeavour. For example; a district could document its thinking and planning (approach) to community engagement, how that has been implemented across the district (deployment), how data demonstrate effectiveness in community engagement (results), and the process by which the district reviews and improves community engagement (improvement). The act of documenting the activities and results in this structure usually leads to the identification of strengths and areas of opportunity for improvement.
ADRI can also be used to evaluate or rate the organisation’s activities and results. The figureabove, ADRI Review Process Poster for Self-assessment, which you can download free of charge from our website, provides a structure for such an evaluation. Each of the four dimensions, ADRI, is evaluated and given a rating, which leads to identification of strengths and opportunities for improvement.
How does ADRI relate to PDSA?
The Plan-Do-Study-Act (PDSA) cycle, provides a step-by-step process to bring about improvement over time. ADRI provides specific dimensions to reflect upon the activities and results of an organisation at a specific point in time.
Both PDSA and ADRI are based on the learning and improvement cycle, also known as scientific method and action research. PDSA provides a method for realising improvement. ADRI provides a structure for identifying where improvement may be required.
There are four types of measures necessary for monitoring and improving systems such as schools and classrooms. These are:
Performance measures
Perception measures
Process measures
Input measures.
Performance and perception measures are generally well understood within schools; not so process and input measures. This post seeks to illuminate the use and significance of each type of measure. It is an edited extract from our forthcoming book IMPROVING LEARNING: A how-to guide for school improvement.
Performance Measures
Performance measures are defined to be:
measures of the outcomes of a system that indicate how well the system has performed.
Performance measures relate to the aims of the system. They are used to quantify the outputs and outcomes of the system. In this way, performance measures relate to the key requirements of stakeholders as reflected in the aims of the system.
Examples of performance measures in a school include:
student graduation and completion rates
student results in national, state and school-based testing
expenditure to budget
student and staff attendance.
Performance measures answer the question:
how did we go?
They are collected and reported for three reasons:
To understand the degree to which the aims of the system are being met
To compare performance of one system with that of another similar system
To monitor changes in performance over time.
Performance measures have two major deficiencies: they report what has happened in the past; and they generally provide no insight into how to improve performance. For these reasons, we need other types of measures as well.
Perception Measures
Perception measures are defined to be:
measures collected from the stakeholders in the system in order to monitor their thoughts and opinions of the system.
Perception measures provide insights into peoples’ experience of the system. Given that people make choices based on their perceptions, whether these are accurate or not, perception measures provide valuable insights that can be useful in explaining and predicting behaviour.
Perception measures are collected and reported for three reasons:
To understand the collective perceptions of key stakeholder groups
To identify perceived strengths and opportunities for improvement
To monitor changes in perceptions over time.
For schools, it is important that data are collected and reported regularly regarding the perceptions of staff, students and families. This data can include:
opinions regarding the school’s services
satisfaction with the school and its operation
thoughts and opinions about specific aspects of the school.
Perception measures answer the question:
what do people think of the system?
Note that care is required to ensure adequate random samples are collected for perception data to be reliable.
Process Measures
Process measures are defined to be:
measures collected within the system that are predictive of system performance and which can be used to initiate adjustments to processes.
Process measures are used to monitor progress and predict final outcomes. Most importantly, process measures are used to identify when changes are required in order to bring about improved performance.
Examples of process measures in schools include:
progressive student self-assessment of knowledge and skills, such as ongoing self-assessment using a capacity matrix
practice tests
teacher assessment of student progress
home learning and assessment task completion rates
monthly financial reports.
Notice that at the classroom level, process measures relating to learning are also known as ‘formative assessments’ – assessment used to inform the learning and teaching processes. These short, sharp and regular assessments are used to identify what to do next to improve the students’ learning progress.
Process measures are collected and reported regularly. This enables intervention when a process appears to be at risk of delivering unsatisfactory outcomes — before it is too late. With student learning this is critical so that mediation — improvements to learning — can be made, ensuring that high levels of learning are maintained.
Process measures answer the question: how are we going?
Input Measures
Input measures are defined to be:
measures collected at the boundary of the system to quantify key characteristics of the inputs that affect system operation.
Inputs to a system affect system performance. In the manufacturing sector, the key characteristics of important system inputs – such as material thickness or lubricant viscosity – can be controlled. Other key process input variables cannot be controlled and process adjustments are necessary based on measurements of these inputs. For example, farmers cannot control rainfall, but they can adjust their rates of irrigation based on rainfall measurements.
Teachers cannot control the prior learning of their students, but they do adjust their classroom processes based on this input characteristic. Similarly, very few schools can control the value that their students’ families place on education, but they can adjust school processes based on data about this.
Input measures — data relating to critical characteristics of key inputs to a system — are required to ensure that appropriate actions are taken within the system to accommodate changes and variation in inputs. They are also required if systems are to become robust to input variation.
The core process in a school is learning (not teaching). The learning process is subject to enormous variation in inputs. Key input variables include students’ prior learning, motivation to learn, family background, home support and peer pressure, to name a few.
Any classroom of students will display enormous variation in these inputs. It is not uncommon, for example, to have a class of 13-year-olds with chronological reading ages varying from seven to 18 years. Understanding input variation is crucial to designing learning processes that cater to the many and varied needs of all students.
At a whole school level, there is variation among teachers. Teachers’ knowledge of the content they are required to teach is not uniform, nor is their knowledge of students’ learning processes or the programs in use at the school. Experience with school and education system compliance requirements can also be highly variable. School processes need to take account of this input variation.
Like process measures, input measures are used to make adjustments to system and process design and operation in order to ensure consistently high performance.
Input measures answer the question: what are the key characteristics of the inputs to the system?
To recap, there are four sets of measures that can be used for monitoring and improving systems:
Performance measures
Process measures
Perception measures
Input measures.
Taken together, these measures provide deep insight into the performance, operation, and behaviour of a system. These measures provide a voice with which the system can speak about its behaviour, operation and performance. Importantly, these measures support analysis and prediction of future system behaviour and performance.
The four sets of system measures can be collected for a whole system as well as for subsystems within it. In school systems, this means that these data can be collected at the state, region/district, school, sub-school, classroom, and student levels.
Watch a video that describes these four types of measures.
At its simplest level, a shared vision is the answer to the question, “What do we want to create?” Peter Senge, 1990, The Fifth Discipline, p206.
A shared vision is a mutual agreement as to the desired future state key stakeholders are working together to create. It helps to align effort, optimise contribution, and to maximise organisational performance and improvement.
Many organisation improvement models recognise the importance of establishing a shared vision. The National School Improvement Tool (ACER 2012) describes the need for; ‘an explicit improvement agenda’. The Australia Business Excellence Framework (SAI Global 2011) defines an excellent organisation as one that; ‘defines its purpose, vision and values for organisational success and ensures it is understood and applied across the organisation’.
How to create shared vision
So, if stakeholder commitment to a shared vision of excellence is critical to organisational wellbeing, how do we make it happen?
Christie Downs Primary School in South Australia has done so – to great effect! The school, of 270 students, includes 90 learners with special needs supported by an integrated Disability Unit. The current school was established four years ago through the merging of two sites, an existing primary and special school. Two different cultures and organisations needed to come together to work as one. The school engaged key stakeholders in creating a shared school vision. The vision would also inform the new school’s four-year strategic plan.
Every student and staff member took part in at least one of a series of workshops to provide their ideas. Parents, families and other community members were invited to attend either a student or after hours workshop to have their say. Students with special needs were interviewed using creative one-on-one techniques that gave them a ‘voice’. Stakeholder input was then collated and a vision drafted by a team comprising students, parents and staff. Students were allowed to lead the team to ensure the crafting of a simple, jargon-free, to-the-point guiding statement. The resulting draft was communicated to all stakeholders, agreed, and used to inform the school strategic plan.
The vision has guided improvement activity and decision-making across the school for the last four years. Leadership and staff attribute the positive culture the school enjoys today to the ownership and commitment generated through this visioning process.
Earlier this year, the fourth in the school’s planning cycle, stakeholders were invited to reflect on achievements and again have input to the school vision to inform the next strategic plan. A refined school purpose, values and behaviours, and graduate profile were also agreed. The process was very similar to that used four years previously with all students, staff and families inputting their ideas. However, this time the school team (again comprising students, parents and staff) chose to summarise and communicate the revised school direction by way of an image: that of a tree (pictured below).
Christie Downs Primary School Vision
All stakeholders are really excited about their tree metaphor! The tree roots are the school’s purpose. The supporting trunk of the tree are the key elements of the school vision – ‘learning, innovation, diversity, and environment’, these underpin the strategies of the school plan. The values of the school are in the hands of the stakeholder who stands beneath the tree branches and leaves – the graduate profile – the skills and capacities, attitudes and behaviours developed by the students of the school.
Congratulations Christie Downs!
So what is different about this approach?
This collaborative process:
involves all key stakeholders of the school community – everyone has a ‘voice’
celebrates diversity, allowing for a richness of ideas to flourish (this is not possible when only a few in the organisation are involved)
builds shared understanding, ownership and commitment.
Many schools are currently promoting ‘student voice’ – a feature of many school improvement models. However our experience shows that this rarely extends beyond a Student Representative Council where a few selected students have the responsibility to collect periodic feedback from students or engage with school fundraising activities. This is a limited view of student voice.
Quality Learning theory emphasises the importance of working together to improve and involving the ‘worker at the front line’ in improving the system. In schools this is the student.
Student contribution begins in the classroom
Students’ potential to lead improvement begins in the classroom. Every student can reflect upon what helps him or her to learn and what hinders learning. Students, with their teacher, can use Quality Learning tools to share, explore and prioritise these driving and restraining forces. A Parking Lot is a good way to collect this data on an ongoing basis. (See our previous blog post Put up a Parking Lot!)
John Hattie in his book Visible Learning (2009) discusses feedback as in the ‘top 10’ greatest influences on student learning. He emphasises the importance of of student-to-teacher feedback (not just the more commonplace teacher-to-student kind).
Based on considered student feedback, teachers and students can jointly design and trial changes to classroomprocesses, with the aim of improving learning. The class can evaluate the effectiveness of these changes over time. The changes can then be:
Plan-Do-Study-Act Cycle, as described by Dr W Edwards Deming
Engaging students in classroom improvement like this has four key benefits.
Teachers can learn a great deal from listening to their students discuss what helps and hinders their learning. This also develops student metacognition and builds capacity in ‘learning how to learn’.
Engaging students in the PDSA cycle of improvement teaches them a practical ‘how to’ approach to improvement, which they can then apply to their own opportunities for improvement. These lessons have practical application beyond schooling.
Engaging students in improving their own situation builds student ownership of the process and outcomes. The teacher has the active support of students in developing, trialling and evaluating a jointly developed theory for improvement.
If the trial is successful, student learning will improve. If it is not successful, students have benefited from experience with the PDSA cycle. An alternative theory can be developed and trialled.
Student contribution to whole-school improvement
In addition to contributing to improving learning in the classroom, students have a significant contribution to make to whole-school improvement. In most schools, this potential remains unexplored, let alone realised.
There are many more students than adults (teachers, administrators and support staff) in most schools. While student-teacher ratios vary enormously by school type and sector, it is generally true that students outnumber adults by more than five to one in most schools. In some places, the ratio is more than ten to one.
The adult populations in schools are diverse; this is even more so for most student populations. There is a rich diversity of backgrounds, languages, cultures, experiences and skills in both the adult and student populations in all schools. This is more pronounced in some schools than others, but it is always present. (Such is the nature of variation in social systems).
Yet in most schools, school improvement is the domain of adults alone.
It doesn’t have to be this way.
The enormous potential of student creativity remains untapped in most schools. Young people have not yet learned some of the constrained thinking that we tend to acquire through life. Students can ask the obvious questions that we don’t even see and have ideas for improvement we could never dream of.
Hallett Cove R-12 School
Student Improvement Teams
Students can lead and participate in improvement teams to address key issues of concern. We have worked with student teams over many years on school, community and industry-based improvement projects. They have never failed to do a remarkable job.
Students from the Student Leadership Team at Hallett Cove R-12 School in South Australia recently participated in a rigorous process of school improvement using the PDSA cycle.
Ten teams were formed looking at issues that affected them within their school.
Student Improvement Teams from Hallett Cover R-12 School, South Australia
The objectives of the process were for students to:
Phase 2 was where the bulk of the work was done. Students analysed a school situation or process, using the PDSA cycle and Quality Learning tools, to understand the system, identify root causes, develop solutions and make recommendations. To do this the used the following process (and tools):
5.2 What are the recommendations for action, including time lines and responsibilities? (Gantt Chart)
5.3 How will the recommendations be communicated?
On the final afternoon, teams presented their findings to the school leadership team. The other Student Teams were also present. Their findings were presented as a written report and a presentation.
The Evaluation Meeting occurred in the days following the report presentation and provided an opportunity to give feedback to the school about their experience.
The many excellent recommendations were acted upon in the weeks that followed, and have made lasting and significant improvements to the school.
In the words of Andrew Gohl, Assistant Principal:
The PDSA cycle provides a structure and a clear process that people can work through, that is inclusive of all voices: regardless of whether you are the very young, the very old, the very vociferous, the very quiet. There’s a clear process there for everyone to have a voice, for everyone to be heard. And, of course, in that inclusiveness, the outcome is one which meets everybody’s needs.
Watch the video that tells the story of the Hallett Cove R-12 School Student Teams.
Students, teachers and families usually have different perspectives regarding the purpose and vision for their school.
Over the years, we worked with families, staff and students at dozens of schools, to identify their aspirations and agree the purpose and vision for their school. This builds a shared direction, ownership and commitment to the school. It also provides a point of reference to inform planning and decision-making into the future.
The process usually involves working with each group to identify and prioritise their views on the elements that comprise purpose and desired vision for the school.
Students at Seaford 6-12 School, South Australia, prioritising the school values
Every school is unique. Consequently, the stated purpose and vision tend to be unique to each school. There are common themes too.
Common themes
A review of data from several Australian schools, with whom we have worked, reveals the following common themes: identified by families, students and staff. (These schools represented a mix of primary and secondary schools across four states and territories.)
A place of pride
Choice of subjects
Clean
Curriculum
Engaged students
Focus on the ‘whole child’
Friends
Fun
Good academic results
Great resources
Great teachers
Passion for learning
Pedagogy
Positive school spirit
Respectful and supportive
Safe and happy
Students future ready
Students learn how to learn
Technology
The majority of schools referred to each of these themes. The wording, expression or nuance may have varied from school to school, but in essence these themes were common.
Different Perspectives
Of particular interest is the degree to which each stakeholder group raises these themes. Some themes, such as Safe and happy, Respectful and supportive and Students future ready are referred to by all three stakeholder groups in roughly equal measure. Other items tend to be raised by only one group. Typically only students identify Fun and Friends as being important elements of their schooling. Similarly, only teachers tend to raise Curriculum and Pedagogy. Other themes are raised by two of the three groups. Students and parents want Great teachers, though the teachers tended not to mention this. Staff and parents want Engaged students, while students tend not to identify this.
The figure below illustrates these different perspectives. Each item is represented by a bubble, the size of which indicates the frequency with which the item is raised; the bigger the bubble the more commonly it is mentioned. The closer the bubble is to one of the three corners (Students, Staff or Parents), the stronger that stakeholder expression of that item. For example, that only students mention Fun is indicated by it being tight in the Students’ corner. Alternatively, Respectful and supportive sits in the centre of the triangle, indicating it was identified by students, staff and parents in equal proportion.
Key messages
We believe these finding illustrate two important points.
Firstly, they highlight the importance of taking the time to find out the perspectives of each stakeholder group, directly from that stakeholder group. Attempts to second-guess the perspective of another group are very likely to completely miss entire themes held dear to that group. Involving each stakeholder group in the development of a shared direction or System Map is a great way to start.
Secondly, there are many points of commonality among these stakeholders, which provide the basis for agreement. Each group mentioned most themes, at least to some degree. This commonality provides a starting point for generating and documenting a shared perspective that is agreeable to all stakeholder groups. Furthermore, once the unique perspectives of individual groups are identified these can be discussed and are frequently incorporated into the new shared perspective.
Discussion
What are the common views among your stakeholders? What are the differences? How do you know? How do you manage the difference?
