Designing for educational outcomes

A large part of the formal education landscape across the world, and also in India, is based on what is called Outcomes Based Education, or OBE for short. But the way it gets implemented on the ground is sometimes mind-numbingly tedious and only promotes a toxic compliance-driven culture, rather than nurturing a culture of inquiry. 

One of the first mistakes we do is to equate "outcomes" with "exam performance" and argue for all of education be designed around examinations. We sometimes hear weird axioms like "assessments drive learning". It is one thing for assessments to assess learning-- but once they start driving our learning, we have lost the very reason why we are educating the population in the first place. This culture has also completely taken out the emphasis on learning, in classrooms. Most of our courses today start as a contractual agreement where we first present the assessment model, exam pattern, etc. and students make a decision based on how well prepared are they to "crack" the exam. 

I remember in one of my classes on distributed computing, many years ago, there was this student who had scored very high marks, who was sitting in the first row. He seemed to be visibly distressed by my teaching. When asked what is the problem, he blurted out, "But what kind of questions can you possibly ask from what we are learning now?" I had to let him know that I was not even thinking of the exam at this time. 

I've also conducted this related experiment. To incoming graduate or undergraduate level students, I ask them about the marks they obtained in Std 10th Board exams. Most of the students remember their marks (and also where they stood with respect to their friends). But when I take a random topic from their 10th standard, like say, explain the axioms of Eucledian geometry, or explain Apollonius theorem, they would have forgotten them. This is a sad state of affairs because the learning is supposed to stay with them-- not the marks. 

We have systematically conditioned our students to perform, but not learn. 

At the core of this mess that we are in, stems from our inability to understand the notion of outcomes. We often equate "outcomes" with "outputs". Outcomes is not the same as outputs. A quick Google search will reveal several articles and videos contrasting outcomes with outputs. 

How do we understand outcomes? Here is a quick definition: 

• Outputs are what we produce 
• Outcomes are what we become 

Consider an example. A car factory adopts the latest robotic technology to manufacture cars. As a result it is now able to build up to 500 cars of different models in a month. This is its output. As a result of its increased and high-quality output, it may become the market leader and it may also become an attractive organization for both customers and employees. These are the outcomes. 

Increased outputs do not necessarily mean better outcomes. Even high-quality outputs also need not imply good outcomes. An organization may be producing a lot of high quality output-- but in order to produce high-quality output, it may have turned into a toxic and exploitative place to work in. Or it may have taken several debts and made bad financial decisions in order to improve the quality of its outputs. Hence, even if the organization creates high quality outputs, the outcome may still be that it ends up bankrupt. Producing something, and becoming something, are two different things altogether. 

Designing for outcomes is not the same as designing for outputs. While outputs do play a role in manifesting the desired outcomes, they are not equivalent to one another. We cannot replace outcomes with outputs. Outputs are visible, while outcomes are typically latent. We have to infer the outcomes from the outputs (and other visible cues). 

In an education setting, it is not what the students produce which should be our focus, but how they are themselves changing and transforming as a result of their educational journey. 

Similarly, outcomes are not the same as capabilities. When educators sometimes understand that outputs are not the same as outcomes, we often see them focusing on students' capabilities instead-- and equating that with outcomes. This is done using frameworks like the Bloom's Taxonomy that categorizes learning based on the kinds of capabilities that the student can demonstrate. 

What a student is capable of doing, is not the same as what they have become, or how they have transformed as a result of their education. A student may have acquired several skills and capabilities during their educational journey, but in this process they might have also become disillusioned, cynical, resentful, and manipulative. We can only imagine how they would put their capabilities to use, based on what they have become. 

So what exactly can we use to design for outcomes? What is the transformation that we seek to manifest in our students as they go through their educational journey? 

To address this, we have to first ask what is the role that education plays in a society. The most general answer that we can give for this is that the role of education in a society is to create an empowered population. 

What is meant by an empowered population? Is it about a population that can meet industry needs and earn their livelihood? Is it about a population that can take ownership and solve problems? Is it about a population that can live independently? Is it about a population that thinks creatively? Is it about a population that can organize, cooperate and build civilized societies? Is it a population that creates jobs? Is it a population that is empathetic and just?

Empowerment includes all of those capabilities, and more. But without a framework to think about empowerment, the above would just be "hand wavy" rhetoric without giving any insight as to how to we design for empowerment. 

In order to address this, we have developed a framework that we call the "empowerment lattice" which is shown below: 

Empowerment Lattice

This framework presents different stages of "becoming" or "transformation" that we can expect from a learner as they go through an educational journey. 

When a learner starts their educational journey, we can think of their state of being as "novice" or beginners. When we provide novices with some skills, they become "capable" of doing something. The entirety of Bloom's taxonomy, including a student's ability to analyze, synthesize and create, are all about capabilities, which is but just the first step in the long journey to empowerment. For someone who is capable of something, we can now provide them with insights and practice. A capable person obtaining insights will become "knowledgeable", while practicing one's capabilities makes the person become "talented". A person who is knowledgeable and talented, can now combine their knowledge and talent in different settings, leading them to become "experts". 

Expertise is what I would call the "base camp" for empowerment. We cannot build an empowered population without first building expertise. Expertise is also the point of independence. All other outcomes below expertise would primarily serve to augment existing workforce in different forms. Some occupations primarily look for capabilities-- be it for laying bricks or writing a program, while some other occupations require knowledge and talent. All outcomes below expertise serve to cater to the existing needs of different occupations. But someone who is an expert in something, is in a position to choose their occupation and even operate independently. 

But expertise is not the same as empowerment. Acting alone, an expert cannot effect much change. When experts are provided with an inter-disciplinary environment that enables them to operate autonomously and connect with other experts from other fields, eventually we can expect this population to become empowered to effect some changes. Empowerment is about inter-dependency rather than about independence. An empowered population comprises of several experts coming from different areas of expertise, with a sound understanding of not just their expertise, but also their limitations. Such a population collectively ends up addressing a larger issue and bringing about desired changes. An empowered population cannot be formed by experts forming their own fiefdoms, operating in isolation, and trying to grab territory from one another. 

Any major challenge that we face as a population-- be it poverty or illness or climate change or whatever, requires an empowered population of inter-dependent experts enriching one another and collectively tackling the challenge. This is what educators need to aim for, 

Education is called a noble profession for good reason. As educators, we not only get to see but also get to affect how students are transformed in their educational journey. It is tempting to use the capabilities and talents of students to serve our own professional goals or our standing in the peer community. But at all times, we need to be mindful of not just what the students are producing-- but also how are they transforming-- what are they becoming. 

The journey of transformation is not simple. It is full of unexpected twists and turns and is a deeply personal journey. To foster transformative journeys for students, it is important for educators to foster a collaborative environment among themselves within and across institutions and countries. Competition is for businesses and athletes-- not for educators. Educators across the world should meet-- not to compare their personal scorecards-- but to enrich one another to help build an empowered humanity. 

Outcomes versus Outputs

 One of the common grouse among serious educators today, is the rampant "factory" model of education, that aims not to empower the student to think and act independently, but to create a homogenised pool of skilled labour, ready for consumption by industry. 

Despite several leading educators being critical of such an approach to education, the factory model is still very strong, and is the de facto model of formal education. One of the most vocal voices against such a paradigm is the Harvard professor Todd Rose. (Actually a lot of educators are against such a model-- but Professor Rose is from an Ivy League university, so our hope is to latch on to his name to get someone to listen to what we're saying 😅). He runs a research program on the science of "individuality" where each individual is modelled as a complex dynamic system, rather than an "averagian" model that is characteristic of the factory. This TED talk here, explains the motivations behind the science of individuality: 

While these developments are underway, there is yet another movement in education that aims to model educational processes based on "outcomes." This is called Outcomes Based Education (OBE), and any resemblance to a certain empire is purely coincidental. 

OBE has been adopted by several countries, India included, which has resulted in a vastly increased bureaucracy around how courses should be specified, taught, assessed, etc.  The entire system of learning now revolves around assessments, and there is even a slogan saying, "Assessments drive learning." And to think, all along, I've tried hard to never let assessments come in the way of my learning. 

The entire activity around OBE is centred around what visible outcomes the students can produce. And outcomes are equated to exam performance. When specifying expected outcomes for a course in the course description, teachers were (until recently) forbidden to say anything about students needing to "understand" anything in the course. Because understanding isn't "visible"-- only what is visible, matters, and nothing else. Kind of, reminds me of John Searle's Chinese Room Argument about what constitutes intelligence. A clerk who doesn't speak Chinese, sitting in a Chinese office, and strictly following a set of rules about how to answer questions, appears to the outside world to be understanding Chinese. But all that the clerk is doing is to follow well-defined rules to produce some visible "outcomes." 

OBE seems to be the factory model on steroids. Individual agency, empowerment, awareness, etc. doesn't matter. Only what the individual can produce, matters. 

This is all the more strange, because, OBE itself says: 

There is no single specified style of teaching or assessment in OBE; instead, classes, opportunities, and assessments should all help students achieve the specified outcomes. The role of the faculty adapts into instructor, trainer, facilitator, and/or mentor based on the outcomes targeted.

Something seems to be amiss in the way OBE is implemented.

 In my opinion, OBE, at least the way it is being implemented in India, seems to be confusing between "outcomes" and "outputs." The idea of outcomes is not unique to education. It has been used in management theory for much longer. And there are several articles written, explaining outcomes versus outputs. 

Outcomes pertain to how the collective system state changes as a result of some activity. The change in the systemic state may or may not be "visible" directly-- but can be inferred by observing certain outputs or evidence. And while individuals in a system produce outputs, the outcome is about the system as a whole, and the change in its state.

Let us take an example. Assume that a new car factory has been set up in a state, which adopts the latest state-of-the-art manufacturing systems. As a result, it produces a lot of cars. This is the output of the activity. The outcome of this output is that-- in the larger system, car prices may go down, more people may start using cars, there will be more demand for fuel, more garages may come up, and ecosystem like motels, drive-ins, etc. may start emerging. And so on. 

The outcome is about the system-- and about what systemic changes (operational, structural, latent, etc. changes) are brought about by the individual outputs that are visible. 

Outcomes are much more comprehensive, much more deeper, and manifest over the longer term-- in comparison to the immediate outputs. In order to see outcomes, we need to have an idea of what was the state of the system before our intervention, and how did it change subsequent to our intervention. 

To understand what outcomes mean in an educational setting, consider this analogy between "irrigation" and "harvesting."

Consider a barren area, where we are nurturing a wide variety of plants. We irrigate them, provide them with soil, air, manure, etc. and each plant goes on to become a tree and produce its own outputs. If each plant were different we end up in a diversity of outputs for the same kind of nurture provided. 

The outcome of such an activity is that, we end up in a vibrant ecosystem, where outputs from one tree becomes inputs for the other, or may in some sense augment one another, resulting in a rich, self-sustaining ecosystem. 

Education is supposed to be somewhat like that. The outcome of education is supposed to be the creation of an interdependent, harmonious, empowered and aware population, that can take on problems faced by the society, and provide credible solutions. 

But that is not what is happening today. Graduates from our educational institutions are even more insecure after their education than before. When placements start, everything else takes a back seat. Hardly anyone seems to be looking around and asking how can we solve this problem-- or that problem. No one wants to solve problems-- they only want salaries-- not even jobs. In fact, I'd even seen this cheeky slogan in the email signature of a student once: "All along I thought I wanted a job, but what I really wanted was a salary." 

The reason for this state of affairs is that, education today, is modelled more as a "harvesting" activity, rather than an "irrigation" activity. 

In harvesting, we are interested only in specific outputs, and the quality of the outputs. We evaluate each plant based on the output it provides, and inject them with boosters to increase their output. We discard "bad" plants which don't provide the output we desire, or the quality of the output we desire. 

The outcome of such an activity is that the ecosystem is greatly denuded because of the narrow focus of our intervention. There is no diversity of outputs to supplement, complement and augment one another. Because natural interdependencies are broken, the system depends more and more on our intervention. The soil loses its potency over time, insects and other parasites start thriving, and we will need more and more potent toxins just to keep our outputs coming. 

Currently, the way OBE specifies and controls pedagogic activity, it resembles more of a harvesting process, rather than an irrigation process. 

Educational outcomes can only be seen over the long term. The investment we make over several years, in promoting original thought and problem-solving skills, will only produce outcomes after a decade or so. These outcomes need not be in the form of Nobel prizes or Olympic medals (but may also be), but in the form of greater ownership from the population towards our society and country, greater involvement in its activities, better work practices, greater acceptance of other subcultures, etc. Nobel prizes and Olympic medals resulting as one of the outcomes of our society developing a scientific temper and a sporting culture is one thing. But targeting them as intended outputs loses the entire picture. If we make Nobel prizes and Olympic medals as our intended outputs (and call them outcomes), we will be creating a harvesting system that can make whatever supporting ecosystem we have, collapse under its own weight. 

Of course, institutions are jittery about their survival, and want measurable outputs to pitch for funds and support and ensure their survival. This is a very valid concern. But we risk a much greater collapse, if we remain focused on specific outputs and remain impervious to the larger outcomes. We need better models to understand how to recognise outcomes and design for outcomes, rather than for outputs.