Better Best Practices
Organisations often introduce Best Practices as part of a change program or quality initiative. These can take a number of forms, from “cook books” and cheat sheets to full-blown consultant-led methodologies, complete with the requisite auditing and accreditation.
This article introduces the Dreyfus learning model to challenge the strategy of naively applying Best Practices, and shows how they can not only fail to help, but even have a severe negative impact on your top performers.
People implementing a business change program usually have good reasons for rolling out Best Practices across their organisation.
- They ensure consistency. We are introducing [insert initiative] and we want to ensure that everyone goes about it the same way. We don’t want to abandon people without offering them any direction and the alternative would be chaos.
- They support learning. We are trying to get everyone up to speed on this new approach with the minimum of fuss, and having a standard set of well-structured material means people can see exactly what they have to do, and ideally how well they are adapting.
- They help limit (potential) impact or damage. (Now we are starting to see their true colours.) In any organisation there’s a bell curve of people’s abilities, and we know that a small but significant number of people will be at the wrong end of it. Implementing this program badly could leave us exposed to significant financial or legal business risks, so we’re going to need very clearly defined practices to protect ourselves. They are called Best Practices because they are tried and tested so I can be reasonably sure they are going to work.
- They help to build a more mobile and flexible workforce. In these fast-moving times, projects can materialise or be cancelled almost overnight. People move between project teams, projects move between offices, countries or timezones. With all our people trained up to use the same Best Practices we—and they—have many more options in terms of career mobility. We call it “commoditising resources”.
- They allow us to enforce control. In a large, hierarchical organisation, this is the real crux of the matter. A division manager or vice president may well be responsible for thousands of people. The only way to provide accountability on that scale is to have a system of clearly-defined and strongly-enforced Best Practices.
In other words, best practices are used as a device to manage risk.
The nature of risk ¶
Risk is an interesting phenomenon. In business terms, “risk” usually means the probability of things going wrong. We assign a likelihood (high, medium or low; percentage points; statistical quantiles) to various events we are concerned about.
But what is risk other than a formalisation of fear? We consider things we fear as risky, and similarly we are less concerned (i.e. associate less risk) with things we don’t fear. In other words, with apologies to George Lucas:
- Fear leads to Risk
- Risk leads to Process
- Process leads to Hate (and meetings, and Gantt charts)
Now fear comes in two flavours, namely rational and irrational. Rational fear is healthy and useful. Lions are bad. Fire is hot. Trucks hurt. This is fear based on good information, either wired in via evolution or learned through cultural or individual development. Irrational fear is the source of prejudices, phobias and knee-jerk reactions. The earth is flat and you’ll die if you go over the edge. Hanging up dead rats by their tails wards off the plague. Putting coconut shells on your ears makes the freight aeroplane appear.
The problem is that from the inside they both appear the same. The sense of fear of spiders is no less intense or real to an arachnophobe than the sense of fear of a lion.
At a basic level, most fear is irrational, especially in a business context. If a system is delivered late, in most cases no-one is actually going to die. You probably won’t even risk injury or go hungry. At worst you might get a slightly critical review.
Pretty much all the software you are using in your everyday life—the office suite you use at work, the games console at home, the browser you are reading this article with—it’s likely that all of these were delivered late. And not working properly. And there was a hotfix / service pack / automatic update that appeared a few days or weeks later to fix some glaring functional omission or security hole. Or it still hasn’t appeared but your vendor is keeping you blissfully unaware of how exposed your phone/computer/console is right now.
Rational fear is good: it stops us getting killed. We should embrace it and cherish it as a guardian of our well-being. Irrational fear is just a drag, but fortunately we have a strategy for dealing with it:
Because irrational fear comes from ignorance, we can learn our way out of it!
Introducing the Dreyfus Model ¶
In the late 1970s two brothers spent some time exploring the nature of learning. They were interested in the nascent discipline of Artificial Intelligence, and they wanted to program a computer to learn a non-trivial skill like playing chess. It turned out there was very little by way of established knowledge around learning, certainly nothing they could use as the basis for a computer program, so they set about researching the learning process themselves.
The result was the Dreyfus Model of Skills Acquisition, which describes how people go from being brand new at something to being able to do it quite literally without thinking.
There are many models of learning and acquiring skills, but there are a couple of things that makes the Dreyfus model stand out. First, it is based on real evidence and experience. It was used to great effect in the early 1980s when the US health service was facing a nursing crisis. Second, it isn’t just a passive observation of development. It also describes what people respond to at each stage and therefore how to help them grow.
When you start learning a new skill, you don’t have any understanding of context yet so you require specific direction. In other words, you have no instinct and you have to be told what to do. As your contextual awareness increases, your need for direction diminishes. In fact you could consider the nurturing of this awareness as fundamental to skills acquisition.
The Dreyfus model divides the learning process into five distinct stages or levels:
A Novice needs detailed instructions, step-by-step recipes. Novices can’t tell whether the instructions are working or not or which ones are important because they have no context to assess them against. Because of this the novice wants frequent quick wins and regular feedback. A good recipe book has plenty of pictures and lots of reassuring messages.
An Advanced Beginner is familiar with the basic steps—the individual tasks—and can put sequences of them together. The advanced beginner is still very much task-oriented rather than goal-oriented, but they are starting to get some perspective. This is the stage at which a learner is most dangerous: They know enough to think they know more, but not enough to keep themselves out of trouble! Toddlers are advanced beginners at lots of things.
With experience, an advanced beginner becomes Competent. At the competent level they are goal-oriented; they can figure out a sequence of tasks to accomplish a goal. It might not be the best sequence, but it will usually work. Competent people like to be given a goal and then to be trusted to achieve it. Conversely you can upset a competent person by trying to give them detailed instructions, rather like back-seat driving. Most people don’t get beyond the competent level at most skills, even those they use in their everyday work. This is a basic human trait; we don’t like to expend energy once we have achieved an outcome, and for most activities the outcome is simply getting the job done.
At the Proficient level solutions start to “just appear”, usually fully-formed, in the person’s mind. They have developed enough of an instinct to firstly pick out the salient details of a situation, and then to match them to their bank of prior experience. A proficient person wants to understand the wider context of their actions, and enjoys metaphor and maxims (and their counterpart in anti-patterns). They will still refer back to their rule-based training to verify their actions, but by this stage they are learning to trust their own judgement. Where the progression from Novice to Competent is fairly linear, the transition to Proficient represents a step change. For a start, it has to be an active choice. You can become competent at something just by doing it enough times, but you have to want to become proficient.
As with the transition from competent to proficient, the transition to Expert is also non-linear. It may take many years of dedicated effort to become an expert in a particular discipline. These people work almost entirely from instinct, and are rarely wrong.
An expert lives in a world of ambiguity. She takes a pride in her ability and likes to calibrate and brush up on her skills by spending time with other experts. Interestingly, people at more junior levels tend to overstate their ability, and those at the higher levels are more modest.
When worlds collide ¶
An unfortunate characteristic of an expert is that they are unable to explain their decision-making process. This makes sense when you consider they are operating on autopilot: the solution is being presented to them, fully-formed, by their unconscious. A professional tennis player just “knows” how to return a ball with a topspin. A professional musician sees their instrument as an extension of themselves rather than as a device they manipulate.
Remember that most people are advanced beginners or competent in most of the things they do, and this is also true in a professional context. This means an expert programmer—or tester or analyst—will have a boss who is at best competent and most likely an advanced beginner. (How many project managers do you know who were talented programmers until quite recently? The Dreyfus model applies to them too.)
These people are thinking in tasks, or at best in goals made up of an identifiable sequence of tasks. This frame of reference is not comfortable processing something like “I’ve been delivering applications for 15 years and I just know we shouldn’t be using an ESB”. (There’s that word “just” again.) Of course the proficient or expert manager would be more than happy with that explanation, and in fact would actively seek out experts to provide them with the instinct-level solutions in the first place, and then vigorously defend their decisions.
Best Practices and the Dreyfus Model ¶
Armed with our novice understanding of the Dreyfus model it’s time to revisit the term Best Practices. What does the phrase even mean? Practices are things you do. They are specific behaviours and activities. They are prescriptive. “Best” is an absolute qualifier (as opposed to the conditional “better” or the more moderate “quite good”). It is context-independent and unambiguous. It is a declaration that there are no better practices than these ones.
So by this definition, Best Practices are a set of context-independent, unambiguous, prescriptive activities. They are the tasks that form the basis of task-based learning. So how do the different skill levels respond to a model like this?
Best practices help beginners ¶
The novice needs best practices. They can’t function without them. Practices show the way—the more detail the better.
The advanced beginner uses best practices. These help define the edges, the boundaries over which the advanced beginner will doubtless step in their error-prone journey towards competence.
The competent person defines best practices. Remember, they are goal oriented but still rely on sequences of steps to achieve those goals. They remember being an advanced beginner. They remember all the mistakes they made, and they want to protect the next generation of novices from themselves. More about this later.
Best practices constrain your top people ¶
A proficient person refers back to best practices. They are learning to trust their instinct but their instinct isn’t always enough. Best practices can be both a help and a hindrance at this stage. If your instinct is to do something off limits, you stop yourself out of a misplaced loyalty to the written rules.
Finally, the expert simply doesn’t use best practices, because they don’t use any practices at all! In fact the expert subverts best practices in order to get work done (or rather, subverts the policing processes that inevitably accompany best practices). Best practices are a necessary evil that must be lived with but should not be allowed to hold them back.
These last two levels, and especially the expert, can quickly come to resent practice-based initiatives, in exactly the same way that the competent person resents being given task-level detail. (“Put the red eight there, under the black nine, then you can free up that ace.”). You aren’t just back seat driving when you constrain an expert to rules, you are invalidating their hard-won instinct and intuition.
Best practices are driven from the centre ¶
Most practice-based process initiatives come from competent-level people. I don’t have any statistical evidence for this but I am convinced it is true. At the competent level, you believe that the mistakes you made at the lower levels must have been down to a lack of rigour or completeness in the description of the tasks you were taught. If we can only get the practices right, we can get people from novice to competent without all that wasteful stumbling through the advanced beginner stage. The competent person hasn’t yet tasted proficiency at this skill (and remember, you start as a novice at everything, including learning about learning, and learning about teaching) so they have no reason, or evidence, to trust instinct or intuition. Whatever the experts are doing, it must be possible to decompose it to a set of repeatable practices—-to “de-skill” it—-so that our less skilled, and let’s face it, less expensive, people can perform the same task.
We could have a small group of experts defining best practices from a Centre of Excellence or Best Practice Group, and we could roll these out across the enterprise. In this context, “expert” doesn’t mean expert in the Dreyfus sense, but someone who is very competent. There is an important distinction here. A computer such as IBM’s Deep Blue that can beat a world grandmaster is not an expert. It is simply able to cover a vast amount of possible search space very quickly The Deep Blue example is significant because chess was the original problem the Dreyfus brothers chose to explore with artificial intelligence, and Deep Blue beating Garry Kasparov in 1997 was obviously a landmark event in computerised chess. In the Dreyfus sense it is just very competent!
Recent neurological studies showed that experts in a particular field don’t process things faster than non-experts. Instead they process fewer things. There is less brain activity in an expert looking at the same problem as a non-expert. In other words they instinctively pare down the solution space and have a well-developed sense of which parts of it are most likely to yield results. (The chess computer example comes up a lot because that was the original problem the Dreyfus brothers chose to explore with artificial intelligence, and Deeper Blue beating Garry Kasparov ).
Better Best Practices ¶
In the context of the Dreyfus model, we can start to establish a more suitable model for propagating knowledge without disempowering our experts, the very people who are the inspiration for the next generation of knowledge workers in your organisation.
First you should provide context, both for the practice and the practitioner. You can describe the same principle from several angles and at different levels of abstraction depending on the Dreyfus level of your intended audience. One thing is for certain, you won’t be able to speak to them all with a single definition of a practice.
Second, the practice should be descriptive rather than prescriptive. In other words describe the desired outcome and maybe illustrate several alternative ways to achieve it. For the novices it is helpful to identify a preferred one. Feeding the steering wheel hand-to-hand is a very safe way to steer a car. As a competent driver it might not be the most effective way to turn quickly while reverse parking, but they’ll get to that later. What you don’t want is to have your expert having to trade their instinct against possible repercussions right in the heat of a crisis when they most need their wits about them.
It should describe the pros and cons of the practice. I can’t think of any practices that are all upside. There is a trade-off for most things. Again, your novices won’t benefit much from this, but you are building in the wiggle room for your proficient and expert-level people to be able to perform whilst remaining inside “the rules”.
Patterns as Best Practices ¶
In fact, it seems the more you adapt a best practice to be applicable further along the Dreyfus range, the more it starts to look like a pattern!
An Alexandrian pattern—named after the architect Christopher Alexander—describes a context, the various “forces” (external factors) it resolves and introduces, and typically several examples. In Alexandrian terms, patterns only work in conjunction with one another, creating a system of interacting forces that need to be balanced to create harmony.
Applying this philosophy to Best Practices would allow you develop a whole variety of interdependent, co-operating practices that the novice can use in a cookbook fashion, the intermediate levels can experiment with and the experts can argue over. In fact, the process of providing a context in the first place can be a very useful exercise in terms of identifying the value of a particular practice or set of practices.
Conclusion ¶
People are risk averse. In layman’s language that just means people have fear, and that’s ok. For the things we should (rationally) fear, it is perfectly appropriate to want to assess the likelihood of encountering them, and to spend effort averting them.
Best practices are about providing guidelines for novices. People will make mistakes as they learn because making mistakes is learning! However tempting it seems, you can’t sidestep the advanced beginner stage and go from novice to competent without ever stepping outside the lines. The human learning process simply doesn’t work like that. Software development is a skill, whichever part of it you are involved in—as a programmer or tester, as an analyst or project manager—so the highest performers will work predominantly from instinct. It’s a pretty good indication of their expertise if they are unable to explain their decisions. (Of course you shouldn’t confuse this for a lack of basic communication skills!)
To create consistency across a multi-skilled organisation, Alexandrian patterns provide a much more workable strategy than the clear-cut absolutes of Best Practices.
The best Best Practices are neither Best nor Practices.
Acknowledgements ¶
Former ThoughtWorks colleague Robin Gibson was the first person to introduce me to the Dreyfus model. PragDave Thomas gave an excellent talk at QCon 2007 about it which led me to Patricia Benner’s book “From Novice to Expert”, which is an informative and uplifting book full of anecdotes from the nursing profession. Michael Tiberg, the organiser of Øredev, kindly allowed me to do a talk at his conference and gave me Best Practices as a topic, which was the origin of this article. While I was there I got to spend time with Andy Hunt, whose wife seems to be responsible for introducing the software community to the Dreyfus model in the first place! Andy is currently working on a book that will feature the Dreyfus model heavily as well as lots of other goodness about how people think. And of course thanks to Niclas Nilsson for suggesting I should write this up and offering to publish it.
History: This article first appeared on InfoQ in March 2008.
