Deliberate practice, calibration, and the development of professional judgement.
We tend to assume that professional expertise develops through experience, that the accumulation of cases, encounters, and outcomes produces, over time, a more reliable professional judgement. The research on expertise development tells a more complicated story. Experience without feedback does not produce expertise. It produces confidence. These are not the same thing.
The distinction matters profoundly for how professionals think about their own development and for how organisations think about training, supervision, and quality assurance. If experience alone were sufficient, the most experienced professionals would consistently be the most accurate. The evidence suggests otherwise.
The two conditions for expertise
In a 2009 paper, Kahneman and Klein, two researchers who had spent careers largely in disagreement about the reliability of expert intuition, published a joint paper identifying the conditions under which expert intuition can be trusted. The paper is notable precisely because it is a synthesis: two opposing camps agreeing on the conditions that determine when fast thinking can be relied upon.
The two conditions are: first, the environment must be sufficiently regular that it is possible to learn patterns from it, chess is highly regular; financial markets are not. Second, there must be adequate opportunity to learn those patterns, which requires timely, unambiguous feedback on the accuracy of predictions and judgements.
Many professional environments fail on both counts. Clinical medicine involves considerable regularity in some domains and considerable unpredictability in others. Social work involves high individual variability and limited feedback, practitioners rarely learn with confidence what would have happened if they had made a different decision. Legal practice involves genuine patterns, but outcomes are influenced by factors outside the practitioner’s control and feedback is often delayed, partial, or confounded.
The implication is that in these environments, experience accumulates but genuine calibration, the alignment of confidence with accuracy, does not happen automatically. It must be actively supported.
Overconfidence: the dominant error of experts
The research on overconfidence in expert judgement is among the most robustly replicated in cognitive psychology. Across a wide range of domains, medicine, law, finance, meteorology, strategic planning, experts consistently express more certainty than their track records warrant. When experts say they are 90% confident, they are typically correct around 70% of the time. Philip Tetlock’s twenty-year study of political and economic forecasting, reported in Expert Political Judgment (2005), produced a striking finding: the forecasting accuracy of experts was not significantly better than that of well-informed non-experts, and in some domains was worse. More striking still, the most confident experts were frequently less accurate than those who held their views more tentatively. Confidence, in expert forecasting, was negatively correlated with accuracy.
Tetlock’s subsequent work, with the Good Judgment Project, identified a population of ‘superforecasters’, people who were substantially more accurate than average and consistently better calibrated. What distinguished them was not greater domain expertise but a specific set of cognitive habits: the active seeking of disconfirming information; the willingness to update beliefs in response to evidence; the habit of thinking in probabilities rather than certainties; and a stance of intellectual humility about the limits of their own knowledge.
Deliberate practice and the development of calibration
Anders Ericsson’s research on expert performance, summarised in Peak: Secrets from the New Science of Expertise (2016), identified the mechanism through which genuine expertise develops: deliberate practice. This is not simply repeated performance of a skill. It is effortful, focused practice at the outer edge of current capability, with immediate feedback on performance and targeted correction of errors.
The critical element is the feedback loop. A musician who practises with immediate feedback on intonation and timing develops calibration between intention and execution. A surgeon who performs procedures and receives detailed feedback on outcomes develops calibration between technique and result. A professional who makes assessments and receives no feedback, or only delayed, confounded feedback, does not develop calibration, regardless of how many assessments they make.
This has specific implications for supervision and continuing professional development. Supervision that focuses exclusively on what to do in a case, the next step, the referral decision, the risk management plan, without attending to the quality of the reasoning that produced the assessment is not developing professional expertise. It is managing the immediate situation without developing the practitioner.
The hindsight bias and the failure of retrospective learning
One of the most insidious obstacles to calibration is the hindsight bias: the tendency, after an outcome is known, to believe that we would have predicted it. Outcome knowledge changes how we remember our previous uncertainty. The hindsight bias has been extensively documented in medical contexts, in legal settings, and in organisational post-mortems. It makes retrospective learning from outcomes unreliable: we update our beliefs about our own predictive accuracy in ways that inflate our sense of competence. The decision journal, a practice of recording the basis for significant decisions, including the degree of confidence and the alternatives considered, before the outcome is known, is one of the most effective tools for preventing hindsight bias from corrupting professional learning.
The role of organisational structure
Individual calibration is important. But individual calibration operates within organisational structures that either support or undermine good professional judgement. James Reason’s work on human error, particularly Managing the Risks of Organizational Accidents (1997), introduced the Swiss cheese model: the idea that catastrophic failures typically result not from a single human error but from an alignment of multiple smaller failures, each of which is individually manageable but which, when combined, create a path to disaster.
For professional practice, this means that the question ‘how do we help individual practitioners make better decisions?’ is necessary but not sufficient. The parallel question, ‘how do we design professional systems that reduce the consequences of individual error?’, is equally important. Mandatory second opinions in high-stakes assessments; structured challenge mechanisms in multi-agency decisions; routine calibration exercises in which practitioners compare their probability estimates with outcomes over time, these are structural responses to a problem that individual training cannot fully address.
What developing expertise actually requires
The account of expertise that emerges from this research is demanding. It requires not just experience but deliberate practice. Not just training but feedback that is timely, specific, and accurately attributed. Not just individual improvement but organisational structures that support reliable reasoning and catch errors before they propagate. The work of Annie Duke on decision quality, including the crucial distinction between a good decision and a good outcome, provides a practical framework for developing the kind of epistemic discipline that genuine expertise requires.
Professional development that takes this research seriously looks different from professional development that does not. It focuses not only on knowledge and skill but on the metacognitive habits, the awareness of one’s own reasoning processes, that support reliable judgement. It builds feedback loops. It designs structural safeguards. And it cultivates the professional culture in which intellectual honesty about the limits of one’s own certainty is valued rather than penalised.
Further reading
Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise. American Psychologist, 64(6).
Tetlock, P. E. (2005). Expert Political Judgment. Princeton University Press.
Reason, J. (1997). Managing the Risks of Organizational Accidents. Routledge.