Why the senior lead has to be the engineer.
Large technology programmes fail in a fat-tailed, existence-threatening way. The published evidence, read against the economics of the consulting pyramid, points to one structural fix: put authority and technical judgment in the same person.
Two of the most careful datasets ever assembled on large technology programmes agree on the shape of the risk. Across more than 5,400 projects studied by McKinsey with Oxford's BT Centre for Major Programme Management, large IT projects (over USD 15 million) run on average 45% over budget and deliver 56% less value than predicted, and 17% go so badly that they threaten the existence of the company. Across 1,471 projects analysed by Flyvbjerg and Budzier, the average overrun is a tolerable 27% — but one project in six is a black swan with a 200% average cost overrun. The risk is not the mean; it is the tail. This article argues that the tail is manufactured by a specific organisational choice — separating commercial authority from technical judgment — and that the choice is not an accident but the profit engine of the leveraged professional-services firm, as David Maister documented three decades ago. The counter-model, in which the senior engineer is the engagement lead, has a public proof of existence in the forward-deployed engineering pattern Palantir described in its 2020 registration statement. We state the argument, the evidence for each link, and what it implies for how buyers should contract.
- 01
The failure distribution is fat-tailed, and the tail is existential. McKinsey–Oxford: large IT projects average 45% over budget, 7% over time, 56% less value than predicted; 17% threaten the company's existence; black swans are defined at over 200% overrun, reaching 400% 1. Flyvbjerg–Budzier, using a different black-swan definition: average overrun 27%, but one in six projects is a black swan averaging 200% cost and nearly 70% schedule overrun 2.
- 02
Tail failures are integration failures, which concentrate at the top of the delivery organisation. Both studies locate the damage in projects that lose contact with business value and compound mid-flight 12 — the class of error that someone with simultaneous authority and technical understanding is most likely to catch early. That attribution is our inference from the published findings, and we label it as such.
- 03
The pyramid is not a staffing accident; it is the profit formula. Maister's economics of the professional firm show profit per partner as the product of margin, billing rate and leverage — the ratio of juniors to seniors — which makes keeping seniors off the tools a financial objective, not a lapse 3.
- 04
A working counter-model exists in public documentation: Palantir's forward-deployed engineers, described in its 2020 SEC registration statement, place senior engineers directly inside customer institutions to configure and extend the product against live operational problems 4.
- 05
The buyer's lever is contractual, not rhetorical: name the senior engineer-lead in the contract, require their hands-on time on the file, cap the junior-to-senior ratio, and measure decision latency — the days between a technical issue surfacing and a binding decision. Each of these is checkable; none is standard in GCC technology procurement today.
The claim, stated precisely
The claim is narrow. It is not that juniors are bad, that pyramids never work, or that every project needs a famous engineer. It is this: in complex technology programmes, the decisions that create tail risk are technical-commercial hybrids — re-architect or patch, cut scope or extend, integrate or replace — and when the person with the authority to decide cannot personally evaluate the technical content, those decisions are made late, by negotiation, on filtered information. Lateness and filtering are precisely the conditions under which a 27% overrun becomes a 200% one. Therefore the engagement lead must be able to read the system, not only the status report.
What the failure data shows — and what it does not
| Study | Sample | Headline findings | Reference |
|---|---|---|---|
| McKinsey & Company with the BT Centre for Major Programme Management, University of Oxford — 'Delivering large-scale IT projects on time, on budget, and on value' (Bloch, Blumberg, Laartz), 2012 | More than 5,400 IT projects | Half of projects over USD 15m massively exceed budget; averages: +45% cost, +7% time, −56% value vs predicted; each additional planned year adds 15% to expected overrun; 17% of projects threaten the company's existence; black swans defined at >200% overrun (up to 400%) | 1 |
| Flyvbjerg & Budzier, 'Why Your IT Project May Be Riskier than You Think', Harvard Business Review, September 2011 | 1,471 IT projects (average cost USD 167m) | Average cost overrun 27%; one in six projects a black swan (on a definition different from McKinsey's): 200% average cost overrun, almost 70% schedule overrun; public and private sectors indistinguishable | 2 |
| Maister, 'Managing the Professional Service Firm', 1993 | Economic analysis of professional firms | Profit per partner = margin × rates × leverage (junior-to-senior ratio); leverage is the structural profit driver, which pulls senior practitioners away from the work itself | 3 |
| Palantir Technologies, Form S-1 registration statement, US SEC, 2020 | Company operating-model disclosure | Forward-deployed engineers embedded with customer institutions, working on live data and operational problems — a documented senior-on-the-tools delivery model | 4 |
The two datasets in Exhibit 1 are complementary. The McKinsey–Oxford study of more than 5,400 projects establishes the averages and the existential tail: half of all projects over USD 15 million massively exceed budget; the averages are 45% over budget and 56% under value; every additional year of planned duration adds 15% to expected overrun; and 17% of projects threaten the company's existence 1. Flyvbjerg and Budzier's 1,471-project sample, published in Harvard Business Review, establishes the distribution's shape: a modest 27% average overrun masking a one-in-six black-swan rate at 200% average cost overrun and almost 70% schedule overrun — with no meaningful difference between public and private sectors 2. The two studies draw the black-swan line differently: McKinsey's black swans exceed 200% overrun, whereas Flyvbjerg and Budzier's average 200% 12.
What the data does not show, and what we do not claim, is a measured coefficient linking leadership structure to tail outcomes; neither study instruments for who led the project or how. What both studies do report is where the damage comes from: projects that drift from business value, that compound problems mid-flight, and that are managed to budget and schedule rather than to the system being built 12. Those are failures of integration — of holding the technical state and the commercial state of the programme in one head at the same time. Our inference is that the organisational design most likely to catch them early is the one in which that head exists: a senior lead who is also the engineer. Readers should weigh it as an inference consistent with the evidence, not a finding of the cited studies.
The leverage machine: why the incumbent model cannot supply that person
If senior-on-the-tools leadership reduces tail risk, why is it rare? Because in the leveraged professional firm it is unprofitable by construction. Maister's classic treatment of professional-service economics decomposes profit per partner into margin, billing rates and leverage — the ratio of junior to senior staff — and shows that, for a given practice, raising leverage is the most direct way to raise partner profit 3. The organisational consequences follow mechanically: the senior's highest-value use is selling and supervising, not building; delivery is pushed down to the cheapest staff who can plausibly perform it; and the senior's technical contact with the work decays into review meetings. None of this is misconduct. It is the business model performing as designed — designed, as Maister notes, around the economics of the firm rather than the risk profile of the client's programme 3.
| Dimension | Leveraged pyramid | Senior-led pod (forward-deployed pattern) |
|---|---|---|
| Where technical judgment sits | Distributed among staff; seniors review | In the engagement lead, daily, on the tools |
| Where commercial authority sits | In the senior, off the tools | In the same person as the technical judgment |
| Path from issue to binding decision | Staff → manager → senior → client steering committee | Lead sees the issue in the code and decides with the sponsor |
| What the economics optimise | Leverage: junior hours billed per senior (Maister) 3 | Output per senior, amplified by tooling |
| Right habitat | Decomposable, repeatable work at scale | Integration-risk-dominated programmes — the fat-tailed class in Exhibit 1 12 |
| Failure mode | Late discovery; decisions by negotiation on filtered information | Key-person concentration — mitigated contractually, not structurally |
The counter-model has a public proof of existence
The alternative is not hypothetical. Palantir's registration statement, filed with the US Securities and Exchange Commission in 2020, describes forward-deployed engineers as a core of its operating model: software engineers embedded directly with customer institutions, working against the customer's live data and operational problems, with the seniority and authority to reshape the deployment as they learn 4. Whatever one's view of the company, the pattern it documented — senior engineering judgment stationed at the point of decision, inside the client — is precisely the structure our reading of the failure data argues for, and it has since been borrowed across the industry under the 'forward-deployed' label. The pattern's economics differ from the pyramid's: revenue per person is higher, leverage is lower, and the firm's margin depends on the senior's productivity with modern tooling rather than on junior markup. That trade has become easier to make every year as AI-assisted engineering has raised what one senior engineer can produce — which is why the model is spreading now.
What buyers should do with this
- Name the senior engineer-lead in the contract, with a minimum share of their hours on the file — and treat substitution as a change requiring consent, the way key-person clauses already work in fund documentation.
- Cap the junior-to-senior ratio for the delivery team in the request for proposals, and ask each bidder to disclose theirs. The number is knowable and almost never asked for.
- Measure decision latency: log the date a material technical issue is raised and the date a binding decision is made, and review the distribution monthly. Latency is the earliest observable symptom of the authority-judgment split.
- Ask the one diligence question the pyramid cannot answer well: 'Walk me through the last time the person who signed this proposal personally changed the architecture of a system in production.'
- Weigh the tail, not the day rate. On the published distributions, the expected cost of a programme is dominated by the one-in-six scenario 12; a delivery structure that plausibly compresses the tail is worth a premium on the blended rate many times over.
An argument from published sources, completed 9 October 2026. The empirical base is the McKinsey–Oxford large-IT-project research (2012) and Flyvbjerg and Budzier's HBR analysis (2011), read as published; the economic mechanism is Maister (1993); the counter-model documentation is Palantir's 2020 SEC Form S-1. The causal link between leadership structure and tail outcomes is the authors' inference and is labelled as such wherever it appears.
- Neither failure study instruments for delivery-team structure; the structural attribution is an inference consistent with, not proven by, the cited findings.
- The failure datasets predate modern AI-assisted delivery; base rates may have shifted, though no comparable newer dataset of equal scale was found.
- Maister's model describes incentives, not inevitabilities; individual firms deviate from it.
- The Palantir citation documents that the counter-model exists and how it is described by its practitioner; it is not an endorsement and reports no comparative outcome data.
- 01Bloch, M., Blumberg, S. and Laartz, J., 'Delivering large-scale IT projects on time, on budget, and on value', McKinsey & Company in collaboration with the BT Centre for Major Programme Management, University of Oxford (McKinsey on Business Technology, No. 27, Fall 2012): research on more than 5,400 IT projects. https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/delivering-large-scale-it-projects-on-time-on-budget-and-on-value
- 02Flyvbjerg, B. and Budzier, A., 'Why Your IT Project May Be Riskier than You Think', Harvard Business Review, vol. 89, no. 9, September 2011, pp. 23–25 (sample of 1,471 IT projects; preprint at arXiv:1304.0265). https://arxiv.org/abs/1304.0265
- 03Maister, D. H., 'Managing the Professional Service Firm', Free Press, 1993 — in particular the treatment of leverage and the economics of professional-firm profitability.
- 04Palantir Technologies Inc., Form S-1 Registration Statement, US Securities and Exchange Commission, filed 25 August 2020 — description of the company's forward-deployed engineering model. https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0001321655&type=S-1
- AI
- Artificial intelligence
- GCC
- Gulf Cooperation Council
- HBR
- Harvard Business Review
- IT
- Information technology
- SEC
- US Securities and Exchange Commission
- S-1
- SEC registration statement for a public offering
wesolutions Research. An argument from published evidence; the studies cited are named, and the inference drawn from them is labelled as ours.
Public.
Every exhibit in this report can be downloaded as a CSV file, with its source line, for independent re-scoring.