---
title: Post-mortem : bug de sous-évaluation des coûts du cache de prompts
description: Un bug de calcul introduit le 19 juin 2026 a sous-évalué, parfois jusqu'à rendre négatif, le coût facturé par Anthropic dès que le cache de prompts était actif sur qgen.
date: 2026-09-16
lang: fr-FR
author: Julien Béranger
model: Claude Sonnet 5
source: https://julienberanger.com/qgen-cost-under-reporting-sept-2026
---

# Post-mortem : bug de sous-évaluation des coûts du cache de prompts

## Résumé

Ce rapport fait suite à une question simple : "j'ai l'impression que [Claude Haiku](https://platform.claude.com/docs/en/models/haiku-4-5/overview) coûtait moins cher avant le 9 septembre 2026". L'investigation sur [qgen](https://github.com/batappli/qgen), qui tourne sur [NestJS](https://nestjs.com/) en [TypeScript](https://www.typescriptlang.org/), a fini par mettre au jour deux problèmes distincts.

Le premier est lié à la date en question : la [PR #72](https://github.com/batappli/qgen/pull/72) ("Fix instructions optimise for Haiku"), testée en direct dès le 9 septembre au matin et mergée le 10, a considérablement alourdi `data/contexts/batappli/instruction-file.md` en imposant à Haiku d'écrire une chaîne de calcul détaillée (`commentaire`) dans chaque article d'un devis. Conséquence mécanique : plus de tokens de sortie, donc un coût de génération réellement plus élevé (~+10 à 15 % sur le coût combiné en production).

Le second problème, plus grave et sans rapport avec le 9 septembre, est celui détaillé ci-dessous : un bug de calcul faisait sous-évaluer — parfois jusqu'à l'absurde, avec un total **négatif** — le coût de toute requête [Claude](https://www.anthropic.com/claude) utilisant le [prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching). Ce bug existait depuis la [PR #52](https://github.com/batappli/qgen/pull/52) ("Add prompt caching", mergée le 19 juin 2026) et n'a été corrigé que par la [PR #87](https://github.com/batappli/qgen/pull/87) ("Fix under-reported costs with prompt caching", mergée le 15 septembre 2026) — soit trois mois durant lesquels `data/costs.json` et les rapports hebdomadaires de `data/archive/costs-prod/` ont affiché des chiffres faux.

### Le calcul fautif

Dans [`src/anthropic/anthropic.service.ts`](https://github.com/batappli/qgen/blob/main/src/anthropic/anthropic.service.ts), la méthode `calculateCost` faisait :

```ts
const regularInputTokens = inputTokens - cacheCreationTokens - cacheReadTokens;
const inputCost = (regularInputTokens / 1000) * rates.inputCost;
```

Le problème : `inputTokens` correspond au champ `usage.input_tokens` renvoyé par l'[API Claude](https://platform.claude.com/docs/en/api/messages), qui **exclut déjà** les tokens de cache — ceux-ci sont comptés séparément dans `cache_creation_input_tokens` et `cache_read_input_tokens`. Le code les soustrayait donc une seconde fois. Comme le contexte mis en cache (les fichiers de `data/contexts/batappli/`) pèse des dizaines de milliers de tokens contre quelques dizaines pour le message utilisateur, `regularInputTokens` devenait systématiquement négatif.

### Exemple chiffré

Avec les tarifs de [`data/pricing.json`](https://github.com/batappli/qgen/blob/main/data/pricing.json) pour `claude-haiku-4-5-20251001` (input 0,001 $/1k, écriture cache 0,00125 $/1k, sortie 0,005 $/1k) et une requête réelle (20 tokens d'input, 39 678 tokens écrits en cache, 689 tokens de sortie) :

| | Calcul correct | Calcul bugué |
|---|---|---|
| `regularInputTokens` | 20 | 20 − 39 678 = **−39 658** |
| Coût input | 0,00002 $ | **−0,0397 $** |
| Coût total | ≈ 0,0531 $ (**0,0459 €**) | ≈ 0,0134 $ (**0,0116 €**) |

Soit un coût affiché **4 fois trop bas** — exactement le facteur annoncé dans la description de la PR #87 ("3-4x too low").

Sur une **lecture** de cache plutôt qu'une écriture, l'effet est pire : le tarif de lecture (0,0001 $/1k) est dix fois inférieur au tarif d'input normal (0,001 $/1k) utilisé à tort dans la soustraction. Quand une conversation relit un contexte déjà caché (dans la fenêtre de 5 minutes du cache), la perte artificielle dépasse le coût de sortie et le total devient négatif. C'est exactement ce qu'on retrouve dans les archives : `-0.0910481` € le 5 septembre et `-0.08722948` € le 10 septembre, dans `data/archive/costs-prod/weekly-report-2026-09-06T00-00-00.032Z.json` et `weekly-report-2026-09-13T00-00-00.021Z.json`.

### Bug annexe

Un second problème, dans [`src/memory/cost-tracking.service.ts`](https://github.com/batappli/qgen/blob/main/src/memory/cost-tracking.service.ts), aggravait la sous-évaluation en amont : `CostTracker` recalculait le coût à partir des seuls `inputTokens`/`outputTokens` bruts, sans jamais voir les compteurs de cache — le coût d'écriture/lecture cache n'était donc jamais facturé sur ce chemin-là non plus. La PR #87 corrige les deux en même temps, en faisant transiter le coût déjà calculé (et corrigé) par le service du provider jusqu'au `CostTracker`.

### Portée

Ce bug touchait uniquement le provider [Anthropic](https://www.anthropic.com/) (via [`AnthropicService`](https://github.com/batappli/qgen/blob/main/src/anthropic/anthropic.service.ts)) — [Mistral](https://mistral.ai/) et [OpenAI](https://openai.com/) ont leur propre logique de coût, touchée séparément par le second bug (absence de tracking du cache côté `CostTracker`) mais pas par la double soustraction. Il n'a affecté que le chiffre reporté dans `data/costs.json` et les rapports dérivés — jamais la facturation réelle par Anthropic, qui elle est correcte de bout en bout.

---

## Post-mortem — Prompt caching cost under-reporting

### Status

Resolved. Fixed by [PR #87](https://github.com/batappli/qgen/pull/87), merged 2026-09-15.

### Summary

From 2026-06-19 to 2026-09-15, [qgen](https://github.com/batappli/qgen)'s internal cost tracking under-reported the cost of every [Anthropic](https://www.anthropic.com/) [Claude](https://www.anthropic.com/claude) request that used [prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching), by a factor of roughly 3-4x, and in some cases reported a **negative** total cost. Real billing from Anthropic was never affected — this was a bug in the application's own cost-accounting logic, isolated to the numbers written to `data/costs.json` and the derived reports under `data/archive/costs-prod/`.

### Impact

- **What broke**: `AnthropicService.calculateCost()` in [`src/anthropic/anthropic.service.ts`](https://github.com/batappli/qgen/blob/main/src/anthropic/anthropic.service.ts), and `CostTracker.trackUsage()` in [`src/memory/cost-tracking.service.ts`](https://github.com/batappli/qgen/blob/main/src/memory/cost-tracking.service.ts).
- **What did not break**: the actual [API](https://platform.claude.com/docs/en/api/messages) calls, the [pricing](https://platform.claude.com/pricing) charged by Anthropic, and every other model provider's billing.
- **Who was affected**: internal cost observability only — anyone reading `data/costs.json` or a weekly cost report during the affected window saw numbers that could be off by 3-4x, occasionally negative.
- **Duration**: ~88 days (2026-06-19 → 2026-09-15).

### Timeline

| Date | Event |
|---|---|
| 2026-06-19 | [PR #52 "Add prompt caching"](https://github.com/batappli/qgen/pull/52) merges, introducing `calculateCost(inputTokens, outputTokens, cacheCreationTokens, cacheReadTokens)` with the faulty `regularInputTokens = inputTokens - cacheCreationTokens - cacheReadTokens` subtraction ([commit `a24c1414`](https://github.com/batappli/qgen/commit/a24c1414e9f0a63bdcd2bd6303a54249e02026a4)). |
| 2026-06-19 → 2026-09-15 | Every cached Anthropic request logs an under-reported (sometimes negative) cost, unnoticed. |
| 2026-09-09 | Unrelated: [PR #72](https://github.com/batappli/qgen/pull/72) is tested live on the `instruction-file.md` context, genuinely increasing Haiku's output-token count and cost (~+10-15%). This is what first drew attention to costs, but is a separate issue. |
| 2026-09-15 | [PR #87 "Fix under-reported costs with prompt caching"](https://github.com/batappli/qgen/pull/87) merges ([commit `c1aabc0`](https://github.com/batappli/qgen/commit/c1aabc017a7965a07c1c2e1e012326786becf545)), fixing both the double subtraction and the missing cache-cost propagation into `CostTracker`. |
| 2026-09-16 | Retrospective investigation (this report) traces the perceived "it was cheaper before" back to two independent causes, only one of which — this bug — turns out to be date-independent. |

### Root cause

Anthropic's [Messages API](https://platform.claude.com/docs/en/api/messages) reports `usage.input_tokens` as the count of **non-cached** input tokens; cache writes and reads are reported separately as `cache_creation_input_tokens` and `cache_read_input_tokens`. The buggy code assumed `input_tokens` still included the cached tokens and subtracted them a second time:

```ts
// src/anthropic/anthropic.service.ts, before the fix
const regularInputTokens = inputTokens - cacheCreationTokens - cacheReadTokens;
const inputCost = (regularInputTokens / 1000) * rates.inputCost;
```

Because [RAG](https://en.wikipedia.org/wiki/Retrieval-augmented_generation) context files under `data/contexts/batappli/` dominate the prompt (tens of thousands of tokens) while the user message is a handful of tokens, `regularInputTokens` was reliably negative whenever caching fired. The resulting negative "input cost" partially or fully cancelled the legitimate cache-write/read charge:

- On a **cache write** (rate 1.25x the base input rate), the negative term is close in magnitude to the correct write cost, leaving the total ~3-4x too low.
- On a **cache read** (rate 0.1x the base input rate — a 90% discount), the negative term vastly overshoots the tiny correct read cost, driving the total **below zero** whenever output cost is small. This is the direct cause of entries such as `-0.0910481` (2026-09-05) and `-0.08722948` (2026-09-10) in the `data/archive/costs-prod/` weekly reports.

A second, compounding bug lived in [`CostTracker.trackUsage()`](https://github.com/batappli/qgen/blob/main/src/memory/cost-tracking.service.ts): it recomputed cost from raw `inputTokens`/`outputTokens` using flat per-model rates, with no parameter through which cache token counts could reach it at all. Cache costs were simply invisible on that path, independent of the first bug.

### Detection

Not caught by monitoring or tests — surfaced indirectly, during a manual investigation into a perceived Haiku cost increase around 2026-09-09, by comparing `data/costs.json` entries and `data/archive/costs-prod/weekly-report-*.json` snapshots and noticing negative `combined_total_cost` values that predated the date under investigation.

### Resolution

[PR #87](https://github.com/batappli/qgen/pull/87):

1. Removed the double subtraction — `inputCost` is now computed directly from `usage.input_tokens`, per [Anthropic's documented semantics](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
2. Added an `extras.cost` parameter to `CostTracker.trackUsage()` so it reuses the provider-computed cost (cache-aware) instead of recomputing from token counts.
3. Started recording the [RAG](https://en.wikipedia.org/wiki/Retrieval-augmented_generation) file-selection call as its own `data/costs.json` entry, which was previously untracked entirely.
4. Persisted `cacheCreationTokens`/`cacheReadTokens` alongside each cost entry for future auditability.

Existing history in `data/costs.json` was left uncorrected — the PR fixes forward-looking accuracy only, per its own description.

### Lessons learned

- **A cost-accounting bug can hide in plain sight for months** when nothing alerts on an implausible value (a negative cost is never legitimate and could have been asserted against at write time).
- **API field semantics need a citation, not an assumption.** The bug came from treating `input_tokens` as "all input tokens" instead of checking the exact definition in the [API reference](https://platform.claude.com/docs/en/api/messages).
- **Derived aggregates (weekly reports) inherited the bug silently.** Because `combined_total_cost` in `src/app.service.ts` simply summed provider-reported costs, nothing downstream could catch the error independently.
- Consistent with general incident-review practice as described in Google's [SRE book chapter on postmortem culture](https://sre.google/sre-book/postmortem-culture/): the fix was blameless and mechanical, but the multi-month detection gap is the real finding worth acting on.

### Action items

- [ ] Add a unit assertion that `calculateCost(...).total_cost >= 0` for any valid input, so a regression fails fast.
- [ ] Add a sanity check (or test fixture) built directly from a captured Anthropic `usage` object, to prevent a future misreading of `input_tokens` semantics.
- [ ] Consider backfilling or annotating the pre-2026-09-15 entries in `data/costs.json` as "unreliable" rather than leaving them silently mixed with correct data.

## Pour aller plus loin

- [Prompt caching — Claude Platform Docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
- [Claude API reference — Messages](https://platform.claude.com/docs/en/api/messages)
- [Claude pricing](https://platform.claude.com/pricing)
- [PR #52 — Add prompt caching](https://github.com/batappli/qgen/pull/52)
- [PR #72 — Fix instructions optimise for Haiku](https://github.com/batappli/qgen/pull/72)
- [PR #87 — Fix under-reported costs with prompt caching](https://github.com/batappli/qgen/pull/87)
- [Google SRE Book — Postmortem Culture: Learning from Failure](https://sre.google/sre-book/postmortem-culture/)
- [cost-investigation-2026-09.txt](cost-investigation-2026-09.txt) — notes de l'investigation initiale sur la hausse perçue au 9 septembre