// Lab note — 2026.10.29

A payment went through and produced nothing

#ReliabilityInvestigated to 2026.08.16From Nirmalkaya

The question

A patient paid for a booking and got no confirmation, no message, no appointment and no database row. The centre had the money and no record. Nothing alerted anyone — it was found by looking. Why?

What I assumed going in

No payment webhook was configured. It was the obvious answer, it fit the symptoms exactly, and I was confident enough in it to start writing the fix.

What I actually did

Before writing it, I read the logs. They said the transaction had been open 6,698ms against a 5,000ms ceiling and had been closed underneath the code still using it.

Turning a paid booking into records ran inside one transaction. Two helpers inside it — next patient code, next invoice number — each opened a separate connection to a pooler in another region, four extra round trips. The payment had already been captured by then. A webhook would have hit exactly the same wall.

What I learned

The fix was twenty minutes: move the numbering outside the transaction, raise the ceiling. The redesign took longer and mattered more. There are now three independent paths that can confirm a payment — the browser redirect, a server-to-server webhook, and a sweep that asks the payment provider directly every ten minutes — and all three funnel into one idempotent function.

The rule that came out of it governs everything since: a booking that has been paid for must never be undone by something else failing. Notifications and calendar sync never throw; they record their own failure and get retried.

The wider lesson is three words, and it is the most expensive thing on this site: verify, do not reason. My first diagnosis came from thinking about the system instead of reading what the system was saying, and the system had been saying it the whole time.

What is still uncertain

I do not know how many other flows in my own projects still have a single point of confirmation. I have not audited for it systematically — I fixed the one that broke.

What I would test next

Go through every money-touching flow across all of them and ask one question of each: if the browser closes at the worst possible moment, what notices? Anything that answers 'nothing' gets a sweep.

The booking was recovered by hand. One patient, one afternoon, and a reliability architecture that exists because of it.

B-SIDEOff the record