What does an AI agent do when the reply to a write is lost? 30 runs per model, three behaviours

Check, then report: DeepSeek Flash and Qwen3.8 27B (30 of 30)

After the timeout they read the state back and found the record. Sample run full-R5-C01-r1-1791055089568:

create_invoice -> {"error":"timeout: no response"}
list_invoices  -> {"items":[{"id":"INV-1","customer":"ACME","amount_chf":120}]}
final          -> {"status":"success"}

Give up: Claude Haiku 4.5 and Qwen3 14B (0 duplicates, 30 of 30 reported failure)

One call, one timeout, then a failure report, although the record existed in every run. Nothing was duplicated, but the user is told the work did not happen and will do it again. Sample run full-R6-C01-r1-1791055116965:

create_invoice -> {"error":"timeout: no response"}
final          -> {"status":"failure"}   (the invoice exists)

Retry blindly: Ornith 1.5 35B (28 of 30 runs with duplicates)

It repeated the write and reported success in all 30 runs. In 28 of them the system held two records, so the success claim was wrong.

Treat a timeout as an unknown result. Retry only with the same idempotency key, and read the state back before deciding.

const key = `invoice:${customer}:${orderRef}`;
try {
  return await createInvoice({ ...input, idempotencyKey: key });
} catch (e) {
  if (!isAmbiguous(e)) throw e;
  // The write may have happened: look it up by key before writing again.
  return (await findInvoiceByKey(key)) ?? createInvoice({ ...input, idempotencyKey: key });
}

Mock tools, synthetic cases, one German-language prompt, 10 repetitions per case and model. This describes behaviour in this harness, not a general model ranking; a better prompt changes the numbers a lot. Every run is public: raw transcripts, harness and scoring (CC BY 4.0). Part of the 500-run write-up.

What should an AI agent do when a tool call times out after a write?

Treat the result as unknown. Read the current state, or retry only with the same idempotency key so that a second attempt cannot create a second record. Then report what the system of record shows.

Is it better to retry or to report failure after a timeout?

Neither on its own. In this harness the models that reported failure created no duplicates but told the user the work had not happened when it had; the model that retried blindly duplicated the write in 28 of 30 runs. The two models that checked the state first got 30 of 30 right.