After SOAR

Mate Labs publishes the Blue City benchmark, without Mate in it

3 November 2026

Mate Labs published Blue City on 2026-11-03, a simulated-enterprise benchmark for AI defenders. It compares five out-of-the-box agents, three Codex and two Claude Code setups, over 32 scored incidents each. Effectiveness ranges from 40.2 to 80.1 percent, cost from USD 0.45 to 4.05 per incident, and traceability from 0.0 to 3.1 percent. Mate publishes the method in full, including eight scenario families, twin pairs, a damage formula and a cost model.

Mate's own system is not attached, so no published benchmark of Mate exists. Blue City is not validated against real enterprises, the damage weights are unpublished and no independent review was found. The evidence grade on the scoring page puts the benchmark's method at B (vendor-measured, with sample and method stated), and that grade applies to the test of other agents, not to Mate.

Source: mate.security, read 2026-11-03.

What it changes for a buyer

It is a published method for testing agents, and it tests other agents. Mate's AI row does not change, and the accuracy figures stay unscored.

Other dispatches

More notes in Dispatches. To see the platforms side by side, start from the card.