A parallel run you can audit: parity reports, rollback and exit criteria

Migration is finished when someone else can verify it

A project plan says the migration is done when the last playbook is rebuilt. An auditor, a CISO or your own future self needs a different test: show me that the new system does what the old one did, on the same inputs, and show me how you would go back.

That is a parallel run. It costs more than a cutover and it is the only version that produces evidence. The vendors know it, and several describe pieces of it. Read each piece as a claim until you have run it yourself.

What vendors say about parity, shadow runs and rollback

D3. D3's Legacy SOAR Migration Program covers six named source platforms: Cortex XSOAR, Splunk SOAR, QRadar SOAR, Siemplify, Swimlane and FortiSOAR. It says it produces a parity report from replayed incidents, offers a rollback path, flags custom code and in-house connectors for human review, and guarantees no percentage (D3 playbook migration) (read 2026-11-03, vendor claim). It also says migrated playbooks arrive in Deterministic mode. The pricing page calls the program free (D3 pricing) (read 2026-11-03, vendor claim). D3 sells the destination, and we found no third-party check.

Mate. Mate says it tests a Gamebook against past cases and runs it in shadow mode before a person signs it off (Mate Gamebooks) (read 2026-11-03, vendor claim). Its SOAR page says playbooks are imported and migrated into Gamebooks, and validated before activation. It also tells of one Fortune 500 SOC that moved 711 playbooks into 10 Gamebooks in three weeks against nine months expected, from an unnamed customer (Mate, replace SOAR with AI) (read 2026-11-03, vendor claim). No page names the source formats, what fails to convert, or whether a Gamebook can be exported. Mate is not a SOAR and does not claim to be a playbook engine, so scoring it on migration means scoring what it documents: 6.0, partial.

Blink. Blink claims import tooling that maps playbooks, custom functions and Splunkbase apps, in a four-week plan (Blink migration page) (read 2026-11-03, vendor claim). It gives no accuracy figure for the mapping.

Torq. Torq advises redesign over lift and shift, a consolidation of processes, and a proof of concept first (Torq blog, 2025-03-24) (read 2026-11-03, vendor claim). Its Kill Your SOAR guide is behind a form and we did not read it.

Duration. D3 says migrations in the industry often run 6 to 12 months (D3 Splunk SOAR guide) (read 2026-11-03, vendor claim). Mate says three weeks for one customer. Blink says four weeks. These are three sellers describing three different scopes. Do not average them. Scope your own.

Import behaviour is your first rollback tool

Look at what each platform does when a file lands on top of an existing one, because that is where a rollback starts.

• Torq asks you to choose keep original, override original or duplicate, and automated imports default to keep original (Torq knowledge base) (read 2026-11-03).
• Tines shows a Replace option when a story with the same name exists (Tines help) (read 2026-11-03).
• Splunk SOAR imports as a new playbook, and Force Update replaces a same-name playbook (Splunk SOAR Cloud export and import) (read 2026-11-03).
• Swimlane imports arrive disabled by default, with an option to import playbooks and webhooks in enabled state (Swimlane docs) (read 2026-11-04).

Disabled by default is the safest behaviour for a parallel run, because an imported playbook cannot fire until you turn it on. Keep original is the safest for an overwrite. Replace and Force Update are fine, and they need a backup.

Timing differences will show up as false mismatches

If the old and new systems do not act at the same speed, your parity report will show differences that are not logic errors. Microsoft documents this for Sentinel: batched incident updates of 5 to 10 minutes, and up to 10 minutes before automation rules run in the Defender portal (Microsoft Sentinel automation) (read 2026-11-03). Define the comparison window before the run, and compare outcomes, not timestamps.

Build the parity report yourself

You do not need a vendor to define parity. Use this method.

1. Pick a replay set. Take 30 to 90 days of closed alerts across every category, including the ugly ones and the rare ones. Store the raw alert, the old system's actions and the final disposition.
2. Freeze the set. Do not change it after the first run, or you will chase a moving target.
3. Run the new system in shadow. Have it propose actions and dispositions without acting. Record them against the same alert IDs.
4. Compare on three levels. Action parity: same actions proposed. Outcome parity: same disposition. Evidence parity: the new system reached it with evidence an analyst accepts.
5. Review every mismatch by hand. Label each as new-system error, old-system error, or acceptable difference. Old-system errors count, and they are a result of the migration too.
6. Write the numbers per category, not one overall percentage. A 98 percent overall can hide a 60 percent in the category that matters.

Keep the report. It is your audit artefact.

Write exit criteria before you start

Agree these in writing with whoever signs off. Examples to adjust:

• Outcome parity at or above your threshold in every high-risk category over the replay set.
• Zero unexplained mismatches in destructive actions.
• A live shadow period of at least a fixed number of weeks with no severity-one miss.
• Every integration in use confirmed against the new platform with one real action.
• Credentials re-created from the vault, not copied from files.
• A tested rollback: a named person has switched back in a drill, inside a stated time.
• Case history exported, retained and readable.
• Old platform support and licence end dates are after the exit date. For on-prem Cortex XSOAR 8, version 8.11 reaches end-of-life on 2026-11-13, so check your own version's date (Palo Alto lifecycle page) (read 2026-11-03).

Rollback is a drill, not a document

A rollback plan on paper fails on the day. Practise it. Decide the trigger, such as a severity-one miss or a failed integration. Decide who calls it and the time they have. Keep the old system warm, licensed and patched, through the whole shadow period and for a stated time after cutover. Swap routing at the source: if your SIEM sends alerts to a webhook, make that webhook the only switch.

Checklist for an auditable parallel run

• Choose 30 to 90 days of replay alerts across all categories and freeze the set.
• Define parity at three levels: action, outcome and evidence.
• Set the comparison window so timing delays do not look like errors.
• Import playbooks disabled, and turn them on by category.
• Run the new system in shadow, and keep the old one acting.
• Hand-review every mismatch and label the cause.
• Report parity per category, not one total.
• Write exit criteria and get them signed before the run starts.
• Drill the rollback with a named person and a time limit.
• Keep the old platform licensed until the exit date has passed.
• Archive the parity report, the replay set and the sign-off.