About & Methodology
How the Bidirectional AI Failure Mode Taxonomy was derived.
What This Is
The Bidirectional AI Failure Mode Taxonomy is an empirically derived classification system covering failure modes in human-AI collaborative work. It is bidirectional because it captures failures in both directions of the collaboration: AI errors (Direction A) and human-process errors (Direction B).
Unlike theoretical AI safety taxonomies, this taxonomy is grounded in 247+ qualifying incidents documented across 19 Claude project spaces over approximately 14 months. 129 of those incidents were Tier 1 structured — formally classified with severity, category, and project attribution.
The result: 104 named failure modes, 9 cross-project patterns, and a classification system that was validated against a hold-out set of incidents before publication.
Research
Research by Stahl Systems — an independent AI research practice. The taxonomy is the product of longitudinal naturalistic observation: real AI-assisted work projects, documented in real time, classified post-hoc using a structured incident methodology.
This explorer was designed and built by Krystal Martinez.
Methodology
Incident Collection
Incidents were collected from structured AI project documentation across 19 project spaces. A qualifying incident required: (1) a documented AI or human-process failure, (2) sufficient context to classify the failure type, and (3) assignment of severity using a defined rubric (CRITICAL/HIGH/MEDIUM/LOW).
Taxonomy Construction
Failure modes were derived bottom-up from incident clusters, then organized into 9 categories. Categories were not pre-defined — they emerged from the data. The taxonomy went through two major versions (v01: 93 FMs, v02: 104 FMs) as new project spaces were added.
Instance Counting
Each FM was counted across five source corpora: VD (Validation Dataset), ECT1 (Evaluation Corpus Tier 1), ECT1M (ECT1 Marginalia), GEM (General Evidence Module), and ECT2 (ECT Version 2). Both unweighted and weighted counts are provided; the explorer displays unweighted totals.
Severity Assignment
Severity is assigned per-incident, not per-FM. The same failure mode can manifest at different severity levels depending on context. The explorer displays the highest severity reached across all documented instances of each FM.
Cross-Project Pattern Analysis
After per-FM classification, cross-project patterns were identified by examining which failure modes co-occurred across multiple projects, whether they formed identifiable compounding dynamics, and whether they produced rules or remediations. 9 patterns were named and documented.
Source Documents
The taxonomy and this explorer draw from internal Stahl Systems research documents:
- Master AI Failure Mode Taxonomy v02 (2026-04-11)
- ECT v2 Failure Mode Taxonomy — Evaluation Corpus Tier 2 (2026-04-06)
- FTM v02 Instance Counts (2026-04-11)
Source documents are internal research artifacts. A preprint summary is forthcoming.