A/B testing

Generative AI breaks the foundational assumptions of traditional A/B testing, but Pearl EVP Rob Ellison argues that nondeterminism creates new experimental surface area rather than eliminating experimentation. ECI Research analyst commentary examines the LLM-as-judge pattern, its real limitations, and what evaluation infrastructure should look like in a compliance-constrained environment.