Are We Optimizing for Interpreters?
TLC Connector | Teach. Learn. Converse.
Hi TLCers,
Last month, we talked about how the funnel had moved and how AI systems are increasingly shaping discovery, comparison, filtering, and decision-making before users reach your website.
This month, I want to talk about something related, but maybe even more uncomfortable: organizations are trying to figure out how to make themselves discoverable to AI systems, and in the process, they’re rediscovering accessibility and technical SEO recommendations that have existed for years.
AI discoverability, accessibility, and experimentation are converging. Are we optimizing for users, or for the interpreters shaping discovery upstream? And maybe the more important question is: should we be optimizing for both?
KEY SHIFTS
Structure content clearly. Use semantic HTML. Improve readability. Reduce friction. Simplify navigation. Expose information cleanly to machines.
Accessibility experts have been saying these things for years. Technical SEO experts have been saying these things for years. Now Google’s AI optimization guidance is saying many of the exact same things because, increasingly, we are designing simultaneously for humans, screen readers, crawlers, accessibility tools, AI systems, and AI agents.
And that matters for experimentation teams more than people realize.
Because this is no longer just about usability. It affects discoverability, visibility, AI retrieval, and increasingly, who even makes it into your funnel in the first place.
Last month, we talked about how users are increasingly arriving already persuaded because AI systems are doing more of the discovery and evaluation work upstream. But if those systems struggle to interpret your content, structure, navigation, or meaning, your problem may begin even earlier than that: you may never become part of the consideration set at all.
That changes optimization.
Honestly, I think there’s a deeper irony sitting underneath all of this. A lot of the “new” AI discoverability recommendations are decades-old accessibility and technical SEO recommendations that organizations largely ignored the first time around. The checklist barely changed. The incentives did.
And the business impact is getting harder to ignore. Research analyzing 10,000 websites found that WCAG-compliant sites gained 23% more organic traffic and ranked for 27% more keywords than non-compliant sites, while organizations are already reporting traffic losses as search behavior shifts toward AI-driven experiences.
I go much deeper on this here:
And if you want to dig into some of the conversations & research shaping this thinking:
Join the Jun 12: TLC Panel: Accessibility, AI Discoverability, & SEO with Shirley van Haalem, Alisa Scharf, and Sani Manić
Stop Chasing AI Rankings Before You Fix How LLMs See Your Brand
Headless Websites and the Cost of Engineering Vanity
Once you start looking at the web this way, the rest of this month’s articles start connecting together differently.
Privacy. Measurement. Accessibility. AI discoverability.
They all point toward the same underlying shift: we are increasingly designing systems that must communicate clearly across humans, machines, models, and agents simultaneously. And that changes what good optimization looks like.
Agentic commerce isn’t a traffic story with Kat Ribant of
I’ll admit it: until I sat down with Kelly Wortham for this latest episode of Knowledge Distillation, I was framing agentic commerce the same way most people are — as a traffic story.
Kelly reframed it for me in a way I haven’t been able to unsee since. The persuasion moment is moving upstream, out of your site, and into the LLM. By the time AI-referred visitors arrive from ChatGPT, Perplexity, or Gemini, they’ve often already done their comparison shopping and decision-making elsewhere.
And because most teams aren’t segmenting AI-referred traffic yet, those visitors are quietly sitting inside both arms of our tests — adding noise, flattening lift, and changing what our programs are actually measuring.
Kelly’s practical advice: start segmenting AI-referred traffic now, even if the data is messy. The teams that build the muscle early will have usable signal when the volume gets serious.
She also introduces an idea worth paying attention to: brand impact tests. Experiments that happen entirely off your site, in reviews, Reddit threads, and third-party content, help train the models that do the recommending.
And there’s a genuine upside hiding in all of this. What LLMs need to interpret content well overlaps heavily with what humans need to understand it clearly. After a decade of dark patterns and conversion theater, optimizing for machines may finally pull the web back toward clarity.
If you’ve been wondering whether your program is still measuring the right thing, go listen.
Catch up on the full podcast with Kelly or on your favorite podcast platform:
Try Prism at ask-y.ai ...because Bots won’t win, AI Analysts will.
Chasing Velocity in A/B Testing: Why More Experiments Can Mean Less Learning by Graham McNicoll of
Experimentation teams love velocity.
More tests. More launches. More dashboards. More movement.
But speed alone does not guarantee learning. In many organizations, the pressure to increase throughput quietly undermines the quality of decisions. Because when teams optimize for shipping experiments rather than understanding outcomes, experimentation can slowly drift from the learning system to the production pipeline. Read the full breakdown.
Velocity Is Not Your Goal. It’s a Symptom. By Kelly Wortham
A lot of organizations mistakenly confuse activity with progress.
More dashboards. More releases. More experiments. More reporting.
Everybody looks busy. Everybody feels productive. Meanwhile, nobody has aligned on what success actually looks like. What have we learned? Are we actually making anything better?
The problem isn’t the output. Outputs matter.
But outputs are not the same thing as outcomes.
When dashboards focus too heavily on outputs, organizations end up optimizing for activity rather than creating meaningful learning or business outcomes. And over time, that changes how teams behave. Read my full $0.02 and get guidance on how to build a dashboard that actually measures if you’re making things better, instead of just different, perhaps faster? (thanks, Erin).
Not a coordinated sponsor post.
Privacy by Design A/B Testing by Dominika Gruszkiewicz of
Privacy is usually treated like a compliance problem. Cookie banners. Governance reviews. Legal checkboxes.
But what if privacy is actually a systems design problem?
As AI, personalization, and experimentation become increasingly intertwined, the architecture for collecting, storing, and using customer data matters more than ever. Better experimentation does not have to come at the expense of customer trust, but getting both requires intentional design from the beginning. Read the full post here.
When Less Really Is More: Privacy and A/B Testing by Kelly Wortham
The industry keeps framing privacy as a tradeoff against optimization.
Less tracking. Less personalization. Less insight.
But thoughtful constraints often force teams to build better systems, cleaner processes, and healthier relationships with customers. The organizations that adapt best may not be the ones collecting the most data. They may be the ones learning how to work responsibly with less of it.
My $0.02 on Privacy-first Engineering.
Not a coordinated sponsor post.
TLC Chatter 3.0: Convert-sations brought to us by
Every month, TLCers share practical ideas, lessons learned, and perspectives from the field through Convert-sations with Convert.
Want in?
Head to the #convert-sations Slack channel and join the discussion.
This Month’s Winners:
🐣 First-Time Contributor: Allyson Marks (TLC: @Allyson Marks)
Welcome to the conversation, Allyson! It’s lovely to have you!
💡 Most Actionable Insight: Florent Buisson (TLC: @Florent) answered the Community question, “How do you know when a company or project is big enough to make having a testing program worthwhile?”
IMO, if you think of an “experimentation program” as an all-or-nothing, big-block thing, the answer is easy: it’s never worth it! Experimentation is a journey where each step is driven by the frustrations of the previous step:
We don’t know how the business is doing => we need good reporting based on decent data
We’re afraid of directly rolling out this thing at scale => pilot with synthetic control
There is too much noise in our reporting data for clear attribution => randomization
I remember once someone told me they had asked a stakeholder at the beginning of a project, “How often do projects fail in this department?” And the stakeholders answered, “I don’t think that ever happened”. To echo what Ezequiel said, it’s not about size, it’s about culture.
👋 Getting to Know You: Eddie Aguilar (TLC: @Eddie Aguilar)
Depends on if I get sponsored to wear gear or not. (oh, Eddie, we know.)
🚀 Convert Contribution: Cory Underwood (TLC: @cunderwood) answered the question, “What research should savvy teams conduct before creating landing pages?”
The purpose of the landing page and how it will be advertised. A landing page that doesn’t serve its intended purpose is effectively useless.