2023 Reflections, Search-Ranking Experiments, and a GDPR Dad Joke
The TLC Connector | Teach. Learn. Converse.
Hello TLC friends and members! What do Die Hard as a Christmas movie, countdowns, and the end of the year all have in common? December is here and it seems like a great time to reflect on an amazing one: the TLC 10th anniversary year!
Before we get started, Merritt Aho will kick us off with a little dad joke, but you’ll have to read to the end of the newsletter for the punch line.
Q: Why does everyone hate GDPR lawyers?
TLC Chatter: A Year of Reflections
January: June Dershewitz shared great resources with all of us during her first talk about what to do once you’d convinced the C-Suite that Data matters: DAA, Data org structure examples, Mktg vs. product analytics. And Dylan Lewis reminded everyone how to start their own Web Analytics Wednesday (WAW) - something that June, herself, had a part in inspiring the creation of!
February: Matt Policastro talked about Sequential testing and the sequential design tool originally created by Merritt with updates made by Matt now available here. Georgi shared his thoughts and we shared a link to his amazing toolkit in the TLC Slack Community.
March: Matt Gershoff shared his talk about the inspiration for the addition of customer insights to the Conductrics platform - ironically, from inspiration from another TLC member, Paula Sappington, and her talk about combining the KPIs and the KPwhys (the quant and qual) for better decision making). And Ronny Kohavi shared the real story behind the Abe Wald story of survivorship bias.
April: Ton Wesseling shared his talk about finding more winners and there were several tools shared including Visualping.io, Builtwith.com, and pagefreezer.com
May: The TLC 10th! AI image prompt by Tim Wilson: “Midjourney: A bunch of nerds on a Zoom call talking about analysts and optimization, where each attendee is a cartoon character located in outer space"
June: We had so much fun talking User Research and dangerous mushrooms with Els Aerts. Best quote of the day, “smashing time with Tony Hawk” 🤣 Though Puja Ghosh’s comment re translation concerns related to butt dials and booty calls was quite hilarious as well.
July: In July we welcomed Lucia van den Brink to talk about A/B testing content and blogs. Shirley Lee asked for some tips on how to get multi-disciplinary teams to contribute ideas - which Lucia answered, leading to further work and expansion that became the collaboration bonus.
August: Jenn Kunz joined us for one of our most popular conversations of 2023 to tell us all to stop investing in (the wrong) tools. Apparently, the tools aren’t always the problem (or the solution). But having the WRONG tool can be. Who could have known? (Jenn - Jenn knows. She knows it all). And because ONE conversation wasn’t enough - we also talked privacy maturity in a lovely conversation with Fosca Fimiani & Siobhan Solberg. And helping me moderate was Juliana Jackson
September: Then, Lukas Vermeer was telling us all to just calm down and worry a little less about interactions, but if you just had to worry, he gave us some methods we could use to measure for interactions. Plus - Lukas gave us three great blog posts for learning more (interaction effects, avoiding interactions, and detecting interactions.)
October: Geddy van Elburg talked to us about cross-cultural marketing analysis. She shared the Hofstede Cultural dimensions and how they can be applied to the way we look at digital customer behavior - especially when dealing with international audiences. Best quote “Fish can’t see water”
November: Dale Bertrand joined us to share his AI expertise and how it is changing the way content marketing and SEO is done. And there were looooong debates in the #advanced-testing channel about whether or not Matt Gershoff was a “stats guy” which led to Matt writing a blog post about’ Privacy-first Engineering in A/B testing which was fascinating.
December: Jonny Longden joined us to explain that no one really understands experimentation statistics and even the experts disagree with each other. Ronny Kohavi disagreed and suggested maybe Matt Gershoff understood the stats (see above re “not a stats guy”). 😆 Man, I love this gig.
How to Run Search-Ranking Experiments. Brought to you by
For marketplaces, e-commerce platforms, and collaboration tools, optimizing the search-ranking algorithm can drive millions of dollars in revenue—and the only way to properly evaluate your algorithm's success, of course, is with an A/B test.
We asked Aaron Maurer to share his experience at Slack and Airbnb and give advice on running successful search-ranking experiments. Read on to learn Aaron's recommendations.
Coming Soon: TLC Conversations & Industry Events
Jan 12 - (TLC Conversation): Craig Sullivan & Oliver Paton
Jan 26 - (TLC Conversation): Erin Weigel
Jan 29-Feb 2 SUPERWEEK TLC LIVE: Engineering the Future — Tackling Data Challenges in a Challenging Landscape (a TLC panel conversation with Simo Ahava, Jenn Kunz, Ken Williams, & Kat Ribant - join the conversation LIVE!)
Feb 5-15: Save $500 Accelerating Innovation with AB Testing w Ronny Kohavi
Feb 9 - (TLC Conversation): Will Critchlow
Mar 15 - (TLC Conversation): Karl Gilis
Apr 26 - (TLC Conversation): Daphne Tideman
To add any of the TLC Conversations above to your calendar, simply click the event in the (parentheses) and you will be directed to the event page for that TLC Conversation. You can also go to the Test & Learn Community website directly and subscribe to the entire event calendar on the left side of the page if you are on a desktop computer or scroll down after the newsletter subscription box on your mobile device.
A: By requesting the answer to this joke, you agree to the storing of information related to your identity, your joke preferences, and history. We will use this information to improve our jokes, analyze joke responses, and assist in our joke marketing efforts. Your response will be stored and any data collected from your interactions with us will be used in accordance with our Privacy Policy, and will be stored according to our Data Storage Policy. You are given the choice to unsubscribe to this joke, but we’ll remember that too. Uncheck this box to agree with this statement. ✅