In part 4 of this 5-part blog series about ‘Building a Culture of Optimization’ I’m going to talk about evangelizing your process within your organization. You can see part 1, part 2, & part 3 here. Part 4: Evangelize the process Process is important. Process leads to consistency, repeatability, and authority in a testing program. Sharing that process and getting others in your organization bought in and supportive is even more important. One source of truth One of the best ways to make your optimization program better known within your organization is to evangelize it via a widely accessible & visible roadmap. Here’s an example roadmap that I use within my organization: I host this roadmap in a Google doc that is accessible to everyone in my organization, from analysts
Continue reading...
In part 3 of this 5-part blog series about ‘Building a Culture of Optimization’ I’m going to talk about the importance of bringing your organization up to speed on the math behind the tests. You can look back and see part 1 on the basics and part 2 on good test design. Part 3: Know the Math! Give your peers a short stats lesson (but keep it light)! What does someone running an A/B or MVT test need to know about math? How detailed should they be? Here is what I tell all of my coworkers: Statistical Confidence = confidence in a repeated result The confidence level, or statistical significance indicate how likely it is that a test experience’s success was not due to chance. A higher confidence indicates that: – the
Continue reading...
In part 2 of this 5-part blog series about ‘Building a Culture of Optimization’ I’m going to talk about the importance of teaching good test designs. You can see part 1 about educating the basics here. Part 2: Good Test Designs You can’t have a good test without a good test design. One of the first things I do when a new test idea surfaces is sit down with the key stakeholders & test proposers to understand the details of what they’d like to test. We’ll talk through the variables that are going to be tested, how best to setup & design the test, and ensure we are on the same page in terms of potential test outcomes and how to ensure we are testing in a clean and consistent manner.
Continue reading...
I’ve presented at several conferences, webinars, and meetings recently on the topic of ‘Building a Culture of Optimization.’ It’s a great topic and something our industry as a whole is still in the process of nailing down – most organizations I see haven’t yet reached maturity here. Given the importance of building such a culture in order to drive testing and optimization across our businesses I’m writing a mini-series of posts on some of the most important things I see, do, and recommend. This will be a 5-part blog series on ‘Building a Culture of Optimization’ with parts 1-3 focusing on education, part 4 on process, and part 5 on spreading the word and shifting the culture. Part 1: The Basics – Educate educate educate! Educating your teammates is the
Continue reading...
How much time do visitors spend on our site? If we make this change, how much more time will people spend looking at our website? Did that change have an impact on bounce rate? As an analyst, do you get these questions a lot? I know I do. And my response is (almost) always the same… I will not report on those metrics and we should look elsewhere for something more meaningful. Why? Well… for many reasons. But first some definitions. Time on page: calculated by the first time stamp of landing on a page subtracted from the next time stamp when going to a new page (alternatively, the time stamp on exit of that page when continuing on to another page tracked within the same analytics account). Time on
Continue reading...
Summary & Key takeaways from Adobe Summit 2014 The Adobe Summit has exploded over the past 3 years as it moved away from the Grand/Little America, where it was still hyper focused on analytics & testing (~2,000 people attending 1-2 years post Omniture acquisition), to a large scale (6,000+ people) pan-digital marketing showcase hosted at the Salt Palace Convention Center. The growth and integration with other tools in their suite has allowed them to shift focus from solely an analytics/optimization product line to a ‘Marketing Cloud’ of tools marketed to the CMO and used across the digital/marketing organization. Here are my top high level takeaways from this year’s Summit: Key themes: Testing, Platform, & Personalization Takeaway #1: Over the past 2 years Adobe has pushed hard on integration & product
Continue reading...
This morning I woke up to the following email in my inbox: Dear Krista, Congratulations! You have been at Google for 365 days, 1/10th of a decade, >3.15 x 1010 milliseconds, 20% of a lustrum or 0.001 of a millennium and today is your 1st Google Birthday 🙂 A big Thank You for all your contributions over the past 12 months. Why not take 60 seconds to recall your first Google year and enjoy a swift walk down recent memory lane enjoying all you have achieved in this time. Happy Birthday! So, in that spirit, here are some of my reflections on my first 365 days, 1/10th of a decade, etc at Google. 1. Where did the last year go? The pace of life/work/innovation around here is insane. I find it really hard
Continue reading...
Several weeks back I posted a campaign tracking guide on the Official GA blog, now cross-posting it here. See the original post here. As an analytics practitioner, one of the most important things I try to teach marketers is how to properly tag their campaigns so we can report on the success of their efforts. To do this, I’ve created a guide for them to follow to make it easy to choose the proper UTM codes to have consistent campaign tagging across the business. This allows us to begin to assign source and medium values to finance channels and usage metrics to really understand how each campaign performs in terms of our bottom line business metrics. Overview: Setting up tracking and reporting on your marketing campaigns is simple and fun. This guide will
Continue reading...
Do you have large data sets being collected via Google Analytics? If you’ve got more than 250,000 visits within the time period you are analyzing, then you will likely encounter sampling in Google Analytics. Why? Sampling occurs for data sets larger than 250k visits in a single property (or 500k if you maximize the sampling ratio) in order to reduce processing times for data via the interface and allow marketers to view their data and make decisions more quickly. Sampling alert message: While the speed to insight is great, at times, the sampling ratio can make it more difficult to analyze your data in the most efficient way. In order to get the most out of my data, I’ve invested time and effort in various ways of reducing
Continue reading...