We have to ask ourselves, are the metrics we’re tracking the right ones? Am I tracking something that’s valuable or vanity? And when does a metric go from being valuable to being vanity?
This question was brought up on a recent The Black Line Podcast episode, where Doug and Mike shared a lot of information, so we’re breaking it down for you here. That way you can determine for yourself if the metrics you’re tracking need to be updated. (If you would like to watch the entire episode, you can check out Episode 75: The Importance of Business Acumen & Valuable Metrics here.)
Valuable metrics give you insight into your job, what you’re doing, and how you are performing. They help you make better decisions, and they help guide you. They can serve as waypoints to know if you are on track to reach your goals. You know you’re tracking valuable metrics when you can act meaningfully with the information you receive from them.
A vanity metric is just the opposite. It’s a number with no context behind it. Vanity metrics get tracked because people are attracted to how easy they are to measure and control. You can game the system with these metrics, presenting them in a positive light for your benefit. (I must note that if you track metrics correctly, you should never have to game the system.)
For us marketers, there are 10 vanity metrics that most businesses use--but that, unfortunately,are also the worst ones. They are as follows:
Why are these considered vanity? They fall into the vanity category because they make you feel good about what you are doing, but they are not indicators of success or achievement. They’re ineffective and end up being tracked because people just happen to measure the wrong things. These metrics are easy to track, and metrics that are easy to track aren’t always going to be the right metrics to track. I’m not saying these metrics aren’t important, they just don’t give you any real substance to do your job. They’re nice to know and be aware of, but they aren’t items you would necessarily want to obsess over.
From the way that everyone talks about vanity metrics, you would think they are the worst things in the world. In reality, vanity metrics aren’t necessarily bad, and at the same time they aren’t the best either. They’re only bad if you make poor decisions based off them, or are only tracking them.
Just as Goodhart’s Law says, the moment that you take a valuable metric and make it an objective, that’s when it becomes vanity. The biggest question you need to ask yourself is, “What are you doing with the metric?” If you are tracking vanity metrics, be cautious. You shouldn’t allow the metric to be the driving force, you should let it be a key determinant of what you do next based on the outcomes you receive from the work you previously did.
Yes, metrics can be both vanity and valuable. How is that possible? There are a few different reasons for this.
One reason this happens is dependent on the company size. The same metrics that could be considered vanity in a smaller company can be valuable in a larger company because you can only manage a certain number without getting overwhelmed.
Metrics can also be both vanity and valuable depending on what you are trying to measure. What do I mean by this? Here’s an example. Lift's CEO (you should know him by now, Doug Davidoff) doesn’t care about how many visits we get to the website, but he cares about how many visits we get to the website. Stay with me a little longer; it’ll make sense, I promise. What he means is that he doesn’t care about the number of visits, he cares about what that means in terms of our growth and how we’re expanding our platform.
All things being equal, an increase in traffic is a good thing, but it isn’t the end of the world if it’s not constantly increasing. It isn’t a hill anyone is going to die on. Traffic trends tell us a lot about what’s going on. If traffic goes down 20% and nothing has changed in terms of what we’re doing with content, promotion, or distribution, that’s a signal for us to create and/or adjust hypotheses to see what will bring that traffic back up. You shouldn’t go into panic mode. Instead, you should form hypotheses and get to testing.
How do you determine what the metrics you’re tracking are to you and your position? To start, you need to ask yourself these questions:
How is it being used? Are you tracking the metric just to track it? Or is it an indicator of how you are progressing towards a goal? If the metrics you’re tracking aren’t being used to help you achieve success, rethink what you’re doing and what you are tracking.
Is it actionable? Can you make good decisions based on what your metrics are telling you? If your metrics are giving you an indication of what next step you need to make on a project, campaign, or other task, you’re more than likely tracking a valuable metric. Once you reach a point where you can’t make a decision off of it or do anything that will make a meaningful difference, it’s time to rethink what you’re doing.
Can you tie a question or a hypothesis to the metric? As I mentioned before, can you do something about the metrics you’re tracking? For instance, those email opens that you noted on your last marketing blast: what does that tell you for the next time you send an email out? Do you know why people are opening the email? In most cases you won’t know. Opens by themselves don’t tell you what you need to form strong hypotheses.
Now if we’re looking at email engagement, you can see what people were clicking on and engaging with. So the next time you go to create an email you can tie it to a hypothesis along the lines of, “if we include more content around this specific subject, then we can expect for the next email to receive higher engagement rates.” Granted that isn’t a super strong hypothesis, but you can see the difference.
People fight all the time over this topic because a metric can either be vanity or valuable depending on the situation. You need to take a lot of determinants into consideration before you can decide what type of metric it is you’re tracking. It comes down to this, if the metrics you are tracking are actionable and meaningful, you’re on the right track. It’s when they are used in isolation and don’t coincide with the above three questions that you’re more than likely dealing with a vanity metric. So before you go obsessing over metrics, figure out if you’re stressing out over the right ones.