Dashboards:  Is Real-Time Reporting The Answer?

I’m a data fanatic.  Everyone on my team knows that the first question I ask when someone brings up an idea, points out a problem or suggests we change paths is, “What data do you have that brings you to that conclusion?”

I’m also rather OCD when it comes to reporting (as my friend and Databox’s CEO, Pete Caputa can attest).  I want a report for just about everything (I think the only thing I like better than reports is a good deck), and I’m maniacal about what’s being reported and how it’s reported.

For years I’ve pieced together my version of KPI dashboards to keep track of just about everything that mattered (and my Director of Operations, Jess, would probably add that I followed things that didn’t matter so much).  

In addition to the reporting and analysis we do for Imagine, it’s a primary part of our value proposition to our clients.  I probably don’t have to tell you that Monday’s were pretty hectic as we filled in multiple spreadsheets to track our weekly KPIs, and the beginning of the month was an insane rush of taking screenshots, copying and pasting items into powerpoint decks for monthly reports.  

Frankly, by the time our monthly reports were ready, they weren’t particularly valuable because we were already 1/3rd of the way into the next month.  To call the process frustrating would be an understatement.

Like many others I wished for the day we could truly automate all of our reporting.  The dream was that we’d push a button and the precise metrics would appear at a moment's notice.  I spent quite a bit of time investigating any number of business intelligence and dashboard tools.  Every time I reviewed a tool, I found three problems with every single one of these tools:

• They could not pull from the places where our data lived (marketing automation, CRM, Google analytics, etc.) so we would have to build spreadsheets or other hacks to address all reporting requirements, or we’d have to use multiple tools.
• Setting up these systems made me long for the days of high school Geometry - and I sucked at geometry.  I’ve never been a fan of implementing a solution when the solution is more painful than the problem.
• The only tools that came even close to solving the problem were prohibitively expensive.

The Promise of Real-Time Reporting

Reporting has been a hot topic in the demand generation world since, well the beginning of demand generation.  A core promise of demand generation is the ability to make data-backed decisions, so access to timely data has always been crucial, and (for the most part) an unfulfilled promise.

That began to change two years ago when HubSpot announced a new reporting add-on feature that promised to give the flexibility and depth needed to address this need, in a manner that would not be too complex or burdensome.  While the reporting tool did not (and still does not) fully live up to its billing (or price), it was certainly a step in the right direction.

Late last year Databox got into the game seriously.  With the addition of Pete Caputa as their CEO, they got to work on developing a variety of integrations with multiple data sources with the flexibility to design insightful dashboards delivered elegantly in real time.

It was the real-time component that got me excited.  And for us, we’ve found that the combination of HubSpot’s reporting tool (which provides real-time analytics as well) and Databox’s dashboard visualization tool was sufficient to cover the core metrics we feel are important to track.  I should note that with Databox’s broad set of integration and the ability to easily customize dashboards, we’ve also been able to create additional dashboards that allow us to monitor deeper, more strategic issues.

About four months ago we launched the dream.  We got rid of the spreadsheets, eliminated the copying and pasting and said goodbye to the monthly reports.  Through a combination of dashboards, we could access anytime, and a scheduled email every week of the primary dashboards became our new standard.

Finally, I’d have the data and insights I wanted, when I wanted, and my team wouldn’t have to do anything to put them together.

Guess what happened?

It didn’t work

Don’t get me wrong; there have been benefits.  I shared an example of one earlier this week.  We’re better off with the ability to see what’s happening when we need to see it.  Additionally hours of mindless work putting monthly reports together have been eliminated, enabling team members to spend that time on more important and higher value activities (and for Ellen to get more coffee).

We encountered three problems with the shift to real-time reporting:

• The biggest benefit of this approach has we had the data when we needed it.  That was also the problem.  We only saw the data when we felt the need to see the data.  This caused to either lose the flow of certain metrics or to spend too much time looking at items (which is something I’ve been known to obsess about).  While we set alerts, we still found that we ended up looking at data more myopically.  The fact that our reporting was always there for us (so it was always visible) had the unintended impact of becoming invisible.  When you see a leaderboard/dashboard every day, pretty soon you stop “seeing it.”
• We lost the plot of the reporting rhythm.  I’ve always been a believer that the number is far less important than the trend.  A problem inherent with just about every real-time or data visualization solution is that it only focuses on a single period.  I can’t, quickly and easily, see how the performance for a period compared to other periods to know what, if any, trends are emerging.  It’s also frustrating that all of these tools use mountain charts and don’t support trailing period charts.
• The overfocus on real-time reporting (which as I shared above also led to short-term analysis) increased misalignment and we lost some of the deeper conversations we’d have during monthly report reviews.  

And So Was Born The Reporting Cadence

Don’t get me wrong; I’m still a big fan of real-time reports.  On their own, they’re just not enough.  The answer is to mix real-time reporting, with more period reporting to ensure you look at your business thoroughly and effectively.

Here’s the cadence we’re introducing at Imagine:

• Monthly Reporting:  The focus for this report is more about trends and progress.  We’re focused more on the main themes and progress of tests, experiments and initiatives.  We’re going back to creating monthly reports, but the process will be much easier with the reporting we now have.  The focus here is about on alignment and what we’ve learned in the last month, how we’ll apply that learning going forward, and ensuring clarity on the hypothesis, tests, and initiative for the next month.
• Weekly Reporting:  This is more about KPI check-ins.  The focus is more, “What do we see here that’s out of the norm?”  I’ll also admit the focus is about 75% of what’s happening that says there may be a problem, with about 25% of what are we seeing that may indicate there’s an opportunity.  Our opportunity focus is more geared to the monthly rhythm.
• Real Time/OnDemand:  This leaves a whole series of dashboards and reports geared to monitor areas and actions that move the business forward.  Some of these reports are used as part of our weekly or monthly reviews.  Most are there for when we need them.

As with most things, reporting is not an either or type of initiative.  The best results come from mixing the best of a multitude of approaches.