After countless hours researching and examining the pros and cons of various web analysis tools, you may have finally found the perfect analytics resource. All has been going great since implementing the use of your web analytics tool. But what happens when that tool you thought was so perfect, well, isn’t? It’s time to compare website analytics tools.
Let’s take a scenario that recently occurred. Keep in mind that the names of the people have been changed.
John the analyst: Wow! This new feature on the website has really taken off. Twice as many visitors are getting into the funnel. This is a huge win for us!
Carrie the VP: This is fantastic work! However, if we’re getting twice as many visitors, how come our daily revenue hasn’t changed just as much?
John the analyst: Uhhh, let me get back to you.
Three weeks later….
John the analyst: Remember that major win we had? We were actually double counting visits to the page. Everything we have been reporting since then has been incorrect.
Sound familiar? A web analytics solution is only as good as the data that is being passed to it. Sometimes the data that is being passed is “bad.” Tags break, pages are coded incorrectly all the time, you name it. So what’s really the problem, and how do you fix it?
Compare Website Analytics: The Problem
In talking to CEOs, CMOs, and CXOs of all sorts, one recurring analytics theme I have heard is that they don’t have confidence in what their web analytics solution is telling them. The reasons for this are multi-fold:
- The analyst pulling reports made critical mistakes a couple of times, and now the organization doesn’t have confidence in the numbers (Easier to blame the tool, isn’t it?).
- There is no governance on sitewide change processes, and web analytics tags mutate and go missing across the site without warning. This causes data discrepancies that cannot be explained.
- There is no alternate baseline to compare the web analytics numbers.
All the above problems are very serious ones and can cause even the most ardent fans of analytics to get disillusioned. While I will later address the first two points, for now I just want to address the alternate baseline problem stated in #3.
All metrics are pretty useless without a baseline to compare to. Hearing your analyst excitedly tell you one day that “We got 1,000 online transactions yesterday!” may sound great, but many questions related to context should run through your head. Compared to what? Why is this important? Have we never hit 1,000 ever? Or on a Wednesday? Or in the summer? Without context, all metrics are meaningless.
But that’s not to say that the context of the metrics is the only thing your analysts should be focused on. Your analysts probably use week over week or year over year reviews, but once they see a crazy dip in home page engagement and tell everyone about it before diving deeper into the issue can send everyone into a frenzy. The user experience team could speculate that the change they were opposed to making is causing the dip. The product team might guess that marketing channel changes are driving the dip. The marketing team finds someone else to blame. Sounds familiar? I hope not. Hopefully, you have a sane set of web analysts who can logically eliminate the possibilities by segmenting the web analytics data.
One thing they cannot do with all their segmentation is check whether there is a problem with data collection. One quick way to validate this is by comparing the metrics to those being collected by an alternate web analytics solution (the alternate baseline). I have personally witnessed backup analytics solutions ending hot emotional debates in about 5 minutes.
Another reason to use a second solution is if your primary web analytics solution does not provide you all the information you need to measure performance effectively. For instance, Google Analytics does not provide IP address information of the visitors to your site. It’s annoying, of course, but that’s their decision. So if you experienced a spike in traffic because of a new bot that is now scraping your site, your options in leveraging Google Analytics to find this bot are fairly limited. If your backup web analytics solution allows for viewing visitors by IP (among many other technical characteristics), you will be able to quickly decide what needs to be done to eliminate the bot from your data set.
Case in point, earlier this year we saw spikes in our traffic. We started getting about 20 percent more traffic a week. Of course, I was elated that all our marketing efforts were paying off. And I nearly committed one of the four deadly sins in web analytics – confirmation bias (more on this in another post). Then I checked Piwik to see what was driving these visits. In one click, I was able to find that ‘semalt.com’ was now aggressively crawling our site. One click was all it took. We now filter out traffic from that referrer in Google Analytics. Would I have found this using GA? Probably. But only after slicing and dicing the data multiple ways.
I cannot say enough about how important an alternate web analytics solution is to run any website effectively. I hope these two real world examples gave you enough fodder to convince decision makers at your organization to at least seriously consider a backup web analytics solution. The requirements of a secondary web analytics solution need not actually be the same as those of a primary solution, as described later in this point. So as you continue reading through the characteristics of a backup platform, keep in mind that we use Google Analytics as our primary web analytics platform and Piwik as the backup.
Characteristics of a backup web analytics platform
Now that we have concluded that a backup solution is important, let’s review the key characteristics of this solution.
- Ease of use
- Ability to measure the critical checkpoints
- Ability to deliver more insights
Let’s dive a bit deeper into each of these aspects.
1. Ease of use
The backup solution cannot be a pain in the neck to implement, maintain, or use. If it is, then someone is going to drop the ball by forgetting to duplicate changes to support the second tool. Before you know it, the backup solution will be so far behind in updates that it will be discarded and ignored. Think of all that time wasted even setting it up.
So if your primary web analytics solution is Google Analytics, then your backup solution needs to be able to provide a similar level of sophistication to create and track goals, define custom variables, track campaigns, and do anything else that you deem to be critical.
For our website, we have just one goal – Track how many visitors saw the Contact Us page. The reasoning is that we get contacted primarily by the phone number that is on this page, so we created this goal in Google Analytics. Since this is a critical goal for us, our backup solution had to be able to support this sort of goal, as well. And Piwik had us covered. We were able to easily create the same goal in it. In fact, here’s a screenshot of goal tracking in Google Analytics and in Piwik.
| Goals in Google Analytics | Goals in Piwik |
 |
 |
Of course, these numbers don’t match exactly across the two. And you shouldn’t care that the metrics between your two web analytics solutions do not match exactly. You should, however, care that the variance between the two analytics platforms is constant over time. In other words, although each solution has its own nuances, the results should vary by the same amount over time.
But I digress. Bottom line is that both the platforms need to be easy to maintain and use.
2. Ability to measure critical checkpoints
As a business, you probably have certain KPIs defined that you use for measuring the day to day performance of your business. 
To ensure the validity of your web analytics solution, you need to identify the handful of checkpoints that are important to validate the accuracy of your primary tool. For example, visitors would be an important one to have on the list. Otherwise, it could be visits/sessions if that’s what you use to track traffic. On the other hand, if you are tracking how many times a video on the site was viewed, that should probably not make the cut for a critical checkpoint.
The general guidelines to identify critical checkpoints are
- They must be fundamental metrics (such as visits, transactions, and pageviews)
- They must be used in everyday decision making (such as average transaction value and type of product purchased)
- They should be in your control (such as excluding organic keywords in a checkpoint)
3. Ability to deliver more insights
When you revert to your backup tool, you are probably in crisis mode. At that time, you want information fast. You don’t want to have to wait to pull web logs and wait days for answers. For example, Piwik will deliver visitors’ technical profile including their IP address, operating system, screen resolution, and a whole host of other information in a single click. That’s how I found the ‘semalt’ bot in a hurry. Remember, you are looking for answers, not elegant reports at this time.
Speaking of additional insights, another really neat feature Piwik provides is a count of actions by visitor. These actions are form submissions, pageviews, and any other engagements the visitor may indulge in on the site. I really, really love this view because it gives me so much information so fast.
Our conclusion about a second web analytics tool
Every business that uses its website as a marketing tool or as a source of revenue needs a second web analytics solution. Furthermore, you do not need to spend thousands of dollars to get an enterprise class cutting edge tool.
Although this is a plug for Piwik, there are other stellar solutions out there including KISSmetrics and Mixpanel. But if you are short on budget and cannot justify spending money on a second analytics solution that will be rarely used, Piwik may be exactly what you need. If you do use it, don’t be so shameless as to use it for free! Do donate to the good cause of supporting Piwik, even if it is just $30 per year.
