Three main reasons a company would want or need a TMS are:  Speed, Reliability, Control.  Double check your needs before you start spending your time and company’s money.

You should consider a tag management system if you are:

• Frustrated with the “one tag, one project, one timeline” model of tag deployment
  
• Switching vendors and looking to gain leverage over future deployments
• Managing globally distributed sites but have little centralized control over tags
• Looking to add Q/A and workflow management to your tag deployments
• Concerned at all about the quality and data accuracy from your web analytics

If you’re looking for TMS vendor make sure they’re providing great TMS product that meets your requirements.  Some TMS vendor may pitch for additional services on top of the main TMS and makes the hole package sexy.  However at the end of the day, if you’re looking for a TMS solution make sure you focus on that, and don’t get distracted and end up getting a so so TMS with many add on services that you are ‘nice’ to have.  Common add on capability I came across is attribution tracking.

Service, training, and support.  You want a partner is going to hold your hand really good and tight.  If the chemistry with the vendor is not jiving during POC, make sure you have a vendor has enough experience who is willing to understand your web site and gets it when they work with engineers.  It is also nice to know if your TMS vendor has some kind of training package or session for their clients, and if they do, it is a good sign they’re willing to grow their relationship and not just selling a solution.  TMS is like content management system (CMS), where your important tags and data are going to be managed.  Don’t let your data be managed  by a solution you’re not going to be happy with.

Prepare your questions up front, and get technical.  If you’re not technical, but need to choose TMS vendor, then make sure your engineer is with you in the POC.  All TMS vendor is obviously going to be able to support hosting your tags and fire off tags.  The differences are going to come out a lot during the technical evaluations.  Just some high level list of things I’d ask…

• Control to be able to load tags earlier or later.  For example, many a/b testing tools would want to be able to load the test version before the original page gets loaded so consumers don’t see the original page being swapped with test version.
• Staging to production workflow control.
• How the TMS tool works to capture meta data from the data layer, and re-define that value to various analytics tool
• Make sure your concerns on mobile site and apps tagging are answered.  How mobile site is being architected varies by company so make sure you understand what the TMS vendor can or can not provide regarding mobile/tablet sites.
• How TMS vendor caches the tags.  Pure cloud, hybrid, or client side.  Make sure your engineer is involved in understanding this so you don’t only have the right too, but a TMS that complies with your data governance and policy.

TMS related articles and sources:

http://www.gilliganondata.com/index.php/2012/11/06/tag-youre-it-one-more-analysts-thoughts-on-tag-management/

http://support.optimizely.com/customer/portal/questions/711631-google-tag-manager-optimizely

I’ve been using ClickTale and I’d like to share how I’ve been leveraging the heatmap to make better decisions on what needs to be tested on the page to improve the desirable outcome of the website or web page.

* images are from www.clicktale.com

Heatmap (mouse move)

Mouse move heatmap doesn’t reflect clicks so you have to be careful to not set wrong expectation.  It is really a good way to see what page content or elements people are engaged with.  In many cases, I use mouse move heatmap to see which content are important to people so I can draw some kind of possible connection between VOC survey.  For example if VOC survey says, customers find XYZ content helpful in making decision to buy, but that content XYZ was not getting any heatmap activity, then it is worth setting up a test around getting XYZ content more prominent visibility.

Click Heatmap

Click heatmap is very useful.  It actually shows you where your site visitors are really clicking.  If you have high engagement identified from mouse move heatmap, but no clicking activity on a button placed within that content, then there is a problem.  In most cases, I use click heatmap to identify a link that seems to be important to the users, but not getting a good visibility.  In such case, I can build hypothesis around testing that CTA and see it that’ll drive more conversions.

One important thing to remember is to use Click Heatmap even after you perform testing.  In recent post mortem analysis I performed, the A/B testing indicated a significant uplift to a particular outcome after adding a link to the top of the page.  After it went live, I looked at the Click Heatmap and noticed that it was the most popular linked clicked, and it really gave confidence to the team that the testing yielded a very important and permanent fix.

Attention Heat map

Attention Heatmap is a great way to see how much attention a specific page area gets from the visitors.  This is different from mouse move heatmap as it is independent from mouse movements.  Imagine you scroll down and stop to an area where the content was important to you.  ClickTale will understand that range of area that consumers pause and read.  This can tell you if that particular content should be brought up to higher position for better outcome, so you can go and test that hypothesis.

Segmentation

So Heatmap is very powerful way to capture and understand what your website visitors are engaged and interacting with.  At the end of the day, you’re optimizing to drive better outcome like sales, some kind of downloads, etc.  Just applying the standard Heatmap analysis is not enough.  In many cases, when you segment your heatmap on people who converted and compare to people who didn’t converted, it may show some very different outcome.  What that means is, you may find a very important winning content that leads to the conversion, and because you did NOT applied the segmentation you might have missed building that content into your test plan.  Make sure to have your analysis take into account of segmentation.

Task completion, or what the intent of consumers on site may not always be what the marketers think it is.  It is important to understand your consumers, but it is also important to understand what the objective and goal of what the web page as well.  So as an analyst, it is important that you have a KPI, so that regardless of the problem you’re solving, it is actually moving the needle of your KPI while making consumers happy.

The important parts within the planning process for testing are:
1) Understanding the size of the issue
2) Prioritizing the issue
3) Come up with a great hypothesis to test

Understanding the size of the issue
It is very easy for a marketer or an executive to point out an issue that he/she feels very strongly about a particular problem on the website. However, that may not be (probably in most of the cases) the most biggest problem that the website has. Depending on the problem you’re trying to solve, there would be various tactics to identify the size of the problem.

Taking bounce rate as a KPI example, you’ll probably want to sort the page by highest entrance visit (or landing page by visits), and find the page with the highest bounce rate.

This is a very important part of testing, as you don’t want to be solving a very small problem and wasting precious resource and time.

Prioritizing the issue
Now that you have a list of pages with big problems, but your team has more than one KPI. When you analyze the data, now you have multiple big problems that seems to be equally important. This is why sorting your order of work by prioritizing what to test is important. One way to do that is analyzing the opportunities in common unit or metric. So if the problem is bounce rate, then converting to the traffic lost in terms of dollars could be start so that you can compare that to other conversion opportunity to sales in dollars. Figuring out your methodology to prioritize the problem is going to be the 2nd most critical process.

Come up with a great hypothesis to test
Now that you know what page you want to test, before you jump into involving the design folks and copy writers, do your analysis. You’ll need to start to understand what and why things suck, and see if there are opportunities to improve. What test elements will give you a better success rate in getting a good test results?

Here are few things you can look at:

- Traffic source and any particular things that pop out driving bad/great results
- Heatmap analysis to see what content is being engaged and interacted or not
- VOC: What are consumers saying about that page that could be improved

Don’t try too many things at once.
If you’re trying to start optimization testing for the first time, here is something to seriously take into consideration. Hopefully your a/b testing is not just a one time test, but rather a program that is part of digital marketing’s DNA. That said, you’ll probably want to find out some kind of pattern that works or pattern that highlights what makes your web page fail most of the time.

This is something worth trying at first and continue trying because it’ll give you a good leverage and organized thought process when you’re trying to solve a more complex problem later on in your optimization journey.

Check out this (above) quick diagram I drew. Say you’ll be performing 4 tests in upcoming weeks or months. You know you’ll be testing several different pages, and going to keep it simple by performing an a/b testing. What this diagram highlights is that, even if you come up with randomly different new content per test, if you always kept the method same (consecutively testing circle vs. triangle), you may find a particular pattern contributing to a win.

So 3/4 tests had version A winning the a/b test, and despite each tests having unique content giving you the insights and wins, you might be able to call out in Test #5 that there is something fundamentally wrong with how original pages were set up to begin with. Then you may set up Test #5 with circle Vs. circle with different content to hone in on the real problem or opportunities you may have to improve.

Now, circle or triangle in this example doesn’t mean anything, but if I give you some random real examples like these, it may help you better visualize what I’m talking about
- Short (Circle) Vs. Long Page (Triangle)
- Marketing fluffy content Vs. Tech Spec heavy content
- Black background Vs. White background
- 3 images Vs. 5 images

MVT (multi variate testing) is actually more complex as you may end up with bunch of test element combinations, so keeping this in mind may help you start a successful optimization program. Have fun testing!!

Some of the roadmaps I’ve seen from analytics conferences were very useful, it gave me the guidance to really think about my starting point and recommend tools/skill sets to help the department mature with analytics.  However, a lot of the digital analytics roadmap I’ve seen are really geared towards adoption of tools, adoption of integration of data/platform over time, or maturity of data practices which commonly turns into Optimization like a/b testing or personalization, etc.

The following diagram I created plots level of advancement in data practice along with maturity of of each level.

Each ‘advancement’ level is probably different for each company as the problems every companies solving are different.  I’m sure every team or department are structured differently and the objectives and goals are different, too.

You can take my diagram as something potentially common for digital analytics analysts working in marketing organization.

The most important thing you need to take into consideration when looking at this roadmap is to ask yourself “what business problem is being asked to solve“.  I’ve stated some examples of use cases in the roadmap, and not necessarily brought out the real questions or “business problem asked to be solved”, so that part is something you have to think for yourself.  I left that part pretty vague.

If you read carefully, there are some what of mix or overlap in the advancement levels. For example, in level 3 advancement, I’ve noted that it is about “data working for business”, and some could be asking “why aren’t you making data work for you at level 1?”. That’s because level 3 is less of looking into the rear view mirror, but more forward looking data analytics practice. Same with level 2, where some may feel that integration and advance segmentation on data should take place at level 1. These are very directional and expectations are different for various businesses and industries so make sure you align that with your business landscape and needs.

The maturity axis is not only about analysts’ maturity in data practice or skills, but it is a mix of maturity of tool/software usage as well as it’s applications, customization, adoption, etc.  You can define the scope of the definition of maturity in digital analytics, but in my view, it is really the depth that is applied to each advancement level.

I’ve seem and heard of many companies in various stage of maturity or advancement in digital analytics data practice, and a lot of times it is not just one straight line of advance level. Some companies do really really well in level 1 stuff and they perfect the art of web site analytics and answer a lot of marketing questions. On the other hand, companies who are strong in paid media may have a hard core data practice using media mix modeling and invests a lot in predictive analytics. In that case, they may be an expert in level 3 and very mature, but may lack very basic digital analytics practice on own media metrics such as measuring and analyzing web site or mobile app/sites. In such case the direction of the arrow in advancement line is the other way around for that company.

Hopefully this blog post ignite the idea of reevaluate our digital analytics roadmap. There is no one straight answer, so hopefully my diagram kicks off some new idea for you.

We know great amount of insights would need to come by deep diving into the VOC (voice of customer), analytics, and many other tools. By then you pray that all that stuff adds up to one single story that is big enough to start asking people to share their time (resource) to create image assets, copy, set up a test, etc.

In one the thing we’ve observe after running deep dive analysis, we made some mini improvements to our core KPIs including bounce rate. Like I said, a lot of deep analysis, but little impact. In parallel, techy folks were doing some cool in optimizing one of page’s load speed. I helped measure the impact and the result was some insane improvements to conversion rate, something like ~8% improvement in people not existing, leading to ~20% increase in sales amount. There are quite a bit of case studies out there, and results are super different across sites, brands, page, or site section. I highly recommend you look at your own data and test to find what ‘your’ cause and effect equation is.

So that lead to my idea of looking at page load report in Google Analytics and segment or filter by different data groups and page load. Data comes from some personal site and random date range for the sake of showing the value of this method of analysis under Google Analytics.

Here is a segmented view showing new visitors, returning visitors, non-bounced visits, and organic search traffic.  It is a amazing how much of a difference you’ll find when page load time is smaller for returned traffic who are likely to have the page cached in browser.  Also slightly lower traffic showing lower bounce rate and high page value.

The same situation goes for non-bounced visitors who are experiencing a page load near 1 sec faster load speed with some great data representing the page value.   Similar observations for organic search traffic.  It definitely seems that the page load do impact the engagement and outcome of the site.

Here is another data cut.  I used filter on Site Speed > Page Timings report looking at avg. Page Load Time greater than 4 sec.  Noticed the bounce rate is almost 4pts different, or higher than when page load is filtered at less than 4 sec.  The average time is for the two different time buckets are absolutely different as well, a whopping 5 sec.  Of course the page value is much higher when Page Load Time is much lower.

These kind of data is really actionable as you can just take this data to some manager in charge of development and say ‘there is an opportunity for improvements’.  Great managers will then make that magic happen and help the company generate more revenue, and most importantly make customer experience a better one on the brand’s site.

So here are my few metrics scanning technique to quickly understand the signal from what is consumers are telling you about consumer journey or their experience on your site via mobile (mobile in this case includes tablets). When companies don’t have a marketing plan, objective, goal… they usually ask analyst on what’s happening with mobile traffic. What’s the current state?, blah blah.

I hate analyzing data without objectives and goals, but in reality this is part of the job of analysts. Please remember that data analysis is as good as the business questions. When there is no good business questions to answer, then think of business questions you think fits the business then formulate the answers to answer those questions.

In such data analysis exercise we still need some kind of tactical approach to understand mobile. These are ideas, but I’m also attaching my rationale behind it. Additional ideas are welcome.

Here is the outline of the key analysis points
- By how much is your mobile traffic growing? (questions the relevancy of mobile traffic to your business)
- What is driving the traffic (questions where to potentially invest more or less based on current traffic drivers)
- Are mobile users consuming the content differently (questions where and what to optimize or create content strategy)
- Difference in interaction or task completion, mobile vs. non-mobile segment (questions if your mobile site is working or not)
- Time factors, recency and frequency (questions if your mobile is driving loyal or engaged audience)

By how much is your mobile traffic growing?
This is an obvious one. Given that many research papers are talking about growth of mobile and it’s internet access, how is your website doing? My piece of advice is don’t expect your mobile traffic is growing according to the industry or market trend. One of the thing I’ve noticed is content catered to mobile has do a lot with how your website’s organic reach to mobile users (content in either a form of content written about mobile, or page that is mobile optimized). * note organic, because companies do pay for ads targeting to mobile…

Make sure to look for signs that could possibility drive higher growth rate. In most cases the mobile traffic growth is not just happening naturally, it could be those content you’ve written or syndicated somewhere.

So as you can see, my blog’s mobile traffic growth is pretty significant looking at year on year. My blog traffic is very small, but imagine you see that growth in your company’s website. That growth is something. What percentage of traffic share it represents? You might want to check that out by year over year. I’ve noticed 2010 to 2011 has shown significant growth particularly from tablets (iPad in particular). What about your website?

What is driving the traffic
Things you need to look for is traffic sources and the actual pages driving the mobile traffic growth. Would it be the inbound traffic to website or would it be some content specifically in interests to mobile users’ journey online.

Again, an example from my data.

Wow, so Google is really helping me drive mobile traffic to my blog and year on year, it is driving a pretty significant percentage increase as well as volume (blurred). MR. Avinash’s Occam Razor blog is also driving some good lift in mobile audience year on year for my blog.

So given that search is driving traffic, it is a good timing to look at the content that people are finding via Google SERP. So go to content and segment by organic search. You’ll see which page is really giving you that organic traffic from mobile.

Are mobile users consuming the content differently
Don’t expect mobile user to stay on your website as long as PC browsers. Remember that most likely the mobile users are swiping their fingers and clicking rapidly through various content. Also in their consumer journey, mobile users could physically be in different places from traditional PC environment Hence, consumer mindset on what info they need and expectation from their activity online is very different from traditional PC environment.

Review mobile segment vs. non-mobile segment, and look at page views per visit, bounce rate, time on site, and recency & frequency. This will set the tone on setting up different digital marketing strategy for mobile, in both traffic acquisition, and usability of the website.

Difference in interaction or task completion, mobile vs. non-mobile segment
Do the same with the previous point on content by looking at mobile vs. non-mobile segment. The difference is make sure you obsess about understanding the difference in task completion rate behavior pattern on mobile vs. non-mobile users. Task completion could be anything on conversions such as file downloads, newsletter sign up, sales conversion, add to cart, watched video, etc.

This part of the analysis is very important. Your website exists for a certain reason, hopefully not just to drive traffic, but to drive some outcome that adds value to both your customer/consumer or business. Obsess in analyzing mobile users against conversions.

Random ideas to look for in this exercise…
- Recency & frequency for mobile converters, to look for differences in customer journey pattern
- Landing page difference for mobile converters and page value
- Multi channel attribution on mobile segment vs. non-mobile segment
- Conversion rate analysis by traffic sources
- Customer profiling on mobile converters vs. non-mobile segment (age, gender, etc.) Yes, you can do this if you have your analytics integrated with social graph

Time factors, recency and frequency
I’ve kind of touch this in previous point, but keep in mind that mobile phones are with consumers 24-7. It is personal, and they may have apps they frequently use to access blogs, or get notifications on phone while in their pocket to read things.

As you can see from my recency or ‘day since past last visit’ shows 2 days apart traffic shows a higher traffic distribution on this bucket than non-mobile segment. My blog traffic is obviously going to be different from your site, so check out to see if you are getting some interesting results. It is probably more likely that users on mobile ar revisiting your website more frequently given that you’re adding content occasionally.

Analytics solutions like Mixpanel can also perform cohorts analysis, so you might find your mobile traffic has a better recency conversion from one event to another event (i.e. free signup to premium account signup). Eyeball the time factor in conversion really closely, mobile customer journey is very different from traditional PC behavior so you might find different insights leading to a whole new digital marketing strategy.

Hopefully you can find something interesting about your consumers or customers on mobile. Have fun analyzing!!