Segmentation is at the heart of most real analysis – and in this series I’m focusing on analytic segmentation – not segmentation of large and significant populations for reporting purposes. In the last post, I showed how we used segmentation to isolate, study and improve the behavior of printed catalog searchers coming online to look for a specific product. Today, I’m going to show an example where visitor geography turned out to be the key to effective segmentation and personalization.

Example: Geo-Based Searching

Use Case: A real-estate focused site knew that geography was probably important to visitor behavior, but despite many years of operation there was significant disagreement about the online population’s searching behavior and how it related to where they accessed the internet. 

Client Question: The client wanted to know how visitor geography and search behavior were related. Key questions included:

• Did most visitors use the site to look at properties outside or inside their current location?
• Did visitors search the same locations repeatedly?
• Were there significant differences between visitors searching in their own geography vs. those searching outside their own geography?

Measurement Issues: Search geography was captured as an input search string – making it difficult to consistently resolve since it had many variations.

Tool: We used Omniture’s Data Warehouse tool for the analysis.

Methodology: One of the challenges here was that we didn’t have a direct correlation between search term and visitor geography. Getting that kind of report is one of the reasons we usually recommend that companies deploy a Vista rule to copy the visitor geography into variables. However, with data warehouse, we could get at the data. It was more a question of how to do it conveniently.

Let’s start with how we tackled the first question – did visitors use the site to look at properties within or outside their current geography? Obviously, we knew we were going to see both behaviors, but the relative percentages were hotly debated.

Unfortunately, in web analytics systems as they exist today, you can’t generally create a segmentation based on the comparison of two variables. In other words, we couldn’t use a query that selected all the visits where the internal search term was contained within the SiteCatalyst visitor DMA. In addition, we didn’t have the ability to apply any operators to the various strings involved – so comparison would be inherently hit or miss.

One alternative was to produce a giant data warehouse request that simply gave us all the combinations of visitor geo and search term along with visit count. We could then load this into Access or SQL-Server and have at it.

For an analysis that required comprehensive coverage, that might have been our approach. But this was just a piece of wider site analysis and we really couldn’t afford to spend more than about 8-10 hours on the whole use-case. So what we decided to do was focus on a small set of target markets that we took to be representative – markets like Bakersfield for mid-size cities and Tahoe for vacation destinations.

We then built a segmentation based on search terms that contained the relevant target area. This took advantage of the segmentation builder’s ability to use “Contains “and multiple rules strung together.

Using segments like this, we generated reports on visitor geography for each target area. This allowed us to answer the first question in detail. For each target area, we knew what percentage of the searchers originated in the target area, what percentage were adjacent (from nearby DMAs) and what percentage were outside the target area.

It turned out that on this particular site, for all areas except resort cities like Tahoe, a very heavy majority of searchers were local.

Note that this segmentation was visit-based – in the next step we extended it to include visitor behavior.

This was almost too easy. The most important decision was simply to focus on a set of target markets that represented different potential use-cases instead of trying to answer the question for every market and for every search.

The second case was just a little bit trickier. We wanted to understand whether the site displays could be tuned based on previous visitor search behavior. So the question was did visitors tended to search on the same places when they returned to the site.

For this analysis, we extended our original target market segmentations to look at all visitor behavior when the first visit included a search of the target market. The segmentation looked something like this:

Note that the Visit filter is now nested inside a Visitor filter. This means we’ll get all of the behavior for any Visitor that meets the Visit criteria. To the Visit criteria, we added that the Visit Number for the target market search had to equal one. This gave us a population of visitors who started by searching in one of our target areas.

Now, we generated a report that looked like this:

With this filter, we were able to see all the searches done by visitors in subsequent visits to the site when they began with a Bakersfield search. We used Excel to aggregate the results and with a little arithmetic (or another segment) we could get all the Bakersfield searches who started off by doing something else in their first visit.

It turned out that for this particular site, the vast majority of visitors in all of our target markets tended to repeat their searches identically on subsequent visits.

Conclusions and Recommendations: This analysis answered some fundamental questions about how visitors used the site – but it also suggested some extremely important personalizations. For any site, the faster you can get visitors to the stuff they want, the better and more successful the experience. The analysis strongly suggested that the site could profitably display results for visitors as soon as they arrived on the site – without even waiting for a search. It also suggested that keeping and using previous results to tune session re-entry pages would have similar benefits.

Reflections: One of these segmentations was trivial, the other only a little more complicated. But used judiciously along with the idea of isolating a representative set of target markets and they were able to answer definitively some questions that had persisted within the business for years. In addition, the answer suggested some significant personalization opportunities that the site was missing.

One other really important aspect of this analysis – and one I find repeated over and over – is the importance of visitor level segmentation. In nearly every real analysis, it turns out to be vitally important to be able to extend a segmentation to retrieve all the behavior for a visitor meeting certain behavioral criteria. As an extension of that, the ability to report on behavior by visit number is almost equally important. We use visit number in reports with startling regularity – and because of that we recommend that visit number be kept in Omniture variables (using a Vista rule or plug-in).

Comments

And in web analytics, we often care more about whether traffic is going up more than its true absolute level. But this focus on rates of change, though something of a truism, hasn’t been consistently applied in many types of online reporting. In this short series, I’m going to discuss some areas of online measurement where concepts of rates-of-change and velocity haven’t been widely adapted but are, nevertheless, extremely useful.

Content Evaluation / Editorial Reporting
Nowhere is the concept of rate-of-change/velocity more important than when it comes to creating media metrics for editorial reporting. Raw numbers about article consumption are almost meaningless except when understood in the context of how long the article has been in circulation and what position it has been given during that time. If you’re providing your editorial staff with a daily report on content page views, you are forcing them to put each article in that context: mentally doing the arithmetic of when an article was pushed, what position’s it’s held, and, given those two facts, how that compares to previous content performance. That’s a lot to hold in your head and it makes using those reports much more difficult than it ought to be.

Using concepts of velocity, you have the ability to deliver crisper, far more informative editorial reporting.

This type of report uses an average of previous stories velocity curves (by time & position) to create a predicted interest line (in blue) for each article. It then maps the actual usage velocity (in red). This gives a simple, precise, and accurate depiction of how well a story is performing relative to proper expectations; it shows not only whether the story is more popular than average but what legs it has and how long it should probably stay in rotation.

To me, this is another example of what we at Semphonic call Analytic Reporting – reports that answer questions instead of just raising them.

Of course, a straightforward velocity report isn’t available out of the box in pretty much any web analytics solution – you have to do some work.

First, you need to create some metadata about the article in the web analytics solution. There are typically two ways of doing this. You can push basic information (like publish-date and position) as variables to be captured by the tag in real-time. Alternatively, you can push a content-id as a variable (or as part of the page name) and then transfer the metadata to the web analytics system (for example in Omniture this would be a SAINT file, in NetInsight a DataConduit).

Keep in mind that constructs like SAINT work pretty well for most article meta-data (e.g. push-date, author, service, length) but can be problematic for something like position. SAINT changes all of the data being viewed – regardless of whether it’s past or present. So you can’t capture the fact that an article may have been position 1 on the 24th of July and position 4 on the 25th using a SAINT table for metadata. This means that variables like position that change over time must be captured by the tag in real-time.

Having the push-date and positions for an article makes it easier to understand its performance, but it isn’t enough to make editorial reporting as easy and powerful as it should be. Web Analytics solutions don’t give you much flexibility in report generation and variable calculation – so getting to the data you really want – traffic by position and time since push date is still very difficult even when you’ve captured the push-date and the position in the data stream.

To make reporting easier, I recommend doing a little more pre-processing on the records. In Omniture, for example, you do this using Vista Rules. By combining two Vista Rules, you can get to the information you really need into a report without a lot of complex segmentation or correlation. The first Vista Rule is the TimeStamp Rule. This simple, very common rule, just places the time stamp in a variable for every server call. The second Vista Rule you need is custom; it calculates the difference in hours between the time stamp and the push-date variable and saves this difference in another variable. You can really make your reporting easier by concatenating that difference and the position together in a single variable.

Keep in mind that while I’ve described all this as happening with Vista Rules, it can be done in other solutions as well (with DataConduits and processing rules in NetInsight for example) and it can be done in ANY tag-based solution using coding in the tag. So no matter what solution you are using: from Google Analytics to DoP, it’s possible to derive this simple but powerful metric of content activity by time from push-date and position.

What you get when you do all this work is a variable with content id and a variable with the number of views (and visits) by position by time since push-date. You can correlate this variable with the basic TimeStamp to get the actual day of week and the hour of day – that’s important since content pushed after midnight (for example) is going to have a different velocity curve than content pushed at 8AM.

That’s pretty much all the secret sauce there is to delivering a report just like the one I started this post with.

Actual Content Velocity vs. Predicted Content Velocity is the sort of Media Metric that I think adds real value to editorial understanding and it’s a great illustration of how useful concepts of velocity really are. It’s also the kind of metric that I’d like to see discussed at X Change in our first ever industry-specific Huddle – Media Metrics (led by Turner’s Tom Cattapan). There’s so much more to measuring media than page views and time-on-site and the vast majority of really useful metrics never seem to see the light of day!

Comments