Issues in Analytics
Filed under: Analytics, Education and Training on Thursday, August 23rd, 2007 by Simon Heseltine
…continuing the live RBDRodeo coverage of Search Engine Strategies - San Jose 2007.This session is all about the different issues with analytics, setup, challenges, etc, and is moderated by Alex Bennert.
The first speaker is John Marshall, with a presentation titled: The anatomy of a click - why tracking never matches”.
Assumptions
- PPC ad with tracking parameters
- Basically functional website
- Modern web analytics tool
- At some point you will need to understand why the numbers don’t match
This session will not cover
- ROI breakage
- Cookies and Cookie deletion
- Anything beyond the single, solitary, lone first click
Walking through the single click. Clicking on the PPC ad sends the URL to Google, for Google to redirect to the actual site (Google counts the click). At this point there’s the chance of a 1% chance of an error due to the browser dropping the redirect.
The browser then executes the redirect. This now gives another 1% chance of error due to DNS lookup errors.
If the site redirects to hide tracking URLs, using Javascript just gave you a 100% chance of an error.
When the browser requests the landing page. There’s a 5% chance of an error in log files due to cached pages for sites that set cookies.
Next, the browser renders the landing page, which can give you a 10% chance of an error when using javascript, and a 5% chance of an error when log file.
The more moving parts there are, the more chances there are of an error. The largest contributor to error is implementation of javascript. Logs are better for catching click fraud.
He then recommended the book “Fooled by Randomness” - Nassim Nicholas Taleb
Next up is Eric Enge (who handed out a copy of his Analytics players shoot off report).
Can you trust your data when analytics packages show different results?
He started off by walking through the report, showing charts to illustrate that the various packages al show different results, and there’s no consistency between the packages, one that counts high in one situation may not.
Sources of variance
- Bad or ambiguous data - Some data is thrown out. Packages make judgement calls
- Session tracking timeout - Industry standard is 30 minutes, some use 15 minutes. Is a new SE visit a new session?
- Many judgement call, implemented differently for each package
Sources of error
- Implementation problems
- Misunderstanding of terminology - Visit URL v Entry page URL
- Javascript placement - Issue is the delay before execution, users move on before JS executes, 1.4 second delay loses 2-4% of visits. Best practice: bottom of page
PPC tracking scenario
Cross check and calibrate - track your orders with other tools. Parameter on the URL for PPC viitors. Match up with orders on your backend
Focus on the strength of analytics - Keyword trending, etc
9 step recipe for success
- Accept that analytics is not perfect
- Know that trend analysis works
- Eliminate implementation error
- Learn the teminology of your vendor (may not be the same across all vendors)
- Focus on the strength
- Pick actionable KPIs
- Measure Errors - cookie deletion rates, delay to javascript execution
- Cross checkwith other tools and calibrate
- Use judgement
Next up is Jonah Stein. Jonah has installed 3 tools on one website to show the differences between the packages. His presentation is titled “The best way to measure ROI”.
Objective of analytics: Rational basis for decision making to maximize ability to obtain campaign objectives.
Choosing a package
- Prices range fom zero to a lot
- Integration, testing and deployment ranges from hours to lots of hours
- Each package makes their own assumptions that affect results i.e. Clicktracks only captures revenue for the first transaction in a session.
- Results require interpretation
What do you compare?
- What is your baseline for comparison? Visits? Unique visitors? Page views?
- Conversions!!!
Be aware that different analytics packages may use different timescales (use GMT rather than EST).
Audit Conversions
- Unique identifier for each order
- Create 2 tables
- Join all that match
- Add all unique invoices from each table
Keyword Specific comparison auditing for PPC
- Join tables at the keyword level
- GA will not give you a keyword level report, Clicktracks will, as will IndexTools
- Look at each keyword, you’ll see a difference across the different packages
Conclusion
- Overall results are fairly close
- Tune analytics systems
- Follow the warning labels
- Analytics vendors do too much
- Analytics should not be relied on for ROI calculation
Best way to measure ROI
- Adwords conversion tracker with a 30 day cookie captured te most conversion
- Use a CRM system or orders - product database
- Save marketing data at the earliest touch
- Incentives to determine source at evey touch
- Capture campaign and keyword to your own cookie
Rounding out the presentation was the final speaker - Avinash Kaushik. His presentation talked about the history of analytics, from logging to packet sniffers, to javascript. Javascript is not the option that most people like
People are not paying for web analytics, not because of the free tools, instead it’s because of silos - a lack of business data integration, there is also a general lack of perceived ROI.
So where are we? We are in the very early stages of analytics.
Overview single soundbite for each vendor
- Omniture - Beyond clickstream analysis, automated Action “taking” - bundled data warehouse
- Webtrends - Marketing optimization, give us your search spend - bundled data warehouse
- Visual Sciences - HBX: Custom excel reports, God’s gift to analysis
- CoreMetrics - Got Retail? Lifetime Individual Visitor Experience
- Indextools - Custom reportig, anything by anything, Enterprise for the cost of little
- Clicktracks - Ease of Use, Unleashing the power of segmentation
- Unica - Custom reporting, multichannel campaign tracking
- Microsoft - Visual, free, demographic segmentation
- Google Analytics - Data Democracy, Best of breed search analysis
The WAA has released today 26 new Web analytics Definitions to attempt to standardize terminology.
