wp improving online experience

wp improving online experience - Transcript

white paper

No More Guesswork A Better wAy to MeAsure uNderstANd ANd IMprove the oNlINe CustoMer experIeNCe
suMMer 2007

Executive Summary
Great companies of all kinds work hard to provide an excellent customer experience Leading Web companies are no different but the technology they need to monitor experience levels online has been slow to develop and frequently flawed Fortunately a new approach to understanding the online customer experience is now available Utilizing a method that monitors the user experience from inside the browser this technology promises to help companies Analyze the actual experience of specific customers and segments and understand which buyer segments have poor performance and why Know which variables on a company s website contribute to or detract from a favorable customer experience including third party content providers use of multiple browsers and rich media types and Make better decisions as to what Web components to monitor and improve by analyzing performance in the way that matters most as users experience it from inside their browsers This white paper will help business and IT managers learn how browser tools work to document the customer experience enabling managers to take proactive steps to increase revenue reduce risk and improve customer satisfaction

whIte pAper

No More Guesswork A Better way to Measure understand and Improve the online Customer experience

Contents
Introduction 3 Three Passive Approaches 4 A Closer Look at Browser Based Monitoring 4 Combining Active and Passive Monitoring 7 Conclusion 8

The information contained in this document represents the current view of Gomez Inc on the issues discussed as of the date of publication Because Gomez must respond to changing market conditions it should not be interpreted to be a commitment on the part of Gomez and Gomez or its respective suppliers cannot guarantee the accuracy of any information presented after the date of publication This white paper is for informational purposes only GOMEZ AND ITS RESPECTIVE SUPPLIERS MAKE NO WARRANTIES EXPRESS OR IMPLIED AS TO THE INFORMATION IN THIS DOCUMENT Complying with all applicable copyright laws is the responsibility of the user Without limiting the rights under copyright no part of this document may be reproduced stored in or introduced into a retrieval system or transmitted in any form or by any means electronic mechanical photocopying recording or otherwise or for any purpose without the express written permission of Gomez Inc Gomez may have patents patent applications trademarks copyrights or other intellectual property rights covering subject matter in this document Except as expressly provided in any written license agreement from Gomez the furnishing of this document does not give you any license to these patents trademarks copyrights or other intellectual property 2007 Gomez Inc All rights reserved Gomez Actual Experience XF the Gomez ExperienceFirst Platform and the Gomez logo are either registered trademarks or trademarks of Gomez Inc in the United States and or other countries The names of actual companies and products mentioned herein may be the trademarks of their respective owners This document contains information of a proprietary nature All information contained herein shall be kept in confidence and shall be for the original recipient s use only Any unauthorized reproduction by any other party shall constitute an infringement of copyright

2

Gomez Inc

10 Maguire Rd Lexington MA 02421 3119 www gomez com T 781 778 2700 F 781 778 2799

whIte pAper

No More Guesswork A Better way to Measure understand and Improve the online Customer experience

Introduction
Top retail stores and leading service providers like banks and other businesses have long realized the critical need to provide a superior customer experience So it s not surprising that with so much business shifting to the Web the need to better understand the customer experience online is growing fast too Business managers closest to the customer are looking for ways to capture specific information about how real people interact with or experience their websites and site services Armed with this information they can Reduce the time it takes to understand the scope and impact of performance related issues on customers A Major Source of Downtime Application performance is one of the major sources of downtime and brownouts as well as the most difficult problem to resolve Resolving these issues in production by throwing hardware at them is the most costly and ineffective approach possible More often than not it will have the effect of aspirin on a broken leg temporary relief at best Forrester Research Validate third party provider effectiveness with an unbiased view Better understand the customer s experience of Web 2 0 rich media sites and Make better decisions as to what to monitor and improve Ultimately of course they want to champion incremental improvements that provide an excellent customer experience This in turn will drive greater revenues increased profits brand loyalty and repeat online business for their organizations technology in transition Rapid technology changes make it difficult for IT to keep up with the data demands Today s sites are a daunting mix of distributed architectures application accelerators smart clients fat browsers and third party content This complexity renders many traditional and more centralized measurement techniques useless for capturing the real customer experience Many of the Web 2 0 browser components that make a site compelling AJAX Flash animation and Flex to name a few are particularly difficult to monitor using the centralized server tools familiar to most IT shops Fortunately newer tools now available are specifically designed to measure and ultimately improve the customer experience The purpose of this white paper is to understand how these passive technologies work to solve the monitoring problems identified in the Web 2 0 world and how they differ from active or synthetic monitoring tools It also explores the benefits of combining both testing techniques to better understand the relationship between performance and the customer experience

three passive Approaches
The need for business to understand the customer experience is driving interest in passive monitoring today And for good reason passive techniques are effective at helping organizations better serve their customers Three types of passive monitoring technologies are currently available They are 1 hardware based appliances These devices work by reconstructing user sessions from traffic at the switch While hardware devices or sniffers are fine for monitoring the user experience of a simple site delivered from a single domain they cannot measure outside service providers Nor can appliances measure the user experience from inside the browser itself This rules out their use with rich and increasingly popular programming techniques like AJAX and Flash 2 the Internet panel approach Deploying a large population of actual Internet users to test websites is another passive monitoring approach The few testing companies using this technique typically recruit users who download a plug in for use in their browser While Internet panels have merit privacy concerns make it difficult for providers to recruit a sizable population of users willing to let a company monitor their Web surfing activities In addition questionable recruitment methods can introduce bias when companies scramble to get a large enough sample panel size Finally smaller regional sites with low traffic also pose problems for panels as it s difficult to gather meaningful results



whIte pAper

No More Guesswork A Better way to Measure understand and Improve the online Customer experience

the browser based approach One of the newer ways to monitor and analyze the user experience is to understand what the user is doing from inside the browser itself Employing a distributed methodology similar to the path blazed by Web analytics tools this approach uses code snippets which are inserted on each Web page Tags gather anonymous user experience information from inside the browser and data is sent to a central repository for analysis One of the first products using this technique is the Actual Experience XF offering from Gomez It is designed from the ground up for analyzing customer satisfaction and the user experience

A Closer look at Browser Based Monitoring
The rapid adoption of Web 2 0 technologies is quickly changing the way applications are made and measured Many companies view Web 2 0 as a way to distinguish their sites from competitors The good news is that well executed browser applications approach the interactivity levels and overall user friendliness of standalone PC applications At the same time as more code moves to the browser it becomes increasingly difficult for IT teams to measure performance That s where browser based tagging comes in Unlike the more limited sniffer or panel approaches a browser solution can provide benefits at the front end of the Web application development and QA testing process long before the application is deployed to identify underperforming Web 2 0 code based on AJAX Flash Flex and other tools This ensures the customer experience is built into the application from the get go so the cost of fixing a problem is drastically reduced First look at third party content Portal and content rich websites rely heavily on content delivery networks to accelerate performance Many also depend on content partners to contribute articles stock charts banner advertising and more Yet few companies effectively relate this third party content to the overall user experience As a result many Web managers still don t realize how much their brand is affected by the ways online visitors experience third party content Measuring how these contributors impact the customer experience in the browser is critical Consider for example the fact that the number of third party hosts used by each of the top 25 online retailers averages nine per website Most visitors to these sites don t know it but they are seeing composite pages composed of pieces and parts from multiple sources While some content originates from the site s central servers many other components are served up by third parties What s important for these retailers to understand is that each content component is in fact being delivered on time and at performance levels up to company standards A major advantage of browser based monitoring is its ability to measure the effectiveness of these third party providers With so much page content today assembled inside the browser locally rather than a central source it is the only technology that provides real insight on how well each content component contributes to the customer experience page performance Passive monitoring provides unique insight into how well a page is performing under a wide range of real user conditions A good example is the time it takes for the visible portion of a popular page to load In this case every user s experience will be different depending on such factors as their browser type and version window size connection type and speed Not knowing this information can make performance tuning difficult if not impossible for developers While the number of Web permutations is daunting developers have the ability today to focus just on the variables that matter to their customer base By understanding where actual users are clustered in regard to operating system geography connection type and speed developers can measure real world performance and to take steps for improvement A good example is the recent release of Microsoft s Internet Explorer 7 0 browser Many sites were not fully tested prior to the release and in fact broke down as many Internet users discovered first hand Browser based passive technology removes the guesswork For example companies using passive tools leading up to IE 7 could have tested their sites during the beta period and understood how many of their users would likely be running IE 7 after its launch Drilling down



Gomez Inc

10 Maguire Rd Lexington MA 02421 3119 www gomez com T 781 778 2700 F 781 778 2799

whIte pAper

No More Guesswork A Better way to Measure understand and Improve the online Customer experience

a level they would have known how well their site was performing on IE 7 desktops running Windows XP at various screen resolutions As companies better understand which variables constitute a satisfactory experience on each page sites can be optimized to better serve their largest and most important user populations Other performance monitoring techniques such as synthetic performance tests can approximate customer satisfaction based on load time averages Traditional Web analytics tools can also reveal behavior but say nothing about what the actual experience is like when a user interacts with the site Browser based monitoring is best suited to determine if there are related performance issues when an experience occurs Gain new customer insight Analyzing the actual experience of specific customers and segments can be of tremendous value to a company Take for example first time versus repeat customers Using browser based passive techniques companies can segment these two groups easily and analyze how well certain features are performing for each audience Understanding customer behavior and satisfaction as it relates to defined customer segments can be extremely useful Say for example that testing of a new cross selling feature using advanced hover over functionality reveals surprisingly slow performance on the desktop Follow up tests then indicate the problem is acute for repeat customers using the Firefox browser running under Windows XP Armed with this information management can prioritize the problem in relation to its impact on the business and act accordingly to fix it learn more about your traffic Business managers are often surprised by the rich amount of geographical information that browser tags make available for analysis Well placed tags on key pages can help managers segment customers by country region or U S state Knowing this companies can better understand how well third party content is received in key markets and which content provides the best value to customers Passive monitoring is also very good at measuring the impact of high traffic load at the user level and helps companies determine how many visitors are having a bad experience and when understand conversions and abandonment Equally important browser based tagging lets managers analyze conversions at a deeper technical level than before possible Making a purchase filling out a form or completing an account balance operation are all considered successful conversions But getting customers to convert isn t always easy Using passive monitoring managers can better understand how users react to every step in a conversion funnel and take proactive steps to tune each one for success The opposite of conversion is abandonment Often poorly understood abandonment rates measured in aggregate are important indicators of the user experience They can tell you if there s some point in your process that is causing user frustration Knowing for example that 25 of users quickly navigate away from your search page can be critical to the success of your business Understanding user satisfaction in relation to abandonment has never been easy to achieve Using browser based measurements however companies today can track the three main types of abandonment 1 2 Conversion funnel drop off This happens when users suddenly leave the buying funnel set up for them hard abandonment This occurs when users push the stop button in the middle of a transaction such as a shopping cart purchase and Interrupted loading This happens when users navigate away before the page completes loading

Prior to browser based measurements few IT tools could tell you how a user exits a page for example by closing the browser or hitting the back button Or how long it took before they clicked away By watching or tracking the user s actions from inside the browser companies can gain a firmer grip on satisfaction levels and deliver incremental improvements to their websites over time



whIte pAper

No More Guesswork A Better way to Measure understand and Improve the online Customer experience

Combining Active and passive Monitoring
Although its strengths are clear browser based tagging is not a replacement for the more traditional form of synthetic or active testing Testing a website on a timed incremental basis throughout the day using agent based testing scripts continues to be the most popular form of testing available and for good reason The need for IT teams to ensure that site availability and response times are under control must always come first and that s where synthetic testing excels Other important characteristics of active testing include deep diagnostics Active testing provides an incomparable set of deep diagnostics at the object level each of which is needed at times by technical teams to diagnose performance problems and implement solutions to overcome them Backbone testing Global testing across multiple networks and geographies provides the lab measurements needed to baseline performance and benchmark yourself against the competition Alerting While alerts can be triggered from either active or passive techniques synthetic sounds the alarm as problems are developing and when there are no visitors to the website and establishing contract baselines For enforcing Service Level Agreements SLAs and for collecting performance metrics synthetic testing is the most viable option putting one and one together As good as passive and active testing are alone even better is what s possible when the two combine in a single development environment Consider for example the money that leading Web companies spend each year on load testing While synthetic load tests have value no automated scripted test can predict with 100 accuracy what real users experience when a site comes under heavy load But when synthetic tests results are combined with real world user metrics from browser based passive tests managers receive a true measurement of load not available anywhere else today

Conclusion
Understanding the customer experience down to the smallest detail has always been a hallmark of top companies As more business shifts to the Web it s not surprising that leading Web managers are looking for similar information about their online customers By better understanding customer satisfaction levels companies are equipped to make incremental site improvements that over time to provide an excellent customer experience The ultimate business payback of course is brand loyalty greater revenues and increased profits Keeping an eye on the customer experience isn t always easy however Major shifts in the way websites are developed and the technologies used to develop them have made it nearly impossible to monitor customer satisfaction using traditional centralized management techniques Fortunately a number of new passive technologies are emerging and are specifically designed to measure customer experience While several passive solutions are available only one stands out a relatively new tagging based technology that analyzes the user experience from inside the browser itself Key areas where browser based monitoring is uniquely capable include Analyzing the actual experience of specific customers and segments Knowing which factors on a company s website contribute to or detract from a favorable customer experience including use of third party content providers and Making better decisions about what Web components to focus improvement efforts on by understanding how real customers and prospects experience the organization s site through their browsers Combining both actual user browser based and active monitoring agent driven tests provides a complete view of customers Web experiences providing Web managers with a 360 degree view of their customers online Web application experiences

Gomez Incorporated 10 Maguire Rd Bldg 3 Suite 330 Lexington MA 02421 General 1 781 778 2700 Sales USA 1 781 778 2760 UK 44 0 207 554 5154 Germany 49 0 40 53299 167 www gomez com

2007 Gomez Inc All rights reserved Gomez is a registered trademark of Gomez Inc The Gomez logo ExperienceFirst ExperienceFirst SCoE ExperienceFirst Network Active Network XF Private Network XF Active Last Mile XF Private Locations XF Actual Experience XF the taglines Ensuring Quality Web Experiences Quality of Web Experience and QoWE are trademarks of Gomez Inc All other trademarks are the property of their respective owners Specifications may change Consult the technical documentation for the most current information Some features described are extra cost options Ask your sales representative for details