RETURN ON TECHNOLOGY: Testing is secret to worthwhile Web site tinkering

I’ve just been reading yet another article giving advice for building commercial Web sites. It’s by a recognized authority named Saul Carliner, at Boxes and Arrows (www.boxesandarrows.com), itself a pretty venerable site for Web-site builders. The article draws parallels between brick-and-mortar retail stores and Web sites. For example, Carliner recommends that your site give some personal customer attention, because that’s how retailers like Nordstrom have become so successful. Store success, Web success. The implication is simple. Except that it’s not.

Carliner’s article is just one piece of a choking swarm of advice by people who don’t have to make profit from their sites. Carliner, for example, would like you to consider making your site into a user community, into a “destination” instead of a simple Web site, and regularly redesigning your site. In all, he has nine big suggestions for you. All of them are right, and all of them are wrong.

Any suggestion might work. But then, if you walk in the wrong direction long enough, you’ll eventually end up across the street, too. What almost every wise sage of the Web leaves out of his or her advice is testing. The fact is, if you tried their advice and tested the results for both customer satisfaction and profitability, most changes would be neutral or negative, which means most changes are expensive failures.

Few designers design sites the way engineers design bridges. The sad reality is that most Web “design” is mere blind fumbling. Somebody gets an idea for a cool thing to do and puts it up on the site. Then somebody else decides on a redesign a month later with some other cool thing or another, and puts it up. And so it goes.

This is tinkering, not business. I’m convinced that if you don’t measure outcomes of your actions, you’re in a hobby, not a profit-making enterprise. And if you listen to gurus, designers and fluffmeisters without having a testing mechanism to distinguish the hits from the hurts, you’re not serious about your Web assets. As they say, trust, but verify.

Web analytics are just the Web version of business metrics. You probably measure business operations right now. If nothing else, your accountant will make you come up with simple financial metrics. Successful businesses generally measure things that matter most to them, such as shipping or packing rates, machine capabilities or quality.

Web analytics extend those measurements to Web sites. Modern sites overflow with things you can measure, aside from the superficial figures like the number of pages viewed. You can also see how users flowed through your site, how many clicked on things you wanted them to click on, and where they came from. When it’s time to take some advice, reshape your site, then test the result. Don’t neglect the second step. And decide in advance what you’re going to measure, so the test is fair.

For me, rather than extracting Web lessons from mall walkers like Carliner, I prefer to learn from researchers like Paco Underhill. Underhill is the author of “Why We Buy-The Science of Shopping.” He pioneered the use of observing actual shoppers in retail stores to see how they actually behave. His cameras have revealed amazing facts about how customers move through stores, what they see and how they like to interact with merchandise.

He carefully documents all his research so it can be analyzed and checked in several ways. He completes the ideal consultants’ loop: examine, suggest, implement, evaluate. It’s that last step, evaluation, that too many Web-site owners don’t take. For Underhill, what matters most is what real people do. Web designers could do worse than take his advice as a starting point.

For example, he has researched the differences between male and female shoppers. While there are variations within each group, the differences between the ways men and women shop are profound. This difference might be enough to affect the way men and women react to an ecommerce site. You can read more about Paco Underhill at www.envirosell.com.

When Underhill is working with a particular store that has made his recommended changes, he checks the results to see if he’s predicted correctly. Any Web site owner should do the same thing. Granted, it’s harder to do. Web logs don’t show whether a visitor is male or female. In general, the identities and behaviors of Web site visitors will be murkier than those in a retail store. But it’s still worth completing the loop wherever possible. Trust, but verify.

*

Altom is owner of Element Associates, a technical communications consulting firm (www.elementassoc.com). His column appears every other week. He can be reached at tim@elementassoc.com.