Lies, Damn Lies, and Statistics

The other day, I was on Yahoo, checking my group mail, and they asked me to take a survey. They showed me several different Yahoo search results and asked me about my assumptions when I saw each one. But they never did ask me what I thought was the main question — namely, would I ever in a million years be using Yahoo search?

It’s been a good long time since I’ve participated in a decently designed survey. I did a write-in answer during the infamous poorly-written piece of crap survey about “romance being one man, one woman” (leaving aside the clearly political bent to the survey, it said nothing about love triangles, or about shutthehellup and give me a love story).

A few months ago, eHarlequin held a survey to ask what we thought about Blaze covers. Maybe you participated. They showed us two different Blaze covers. One had the standard splash of red silk, across which, the “Blaze” was scrawled in a quick, italic script. This is the cover you usually see. The other one was truly gorgeous, a much more subtle “category” look. I thought the design was fantastic. HOWEVER, the font they used in their subtle hint that it was a “Blaze” was some sort of digital computer font. It looked like the kind of font you see on scrolling highway signs or 80s computer screens. Not sexy at all. And then they asked us which one we liked better. I picked the current design, because digital computer font does not say “sexy, passionate read” to me. Had they showed me the screens without font, I would have definitely picked the more subtle design.

I heard that the results of said survey told the powers-that-be that people wanted category books to look more like category books. No, they said that people didn’t want their sexy reads to look like Toshiba instruction manuals!

You can’t go to eHarlequin without seeing a survey. Do you like SEALs or FBI agents? Buffy of Sidney Bristow? Category books or Single Title? Trade or Mass Market? I used to answer them without a thought, but now I wonder. Are they actually assuming this is market research? Are they going to the editors and saying “Buffy scored a hell of a lot higher than Alias, so stop buying spies and stock up on vampire slayers,” when what I was saying when I chose the survey was that I preferred Joss Whedon’s sense of humor and skill with creating multi-layered ensemble casts to the frenetic and single minded plot line of Alias? I’ll watch anything Aaron Sorkin writes. Doesn’t mean I particularly like sports shows, political shows, or shows about SNL. I like Aaron Sorkin’s characters. I never watched that Geena Davis-as-president show. I wasn’t watching The West Wing for the politics.

I think I need more surveys where they asked me why I picked what I picked.

When I worked as a waitress, I learned that those customer satisfaction surveys they give out at restaurants, hotels, and car dealerships are often scored on an all or nothing basis. They’ll give you a hseet at which you rate your service on a scale of one to five (5 is highest) but when they tally the scores, anything that’s not a five is given the same weight as a 1. This shocked me. I know a lot of people who don’t give out fives unless you’re serving your first born on the plate to the customer. Why “very satisfied” could possibly be equated with “very disappointed” because it was “superlatively satisfied” is beyond me. And let’s not even talk about people who give ones because they thought the food was too expensive. It reminds me of the Amazon reviews where they give the book one star because Amazon lost the shipment or sent the wrong book.

I don’t know if I trust market surveys. There are too many variables, design flaws, idiot respondants, and worst off all, it leads to mediocrity. It’s like talking about writing contests where an entrant with a strong voice scores at the top and bottom, but is beat out by someone who scores in the middle, and wins because the story offends no one.

Which do you prefer?