Seems like a funny way to start a blog post on a blog that purports to be about the value of data, doesn’t it?
Those of us who work with data sometimes have to be reminded that we cannot always take data on blind faith. Data can sometimes mask the truth or intentionally mislead. But, it’s NOT the data’s fault. The way we collect data is just one of many reasons why the numbers sometimes lie.
I was reminded of this fact this morning at my local Starbucks. A difficult customer was before me in line; I’d seen her there before – you know who I mean, the one asking when the prices had increased (they hadn’t) and saying that she’d meant to order soy milk in her latte (she hadn’t) all the while looking disdainfully at the Barista’s spacers in his ears!
and make sure they understand your motivations.
When it was my turn, I ordered my Grande Bold with a smile and was handed a slip of paper and told “you have been selected to answer the survey; go to the website and tell us about your experience today and you’ll get a free drink.”
What a lucky break for the Barista that it was me who got the survey, not the complainer before me. Luck or not? Strangely, my ‘survey’ did not come out of the till with my receipt but was already printed and on the cash register. Hmmm… perhaps this survey was intended for the previous customer, but the Barista overrode the random selection in favour of someone who might give a more glowing review? Maybe I’m a cynic (I can be, sometimes!), but maybe getting those good reviews is directly correlated to the Barista’s performance reviews.
I remember another case where post campaign analysis indicated an unexplainable very large response group in New Brunswick. It was only after many questions that we discovered the reps in the call centre were using their own postal codes in a mandatory field on the database!
Careful planning and training – along with a clear understanding from everyone, especially the front line – of the reason for and value of the data can avoid many of these problems. Put on your cynical hat, understand the motivations on the frontlines and make sure they understand your motivations. The Barista may simply have wanted to “reward” a good customer with a free drink – rather than taint the survey results.
I bet you have a few stories of your own of when data “went wrong” due to something that happened in the field. We welcome yours here!
Funny, as it reminds me a similar experience.
I was overviewing a survey for a reknown Canadian coffee distribution brand.
The company came back stating that the results were off compared to their own sales numbers.
Make the long story short, we figured out that the issue was with the survey. The survey was coming from the US. There is a big cultural difference between Canada and US coffe consumers: Americans always eat with their coffee, when Canadians like their “double-double” alone. The question was refering to consuming coffee “with a snack”, hence the biaise in the answers, understating the real consumtion. We had to rewrite the question, and make several tests before we got the question appropriately written.
Good learning experience.