Sunday, 8 July 2012

Data Quality - S.E.P. ??

Douglas Adams' book Life, the Universe and Everything waxed lyrical about a fantasy piece of technology called a S.E.P. field. This extremely useful piece of kit was used like camouflage. It enabled the characters to land on Earth in a massive UFO without anyone noticing.

The S.E.P. field did not make you invisible. Everyone could still see you perfectly. It just made them totally ignore you by convincing them that you were just somebody else's problem.

Amusing, it may be, but it is based upon a large body of philosophical and psychological studies that suggest people, organisations and whole societies can completely disconnect themselves from recognising a critical issue - even though it is staring them in the face. Here are some well known reasons why you may be the last to hear about a data problem in your organisation.

Optimism bias - This is a self-serving trait where people spot a potential risk, but ignore it by assuming it will not happen. The reason why they do this is because it allows them to continue on their chosen pattern. e.g. - Smokers are less likely to think that cigarettes will give them cancer. This bias is most often seen in high-pressure areas where risk-taking is rewarded. 

Diffusion of responsibility - The colleague is less likely to report a problem because they do not see it as their job to raise it. This is most common in hierarchical organisations where roles are very sharply delineated or processes are very mechanistic and rigid. It is perceived as too much trouble to deviate from the norm.

De-motivation - The colleague does not see the benefit in reporting or fixing the problem. It's far too much effort for something that they know will not help them in the long run. They may even think that fixing it is impossible. De-motivation is a symptom of low self-esteem, cynicism or excessive work loads.

Herd mentality or bystander syndrome - If no-one else is dealing with the problem, then why should the individual? I call this "collective irresponsibility syndrome". It is usually driven by fear of being punished for standing out from the crowd.

Your data quality measures can't be everywhere. Sometimes you have to rely on your colleagues to keep their eyes and ears open and report what they know. Be aware that it can be a challenge for people to come forward and say something has gone wrong - even if they know it wasn't their fault. Things you can do:
  • Start a no-blame culture
  • Encourage colleagues to take personal and social responsibility
  • Build a customer centric ethos
  • Allow people to report the problems they discovered in different ways. This enables them to choose the most comfortable  method of reporting
  • Recognise colleagues for spotting data quality issues

So is there a S.E.P. field around your quality issues? 


  1. Bravo, Rich!
    If we know WHY the S.E.P. field works, we can devise get-arounds! Might I add "Learned helplessness" and "No words to describe" to the list of reasons.

    - Optimism Bias
    - Diffusion of responsibility
    - De-motivation
    - Herd mentality
    - Bystander syndrome
    - Learned helplessness
    - No words to describe

    Are there others?
    Gwen Thomas @gwenthomasdgi

  2. Thanks Gwen. I am happy to add your categories when we get a more complete list. Let's keep this going and see if we can bottom this out with a complete list of strategies. Perhaps just giving them this list might help.



  3. Hi Rich
    Great post, and anytime you can cite Douglas Adams you know you're probably on the right track!

    I like Gwen's additions and I can recommend one more, "Once bitten, twice shy".
    I think this is a key factor that creates more and more "de-motivated" individuals. These are the formerly passionate data evangelists that actually witness a problem, and shockingly try to do something about it! They raise the alarm, try to get sponsorship and priority to mitigate - and get no support whatsoever. So what happens next time a new problem comes along? They'll look the other way.

    Rob Karel

    1. Hi Rob,

      Totally agree... as well are the ones who tried to fix a problem without doing it right and make more problems for themselves. They're all pretty big de-motivators.

      Thanks for contributing.