4 New Years Resolutions For Your Data

It is traditional - at this time of year - for everyone to think about what they have been doing wrong in the previous year and come up with some resolutions as to how we can make our New Year better. 

For some, this might be a new fitness regime, a change of job, more time with the family. For others, it could be as mundane as a new hair style. For me, I think I need to look at my diet.

So how can you make it a good year for your data? Here are my top tips to make 2016 a happier year for you and your data:

1.  Make data your priority:

For many businesses, the direction has been about platforms, systems and sales. Data has been collected and moved from one system to another - each time believing that the new system will magically fix the data that it carries. Make 2016 the year when data is valued as a strategic corporate asset, and more importantly, the reason for why the system is there. Make 2016 the year where the data is fixed and optimised. Make an effort to fill the missing data left by inefficient system migrations. Route out default and wild-card entries and engage everyone to get those gaps filled.

2. Re-negotiate with your suppliers

How about raking over all of the 3rd party data suppliers' contracts? Speak to the consumers of the data and find out if there are any common problems. Make a schedule for when the contracts need to be renewed, and see if you can set some data quality standards into the new contracts.

3. Make 2016 a year of documentation

You have your procedures, you may even have some entity diagrams knocking around. But do you know your data lineage? Are your data domains known and controlled? Are your colleagues aware of their responsibilities and accountabilities? Make 2016 the year when the documentation becomes as important as the systems. It always makes me laugh when new architects join a company and immediately start asking everyone about the systems. That tells me that the documentation is not up to scratch.

4. Stop trying to sneak data governance into unrelated projects

Make 2016 the year when good data governance becomes business as usual, rather than trying to shoehorn short-term change on the back of a large-scale programme that has better things to do with their time. Lets campaign for more investment in permanent jobs in this sector, and give our colleagues the world-class support that they deserve.

I hope you find these suggestions interesting. Let me know if you have any more.

To software, or not to software

The oldest computers were not easy to use. Every time they were switched on, they had to be programmed before they could be operated. My first computer was a Sinclair ZX81. It had 1 kilobyte of RAM, and no hard drive. You had to either write commands and programs in BASIC, or you had to load a program from a cassette tape.

Compare this to a modern tablet computer, and the difference is astounding. You switch it on, and right away you can perform tasks with the glide of your finger across the screen. This is all due to the magic of modern software.

Software are computer programs that enable a user to carry out tasks without having to program the computer first. So, your computer/tablet/phone's operating system (Windows, OS-X, Linux, Android, Unix, Chrome etc.) is software. You will also have a number of programs that you may have added to help you do common tasks, like image editing or playing videos.

However, when you take a serious look at what a lot of the software is doing (particularly when reading that huge license that appears when you first install the software), it is not hard to come to the conclusion that not all of it is ideal!

• Many operating systems share your data with government agencies (Snowden revelations).
• Google/Andriod uses your data for advertising purposes.
• Apple's digital rights management means you cannot play their movies using non-apple software, or using non-apple portable devices.
• Some Android apps collect your location data and contacts details. 
• Many specalised programs have their own proprietary formats that cannot be used by other software.

So it is clear that not all software (particularly when it comes free) is working in your best interests. Even when you pay, you may find that the commercial interests of the company that created your software is enforced.

The most common large scale problem I have seen, is when a company buys an operational system, only to find out that they cannot easily obtain management information from the data, due to vital data being over-written or not saved. They have to go back to the software company to buy their proprietary management information layer to fill in the gaps.

I have also seen projects derailed because a supplier would not share their entity, or how they were going to administer their process.

So to be truly free in computing, it is clear that you need to be able to write your own software using a programming language (Java, C, Python, etc. etc.). A programmer can deliver bespoke systems that enable your organisation to do exactly what it needs to do, without conflicts of commercial interference from suppliers.

Many organisations with vision are now seeing that we need to be generally a more tech savvy society. The BBC, the UK's government, Google and Barclays Bank are just a few organisations that are investing in programming education for children. They now see that programming is a high value skill that benefits everyone.

Data - who cares?

Your data is important to your organisation. Everyone is clear on that. Your comms strategy is going great. You have profiled all of your important data sources, built scorecards that highlight where the bad data is. You take your reports to key people and ask for resource to fix the data. The reply is "so what?"

If there are no accountable owners of the data, you are going to find it hard to actually fix things. The buck has to stop somewhere, and if things are going wrong, someone has to put their vanity projects on hold and get their hand in their pockets. So when the rest of the organisation is in flight, how do you go about assigning data owners - and making them face up to their responsibilities?

1. Go to the top

Get buy-in from the most senior people you can find. Use their names, whenever you experience inertia.

2. Categorise the 4 Cs
In all organisations, data is Created, Changed, Controlled (including retirement) and Consumed. Creators put data into systems. Changers make amendments, Controllers monitor data and perform mandatory work, like data retirement,  data quality and data governance. Consumers are people who use the data. They are important, as they are instrumental in setting standards for data quality. Very often a natural data owner will arise, as they will have oversight over all of these people and control their budgets.

3. Categorise your data on its nearness to source

Your data is either Primary, Secondary or Tertiary. Primary data sources are operational databases. Data goes directly into them. Secondary data sources are data marts that are created from primary data sources. They are for a specific purpose. Tertiary data sources are created from Secondary or other Tertiary data sources. Why is this important? Your initial focus must be on getting ownership of Primary data sources. Very often, people will obfuscate by drawing attention to secondary and tertiary use of primary data. Do not get sidetracked until all of your primary data is owned.

4. Categorise your data from a business perspective

Look at how your business is structured. Categorise your data so it can fall under the present organisational structure. Very often, data owners will arise from this process.

5. Record what the owners are accountable for

Very often this is missed. Document what they are accountable for. Have the accountabilities written into their role profile. Have their performance measured against these accountabilities.

Acquisitions in Data

It has been recently announced that Informatica (a software company that provides data management tools) is to be acquired by Permira Funds and a Canadian pensions investment fund. I have been witness to many takeovers within the financial services sector. It is one of the inevitabilities of our society. Whatever it is... if it's for sale, someone will buy it.

I have seen companies purchased for many reasons. I remember one software company was purchased by their customer because they were due to renew their licenses, and it was cheaper for the customer to acquire the company.

While the takeover of Informatica has been received positively by their customers, there are definitely some things to consider when one of your tech suppliers is taken over.

• Will your supplier still have the same ethical considerations?
• Will they stay committed to providing service to your business, or are they planning to move to another sector?
• Will they be asset stripped?
• Are they planning to deploy with different technologies (forcing you to acquire more software/hardware to remain compatible)?
• Will they be changing release cycles, and how will that affect you?
• How will the process for engagement with them change?
• Will their prices go up or down?
• How will any relocation plans affect you?
• Will their standards of data quality suffer?

Supplier risk management is an emerging discipline as modern companies become increasingly dependent upon each other for their success.

The key message is:

"Don't wait until you read about your supplier problems in the newspapers."

Windows 10 review

OK, I've been pretty scathing about the windows 8 operating system in the past. Being a power user who does not have a touch screen on my PC, I found the metro user interface problematic. It appears that I was not alone in my opinions.

So when the beta preview of Windows 10 came around, I was intrigued as to what Microsoft's reaction would be. Would they learn and evolve from the public reaction, or would they simply keep digging? Given that they had invested a lot on development of the Microsoft Surface platform, their response will probably say a lot about their future commitment to PCs and standard laptops.

I had  3 main issues with Windows 8 - the tablet oriented user interface, the over-engineering of screen navigation that seemed to slow the experience down, and the general weightiness of the operating system. It seemed stable and reliable, just too cumbersome and onerous to work with a keyboard and mouse.

Firstly, I must say that my testing was done on a virtual machine. It installed flawlessly, and set up a Microsoft email address. The setting up was logical and simple to follow, just like the Windows 8 operating system I last used. No surprises.

I was warmly greeted by the familiar sight of a start button and menu bar. Microsoft pioneered this type of user interface with Windows 95. Almost all other operating systems use a variation of the start menu - with good reason. It works. The start button was the one thing I missed when I moved over to Apple. Clicking on the button revealed a broader start pane, with aspects of the metro panes integrated into it. So the business community is going to be pleased about that. They will get the traditional user interface that won't cause compatibility issues with their proprietary applications. It also works great with vmware (I tested on 'Boxes' and 'VirtualBox').

The general feel of the operating system is one similar to Windows 7 in layout. But there is one trick up the sleeve.. Windows 10 is very lean. I was able to dial the ram down to 2gb, and the menu and explorer still ran quickly. This is going to please another part of Microsoft's followers - PC gamers.

So all in all, from what I have seen so far, Windows 10 is a good step for Microsoft. But this is only the beta preview, and a lot can happen between this test and arrival on the market. So here is my advice to Microsoft about how to keep this improvement going:

1.  Engage with manufacturers to make sure they don't bundle their PC/laptops with ridiculous proprietary applications that can already be done by Windows (media players and disk burning suites, I'm looking at you!)

2.  Find a way to reassure business and retail customers that their data is safe from government spying on a Windows machine.

3.  Engage directly with as many service providers who can deploy over the internet - banks, cloud providers, software developers. There is still a lot that can be done to make Windows into a platform that can integrate data, systems and services for everyone.

5 reasons why they won't let you have the data

The appropriate sharing of data and information is key to any organisation's success. One department may be collecting data that is useful to other parts of the business. 

When you ask for that vital data, many barriers can be placed in your way. One problem may be resolved, only to have others magically appear. (i.e. the firewall won't let the data through, its not their responsibility, hiding behind data protection and data governance, you haven't ticked the right boxes etc. etc.)

When more than one barrier appears, it's time to consider why this is happening, and it's not always pretty. The following are the top reasons I have experienced (in my previous roles) as to why your colleagues are putting you off access to their data.

1.  It's not all there

Yes, some departments stop collecting data and let the rest of the organisation think they are still collecting it. This can happen for many reasons. Perhaps the budget went on something else, or there were cost cutting exercises that affected third party contracts. Sometimes data is lost, due to errors, or key personnel leaving, along with the knowledge on where it is. It's important that you get to the bottom of this.

2.  The quality is poor or unknown

It's very hard for colleagues to admit that the enormous spend on capture has resulted in incomplete or inaccurate data. Sometimes, not knowing the quality can be more disruptive. Colleagues may choose to refuse you access, rather than letting you have the good quality data if it was known.

3.  They are unsure of the measures that use the data

This is by far the most common problem. They won't let you have access to the raw data, because they are concerned that you may find flaws in how their own measures work.

4.  They are protecting job security

Many departments will fear their jobs may be lost if they hand over the monitoring of data to another team. It is important to state the nature of your business, and define the limit of your scope to allay any concerns in this area.

5.  They are covering up catastrophic losses in the business

Never rule this one out. It happens more often than you think.

The function of a data governance department is not to act solely as a blocker. A fully mature data governance team should also be your advocates, cutting out red tape and enabling you to have the correct data to do your job. If you experience any of these problems, contact data governance. If you don't have a data governance team, isn't it about time you got one?

Data Meme

Really great meme here that all programmers and data geeks will appreciate.