Saturday, 14 December 2013

Big data - a conceptual example

I recently took my children on a trip to Jodrell Bank. For those who do not know, it is a very old radio telescope observatory in Cheshire that was build by Sir Bernard Lovell.

Although Jodrell Bank is getting very old, and is being overtaken by space based telescopes like Hubble, it is still a shining example of ingenuity and science in action, and it still has a vital role, gathering information about pulsars around the galaxy.

If we consider that science is the collection of data, it is surely best to acquire the greatest amount of data possible. This is why telescopes got larger and larger - to catch more data. 

But science wants to collect so much more data than just one conventional telescope can manage. So what they have done is to stop building large telescopes like the Lovell telescope at Jodrell Bank, and instead build arrays of smaller dishes.

An array is a set of radio telescopes that are controlled together and pointed at the same part of the sky. The information is collected from all of them and put together. Modern Big Data works like an array - because one large computer is just not good enough any more. Data has become too large, varied and complicated for it.

Big data solutions are arrays of computers that are joined together to process very large and complex data sets. They employ special hardware and software to ensure all of the work is shared across the computers in the array. The end result is that huge, complex data sets can be processed much quicker than before.

No comments:

Post a Comment