SNIA Europe - SNIA Europe Blog

Advancing storage & information technology

SNIA Europe Blog

Introduction to SMI-S

An excellent introduction to SMI-S here from Jeff Goldner.

Click here to read Introduction to SMI-S

Showing 0 of 0 comments

SNIA Europe Question:

What is Big Data? Is it the next new fad? Or is it real?

Big Data - Fad or Reality
from Martin Glassborow
@storagebod on Twitter
http://www.storagebod.com/wordpress/

The biggest problem with deciding whether Big Data is Fad or Reality is defining what Big Data maybe; it is certainly the case that we are storing more and more 'stuff'. This range of stuff is everything from those funny videos which we all get sent by email to the data that is stored in CRM databases to the log-files that we collect from our web-servers. Arguably if you put that all together, you have Big Data!

Now what is becoming more apparent is that we are not talking about Big Data as stuff but what we are really talking about is Big Data Analytics; taking all that stuff we have stored and deriving information from it.

Of course this is nothing new, nearly 20 years ago I was working with a Teradata warehouse that utilised an extract from the main customer database for a retail bank. This was used to target marketing and also as a way of detecting fraudulent account activity.

Then later on, I worked with big Petroleum companies analysing large data-sets as part of seismic processing applications.

These all used MPP techniques to process large quantities of data (well large for the time) rapidly; these same techniques are are coming to the fore as part of Big Data Analytics but instead of it being a specialist field, it is becoming a more widely used and common business tool.

So something which has been around for over 20 years cannot really be said to be a fad but it might just be time for this dog to have it's day.

Return on Investment for Big Data Analytics is going to be the key metric in this; if you are in the commercial world, this is about commercial advantage from asking better questions; if you are working in the public sector, this is about better outcomes from asking better questions. Where it has been traditionally used has had quite simple questions; is there Oil there, is that spending pattern unusual, do they have lots of money in their current account etc; where we are moving too means the questions will be sometimes be fairly abstract to start with and will need honing down. Better questions should result in better answers.

However I do have some concerns about how effective this all might be as a general purpose business tool. It'd be interesting to compare the results of Big Data Analytics with the results of Common Sense or even Expertise; if you analyse huge datasets at a huge cost and make no better decisions than that of an informed person or group of people, you have gained little.