Once you get past the trending hashtags of the latest soccer star or Miley Cyrus update via social media, it is hard to ignore the drums beating a new path for companies called “big data”. In some circles big data and the IOT (internet of things) almost become a blur of inter changing terms concepts and ideas.
To address many frequently asked questions (FAQ) about big data and what impact and benefit it can bring to your business, I have compiled this list to help you define big data for your own company.
Big Data: According to Wikipedia the term is used to describe large and or complex data sets that current data processing applications are inadequate.
Why should I bother with big data?
Today it becomes ever more possible to support “gut instinct” with predictive analysis which can highlight and verify correlations, hidden trends and therefore allow improved decision making platforms.
Will my desktop computer and servers need to be upgraded?
Yes and no. Big data often needs to make full use of hired cloud data processing solutions and yet local servers also often need to be upgraded to support the development and be capable of processing its part of the process.
Is this just an upgrade of my business intelligence system?
No. BI (business intelligence) is a great tool for describing existing firm data and helps identify from this data certain trends. Big data takes these principles to a new level. Now think of having the ability to seek an additional axis for your data, define casual effects and use predictive analysis to produce forecasts on less quality data but higher volume sets. This can provide a deeper insight to behaviors and relationships.
What are the main components of Big data?
- Variety : A range of category information sets where data has been obtained and gathered from i.e. data gathered from your local excel sheets, social media, industry specific white papers.
- Velocity: This refers to the speed of data generation, i.e. 1 data measure per 10 second or 10000 data measures per second.
- Volume: The sheer size of data used, compiled and used. i.e. think in logarithmic scales. If you took 30 incremental steps you may have traveled 30 yards. If you take 30 logarithmic steps you get to the moon.
- Veracity: Quality. We know the cliché of “crap in crap out”, this remains valid and needs to be controlled, considered and managed as part of the output and decisions made through big data use.
- Variability: Consider that not all the data will be gathered in robust consistent manners. You may have quality data but the frequency of measure or simply missing periods of data may exist.
- Complexity: When dealing with many different data sources there is the challenge of data management – hence complexity
So what does this mean to my business?
Your business needs to consider what scale and impact can and will big data have in your future business model. Will you be able to supply data that is consistent with the themes above? What changes to your data accuracy, formats and sources need review and change?
What is your market, competitors, consumers doing with big data?
The answer to the above questions will no doubt be a changing picture so retain and open mind and expect even more change over the next few years.
Let me know your questions, fears, experiences with big data or write a comments below.
JAMSO blog samples 