Towards Big data in a big way - an introduction

In the modern times, people all over the world are increasingly creating digital information through their usage of cellphones, digital cameras, social media websites like Twitter, LinkedIn, Facebook,etc. The digital data deluge is on the rise at a very fast pace. Online streaming of data is a very common thing to do over the internet, which primarily needs huge data transfers across networks. It has been estimated that almost 90% of the digital data residing in the world of ours was created in the past two years itself. Also most of this data (85%) is of the unstructured kind of data.

Few things which might not be known to few, regarding digital data are:

1) Internet has a huge memory and it literally forgets nothing, absolutely nothing - anything and everything that gets created digitally, gets stored in the internet, either through different storage mediums or in raw form as a unstructured stream of ones and zeroes (which can be retrieved as meaningful information when needed) in the network/ memory of the network elements.

2) Privacy issues are increasing with the addition of different internet applications - Marketers and security agencies are extremely interested to monitor practically everyone for their ulterior motives. NSA has been allegedly reported to be following secretive practices to keep record of all the online activities of world citizens over the internet. E-commerce websites use the information content of the web browser cache to understand the consumer behavior and consumer buying pattern.

3) All the big data are getting analysed and collected by different companies at various geographies and capacities - with the increasing demand for analytics among the corporate and industry trend analysts, different companies are increasingly looking for different ways to understand their customers better. While social media websites are a rich source of such information, other Web 2.0 based websites are also important.

With the growing reach of internet and related technology, more and more data will be generated in future and analyzing such data can help the companies to reap tremendous amounts of profits. Data scientists are in huge demand now-a-days and in a way they hold the key to success for many businesses.

Big data have three unique properties of Volume, Variety and Velocity. Along with this time-stamp and location information adds meaning to the data. Analysis of big data is gaining importance primarily because of the fact that unlike analysis of sample based data, analysis on big data takes into effect the entire set of data (as is) and hence all sampling errors can be avoided.

Photo courtesy: intellihq