Big Data

Big data is a term used to describe massive amount of data in both structured and unstructured form that includes day-to-day basis of all business transitions. The amount data that’s being stored on global level is very huge volume and difficult to process, isolate and implement in real-time basis but possible using some latest data tools.

Despite of these problems, big data has potential to help organizations to improve their operations and faster and more intelligent business decisions.

This big data concept is first articulated by Doug Laney, an industry analyst in the early 2000 and this comprises:

• Volume
• Velocity
• Variety
• Variability
• Complexity

Let’s look into brief of these: 
Volume: Organization collects and stores huge volume of data from various sources – business transitions,      social media etc. But storing this huge volume of data become big task for most of the organizations and new technology such as Hadoop have solved this.

Velocity: Data collects in an unprecedented speed and it must be dealt with timely. Maintaining real-time basis.

Variety: Collated data could be number, strings, and images, audio or other format in structured or unstructured.

Variability: increasing velocity of data flow and it is highly inconsistent with time.

Complexity: It is difficult to match or transform to other system.

Who uses big data?
Most of the Multinational Organizations / Banking / Educational institutions / Government / Health care / Retail / Manufacturing industries manage the big data for their day-to-day business transactions.

When massive datasets are dealt, organizations face difficulties in creating, manipulating and managing big data.
This is particularly problem in business analytics because standard tools and procedures are not designed to analyze massive datasets.

How this big data can be stored?

Maintain huge data storage units.

• Use faster processors
• Use open source platform such as Hadoop
• Parallel processing, cluster based, virtualizations, grid system etc.
• Use of cloud computing.

Skills required for managing big data:

• Apache Hadoop
• Cloudera
• MongoDB
• Talend
• SQL
• Statistical and Quantitative Analysis
• Programming languages such as Java, Python etc.

Opportunities:
Requirement of over 4.4 million Data scientist or Big Data Engineer by 2016 worldwide.

Pay scale: 
An average data analyst salary in US: $38,999 – $80,000