As we all know that a Supermarket is something big where we get grocery, food and house hold items and these items are organized in aisles. Every supermarket has its own style. Similarly  private operators who give Big data distributions (grocery Stores) offering a wide range of enterprises big data services with hadoop ecosystem (food and house hold products) to the typical Business Intelligent system. In this blog, we are going to move a trolley across various big data supermarkets.

What are the Big Data Distributions (Supermarkets)?

Cloudera Hadoop Distribution (CDH): A Leading Big data Platform for Apache foundation offering 100% enterprise-ready distribution of Hadoop and related project.

IBM Info sphere BigInsights : Bring the power of hadoop to the enterprise contains enhanced functionality and improved consumability, making it easier to use Hadoop leveraging existing skills, build big data applications and uncover insights in your data.

MapR: Delivers on the promise of Hadoop, making managing and analyzing Big Data a reality for more business users. MapR Distribution brings unprecedented dependability, speed and ease-of-use to Hadoop.

Horton works Data Platform (HDP) : Most stable and reliable Apache Hadoop distribution. Built and packaged by the core architects, builders and operators of Hadoop.

Karmasphere: Provides everything data professionals need to put Big Data to work. From predicting customer needs to tracking user behavior to optimizing development processes, Karmasphere was built to help you turn data into a strategic asset.

Where does the Hadoop ecosystem (products) fit into the typical BI system?

Layers of Big Data:

  • Big Data Store
  • Big Data Processing Engine
  • Big Data Integration
  • Big Data programming
  • Big Data Insight
  • Big Data Management

Big Data Store:

Hadoop distributed file System (HDFS) HDFS is highly fault-tolerant and is designed to be deployed on cheaper hardware. Data is distributed across all nodes
Hive Data warehouse system built on top of the hadoop for analyzing large dataset.
HBase Column-oriented database system for random, real-time read/write access

Big Data Processing Engine:

Mapreduce Programming model for processing a large Cluster of commodity machines. Mapreduce has Parallel processing power of distributed file system with large data set.

Big Data Integration:

Flume Distributed services that can collect data from different sources.
Sqoop Imports data from an RDBMS to hadoop and vice versa
Hiho Moving Data between any database and hadoop
Chukwa The Powerful tool for displaying, monitoring and analyzing results of the large collection of logs.

Big Data Programming:

Pig The Dataflow scripting language of high-level platform and capable of running mapreduce Engine.
HiveQL Querying Language to access the hive.
Jaql An executable program and a built-in annotator library provide the text analytics for Hadoop.

Big Data Insight:

Mahout The Core algorithms for implementing clustering, classification and filtering of large data sets.
Hue User interface framework and software development kit (SDK) for visual Hadoop applications
Beeswax User Interface framework for analyzing hive.

Big Data Management:

Zookeeper Coordination service for distributed applications.
Oozie The Workflow scheduler system for managing hadoop jobs.
Whirr The Cloud-neutral way to run services

Big Data Distribution (Supermarkets) Comparison:

Features MapR IBM Biginsight Cloudera Hortonworks Karmasphere
Name Node High availability Available Available Available Available Integrated with MapR
Connector to Social media Flume Social Data Accelerator Flume Flume No
Administering and Monitoring Not Available Available Available Available Available
On Cloud Google Cloud Platform and Amazon EMR IBM SmartCloud Enterprise No No Karmasphere Analytics for EMR
Machine Learning Mahout Machine data Accelerator Mahout No No
Web User Interface Available Available Available Available Available
Cluster Set up Easy Easy Easy Easy Medium
Product MapR Edition Infosphere biginsight CDH Hortonworks Data Platform Karmasphere Studio
For windows No No No yes Yes
Text analysis INFA Hparser Annotation querying language No No Ability to use SAS, SPSS and R Analytic Models
Posted by Jayaramaraj L
Comments (1)
August 26th, 2013

Comments (1)

Arindam - September 9th, 2013

Thanks Jayaram. The comparison was really crisp & useful for non-BigData people like me. keep it up.. :)

Comments are closed.