Posted by Karthikeyan Sankaran
Comments (2)
October 5th, 2009

“To be or not to be, that is the question” the most famous words from Shakespeare’s play Hamlet aptly summarizes the conundrum faced by BI decision makers faced with the problem of data consolidation within their organization. Essentially, an organization has to consolidate its data repositories for various reasons, viz. Mergers and Acquisitions, Single View of Customer, Regulatory requirements, Resource optimization etc.

The act of creating consolidated physical repositories is a cumbersome task, especially in large enterprises, that such investments need solid financial models, elaborate planning and even then there is no guarantee of success. For many years, the only viable option for enterprises is to perform “physical consolidation”, i.e. create the consolidated logical and physical data models, create hard-wired ETL programs to load data into the consolidated database and use this infrastructure for reporting and analytics. Not anymore! – It is my view that Enterprise Information Integration (EII) provides a viable option for enterprise wide BI consolidation and this post it to initiate practitioners to look at EII as a business solution for problems in the BI domain.

EII falls in the realm of data virtualization and refers to technology behind real-time aggregation of corporate data across multiple, widely disparate data sources. EII delivers comprehensive, reusable “views” that is exposed via SQL and/or web services to whole lot of consuming applications (Reports, Dashboards, etc.). Though the concept of data virtualization is very apt for BI consolidation, there are many other class of problems in which EII technology can play a major role.

At a high level, EII tools work in the following way:
• Designer uses the data connectivity feature of the tool and can potentially connect to a whole lot of data repositories, viz. databases, XML, excel sheets, etc.
• Data Modelers then can combine the required data sets and model them in the EII software
• EII software then creates a virtual data layer and exposes the metadata in the form of views
• The views can be optimized by using the features available within the software
• Views are exposed to consuming applications and at run-time, data is fetched from the underlying data repositories

A Simple yet Powerful Value Proposition!

It is important to understand that EII does not replace EAI or ETL. All 3 technologies can co-exist and each on its own provides solutions for different kinds of problems.

EAI stands for Enterprise Application Integration and is used in cases where different applications (Sales & Inventory application for example) want to talk to each other. Target for EAI is an application.

ETL stands for Extract, Transform and Load and is used in cases where data aggregation is required to provide an integrated, consistent definition of data for decision support process. Target for ETL is a database.

EII stands for Enterprise Information Integration and is used as a framework for real-time integration of data from multiple sources both inside & outside an enterprise, empowering business users to “pull” any kind of data from anywhere in the enterprise. Target for EII is the business end user.

A classic reference architecture in which all 3 tools can play a part is when, transactional applications are integrated thro’ EAI, data from these applications flow into an Enterprise Data Warehouse (EDW) by leveraging ETL capability and then EII tools help to combine data from OLTP applications, EDW, external data repositories and local excel sheets for business decision making.

In terms of technology, there are specialized EII tools like Composite Software and also many of the standard reporting tools have an EII component embedded in them (Data Federator in Business Objects XI, Virtual View Manager in Cognos 8.x etc.). In my view, EII does have a strong business case and BI practitioners would do well to evaluate this option when faced with data consolidation challenges.

Thanks for reading. Please do share your thoughts.

Comments (2)

Shankar V - October 26th, 2009

Nice blog there Karthik. Not sure if you have evaluated Oracle's Hyperion solution - Essbase with ODI plus Siebel Business Analytics (SBA). It provides for some amazing analytics that are streets ahead of its competition - or so I am told. Whether SBA can provide the EII that you are looking at is not something I can answer - but it sure is a powerful tool for end users to create their own reports and analyze the data available in the Essbase database. Maybe, worth a try from your end? -shankar

Asish Supakar - October 6th, 2009

Dear Karthikeyan.... I enjoyed reading the post. It is informative, crisp and lucid. I have been an end user of EII for three years now. So I'll share my thoughts as an end user of EII. I feel its too late to ponder "to be or not to be". In todays times, EII has become more of a basic necessity than a luxury. You were right in stating EAI, ETL and EII can co-exist. This is actually a matter of fact in most of today's organizations. Every organization has the need for EAI so that their various data entry systems like inventory, payroll, marketing, operations can talk to each other. This is followed up by ETL to store the same data to central repositories from where it can be extracted as per requirement. After all this is done, EII comes into play by delivering the decision maker with a customisable view that might extract data from a single database, multiple databases or even OLTP applications. The absense of EII is a serious handicap to any organization having multiple business units n processes. Such an organisation is like a fabulous ship without an anchor to dock in a shore and offload its cargo. How else can one explain all the pain and efforts taken to talk to different applications through EAI and store them to depositories with the help of ETl without having having the capability to extract and display views customised to aid decision making.

Comments are closed.