The Disconnect Of Intelligence and Analytics

转自---http://www.information-management.com/issues/21_1/unplugged-10019478-1.html?pg=1

Big data has finally arrived.

With cheap, seemingly infinite storage and powerful computing technology, the infusion of the Internet and clouds of CPUs, businesses are capturing enormous amount of data.

From clickstreams, point of sale records, call center events and ATM transactions to Facebook, blogs, emails, RFID and mobile device data, the influx of information is now impossible for the human brain to analyze and comprehend.

Being analytics savvy is no longer an option for most businesses but a matter of survival. If a company can gain insight into customer, product, channel, pricing and supply chain and predict the best customers, best pricing and most profitable products by analyzing business data, it will leave competitors in the dust. Analytic competition is increasingly the norm, and gets more sophisticated every day as companies strive to "out-data-mine" their competitors.

Meanwhile, there has been raging debate about business analytics versus business intelligence in recent years. Many pundits and analysts have drawn their own line to differentiate BA from BI. Some have proclaimed the death of BI upon the rise of BA. The debate raises serious question about the current state of BI and the relevancy of BI architecture, as well as the technology, process and organization that surrounds it.

Historically, BI implementations have often stopped at data gathering and reporting and failed to dig further into analytics for actionable information and data-driven decision-making. But BI as a concept and theory never was, nor should be, just about collecting and reporting data. In its full promise, BI provides historical, current and predictive views of business operations that supports better business decision-making. Unfortunately, BI in practice too often fails to deliver on the promise of helping business acquire intelligence and make fact-based decisions. And, all too frequently, IT as the owner of BI systems and technologies fails to understand business from the beginning and does not provide valuable information or help guide business decisions. In short, BI often lacks both business and intelligence.

BA, on the other hand, has been overhyped as the silver bullet to fix most of the problems BI can't address today. As capable and promising as BA is, most information management companies need to overcome common challenges to compete on analytics: misaligned IT and business objectives, data quality failures, data silos, inconsistent data definitions, information latency, a lack of standards and performance problems.

It's important to remember that business analytics isn't a new concept, and has been practiced since the Efficiency Movement in the late 19th century. It was then that Frederick Winslow Taylor postulated that, by analyzing work, the "one best way" to do a task would be found.

However, the "big data" era introduced an astronomical amount of data, the wealth of information that companies can potentially unlock and the depth and breadth in which organizations can perform analysis, enable technology and apply analytical methods of the 21st century.

In the past, BA processes have run mostly in silos. Business managers and analysts often used desktop tools to build models and analytics outside of the BI architecture framework. This approach exacerbates problems in data quality and governance and lacks an enterprise-wide approach for data integration and standards. Furthermore, in order to perform deep analytics, data needs to be captured and modeled at the most granular level, and as a result, the amount of data for processing could quickly overwhelm a personal computer. The scope of analysis and performance of such analytical processes are limited due to the constraints of desktop or laptop PC power.

The good news is that existing BI architecture and its infrastructure, which includes powerful BI and database servers, are able to transform and adapt to meet the new analytical requirements. The intent here is to focus on the architecture aspect of BI and BA and propose a new solution with an integrated approach.

The Traditional BI Architecture

The traditional BI architecture shown in Figure 1 integrates enterprise data from multiple sources, which could be internal and external, into an operational data store. Internally, many organizations have critical corporate data in transactional systems, such as ERP, CRM and point of sale. External data can be sourced from industry benchmarks, census, business bureau and more. Data sources can also be both structured and unstructured, such as emails, Web pages and blogs. Data is then extracted, transformed and loaded from the ODS to the enterprise data warehouse in a formatted structure suited for analysis.

It's crucial that the enterprise solution implemented gathers all relevant data for analytics to provide a holistic view of analyzed subject and a single version of the truth. This could include, for example, customer, product, pricing, sales and revenue data. Additionally, OLAP cubes and data marts can be built to focus analytics on a specific domain. This first step accomplishes data collection and preparation for reporting and analysis.

On top of the data service layer, BI reporting and visualization tools serve as a presentation layer providing financial, operational and regulatory reports through a business semantic layer, which translates technical terms into business language that executives, managers and analysts can understand. For the most part, the BI presentation layer is very heavy on reporting but light on advanced analytics.

This architecture, albeit an enterprise-wide approach, lacks deep analytical and predictive capabilities. Traditionally, this is where the work of IT ends and business analytics starts, with statistical, quantitative and predictive work conducted outside of the framework.

This unfortunate reality has contributed to the myth that BA is something totally different from BI. The vision of BI always includes analytics, and BA is merely a subset of BI focused on analytical parts of business intelligence. Because the traditional BI architecture doesn't lend itself to advanced analytics capabilities, such as statistical modeling and data mining, it's not surprising business users collect data and reports from BI systems and then use their own analytics in spreadsheets they control. This approach is not a viable solution however, because uncontrollable processes and questionable data will seriously hamper a BA effort. Research studies estimate that roughly 94 percent of spreadsheets deployed in the field contain errors, and 5.2 percent of cells in unaudited spreadsheets contain errors.

An Analytics-Oriented BI Architecture

What we need is an analytics-oriented BI architecture that incorporates advance analytics and analytic modeling capabilities into the current BI framework, as shown in Figure 2 (on page 18). Traditional BI vendors need to build more advanced analytical functionalities within their BI offerings. Many major BI tools don't support advanced statistical and quantitative modeling. Some support limited analytics and require highly technical skills (such as SQL) for use, which most business users don't possess. BI vendors need to provide more user-friendly analytics tools with much broader capabilities for statisticians and business analysts to use without lots of IT support. These new capabilities should include predictive analytics, data mining, text analytics, simulation, decision analysis and advanced modeling. Second, traditional analytics software vendors need to embed powerful analytical capabilities into the BI platform and make integration much easier for customers. Most BI applications and BA applications operate on very different platforms. Every company needs to reckon with integration and ROI before investment. BI and BA vendors should work together to make the integration much less painful and help customers unleash the best of both worlds.

An integrated solution combines advanced analytics with powerful data visualization and advanced reporting capabilities to support fact-based and data-driven decision-making. Under this new architecture, advanced analytics will be an integral part of BI. Analytics process and technology could be managed under one unified BI framework and strategy that ultimately should align with a company's business strategy. Initiatives such as data management and governance could benefit both BI and BA programs. Companies that have high quality information that is well-defined and understood across the enterprise already have a solid foundation for BA.

In terms of implementation, there could be different deployment approaches based on the conceptual architecture. For instance, analytic models might be built into a database or data warehouse to leverage its processing power. In-database analytics has lots of advantages - analyzing data where it resides to avoid data movement and duplication. However, in-database analytics can be costly when analytics processes, which are volatile and adaptive in nature (as old models need to be updated or rebuilt with latest data input), are hindering other mission-critical OLTP or OLAP operations. It may lead to a separate environment for development and deployment of an analytic model. Meanwhile, advanced analytics capabilities are better built within existing BI tools for better compatibility and integration with existing BI features. Analytics could also be built into operational systems when less data integration is needed - analyzing data while capturing it. Organizations should choose the best deployment model to fit their business analytical needs.

Lastly, BA needs to be integrated and embedded in business process to be effective. One such example is to create a closed-loop style repeatable process in the normal workflow of business operations to feed the results back into the operational system where the data for analytics is sourced. This kind of decision automation is used in cases where decisions tend to be high volume. For instance, an online retailer can use an analytical model that predicts high probability of a customer buying a certain new product to attempt cross-selling by dynamically displaying ad banners when the customer visits the online store. An online bank can approve or reject loan applications automatically based on the criteria defined by the application processing rules engine using predictive analytics. Only the exceptions (rejected applications) will be sent to loan officers for review and follow-up. The model significantly reduces the cost and decision time for the bank and customers, a win/win for both.

The key characteristics of the analytics-oriented BI architecture are:

• Integrated (data, reporting, analytics);
• Robust and flexible (for rapid changes);
• Evolving and adaptive;
• Consistent (standards in process and data);
• Controlled;
• Transparent (versus black-box
• approach); and
• Embedded (analytics as part of business process).

With the burgeoning demand in advanced analytics and emerging analytical technologies, we will see the convergence of BI and BA in the marketplace. BI megavendors will likely acquire smaller BA players and integrate advanced analytical tools and capabilities into their BI portfolios. At the same time, traditional analytics software vendors will likely push more into the BI platform territory. The reciprocal penetration will accelerate the consolidation, standardization and adoption of analytics while moving toward an analytics-oriented BI architecture.

The opinions expressed here are the views of the author and do not necessarily reflect the views and opinions of Deloitte Consulting. Deloitte is not, by means of this article, rendering business, financial, investment, or other professional advice or services. This article is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business.

George Shen has 16 years of experience advising and designing business intelligence and business performance management solutions for many Fortune 500 clients in financial services, telecommunications and life sciences. He is an Information Management Specialist Master with Deloitte Consulting.（呵呵，原来是德勤的一个DM)