Pervasive Business Intelligence

Operational data warehousing involves new capabilities that enable operational decision making in real time

Tuesday, October 06, 2009

Print

Comment

Email

Digg

Del.icio.us

Reddit

Twitter

Today's enterprise data warehouses (EDW) – high-end analytical computing platforms – continue to provide powerful strategic analysis. However, the newest wave in database-driven insight is operational data warehousing. This involves new capabilities that enable operational decision making in real time – as well as strategic data mining. The use of these real-time centralized data repositories has escalated and expanded. The term most commonly used for this new approach is Pervasive Business Intelligence (PBI).

This is not a future vision but a present reality. By implementing PBI, the best businesses are taking knowledge management to new levels. The day of passive knowledge management is quickly ending. Pervasive BI rapidly evolves knowledge management into competency optimization for employees – who can truly interact with customers and the business with a panoramic view, compressed into actionable tactical intelligence for tangible economic value.

Pervasive knowledge management
In simple terms, pervasive BI delivers data warehouse insights to everyone in the enterprise, not just an elite group of technologists, analysts or knowledge workers. This is truly a revolution in the use of knowledge. You might even call it pervasive knowledge management. The goal is to make thousands of daily tactical decisions better by using facts instead of intuition alone. Imagine every employee making 40-50 small decisions a day. Next, imagine adding the use of Pervasive BI, so that 20 of those decisions are supported by facts instead of guesswork. Generally, the line-of-business managers understand pervasive BI instantly and wonder why they haven't had it all along. Some knowledge management (KM) practitioners don't think of technology as a best practice arena, however, we would propose that this is indeed an emerging best practice. Look at it this way: you distill facts, insights, and information then systematically deliver it to the thousands who need to make smarter decisions. Maybe think of this 'knowledge distribution' – as the purpose of the larger server platform. At times, summary data is all that is required.

However, the use of more detailed, actionable information in real time can certainly provide much more valuable knowledge. Agreed?

Unfortunately, computer technologies were once limited, forming a barrier between the front line user and the EDW data until around 2001. It existed because front line users need 'fresh data' in addition to the historical information normally found in the EDW. While historical data is loaded into the EDW nightly or weekly, front line employees need today's data as well. For example, a call center representative (CCR) needs to know a lot about the consumer calling in: residence, prior calls, profitability score, and the next best offer to suggest.

All this can be calculated nightly, ready to go each morning. But the CCR also needs to know what calls the consumer made this morning and what the consumer did on the company website today. Fresh and historical data need to be displayed while the consumer is still on the phone. Getting that data from the production system, into the data warehouse, and back to the front lines is sometimes called 'real time BI;. Until 2001-2002, the software and computers could not do this.

As it often occurs, some enterprises weren't daunted by the seeming limitations of technology. The business-IT visionaries wanted the benefits of pervasive BI immediately, and they became creative. So in 2001-2002, numerous enterprises found ways to speed up getting data into and out of the EDW. These visionaries were able to connect their point of sale machines, airline gate agents, web site, call center, and dozens of other front line users straight into the EDW. As IT organizations charged ahead, the software vendors saw opportunities to improve their products in the same direction. In a symbiotic relationship, IT experts and IT vendors collaborated to make the technologies work in real time.

Elements in pervasive BI
How did they do it? What were the barriers that were overcome? First, three technology subsystems had to evolve. Second, IT organizations needed new designs and processes. The three technology subsystems – Data Integration Services, Decision Repositories, and Decision Services – are part of an information supply chain. Data originates in raw form, gets transformed and cleansed. Next, it's repackaged, stored and repackaged, then finally analyzed and distributed.

Data Integration Services is a point of convergence, a hub, taking all kinds of data from many sources and production applications, and preparing it for the EDW. To meet the 'service level” goal of 'under one hour' or 'under five minutes,' delivery of the raw data needs to be “guaranteed delivery” to the data integration hub so nothing is lost and everything is on time. Data integration services also must clean up the data quickly or else the inaccurate data will lead to bad decisions. It must remove duplicates like name and address, and swap out 'codes' for more understandable values. All this must occur as multiple streams of data flow into the data integration server continuously or in spurts.

The Decision Repositories (data warehouses and marts) need to load the data and serve it up as fast as possible. This is difficult because data loading can consume the entire EDW server's capacity, leaving no access for the front-line and back office employees. Performance for the front line and back office users is terrible during loading. But around 2002, Teradata began offering two components to solve this: a real time data loading utility and mixed workload management tools. Mixed workload management means prioritizing work inside the data warehouse according to business rules. This enables an enterprise to give the call center workers top priority, reports medium priority, and data loading low priority. The result is data loading tasks perform well while reports run fast, and the call center runs fastest.

Decision Services grabs the EDW data, reformats it, analyzes it, and delivers it to the front line user. Since the front line user doesn't have time to read reports, the requested facts have to be short and specific to the business task. This means delivering analytic information to the entire knowledge network: BI dashboards, employee portals, mobile devices, and modern 'composite applications. 'Since nearly 50% of all IT development projects are now composite applications, the EDW and Decision Services roles must be fit into those developer tools and applications.

The result is the ability to deliver analytic insights in any business process: Pervasive BI.

The IT organization, led by the CIO, also has some critical success requirements. First, there must be a 'service level agreement' between the IT group and the front line users for performance and availability. Negotiating the agreement wrings out all the cost issues, true business needs, and project objectives – resulting in information access priorities. The second success factor is mission critical availability. The entire information supply chain must be 'always on, always available.' Since many business processes run any time of the day, the pervasive BI subsystems must as well. But most companies have not made the end-to-end information supply chain mission critical. This is the area of the biggest risk. The good news is most IT teams know how to do it with their mainframes and UNIX servers. They simply need to apply those principles to the pervasive BI infrastructure.

With the emergence of Pervasive BI, organizations can at last activate the terabytes of data sitting passively in massive repositories. They can convert knowledge into economic value at cyber-speed – in seconds, not hours. They can create limitless adaptability in their businesses and create competitive advantage that is truly as sustainable as the capabilities of their centralized database. Interested in best practice examples? There are examples at many leading companies across industries. If the readers and editors want further information on Pervasive BI – and its contributions to the field of knowledge management, we can report on those instances in the hope that the field of KM can at last welcome data warehousing technology into its sacred provinces with open arms. Again, this is about data being distilled into knowledge and systematically distributed to thousands of knowledge workers.

Pervasive BI is trending upwards, having moved from the early adopters into the mainstream. The visionaries have blazed a path but there is still time to be a leader in your market using pervasive BI. You need a vision, a clear strategy, and a capabilities roadmap, in order to more effectively and economically differentiate your business. Just don't wait until this trend is, well, pervasive.

Ashok Ekbote,Country Manager, Teradata India

Page(s)   1