Push Technology

Push Back in Push Technology
Is Push Technology the Next Big Thing for Data Warehousing, or Is It Just a Distraction?
By Neil Raden

--------------------------------------------------------------------------------

Although supporting the decision-making process with information technology is one of the hottest areas in computing these days, the focus is decidedly technical, often with only lip service paid to the business processes it is supposed to enhance. Instead, we deal with data warehouses, data marts, Web-enabled data warehouses/marts, and, coming soon, push technology. From my perspective, these are just the means to an end: helping knowledge workers perform by improving their ability to deal with information. I have a lingering but largely unquantified fear that each new wave of technology only serves to distract us from providing real value. By diverting our attention, we spend an inadequate amount of time maturing our work efforts in preceding technologies and merely port the unfinished business to yet another platform.

Publish and Subscribe
What should we make of push technology? Richard Hackathorn, in an excellent review of the entire publish and subscribe (P&S) field ("Publish or Perish," Byte, September 1997, page 65), claims that "Applying P&S to data warehousing will move us into a whole new market dynamic for information dissemination." It is quite evident that Web-enabling data warehouses has a dramatic positive effect, but is it sufficient without the added impact of push? Hackathorn feels that the existing client-side technologies for accessing data warehouses, whether they are client/server or Web-enabled, still represent the old way of interacting with a computer: request and reply (R&R). He is quite certain that R&R is inadequate because, in today's complex business environment, we no longer know when or in what order knowledge workers need information. I suspect that we never did and will go a step further: All existing data warehouse methodologies and architectures ignore the value that these "consumers" add to this information. Push may be the best hope yet for finally incorporating the collaboration and consensus building so necessary in the decision-making process.

PointCast Inc. unleashed the push maelstrom on us, and it will surely drive many of our approaches, solutions, and even organizations to the junkyard. The field is already exploding with both start-ups and offerings by the likes of Microsoft Corp., Netscape Communications Corp., and the IBM/Lotus Development Corp. duo. Rather than surveying the landscape and examining the technical nuance of push technology, this article will deal only with the effects of push on data warehousing and decision support.

First, a Few Prejudices
Data warehousing is in danger of losing its relevance if it doesn't improve its track record of providing sustainable value. Buried in all the hoopla surrounding this feeding frenzy are many disturbing stories of failed projects, huge write-offs, and damaged careers. Even projects that enjoy early success tend to bog down over time and become difficult to sustain. To make matters worse, the field as a whole is trendy and full of contradictions, the jargon misleads more than it informs, and the prescriptive information offered by the "gurus" rarely rises above the level of common sense.

Among the many reasons for this are technical issues, such as inadequate planning for scalability, na齰e choices of tools that run out of horsepower as people learn to use them, and architectures and data models that simply cannot live up to the promise of "going enterprise." There are even more dangerous portents beyond the mere technical. Poor grasp of the breadth or complexity of the business processes, rigid workflow imposed by the architecture, overly restrictive security based on outdated "need-to-know" thinking, and an oversimplification of the "requirements" of knowledge workers contribute to the demise of what appeared to be a successful implementation. The antidote to this deadly scourge is to adopt three simple strategies:

Improvement: Look at data warehousing as a means of improvement, not merely record-keeping. Simply to supply data is not enough. It must be packaged within a context of frameworks, analyses, and rules. To be effective, it must challenge people's thinking, not reinforce it, and lead to asking the right questions. Effectively deployed, data warehousing can be a rising tide strategy, raising the level of everything in the harbor.

Collaboration: Close the loop. No decision-support system delivers sustainable improvement unless it fundamentally changes the nature of work. The best data-surfing program in the world will only have a temporary and localized effect if the analysis can't be shared with others for comment and consensus, leading to action. Data warehouse architectures miss this component completely by being overly focused on data and reporting issues instead of the broader issues of questions, content, problem-solving, issue elevation, and consensus that people have to deal with in an organization.

Aggressive inclusion: Information economies turn the Scarcity Theory of Value on its head: Unlike gold, the value of information increases when there is more of it. Rather than finding artificial barriers to access, plan on nearly universal access.

These three strategies of improvement, collaboration, and aggressive inclusion provide the axes of evaluation of push technology. One more thing is needed: a fundamental rethinking of the whole development process. In his excellent book, The Intranet Data Warehouse (John Wiley & Sons Inc., 1997), Rick Tanler speculates that, "...too often, organizations view building a data warehouse as a project ... the team adheres to an implementation plan ... and usually succeeds in that goal, completing the assigned task by the target date." His implication is that the results are not sustainable because the project-oriented approach will always lead to suboptimal results. Instead, a mission-oriented approach is required because, "the primary design goal is flexibility, where the ability to continuously evolve to meet changing user needs is even more crucial than efficiency." This provides the fourth axis of evaluation of push technology: Can it provide the kind of adaptive evolution that decision support needs? But first, a brief description of the push technology field.

What is Push?
At its most fundamental level, the concept of P&S, of which push is a manifestation, is designed to provide three services: coordinating processes, replicating content, and informing people.

P&S is not new, even in electronic terms. Services like Individual Inc. (www.individual.com) with its NewsPage Direct product, have provided daily news summaries, based on each subscriber's predefined areas of interest, via fax and email for a few years. But this "feed" was limited, initially, to a point-to-point feed of static textual information. Webcasting, pioneered by PointCast, was the first to provide a free service (paid for by embedded advertising) that included not only content but the application logic to animate it, as well as live hypertext links to other Web resources. PointCast clients merely specified at what interval they desired an update (every hour, for example), and the messaging between client and server was automatic. All of the essential elements of push technology are evident in this scheme:

By subscribing to a channel, PointCast acts as a broker, negotiating directly with content providers, freeing consumers from the need to maintain their own point-to-point arrangements. Given the volatility of the industry, with content providers appearing and disappearing regularly, this relieves consumers of a daunting task.

Early users of PointCast were pleased with the product, especially given the cost (nothing) and the effort required to maintain it (none). Network administrators, on the other hand, were not quite as sanguine. In some cases, the number of PointCast clients on a company's WAN proliferated faster than Tribbles, and the hourly feeds quickly overwhelmed the network resources. In order to succeed as a commercial product, especially on corporate intranets, PointCast needed to rethink its strategy. Enter the I-Server, a PointCast server product that acts as the single point of contact between PointCast and an organization, eliminating all the point-to-point connections and allowing integration of new, internal "channels" beyond those provided by PointCast. In fact, Microsoft has been pushing a standard, Channel Definition Format (CDF), which will allow anyone using the protocol to become a channel. Technology advances such as these are accelerating the volume of sources and uses of content, as well as making it much easier for anyone to be an author.

There are some variations on this theme. Marimba Inc.'s Castanet uses a client-based "tuner" to distribute software as well as content, performing differential updates of executable programs or applets on the client. Castanet is already embedded in Netscape's NetCaster software. The implications of this technology are enormous, especially for expanding the reach of all types of applications, including data warehouses. Using this approach, clients can "tune in" to "broadcasts" that provide not only content but the application logic required to view or even manipulate it.

Take this a step further: By including the ability to convert every "listener" into a "composer" as well, the push process becomes bidirectional. Instead of being a passive consumer, clients can receive, manipulate, increment, and pass on these broadcasts, publishing their added-value widgets into "channels" that are monitored for differential updates by wide audiences.

Impact on Data Warehousing
Push technology will have a major, positive impact on data warehousing (so positive, in fact, that data warehousing may begin to disappear as content and process become unbundled) beginning as early as the end of 1997, with an accelerating effect through 1998 to 2000. The impact will manifest itself in at least four distinct ways.

First, it will vastly increase the audience for the data warehouse. The increase will be internal, reaching an ever-increasing swath of workers; across the extended value chain of the organization (such as customers and suppliers); and externally, as a source of "push" information for other audiences. Subject to adequate controls of security and usage, the previously far-fetched notion of every organization syndicating its data for commercial use may emerge in a new marketplace. By providing the mechanisms for publishing new information easily and subscribing to it by content, not contract, an unfathomable variety of services and products will emerge.

Second, push technology will change the way people interact with the data warehouse, perhaps even causing the whole concept to evolve into something else, such as an "information network." Rather than being the repository of numeric data to serve spreadsheet transfer, reporting, OLAP, and data mining applications, it will be the focal point of a whole new set of Web-enabled work processes for notification, exploration, analysis, alternatives formation, alternatives testing, decision-making, and implementation. The entire decision-making process can be expanded to include customers and suppliers with off-the-shelf utilities instead of costly and time-consuming development efforts like EDI.

Third, the concepts of agents and alerts -- currently existing in the barren and rocky world of compartmentalized data stores where their skills can find no purchase -- will finally have the enabling architecture that can accommodate them, providing for adaptive networks of computing resources that can host and relay the myriad processes that will be spawned and killed without human intervention. This has broad implications for the data warehousing industry, too. Hardware and network vendors will be delighted with the "push" from push; software vendors will have to scramble. Those tools that have not yet adapted to a network model will be unable to keep up with the demands for integration. "Traditional" relational databases, tools, and methods will see a diminished prominence if not an absolute drop in demand, in favor of adaptive applets that are designed for a widely distributed universe from the start. Data warehouse practitioners that are still tied to data and data modeling will find themselves at sea.

Finally, push technology should provide the irresistible impetus for knowledge workers finally to give up control of their data hoarding/report slave role. Such a move would signal the end of the spreadsheet reality that predominates in most major firms today, with its attendant row-and-column view of the business enterprise.

Adopting Push
Granted, the manifestations described previously may sound just a little too good to be true, but the move to push technology is inevitable. Information technology, satisfied that it has finally begun to wrest control of computing resources back from the desktop, is going to be overwhelmed by what is bearing down on it if it doesn't get with the program. Like Emerson said, "Things are in the saddle and ride mankind." The direction for technology no longer emanates from within the enterprise, but rather from the outside. PointCast's lurking outside the door last year was just the tip of the iceberg. Once the number of channels and broadcasters proliferates, the offerings become indispensable business intelligence, and the products to deliver all this content take form, knowledge workers will connect any way they can. Why? Because push technology is aligned with the way people actually work.

Why Push?
Here are the fundamental reasons why knowledge workers will find push technology irresistible, especially after enduring a decade of wildly unreliable and hard-to-use GUI-interfaced client/server applications, many of which were less capable, less reliable, and less flexible than their text-based forebears.

People want to receive digestible information at regular intervals, especially reports, without having to request them, and especially without having to do work to generate them.
Only some workers at first, but many more as the practices are perceived as reliable in the organization, will limit notification to adjustable exceptions only, such as, "Page me when any of my customers has less than five day's supply on hand," or, "Send me a report of skewing trends in customer response to promotions, and show me copies of the promotion materials involved."
Finally, the most clever of the lot will welcome receipt of information that challenges them with information they have not explicitly requested, such as, "I'm interested in the unexpected success or failure of my competitors," or, "Who is perceived by my customers as our competition?" or, "Send me the same stuff the boss is looking at today."

Conquering Infoglut
No one can digest more than a fraction of the information available to them. Even with search engines, there is a bewildering array of mostly unstructured data available today, and it is proliferating riotously. Infoglut is a fact of corporate life, and it is only going to get worse, so people need new mechanisms for sifting through it. Push can provide the kind of open, flexible, semiautonomous adaptive architecture that can learn what consumers want and how they work, rather than the other way around. The net benefit will be hard to quantify at first. But what is clear is that the only sustainable advantage of any enterprise is the ability to create good ideas. Managing an unmanageable information supply is a compelling motivator.

Adaptive Framework for Innovation
If we've learned one thing in data warehousing, it is (to paraphrase Edwards Demming) that all models are wrong, but some are useful. No amount of planning or modeling can prepare us for the ambiguity, uncertainty, and complexity in even the most innocent-looking business processes. Rather than merely jumping from entity-relationship modeling to dimensional modeling, data warehouse architects are beginning to see that change is the only constant. No structure, no matter how simple or complex, will remain viable as long as we hope it will. Consider this the G齞el's Theorem of decision support: Any designed system, at any level of sophistication, will ultimately fail, precisely because it is designed. The mere act of designing of necessity simplifies assumptions and misses the nuance of any real-world process. Embracing change as the prime mover in this context and accepting the power of contingency are the keys. New approaches are needed to create decision-support systems that are as protean as the business environments in which they operate.

A robust push technology offers the next best hope for adding value without designing at all. By tossing a million interacting agents out into the network and allowing the inevitable adaptive (that is, Darwinian) processes to occur, the best (meaning most useful) ones will survive and grow and the poorly adapted will disappear or change.

Leveling the Playing Field
Actually, I hope that this Darwinian scenario will be the case, rather than that a few players will find ways to dominate and distort. Completely open communication between adults, for example, tends to favor those who are the best communicators, not those with the best ideas. But then, who is really to judge? Getting back to the four axes of evaluation -- improvement, collaboration, aggressive inclusion, and adaptive evolution -- let's see how push technology stacks up.

Improvement and Collaboration
The necessary elements of an enabler of improvement are that it challenges current thinking, provides alternative views, allows different approaches, and provides a mechanism for tracking your results after the fact. Getting the same reports every day, week, quarter, and year can have a stultifying effect. Even if the numbers change in a significant way, the mere act of analyzing from the same templates year after year tends to dampen the appearance of change. Fundamental changes in business may only appear to be aberrations, quickly explained away rather than evaluated from a critical perspective. Push provides the opportunity for workers to create their own presentation of information and share it widely with little effort, vastly expanding the audience viewing things from alternative perspectives.

The ability to be author, publisher, and consumer flattens the hierarchy of an organization. For example, a marketing analyst using today's OLAP tools might follow a thread of reasoning that leads to an insight about the way the company, say, handles distribution. What mechanism exists in today's data warehousing architectures for him or her to share this insight with a colleague in the distribution area? Typically, none does (other than email, which is too limited a medium). But by publishing the analysis, it will get the attention of anyone subscribed to subject areas that pertain to distribution. Any process that can accelerate the flow of ideas in the organization, especially those that come packaged with the ingredients to mix the recipe, will lead to better and faster approaches for improvement.

Aggressive Inclusion
The lower marginal costs of adding and maintaining new subscribers is a sufficient driver to widen the audience. The higher value added per participant is also a given, because each participant can be author as well as subscriber. In fact, it is already well-documented that the economics of the Web in general already exhibit this tendency, and it is only enhanced by adding fundamentally new and expanded capabilities to it, which is precisely what push technology represents.

Adaptive Evolution
Nothing is more complex than a human being except the relationship among several of them. As avatars of knowledge workers, the myriad of objects that will proliferate due to push will exhibit adaptive behaviors that mimic those of organizations. By not artificially constraining the process through barriers to entry, undue influence, or insistence on standards, this community of agents will be able to adapt to change at a greater rate and with lower cost than the expensive development projects we now endure. Will things exhibit elements of unpredictability? Yes, but the ability to adapt quickly means these occasional dysfunctions will have minimal effect, compared to the alternative. ("The data warehouse will be delayed three quarters, and some/most functionality will be pushed back to Phase 2, until we can resolve some fundamental semantic definitions between our three divisions.")

This in itself will require a rethinking of how we do business, favoring speed-of-thought approximate knowledge as opposed to dead data in a drawer, neatly arranged. Don't expect this to happen overnight, or even in this generation, completely.

Future Directions
Going forward, consider push technology to be a way to expand the impact of your Web-enabled data warehouse, and if you don't have a Web-enabled warehouse, get busy. The economics are too good to ignore. However, there are a few caveats: only multicasting is viable, sustainable, or scalable. Do not attempt to employ the technology in a many-to-many connection pattern; your success will be your undoing. Most important of all, adopt a mission-oriented approach, not a project-oriented one, and make your mission nothing less than to enable the decision-support architecture to be driver of improvement in your organization.

 

--------------------------------------------------------------------------------
Neil Raden is president and founder of Archer Decision Sciences Inc., a consulting company with offices in New York, Chicago, and Santa Barbara, California, that specializes in implementing data warehousing and business intelligence. He is widely published and a frequent speaker on these topics. You can contact Neil at nraden@archer-decision.com or 805-966-5145.
--------------------------------------------------------------------------------

What did you think of this article? Send a letter to the editor.

--------------------------------------------------------------------------------

Subscribe to DBMS and Internet Systems -- It's free for qualified readers in the United States
November 1997 Table of Contents | Other Contents | Article Index | Search | Site Index | Home

--------------------------------------------------------------------------------
DBMS and Internet Systems (http://www.dbmsmag.com)
Copyright ? 1997 Miller Freeman, Inc. ALL RIGHTS RESERVED
Redistribution without permission is prohibited.
--------------------------------------------------------------------------------
Please send questions or comments to dbms@mfi.com
Updated October 2, 1997.