messageQ

http://www.messageq.com/ > communication= s=20 middleware

12/24/01

Message-Based Computing: The Fourth Wave of Integration

by Brian Whetten, chief scientist, Talarian = Corp.

August 2001 marked the tenth anniversary of the start of the = Web=20 revolution, which was launched with the release of the first = publicly=20 available software for building Web sites. In the decade since = that=20 introduction, standards-based Web communications have emerged as = the=20 dominant computing paradigm and what might be called the "third = wave" of=20 computing--replacing the proprietary client-server model, which in = turn=20 had displaced mainframes and dumb terminals.

Now the enterprise is poised at the edge of a fourth wave:=20 message-based computing. Ten years from now, we will likely look = back on=20 the first decade of the new millennium as the period when = Web-based=20 computing moved to the next level. But how will this new wave of = computing=20 affect the enterprise? And what are the key standards and business = drivers=20 behind it?

Certainly, message-based computing must uphold the values = afforded by=20 the Web, such as permanent accessibility and true openness. = Vendors must=20 "open up" their technology to offer customers real freedom of = choice.

In 1994, Gartner Group spelled out what most companies already = knew:=20 that building and maintaining client-server applications were much = more=20 expensive than the raw software and hardware costs might suggest.=20 Gartner's calculations of the total cost of ownership of = client-server=20 applications led to a revolution in the way companies viewed the = return on=20 investment from IT and helped accelerate the uptake of Web-style = working=20 based on simple, universal standards and ubiquitous connections. = Yet=20 something was still missing.

The third wave of computing included a major shift from = proprietary=20 network protocols to standard Web protocols, but much of the rest = of the=20 client-server model remains. Browsers can query many hundreds of = different=20 servers, but they must still "pull" information down to the user = at=20 regular intervals by querying servers for data. Contrast this with = media=20 such as radio and television, where users select channels and have = the=20 information "pushed" to them.

In effect, a browser client is like a phone that doesn't = ring--only=20 servers are able to take the equivalent of incoming calls. While = this=20 simple arrangement has provided a powerful foundation for the = initial Web=20 revolution, it has come with a set of limitations. These = limitations are=20 what provides much of the inspiration behind message-based = computing.

The media currently offers many examples of message-based = computing=20 models: peer-to-peer architectures, the real-time interconnection = of=20 business systems within corporations, streaming and content = distribution=20 networks, to name a few. In all these examples, the computers in = the=20 system can both initiate and receive communication "phone calls." = This=20 facility is enabling a whole range of new applications and = business=20 opportunities, allowing applications to communicate in real-time = as well=20 as dramatically reducing the costs of integrating and maintaining=20 disparate applications.

While message-based computing has only recently become publicly = visible, over the last decade thousands of the world's top = corporations=20 have deployed message-based distributed applications for many of = their=20 most revenue-critical applications. A class of communication = software=20 called message-oriented middleware (MOM) has supported these = applications.=20 In the past, MOM products have been deployed primarily using = proprietary=20 interfaces rather than standardized Web protocols.

More recently, however, a number of new standards have been = created to=20 support message-based computing. These standards include Java = Message=20 Service (JMS), eXtensible Markup Language (XML), Simple Object = Access=20 Protocol (SOAP) and Pragmatic General Multicast (PGM). Just as=20 client-to-server communication exploded with the advent of Web = standards,=20 these standardized messaging protocols mark an inflection point in = message-based computing. The fourth wave of computing has = arrived.

Defining the Fourth Wave

Although the basic Web model is suitable for some applications, = it is=20 still inadequate for others. When based on widely adopted = standards,=20 message-oriented middleware provides an extremely versatile = communications=20 system. As the name suggests, messaging systems allow separate=20 applications to communicate by passing messages back and forth = rather than=20 by being programmatically spliced. The first piece of = software--either=20 client or server--generates a request, wraps it in a message and = hands it=20 over to the messaging middleware. The middleware is then = responsible for=20 delivering the message to a remote receiver. The remote system = unwraps the=20 message, answers the request and creates a new message in which to = send=20 its answer.

Messaging software comes in two basic types: message queuing = and=20 publish-subscribe. Both provide the same asynchronous, = "fire-and-forget"=20 model of communication. Message queuing provides reliable message = delivery=20 for transaction processing systems and is used primarily for = non-real-time=20 delivery of messages to a single receiver. It is typically used = when=20 messages absolutely, positively have to be delivered to a single=20 destination, as in financial transactions.

Publish-subscribe software, by contrast, provides delivery to = more than=20 one receiver at a time. It is particularly useful when many = receivers need=20 the same data, when high performance is required or when the data = has to=20 be delivered in real-time.

Messaging software is particularly important in cases where = groups of=20 applications need to be integrated, such as in enterprise = application=20 integration (EAI) and business-to-business (B2B) exchanges. It is = widely=20 acknowledged that up to 40 percent of an IT budget is spent on = application=20 integration. In the client-server model, integrating 100 different = applications requires up to 20,000 different point-to-point = connections.=20 Additionally, in a client-server world, each call is likely to be = unique,=20 making maintenance of the system prohibitively complicated and = expensive.=20

With messaging software, however, each client needs only a = single=20 connection into the "messaging cloud." And instead of hard-wiring=20 applications together using a blocking procedure call (where the = client=20 must wait until the results are returned), the "fire-and-forget" = nature of=20 messaging allows clients to continue processing while the = middleware is=20 delivering the messages. Even more important, the messaging = infrastructure=20 can provide delivery that is both real-time and fault-tolerant, in = a=20 process that is invisible to the communicating applications.

Of course, messaging is not a panacea for all. Since a separate = messaging infrastructure must be deployed, message-based computing = increases complexity for small systems. For larger deployments, = however,=20 this complexity is dramatically reduced.

Message-based computing is crucial for real-time systems, such = as=20 financial exchanges or financial market data distribution. In a = financial=20 system, every time the data at the server changes (typically many = times=20 each second), this data needs to be selectively distributed to all = the=20 computers that need the information. With a client-server system, = each=20 client would have to call each financial server every few seconds = to see=20 what data has changed. This does not meet the real-time = requirements of=20 the financial trading community, and the redundancy of = transmission also=20 makes the process very inefficient.

Large-scale content distribution networks (CDNs), such as = Akamai, and=20 internal enterprise CDNs provide additional examples where content = needs=20 to be pushed in real-time to thousands or even millions of = receivers, with=20 managed performance and reliability. Publish-subscribe messaging = enables=20 many receivers to "subscribe" to a "subject." When a client = "publishes" a=20 message to a given subject, the messaging infrastructure takes = care of=20 routing it in real-time to all the subscribers.

With some publish-subscribe systems, such as Talarian Corp.'s=20 SmartSockets product, the messaging infrastructure provides very = efficient=20 one-to-many distribution--or multicasting--even where messages are = destined for tens of thousands of receivers. When combined with = the fault=20 tolerance and "fire-and-forget" capabilities of messaging = software, this=20 provides a powerful capability. The rule of thumb in choosing=20 communication software used to be "real-time, reliable or = scalable--pick=20 any two." With publish-subscribe messaging, choosing all three = becomes=20 possible.

Messaging software has many other advantages. Since the = separate=20 messaging infrastructure is dedicated purely to carrying messages = back and=20 forth, applications can be built without regard to the details of = the=20 messaging process. Many applications can thus share the same = messaging=20 infrastructure, which can be highly optimized. The messaging=20 infrastructure is inherently symmetrical: a server can easily send = a=20 message to a client, and vice versa.

By removing the direct connection between client and server,=20 message-based systems also allow workload management. At busy = times,=20 messages can be put into a queue and dealt with when resources = become=20 available. As well as smoothing out bursts in traffic, this = facility can=20 allow software to cope with an intermittent connection, as might = be=20 experienced with a wireless mobile device. The device and a remote = wireless server can queue messages when the connection is lost and = deliver=20 them when contact is resumed, greatly increasing the reliability = of the=20 service.

Standards and the Role of the Customer

Messaging has been very successful for high-end enterprise = applications=20 and application integration, but historically it has not seen the = same=20 level of awareness and uptake in the mainstream developer market. = Before=20 1997, influential software providers such as Microsoft and Sun=20 Microsystems were focused primarily on extending existing remote = procedure=20 call mechanisms to the Web. Sun created a technology known as = Remote=20 Method Invocation (RMI) for its Java technology, while Microsoft = extended=20 its Component Object Model (COM) to yield Distributed COM = (DCOM).

The lack of open standards for messaging had been a major = barrier to=20 mainstream adoption. However, this has now been solved through a=20 combination of JMS, XML, SOAP and PGM standards.

Java Message Service

JMS is a standard API that is a required part of Sun's widely = adopted=20 Java 2 platform, Enterprise Edition (J2EE) Web computing = specification.=20 JMS features a single standard API that covers both message = queuing and=20 publish-subscribe messaging products. It provides an application = standard=20 that removes dependencies on vendors, allowing buyers of JMS = applications=20 to avoid vendor lock-in and enabling developers to build skills = that are=20 applicable across multiple vendors' products. There are, of = course, wide=20 differences in the functionality, performance and scalability of=20 JMS-compliant products from different vendors. Nevertheless, the = creation=20 of JMS was a seminal event in the messaging marketplace, and = deployments=20 of JMS products are proliferating at a rapid and accelerating = pace.

JMS lays down rules about the delivery of messages and the = interfaces=20 used to pass messages from an application to a messaging system. = In=20 effect, it is like a standard address label that can be filled in = and=20 affixed to a parcel, as well as a set of business terms that must = be=20 satisfied by any parcel delivery service wishing to carry the = package.=20 With JMS, the messaging system is responsible for delivery and for = notifying the sending application of any failures that might = occur. The=20 biggest weakness of JMS is that, for now, it supports only Java=20 applications.

eXtensible Markup Language

Created by the Web's main standard setting body, the World Wide = Web=20 Consortium (W3C), XML sought to meet the longstanding need for a = means to=20 describe data arriving from one system so it can be understood by = another=20 system, with a minimum of preagreement. XML has proved a = phenomenal=20 success. Many organizations are now involved in efforts to use XML = to=20 create the minimum preagreed formats that will let whole = communities of=20 companies and individuals exchange messages on a completely ad hoc = basis.=20

While JMS provides a way of writing standard labels for = packages, XML=20 provides a standard way of representing the actual contents of the = package. XML is often called the "lingua franca" of the Web. Since = XML is=20 a standard in its own right, XML objects can be delivered over = modern=20 message-based infrastructures or between client-server = applications. Older=20 application integration systems, such as CORBA or DCOM, created = needless=20 complexity and scalability issues because separate applications = had to be=20 tightly coupled together.

Simple Object Access Protocol

Since standards like JMS cover only the programming interface = for=20 messaging systems, another problem remains: How can two different=20 messaging systems intercommunicate? For example, if one enterprise = uses=20 Talarian's JMS product and another uses IBM's, how can they = understand=20 each other?

SOAP is a new standard that is helping to solve this issue. In = its=20 current form, SOAP provides a standard means for any two pieces of = software to pass messages between themselves. It was created = primarily=20 with client-server-style computing in mind, and so some issues = remain for=20 using it with messaging systems. Even so, it appears likely to = become the=20 standard way that JMS products will be able to exchange messages = between=20 themselves or between JMS and other messaging systems.

It can also provide a standard way to connect non-Java or = non-JMS=20 applications to a messaging system. While both could be used to = connect an=20 application to a messaging system, JMS provides standards for = end-to-end=20 delivery, while SOAP only provides a way to hand off the messages = to a=20 neighboring system. By analogy, SOAP standardizes how UPS might = deliver=20 packages to a FedEx pickup point, but it does not yet standardize = the=20 labeling and business terms. This is a gap that will likely be = filled in=20 the next one to two years.

Pragmatic General Multicast

One remaining question is how to standardize what happens = inside a=20 messaging system. While standardizing the entire system is = impractical,=20 standards are emerging for key network components. The most = important is=20 the PGM de facto standard for reliable multicasting.

Just as TCP/IP (Transmission Control Protocol over the Internet = Protocol) provides the core networking system for data delivery on = the=20 Internet, corresponding standards are needed for data delivery to = many=20 receivers at once. IP multicast is a standard protocol that = enables the=20 efficient delivery of information to many receivers. Like IP, = however, it=20 is inherently unreliable, so TCP is necessary to provide = reliability and=20 "congestion control"--the magic that allows everyone to share the = Internet=20 fairly even when one piece is vastly overloaded.

PGM is the equivalent of TCP for multicast. Supported by = Talarian,=20 Cisco Systems and Nortel, PGM provides a protocol for reliable=20 simultaneous delivery of messages to many locations over the = Internet or=20 on a corporate network. With PGM, the author of the broadcast is=20 guaranteed that all data is delivered to the intended receivers or = is=20 informed of any unrecoverable data packet loss. For = publish-subscribe=20 messaging systems, PGM provides a building block for the messaging = system=20 itself, allowing it to offer much higher performance and = scalability for=20 messages that need to go to many destinations. This is useful in = systems=20 such as financial market data distribution, content distribution = networks=20 and database replication, to name a few.

All networks are not yet multicast-enabled, however, so = messaging=20 systems should not depend on multicast. Talarian, for example, = overcomes=20 this limitation with its Adaptive Multicast technology, which = takes=20 advantage of native IP multicast when it is available--and = emulates it in=20 its messaging infrastructure software when it is not.

Selecting a Messaging Vendor

Together, JMS, XML, SOAP and PGM provide the catalyst for the=20 message-based computing revolution. Many vendors currently support = these=20 standards, giving IT managers and developers confidence that their = applications will be portable between vendors. However, the = capabilities=20 of the vendors' messaging products differ tremendously, = particularly in=20 the areas of performance, scalability, fault tolerance, management = and=20 robustness. In addition, the JMS standard currently does not = support many=20 important advanced features--such as security, access control, = firewall=20 tunneling, support for other languages and load balancing--though = this=20 will likely change as the standard evolves.

It is important for IT managers and developers to realize this, = and to=20 understand the questions they must ask their vendors. = Organizations must=20 establish the requirements for their projects and then evaluate = and=20 benchmark the products of the top vendors that appear to meet = these=20 requirements. One of the most important questions to ask is = whether the=20 vendor is actually certified as JMS-compliant, as many are not. = Compliance=20 means buyers can be confident that they can swap the vendor's = products for=20 others with minimal effort, if the products prove inadequate at = deployment=20 time.

Standard messaging communication systems will become an = essential=20 feature as the fourth wave of computing takes hold. As with any = invention,=20 the creation of the Web cannot be undone. Where the Web underpins=20 software, it demands open standards and interoperability. Any = vendor that=20 creates software which cannot readily integrate with other = packages will=20 not find success in the long term. Software vendors must therefore = collaborate to survive. A vendor that refuses to acknowledge the = need to=20 work with partners and rivals when creating new software is doomed = to=20 fail.

About the Author

Brian Whetten, Ph.D., is the chief scientist for Talarian Corp. = Previously, he was the principal founder and CTO of GlobalCast=20 Communications, which pioneered the development of application = layer=20 multicast and was acquired by Talarian in 1999. Dr. Whetten is one = of the=20 foremost experts on reliable multicast protocols and real-time = content=20 distribution, and a leading author in the Internet Engineering = Task Force.=20 He has written several articles for industry and academic = publications,=20 and has spoken at more than 40 conferences and seminars worldwide. = Dr.=20 Whetten holds B.S. and M.S. degrees in computer science from the=20 University of Illinois at Urbana-Champaign and a Ph.D. in computer = science=20 from the University of California, Berkeley. For further = information on=20 Talarian products, visit www.talarian.com/fourthwave.