It happens all the time. A large healthcare provider acquires a new hospital and is faced with the daunting task of integrating the hospital subsystems with its corporate systems. Or, a mature technology company realizes it needs to integrate a legacy system to unlock its hidden data or functionality. These are but a couple of examples of the challenges posed by systems integration.

In fact, lost in all the talk about the promise of Web services built on a service-oriented architecture (SOA) is the reality that few companies have well-behaved, homogeneous IT environments. And before companies can realize the benefits and efficiencies that come with reusable Web services, they first need to integrate the systems they have inherited or acquired.

Sun is no stranger to the integration challenge, having acquired two companies — StorageTek and SeeBeyond — over the past six months. Despite the fact that Sun has already built and deployed an SOA, we are faced with the challenge of linking into the systems at StorageTek and SeeBeyond in order to integrate their applications, databases, and services into our IT environment.

Luckily, SeeBeyond made its mark building a middleware and operating system-independent product that enables integration. In fact, SeeBeyond's product suite now forms the basis for the Sun Java Integration Suite. The Java Integration Suite is designed to solve the problems associated with systems integration. It provides a tool set for integration, management tools for monitoring integrated environments, and reporting tools that can demonstrate the business value of IT transactions.

In fact, Sun is using the Java Integration Suite to link into and access the systems at StorageTek and SeeBeyond. I recently sat down with Ross Altman, CTO of Business Integration Platforms at Sun, to discuss the problems posed by integration, the opportunity offered by an SOA, and how the Java Integration Suite can lessen the problems and help companies realize the opportunities therein.

Vass: Ross, you were the CTO at SeeBeyond, the middleware integration company that Sun recently acquired.

Altman: That's right.

Vass: So you must be an integration expert.

Altman: I've been called that.

Vass: All right. Let's tap that expertise. Starting at the beginning, what is integration?

Altman: There are three ways to do integration. The first is data synchronization. When you have the same data in multiple applications, and you change information in one application, you would naturally want to make sure that the information becomes updated in all of the other applications. To accomplish this goal, you could either have somebody run around and do it manually, or you can automate the process by pulling the data out of the first application, putting it into the right formats, and then loading it into the target applications. That is data synchronization. And we've probably been doing it ever since somebody wrote the second IT program.

Vass: Yeah. People have been doing data synchronization as far back as MQSeries and file transfer, and that only goes back part of the way. What are the other types of integration?

Altman: The other two forms of integration deal with creating composite applications. And there are two different kinds of composite applications. The first is a transactional application, where you ask a question and get a response — except that instead of getting a response from a single database, you're getting responses from two, five, or 25 different databases, files, or applications. So the new application is made up of a little bit of new code and whole lot of code that you're leveraging.

Vass: What is the second type of composite application?

Altman: Again, you're creating a new program that leverages existing databases, files, and programs. But in this instance, the new program executes a long-running process — one that executes over minutes, days, weeks, or even years.

Vass: Do composite applications just bring together all these different applications into a single integrated system, or is there more to it than that?

Altman: Composite applications are a way of creating new programs — programs with their own user interfaces, processing, and data access. The difference between a composite application and a "normal" program is that a composite application gets most of its data by leveraging your existing applications and databases. So these new composite applications are going to look brand new to the user. Composite applications allow you to leverage existing systems without breaking the systems and without breaking the bank.

Vass: So now we understand composite applications and the three kinds of integration. How does an SOA help improve integration?

Altman: In order to build the data synchronization solutions and composite applications, you have two options. You could build each one of those solutions by hand on a one-off basis. Or you can study the problem of integration. You can say: How can I create reusable assets? That's where a service-oriented architecture comes in. With a service-oriented architecture, you view architecture as an opportunity to access existing applications, regardless of whether they're legacy applications or packaged applications. You try to leverage standards whenever possible. You do it in such a way that once you've accessed an application for one composite application or one data synchronization solution, you can reuse the integration parts.

Vass: Can you give us a short run down of some of the standards that are behind this? I assume ebXML and SOAP and others?

Altman: There are a lot of standards. That's part of the problem. We're never at a loss for standards. In some cases, the standards are complementary, which is great. For instance, you have a whole host of open standards that are related to Web services. In order to build an application using Web services, you just need to use a collection of Web services standards. But for communications and interoperability purposes you also have ebXML, which is predominantly a B2B standard. You have AS2, which is also called EDI on the Internet. Then there's RosettaNet, which is a standard that grew up in the high-tech industry and has been extended to other industries.

Vass: We use RosettaNet a lot here at Sun.

Altman: Each of these standards are used extensively, but few standards dominate in that they are the only way that people approach the problem of write-once-run-anywhere portability or plug-and-play interoperability.

Vass: Can you talk about the Java Integration Suite, and what kinds of standards it uses, as well as how companies are using the Java Integration Suite? I know Sun is using it as part of the merger and acquisition of StorageTek to link into their systems.

Altman: As far as the standards are concerned, we leveraged the J2EE portability standards as we built out the Java Integration Suite. The Java Integration Suite can deploy on top of multiple J2EE applications servers. Certainly, it works with the Sun Java System Application Server, but it can also deploy on top of the IBM WebSphere, BEA WebLogic, or JBOSS servers because of our use of the full set of J2EE standards.

Vass: That's great. So it's middleware-independent.

Altman: That's right. It's middleware that's middleware-independent. In a sense, we're taking a dose of our own medicine there.

Vass: That's really good. What about non-Java platforms?

Altman: Our support for the J2EE standards gives us interoperability with all Java applications. For interoperability with non-Java environments, we have mechanisms — which we call either adapters or wrappers — that allow you to connect to applications built on top of a diverse array of protocols. These adapters can handle request/reply protocols, so developers can make Java calls, CORBA calls, or CICS calls. And we also have adapters for a large number of B2B protocols, including RosettaNet, ebXML, and AS2, as well as traditional, old-fashioned B2B protocols like X12 and Edifact.

Vass: So, you have built-in connectors that connect into a CICS or a CORBA application?

Altman: Yes. We call them eWays. We have 85 different eWays. We have eWays that allow you to call CICS programs, IMS programs, CORBA programs, .NET, COM/DCOM programs, and COM+ programs. You can also use all the B2B protocols that we just talked about. You can use MQSeries or JMS messaging. Or, you can use MSMQ in a Microsoft environment for messaging.

Vass: That's great. It sounds like it can help integrate pretty much any environment.

Altman: Well, it takes more than adapters to achieve integration; the adapters only provide for interoperability. This means that you can get letters and numbers from application A to application B. It doesn't mean that the letters and numbers are going to mean the same thing in A and B.

Vass: Right. I'd imagine that is where data standards prove really important.

Altman: Yes. With luck, with each integration project you'll be able to leverage data or content standards. But, there is no market where all of the integration that you need to perform is addressed by content standards. When the content standards exist, there tends to be a lot of variations in their interpretation. There may not be standards for the type of transaction that you want to create. And, the standards that do exist may not be complete or may be interpreted differently. For all of these reasons, you almost always need data transformation capabilities along with your adapters in order to get what we call "true semantic integration" — meaning both sides understand the data in the same way.

Vass: So you're saying that the data transformation capabilities in the Java Integration Suite can define what a "customer" is — even when a "customer" means a company to me and a person to you?

Altman: Right. A "customer" may be a company in one instance and a person in another. So the data may need to be transformed. Or, I may send you information about an item's weight. You may think the weight is indicated in pounds, while I think it's in kilograms. So we would need to execute a translation program to convert the data that you're passing to me from pounds to kilograms.

Vass: With the Java Integration Suite, you can build all that into the interfaces?

Altman: You can build all that into the interfaces without writing code.

Vass: Wow. That's pretty impressive. Sometimes I hear people say: I already did all this in CORBA. How do you respond? How is this different?

Altman: Sure, they did it in CORBA. But in the CORBA context they were only doing request/reply integration between one program and another program. That's part of what you do with Java Integration Suite, but it also does messaging. And, CORBA-based integration is limited to applications built on the CORBA standards. In other words, you need a CORBA program to call a CORBA program. So, you have a bit of a problem when you look around and find that 97 percent of what you have is not CORBA. You need to have flexibility. You can't rely on your underlying applications to be any one thing. The fact of the matter is our IT environments will never be homogeneous no matter what we do.

Vass: So how do you architect around that?

Altman: The only thing you can do is create an architecture that accepts — even willingly embraces — the fact that you're always going to have heterogeneity. That's what the Java Integration Suite is really all about. With the Java Integration Suite you have a tool set, a methodology, and an entire development model that's built around taking any technology and converting it to interact with any other technology. Using this approach, you can integrate everything you've got.

Vass: So, you don't really have to rewrite your legacy systems to pull together a new application based on your legacy environment.

Altman: As a matter of fact, the key concept is non-invasive. The entire approach not only has to embrace heterogeneity, but it has to be absolutely non-invasive. As soon as you start to require changes to the underlying applications — whether they are packaged applications or legacy applications — you are out of the picture. Nobody can afford to do that. Anything that requires an invasive approach to integration is a non-starter.

Vass: There are two things that concern me about linking all my applications together. How do I know that all my transactions or business events are getting done once I've got 20 applications linked together? And, if something starts to run slowly, how do I find the performance bottlenecks in a multi-step process?

Altman: Both questions come back to the same central idea: Once you've built this thing, how do you manage it? Like any application, you have the ability to build it so you log every transaction and so you can create break points for testing and debugging. Also, a lot of composite applications are based on the premise that you have multiple steps and those steps are executed in a prescribed order. At certain points in the flow chart, you might need to make decisions. So you might want to be able to go into a specific instance of the program, or transaction, and say, where am I at now?

Vass: Can you give a real world example of composite application management?

Altman: Say you're a telco with a 43-step process that you go through in order to provision a DSL line. At some point, a customer calls up and says, what's going on with my DSL line? The employees should be able to determine that the provisioning is at stage 38. That kind of visibility is enabled by the Java Integration Suite because the tools have been built to enable rapid development, rapid deployment, easy debugging, and runtime managing and monitoring, as well as logging, billing, and creating alerts.

Vass: I assume there is a management console for looking at this, too.

Altman: There is a management console that can tell you how many transactions are running. It can show you if all of your servers are running and if they are running at top speed. Also, it can deliver business metrics and you can see, for example, the average dollar value of the orders being processed. From the messages that are flowing, you can derive technical data and, perhaps more importantly, business insight.

Vass: How do the alerts work?

Altman: Instead of having somebody sit and look at a screen, you can have the transactions tell you when performance is outside your response time boundaries. You can set up rules that say when the average response time for a particular part of the business process takes longer than three-tenths of a second to execute, put that data into a performance monitoring dashboard. Or, maybe when the entire business process takes longer than 30 seconds, I want to generate an alert and send a message to an operator or manager. We call this business activity monitoring. And, if you had to write that code by hand, you'd never even build that kind of instrumentation.

Vass: What operating systems does the Java Integration Suite run on?

Altman: Like most of Sun's open systems products, it runs on almost every operating system and hardware platform. It runs on the open source Solaris Operating System on top of both x86/x64 (any hardware vendor) and SPARC including the new Sun Fire T2000 systems. It also runs great on open source Linux (both SuSE and Red Hat) and Microsoft Windows (2000 and 2003). Or you can run it on other open systems platforms like AIX and HP-UX.

Vass: Aside from the specifics of the technology, can you think of anything else that people need to know before embarking on an integration project?

Altman: You need a mix of business and technical expertise. One of the smartest things that I've seen people do is take a business analyst and a technical person and team them in an agile development mode where they actually share project responsibility. That way, the person who knows the technology makes sure that all the technical issues and details are addressed in an appropriate way. Meanwhile, the businessperson understands the data and the way the data is used. You can't do it well with only one or the other. You really have to team both areas of expertise.

Vass: Let's say I knew nothing about any of this stuff, and my company just bought another company. Now I've got to do integration. What are your recommendations for somebody who needs to jump into a business integration problem?

Altman: First, even though I can describe the solution quickly, and it sounds relatively easy, there's nothing terribly easy about integration. What our tools do is reduce the time and effort required to build integrated solutions. Integration requires a deep knowledge of the source and target applications. And you need to understand the business context. There are a lot of things we can do to help you approach those issues from a methodology standpoint and from a best practices standpoint. And we have the people and the technology that can make you successful, but the caution is: Nothing is going to make integration easy. We just make it easier.

Bill Vass
Chief Information Officer
Sun Microsystems, Inc.
cio@sun.com