|
|
Java Technology for Business Intelligence
Authors:
Paul Oh - Market Development Manager, Business Intelligence & Data Warehousing
Chuck Mosher - Market Development Engineering
Although companies are familiar with the benefits of data warehousing for improving corporate decision making, business intelligence solutions are emerging as the next wave of applications that can determine success in the e-economy.
Business intelligence makes the enterprise "smart." While not new in and of itself, business intelligence can be seen as the process of transforming data into information and ultimately into knowledge that is valuable to the corporation. Applications such as data warehousing, data mining, Enterprise Information Portals (EIP's), and Knowledge Management Systems (which can all comprise a business intelligence solution) are able to provide insight into customer retention, purchasing patterns and even future behavior. They can also consolidate the presentation of and access to data stored throughout the company. These applications can not only tell you what has happened but why and what may happen given certain business conditions - allowing for "what if" scenarios to be explored.
Business intelligence touches on every aspect of IT such as enterprise resource planning, supply-chain, and customer relationship management. By improving their ability to collect, interpret and act on their information assets, companies realize more-efficient operations and decision making, fueling top-line growth and rewarding shareholders.
It is universally acknowledged that information is a valuable asset and competitive advantage, especially in e-business. However, extracting potentially valuable information from the massive volumes of data collected by operational systems is the biggest challenge faced by most companies in developing business intelligence systems. The priorities for business intelligence system designers and applications developers are interoperability, scalability, and adaptability, however traditional IT practices have focused on corporate or departmental solutions with their own internal standards for data interchange, system access, and security. In order to bridge this gap and allow for the creation of robust and semantically rich business intelligence and data warehousing applications that capitalize upon lucrative new business models such as business-to-business e-commerce, and trading exchanges, participants will need to utilize standards-based development and computing models. These models act as a "blueprint" for new applications and systems development as well as a common set of standards and interfaces through which applications may interact, explore and exchange information.
Back to Top
The Data Interoperability Challenge
Demands on corporate data warehouses have been steadily accelerating for the past several years as businesses generate and collect more and more information over the Web. As a result of this information growth, people at all levels inside the enterprise--as well as suppliers, customers,and others in the value chain--are clamoring for subsets of the vast stores of information--such as billing, shipping, and inventory information--that can benefit them. While collecting and storing vast amounts of data is one thing, utilizing and deploying that data throughout the organization is another.
The technical challenges inherent in integrating disparate data formats, platforms, and applications are significant. However, emerging standards such as the Application Programming Interfaces (API's) that comprise the Java platform, as well as XML technologies can facilitate the interchange of data and the development of next generation data warehousing and business intelligence applications. While Java technology has been used extensively for client side access and in the presentation layer, it is emerging as a significant force for developing scaleable, mission critical server side programs. The Java2 Platform, Enterprise Edition (J2EE) provides the object, transaction, and security support for building robust, adaptable enterprise class systems.
Back to Top
Incompatible Metadata
One of the key issues limiting data interoperability that business intelligence developers must solve is that of incompatible metadata formats. Metadata can be defined as information about data or simply "data about data." In practice, metadata is what most tools, databases, applications, and other information processes use to define, relate, and manipulate data objects within their own environments. It defines the structure and meaning of data objects managed by an application so that the application knows how to process requests or jobs involving those data objects. The problem is that most applications define metadata differently using different programming structures, syntax's, and semantics as well as storing metadata in different data management systems with different file formats.
An example of metadata is the schema or model that database programmers create that define the fields within a table, tables, and table relationships within a database. The database management system uses this metadata to determine which tables and rows to access in response
to an end-user transaction or query. Developers can use this schema to
create views for users. Also, users can browse the schema to better
understand the structure and function of the database tables before launching a query.
Data warehousing and business intelligence developers in particular are familiar with the problems caused by incompatible metadata formats. A typical data warehousing application requires the integration of many different types of tools for the extraction and transformation of data, often from different operating systems and software applications. Data must then be transported in stages to the data warehouse where it is merged with data collected from other sources - each with its own set of metadata. Query, reporting, and analysis tools likewise need to maintain common metadata to ensure that the data views maintained by the tool are synchronized with the associated database schemas. Without a common model for creating metadata, developers must hardwire discrete interfaces between applications to allow for the exchange and synchronization of data. The high cost of developing such a system in terms of development and maintenance can be prohibitive. Companies have been limited in regard to developing solutions that require the exchange of data from multiple, heterogeneous applications in a meaningful way.
To address the metadata issue, a group of companies--including Unisys, Oracle, IBM, SAS Institute, Hyperion, Inline Software and Sun-have joined forces to develop the Java Metadata Interface (JMI) API, which permits the access and manipulation of metadata in Java with standard metadata services. JMI is based on the Meta Object Facility (MOF) specification from the Object Management Group (OMG). The MOF provides a model and a set of IDL interfaces for the creation, storage, access, and interchange of metadata and metamodels (higher-level abstractions of metadata). Metamodel and metadata interchange is done via XML and uses the XML Metadata Interchange (XMI) specification, also from the OMG. JMI defines a Java mapping of the MOF IDL interfaces as well as the contracts necessary to connect to a metadata repository (see J2EE Connector Architecture below). The goal is to overcome the limitation of proprietary systems using different and incompatible semantics, structures and syntax for metadata. The lack of metadata interoperability prevents sharing of data between applications and has limited the development of robust BI systems.
JMI is part of a larger strategy of utilizing Java technology to create an end-to-end data warehousing and business intelligence solutions framework. Through the Java Community Process (see sidebar) industry experts are extending the functionality of J2EE in new areas relevant to data warehousing and business intelligence ISV's and users. Another specification underway is the Java Olap Interface (JOLAP) API that will provide Java based access to OLAP servers and multidimensional databases.
Back to Top
Business Intelligence and J2EE
Metadata management is just one aspect of creating a successful business intelligence solution. As applications become more web-centric and integrated with operational systems such as ERP and CRM - it is important that new applications be developed and deployed using a scaleable, robust and secure development and deployment framework. J2EE was specifically designed to meet the rigorous needs of enterprise computing as well as those for data interchange and interoperability.
While the benefits of J2EE for building enterprise applications are many, of specific interest to data warehousing and business intelligence developers are scalability, multi-tier support, platform independence and security.
Scalability Data warehousing and business intelligence applications typically involve ad hoc combinations and transformations of large amounts of data making scalability of the underlying system critical. As the number of users increase, J2EE can reliably manage millions of transactions during major web surges.
Multi-tier support Multi-tier architectures are composed of tiers of application logic that are separated from the data tier and the client user-interface. Multi-tier architectures bring high levels of scalability and reliability to Web applications as unpredictable demand levels and changes in the application code do not require rewriting the entire application.
Platform Independence Java Virtual Machines are available on a wide range of computing platforms from handheld PDA's to servers to mainframes. As users expectations for information to facilitate decision making increase, there is greater need to distribute information on a variety of different devices and platforms.
Security Java has been designed from the ground-up with security considerations as a central feature. The security architecture of J2EE defines simple, flexible relationships between protected resources, the roles that have access to those resources, and users and components.
A key technology of J2EE is Enterprise JavaBeans (EJB), an architecture for the development of component-based distributed business applications. Applications written using the EJB architecture are scalable, transactional, secure, and multi-user aware. These applications may be written once and then deployed on any server platform that supports J2EE. The EJB architecture makes it easy for developers to write components, since developers do not need to understand or deal with complex, system-level details such as thread management, resource pooling, and transaction and security management. These issues are all taken care of by the EJB server, allowing the developer to focus on writing the business logic. Applications are then composed by combining EJB components, sometimes supplied by different vendors, into modules that can be deployed, managed and executed on any compliant J2EE implementation. This allows for role-based development where component assemblers, platform providers and application assemblers can focus on their area of responsibility further simplifying application development.
Back to Top
J2EE Connector Architecture
While accessing data stored in relational databases is a relatively trivial matter with the JDBC API, most applications will require access to larger amounts of data stored in Back Office applications as well as legacy computing environments. J2EE defines a Connector Architecture that allows access to this data within the Java environment by defining a set of contracts that need to be fulfilled between the back end system and the J2EE platform to support security, transactions and resource management. The connector acts as an interface between the J2EE platform and the targeted data source which allows for transparent connectivity between these two systems. This simplifies the integration of operational systems, data warehouses and mainframe based systems, since only one connector needs to be provided for any single back end data source. As mentioned previously, connectors can be built that access metadata repositories, either locally or remotely on other platforms. By utilizing the connector architecture to access a metadata repository, and using JMI to manipulate the metamodels and metadata stored in that repository, interoperability between applications, tools, services and disparate data sources are enhanced.
Back to Top
Java Technology for Business Intelligence
As we have seen, the J2EE platform provides key benefits for building data warehousing and business intelligence applications, tools and services by providing a solid architectural framework that simplifies complex development and speeds product time to market. By leveraging the J2EE platform, organizations can take advantage of the scalability, multi-tier architecture support, and security of Java which has become the de facto industry standard for building transactional, web based applications.
With the support of industry leading companies in developing extensions to J2EE for the data warehousing and business intelligence marketplace, J2EE is a compelling platform for companies in which to deploy these applications.
Back to Top
Case Study: Business Intelligence in the Travel Industry
In the following scenario, a Java-based infrastructure enables a travel reservation company to develop and deploy applications rapidly, integrate them with existing systems, and interoperate through access to common metadata services.
The company has amassed a great deal of information about its operations in various systems and applications distributed throughout multiple departments. Flight, hotel and automobile reservation information is located in a database being accessed by travel agents worldwide. Another application contains information that must be updated with credit and billing history from a financial services company. Data is periodically extracted from the travel reservation system to spreadsheets where information regarding customer activity is kept for future sales and marketing analysis. Departments have similar systems but based on different databases, applications, etc. The company must integrate the operational systems across the entire company with minimal impact on day-to-day operations and provide sales and marketing analysis to key decision makers across the consolidated systems.
By utilizing J2EE, the company can take an incremental approach that consolidates application development within an EJB container, which can run on a variety of hardware and software platforms so existing databases and applications can coexist with newly developed ones. Furthermore, tasks can be divided by roles according to the part of the system being developed, utilizing resources more efficiently and simplifying application development.
For example, application component developers can focus on the business logic involved in creating a hotel reservation or checking flight departures without worrying about the system level issues of developing an entire distributed, multi-tier application. Similarly, application assemblers and GUI designers can focus on overall application functionality and user interface presentation without having to be concerned with the details of the individual application components.
The company constructs EJBs to model various components important to the travel reservation company's business. Beans are written that encapsulate information about Customer, Hotel, Car Rental Agency, and other attributes. Each bean defines methods that execute various business functions. Hotel has methods that provide name, location, number and type of rooms, and distance to local airports. To model a Hotel Reservation, the company constructs a bean that references these other beans for a particular Customer, Hotel, Car Rental Agency, Date.
Similarly, a Payment bean can interact with and invoke payment from a credit card clearing company and determine billing and payment status. These beans can be developed according to the specific workflow they need to accomplish rather than the underlying details of whether a
reservation is being made for a hotel, for a car, or for an airplane. The Payment bean for example can be used in either case, simplifying application development and enabling component reuse. Also, these beans can be written on top of virtually any underlying data store, simplifying the integration of multiple heterogeneous systems.
Data stored in existing applications can be accessed with either JDBC for relational databases or specialized Connectors for nonrelational systems. Integration and interoperability of these data sources is further enabled by the metadata repository that contains metamodels of
the data contained in the sources, which can be accessed and interchanged uniformly via the JMI API. These metamodels capture the essential structure and semantics of business components, allowing them to be accessed and queried via the JMI API or to be interchanged via XML. Through all of these processes, the J2EE infrastructure ensures the security and integrity of the data through transaction management and propagation and the underlying security architecture.
To consolidate historical information for analysis of sales and marketing trends, a data warehouse is often the best solution. In this example data can be extracted from the operational systems with a variety of ETL (Extract, Transform and Load) tools. The metamodels mentioned above allow EJBs designed for filtering, transformation, and consolidation of data to operate uniformly on data from diverse data sources as the bean is able to query the metamodel to identify and extract the pertinent fields. Queries and reports can be run against the data warehouse, which contains information from the consolidated companies in a consistent, enterprise-wide fashion through the use of the JMI API.
The Java platform, coupled with J2EE's middle-tier facilities, reduces the time required to develop and deploy the application. Because J2EE is an open standard, the company can select the vendor and computing hardware that best fits its needs. In addition, the wide selection of development and content authoring tools that can be used with J2EE gives developers the flexibility they need to build services rapidly, providing competitive advantage to the enterprise.
Back to Top
|
