Scaling J2EE Technology Applications Based on Application Infrastructure Techniques - Part 2
March 7, 2001

by Rakesh Radhakrishnan

This article is second in a four-part series that identifies multiple techniques for scaling applications based on J2EE technology to support many concurrent users.

Scaling Techniques

Mail Server - Mail Multiplexing and Mail Store Partitioning (e.g., iPlanet Mail Server)
Generally, the architecture in many mail implementations is logically and physically a single tier. This presents enormous challenges for scaling the mail system. A very simple implementation of a mail server can include a single physical machine residing in the internal DMZ (IDMZ) running SMTP, IMAP and POP. An internal DNS system resolves connections to the mail system after a connection is received from an external DNS server for that domain.

Single Tier with Proxies
Even in a simple mail implementation, scaling vertically within the box has limitations, and the server will simply stop accepting new connections when system resources are completely consumed by connected users.

Figure 5 shows mail gateway/proxies implemented in the external DMZ (EDMZ) to protect the main mail server in the EDMZ from overload -- like LDAP gateways for the directory server.

Figure 5: Single-tier mail implementation with proxies (Click image to enlarge.)

In this case, there is a certain amount of horizontal scaling achieved. The mail system in the internal network is also the mail store that typically has storage attached (directly or over the network).

Mail Multiplexer
If the mail environment is at peak capacity and requires scaling even further, it is possible to implement a mail multiplexer between the gateway and the mail system that handles the routing of the mail requests based on mail/data partitioning between two or more mail stores. This configuration is shown in Figure 6.

Figure 6: Mail multiplexer (Click image to enlarge.)

In certain cases, the mail-multiplexer can route traffic based on protocol, IMAP, POP or HTTP.

Mail Multiplexer with Mail Store
To the extent possible, the mail store is vertically scaled within a physical server. Data partitioning -- a horizontal scaling technique -- is typically adopted as the last resort/technique to scale, as shown in Figure 7.

Figure 7: Mail multiplexer with mail store (Click image to enlarge.)

Web Server - IPLB and Web Farm Clustering (e.g., iPlanet Web Server)
The proxy server farm handles client connections after appropriate name resolution and authentication/authorization. If the requested content is available locally within the proxy farm cache, the proxy server serves it. If the request requires certain logic (presentation or business) execution, or if the content is not available locally, the Web server is contacted.

Typically a large dot-com Web server implementation has the following characteristics:

• It incorporates horizontal scaling techniques for Web servers.
• The Web servers are usually based on multithreaded and multiprocess architecture.
• The servers support multiple CPUs to scale vertically.
• Typically these systems are deployed on multiple-CPU servers.

There are two techniques primarily associated with Web server horizontal scaling between multiple systems:

• IPLB (IP-based load balancing)
• Web server clustering

IP-based Load Balancing (IPLB)
IPLB is the simple approach, where a level 4 switch routes HTTP connections between multiple Web servers based on a simple load-balancing algorithm (round-robin, uptime, etc.). In this approach, we assume that all Web servers in a farm are configured exactly the same way with exactly the same content. There is usually no communication between these Web servers; the IPLB switch detects new nodes added to a farm and balances HTTP load across the new systems. This approach is shown in Figure 8.

Figure 8: Simple IP load-balancing Web servers (Click image to enlarge.)

Web Server Clustering
Unlike IPLB, Web server clustering is more complex. With the IPLB technique, each Web server runs its own admin server. With clustering, a group of Web servers are managed and administered from a single server. All Web servers in the cluster share the same configuration files. The master server (the one that runs the admin server) manages the other Web servers in the cluster in a master/slave relationship. A simple cluster is shown in Figure 9.

Figure 9: Web server clustering (Click image to enlarge.)

The master server automatically starts failed processes, handles fail-over and manages all initial connections, routing them only to those available slaves in the Web server farm. The master server also maintains connection information about the application servers the cluster can access to execute logic.

In this approach, the load-balancing capability is offloaded from an external IPLB switch to the application code, so these techniques are mutually exclusive -- you would not use both IPLB and clustering together.

Application Server - State/Session Synchronization (S3) Between Clusters and Application Partitioning (e.g., iPlanet Application Server, ATG Dynamo)
Application servers play a very significant role in today's dot-com environment. Application server platforms based on J2EE technology offer application-level scalability by distributing application components across multiple tiers and between multiple application servers (components based on technologies such as EJB, JSP, Servlets, RMI Objects, and so on). Typically these application servers are multiprocess and multithreaded. They are expected to scale vertically anywhere between 4 to 8 CPUs. Application servers are typically deployed on such systems as Sun Enterprise 420s, 4500s, 5500s, 6500s or an E10000 domain.

Two techniques are commonly used for horizontal scaling:

• Application partitioning
• Application-level clustering

Application Partitioning
Application partitioning is the easier approach of the two, but not the most efficient. Using this technique, the scaling is achieved by locating a certain set of application logic on one server and another set of application logic on another.

For example:
The electronic bill payment and presentment logic could be supported by one platform, while the customer self-service application is supported by another, with the e-store logic possibly supported by the third platform. Based on user profiles and the state of the connection, the Web server connects to and invokes logic from the appropriate app server as requested by the client. This simple configuration is shown in Figure 10.

Figure 10: Application partitioning (Click image to enlarge.)

Application-level Clustering
The alternate technique is not mutually exclusive. In many cases the two are combined. This approach, referred to as application-level clustering, has application components (EJB, JSP, Servlet, RMI objects) distributed redundantly between multiple application servers.

The state and session of each client connection is maintained by the primary application server, which accepts all user connections initially, and then balances load between the remaining backup and alternate application servers. The backup will act as the primary in case the primary fails. The state and session information is constantly synchronized. This technique is shown in Figure 11.

Figure 11: Application-level clustering (Click image to enlarge.)

Combining the Two
If these two techniques are combined, application-level clustering is maintained for each application partition. Containers in the app servers offer multithreading, load balancing, connection pooling, database caching and automatic fail-over for code. A implementation combining both techniques is illustrated in Figure 12.

Figure 12: Combining application clustering and partitioning (Click image to enlarge.)