Back to Dot-Com Builder How-Tos Archive
STAR: Strategic Technology Architecture Roadmap
by Rakesh Radhakrishnan "STAR" is Sun's acronym for "Strategic Technology Architecture Roadmap," an end-to-end framework for all the technologies in an enterprise, including those used by its trading partners and external customers. The key phrase here is "all of the technologies in an enterprise": the STAR framework encompasses and ties together the various architectures in an enterprise -- network, systems, application infrastructure (such as database and directories), and functional applications -- toward the common goal of transforming it into a true e-business. Application architectures (specifically those based on J2EE technology) drive the application infrastructure architecture (Web servers, app servers, database servers, messaging servers, media servers, and so on). In turn, the application infrastructure required to run a J2EE technology-based application dictates the network architecture and the selection and design of the compute resources and data storage platform. You can think of the application infrastructure as the "glue" that binds itself with the rest of the infrastructure to offer to its users what are referred to as "systemic qualities." The technologies referred to parenthetically in Table 1 are a partial list.
Table 1: Infrastructure layers and associated technologies STAR: The Business Case The STAR framework originates from the collective realization that an e-business can outperform traditional businesses by adapting more rapidly to market changes; responding faster to customer needs; operating at the lowest possible cost; and managing the creation and termination of market relationships as needed via the Internet. To capture these advantages, an enterprise needs to integrate its systems, including e-commerce, ERP (Enterprise Resource Planning), CRM (Customer Relationship Management), KM (Knowledge Management), SCM (Supply Chain Management), and so on. E-business applications enable an enterprise to move into the world of "always on," using the Internet, intranet, and extranet (or VPN) platforms to protect margins, develop competitive advantages, expand channels, and provide knowledge and access to all employees, regardless of their client devices. Large-scale electronic retail and business-to-business procurement Web sites are growing at an exponential rate. This leads to the capturing of every piece of the data associated with every customer's purchasing experience, which, in turn, leads to more effective data mining, more efficient online buying, better focused product marketing, and a general optimization of supply chain activity. The Architectures in the STAR Framework How does an enterprise architect design a solution that covers all the facets of the computing environment, including applications, systems, servers, storage and the network? This paper identifies a framework for the dominant reference macro architectures that play a critical role in each of these areas and puts them into a single perspective. Note that many architectural models are discussed in today's IT industry, such as "Network-Centric Architecture," "Distributed Component Architecture," "N-Tier Architecture," "Service-Driven Architecture," "Adaptive Compute Architecture," "Storage Network Architecture," "Standards-based Architecture" and so on. The STAR framework highlights the fact that each one of these generic architectures is mainly applicable in one area or the other (such as application, storage, and so on), and when combined they offer powerful synergistic values. This alignment of architectures and infrastructure layers is listed in Table 2.
Table 2: STAR infrastructure layers and associated architectures Layering of an Enterprise Technology Environment Implementing the various architectures previously described requires a combination of hardware, software, and networking technologies. Platform requirements such as processor speed, storage access, network bandwidth, operating system environment, and application infrastructure can vary considerably, depending on the services being implemented, and can change significantly over time. As technology evolves, services become more sophisticated, and transaction volumes explode. Rapid development of a service-driven and network-centric architecture depends on creating a flexible platform (network hardware and software) environment that adapts rapidly in response to changing platform requirements. Service-oriented processing is complex. It requires the development of network-callable services that can handle:
New classes of system software have rapidly evolved over the last several years to support these complex requirements. Web servers, application servers, LDAP servers, ORDBMS, message-oriented middleware, transaction managers, content servers, ORB servers, proxy servers, WAP servers, portal servers, and so on provide most of the fundamental processing requirements, allowing developers to focus more exclusively on the business-specific aspects (functional requirements) of Web-based application development -- for example, business services. Many of these products can be combined to provide a complete application-hosting environment, totally independent of the underlying hardware and operating system environment. Today it is these products, and not the underlying hardware and OS, that present the processing facilities necessary for application development. Keeping this in mind, the e-business technology platform can then be divided into two primary layers:
The Lower/Network Infrastructure Layer This logical and physical layering between consumption of processing power and memory (in the server infrastructure layer) and storage (in the storage infrastructure layer) lays the groundwork for a flexible and adaptive infrastructure. It also addresses many issues faced currently in the e-business world: scalability (support for higher number of nodes in a cluster); manageability (online/non-intrusive backups); availability (isolation of back-up traffic, networked mirroring, networked RAID, and so on); security (isolation of data); and compatibility (support for heterogeneous storage subsystems and compute servers). In other words, the lower infrastructure layer comprises all system hardware that forms the necessary infrastructure to run the upper service layer.
The Upper Infrastructure Layer This upper infrastructure layer comprises all the application software in an IT environment, including management software services that manage change, content, configuration, problems, systems, and the network. Included are basic software services such as application servers, Web servers, mail, FTP, HTTP, DNS, DBMS, messaging, transaction management, and CORBA services; also included are business services, such as bill presentment and payment, accounts receivables, inventory management, and others. Key to this upper layer are portal services for such sites as a B2C portal (for consumers), a B2B portal (for business partners and suppliers), and a B2E portal (for enterprise employees), each offering a window on a combination of services for a particular type of user population. The upper infrastructure layer includes the application infrastructure layer and the virtual application layer. The Application Infrastructure Layer: This layer is the combination of all the basic services that form the application platform, and it includes such services as DBMS, FTP, DNS, e-mail, search engine, doc management, and so on. The Virtual Application Layer: This layer essentially covers the Java technology components, such as presentation logic (JSP/servlets), business/application logic (EJB), and data access logic (JNI, JNDI, JMS, JTS, JDBC, Java IDL, and so on). This virtual layer introduces a virtual machine (in the browser/client side, in the Web server/application server, and in the database server side) between application components and the basic services. Many upper-infrastructure layer products today are available on a wide variety of lower infrastructure layers. Applications written to run on these products can be deployed on any compute/storage infrastructure layer. These applications may remain unaffected by future hardware and operating-system changes because of their independence from the underlying lower infrastructure layer. But there remains the potential problem of their dependence on the upper infrastructure software; this product space is radically changing today. Applications need to maintain independence from the application infrastructure layer, too. Most Popular Application Servers Compliant With J2EE Technology
(Note: There are many more vendors that offer application server platforms compliant with J2EE technology, and almost all of them support their application servers on at least three different operating systems.) It is the requirement for application independence that gives rise to the virtual application layer, which provides standard APIs and specifications. By writing applications so that they are dependent only on virtual application layer APIs, developers can be assured their applications are truly portable. Beyond application servers, vendors also offer other servers that are J2EE technology-compatible: messaging servers based on JMS specifications, transaction servers based on JTS specifications, and CORBA servers based on Java IDL specifications. There are several Java technology application vendors (ERP, CRM, SCM, and so on) who offer their applications as EJB components that will run on any application-server platform. Each of the generic macro architectures discussed earlier plays a significant role in each of these layers. (The storage network architectures and server-centric architectures are most applicable to the lower infrastructure layer.) Whereas the service-driven architecture is applicable to both layers -- the upper infrastructure layer and application infrastructure layer -- component-based architecture relates to the virtual application layer, and the n-tier architecture primarily relates to the application infrastructure layer (as depicted in Figure 1). Relationships between STAR and Generic Architecture Layers
The IT architect must work in conjunction with the systems architect, application architect, storage architect, and so on, to ensure that the guiding principles behind each of these architectures are adhered to and implemented at each one of the layers. There are synergies associated with all of these architectures that need to be harnessed. An obvious example is the storage/server/network consolidation (sometimes referred to as "architecture consolidation"), where the goal is to make the entire network act as one gigantic super computer that scales in storage capacity and processing power as a business grows. Historically, in computing environments where a specific client/server application met a specific need of the organization, hosting this application in a computer (low-end, midrange, or high-end) made sense in an enterprise with a defined set of users, a regular timeframe (for example, 8:00 a.m. to 8:00 p.m., Monday through Friday), and defined geographical boundaries. But in today's highly integrated e-business application, which must cater to the needs of the entire stakeholder community (consumers, other businesses, suppliers, distributors, investors, employees, and others), at all times and at any location, having just one or a few individual computers (even the high-end servers) does not meet these requirements. Therefore, the consolidation of the network, servers, and storage to form a gigantic computer (that might start off with 4 nodes and grow up to 400 nodes) is where the potential synergies of these three architectures at the lower layer lie. Similarly, at the upper infrastructure layer, the service-driven architecture is dominant, as it combines all the services offered by an enterprise into an e-business solution. At the same time, each of these services is based on a component architecture, which is what makes it possible to layer them into multiple tiers in the first place. You can think of this as one computer (the network) and one application (the combination of all interlinked services). The "-ilities" and Why They Matter in E-Business At Sun, when we speak of e-business, we refer to "the -ilities," that is, the requirements, such as scalability, compatibility, reliability, and so on, that are fundamental to doing business on the Web. Let's review these briefly in an effort to tie them to the real world of the enterprise.
Scalability
Compatibility
Availability
Manageability
Security
Adaptability The Virtual Application Layer with the Distributed Component Architecture
In this layer, the application architect focuses on which J2EE technologies to use for a particular application:
Figures 2 and 3 show how these Internet-facing applications are built based on Java components and related components. The application architect building on the J2EE platform is faced with decisions and such choices as:
J2EE technology is an object-oriented, distributed-component-based platform that defines a set of specifications for how applications based on Java technology are to be designed and integrated. The primary benefits of this distributed component architecture, where an application is sliced and diced into manageable modules and components, are compatibility and manageability. Millions of lines of code are written in a modular fashion and thus are divided into manageable pieces. The component model based on object-oriented principles permits pluggable modules under a common framework for applications. A good example of this is the iPlanet Administration Console, a user interface that permits the administration and management of the basic services such as the iPlanet Directory Server, Mail Server, Web Server and so on. The Console shows plugins to the basic framework relevant to the configuration of the applications. If the RADIUS Server (an optional package for remote users with dial-in authentication services) is part of the configuration, it shows up under the iPlanet Administration Console as another component that can be managed. In fact, component-based J2EE applications make it possible to tier the applications in the application infrastructure layer to offer scalability in the application platform layer. The component-based J2EE technology model means you can locate:
This distributed architecture at the virtual application layer allows for scalability within and outside a compute server box, as well as increased availability, because application components can be distributed redundantly. For example, EJB components can be dispersed among multiple application servers with state/session synchronization, a technique that makes all the application servers "aware" of who is connected and what is being accessed and executed by those users. This architecture also addresses techniques such as "sandboxing" security, which allows for distributing and accessing application components in a secure manner. Beyond sandboxing, deployment descriptors (offered by containers of an application server) in conjunction with protection domains offer descriptive authorization between components at the virtual layer. J2EE technology-based applications can also be built with programmatic authorization, offering a fine-grained model for addressing an application's internal "components" security. Message protection for privacy and integrity is extremely critical for B2B applications where XML messages are sent over the Internet from one site to another. This component-level security is supported by mutual authentication at the virtual layer based on digital signatures and certificates offered by PKI-based basic services. Component-based architectures are appropriate and recommended for all application architectures. Each of the services offered in a service-driven architecture, whether it be a management service, basic service, business service, or a portal service, is essentially an application based on the component model. A component-based application provides an architecture you design, module by module, as opposed to an application architecture you inherit. One of the key benefits of this component-based model is the capability to upgrade one component without impacting the others in the architecture. For example, a decoupled application, where the presentation logic is isolated from the business logic, can be migrated to a new look-and-feel without changing a single line of code in the business logic. This is especially important in an architecture with a combination of buy and build solutions. Here's a further clarifying example: Suppose, as part of your company's e-business initiative, you are making bill presentments to your customers over the Internet. Due to competitive pressure, you face an immediate requirement to augment this bill-presentment component with bill-payment capabilities. If the billing service is based on a component model, introducing the new capability to make bill payments is a fairly simple step. The upgrade and introduction of the new payment component is handled on the server side, a seamless integration with the existing bill presentment module. The only change the customer sees is a new icon in the presentment module that leads him or her to make bill payments. You can extend this further by personalization and cross-selling or up-selling along with the bill view, based on the customer's prior purchase patterns. With this modular approach, an e-business can start with a set of critical business services and then add or remove services and components as the business requirements change. The changes are accomplished at the virtual platform layer with minimal impact on the application platform layer. Components communicate with each other via standards-based protocols, such as CORBA and RMI/IIOP. The Application Infrastructure Layer with an N-Tier Architecture The n-tier architecture addresses scalability by tiering an application's logic (presentation, business, data-access, integration, and so on) between application infrastructure services, thus paving the way for vertical and horizontal scalability based on the tier. This layering is made possible because each application infrastructure platform is focused on a service. For example, most of the presentation logic is handled by the Web servers and portal servers; business logic is executed at the application server; common application data (such as user info) is accessed from directory servers; and transaction data is manipulated from database servers. Note that Java technology is capable of executing in all tiers (browser, Web server, app server, and database server), and this addresses typical performance issues associated with code traversal and network traffic. This layered model also allows for the parallel execution of the application code in all n tiers, where possible. This, in turn, results in more efficient utilization of all network resources and better scalability; and, with bottlenecks avoided, uptime and availability are increased. New levels of security can be achieved with unique security policies between tiers and multiple secure zones within a network. Figures 4, 5, and 6 depict the changes in application architectures from two-tier to three-tier to an n-tier model.
Note: Each image of a Sun Enterprise 10000 system represents an E10000 domain and not an entire E10000 server. Typically there are five tiers in an e-business application:
By logically and physically layering an application like this, each tier can offer both vertical (between multiple CPUs) scalability as well as horizontal scalability (between multiple servers), thus bringing true scalability to the application overall. Three-tier architectures were embraced by most e-business vendors (such as PeopleSoft and SAP) with the advent of application servers, which were introduced to act as the middle tier between the client and the back-end resources. This aided in keeping client-side code to a minimum (averting the issues associated with fat clients) with remote calls to server-side code. In addition, the three-tiered (and potentially n-tiered) approach was soon recognized as well suited for distributing applications over a widely dispersed user population because of the reduction in WAN traffic. And, on the server side, due to connection pooling and multiplexed routing of user connections between set and dynamic connection pools, this approach could result in minimal consumption on the resource tier. The introduction of the n-tier architecture to ERP/CRM/SCM applications makes it possible to extend the same systems to the e-business world with the introduction of few more tiers (such as the presentation and integration tiers). For example, PeopleSoft, when introducing multitiered architecture to its suite of e-business applications, used the Tuxedo Application Server from BEA systems to extend its application architecture to WAN users and to address scalability issues. Later, with the need to deploy these systems over the Internet, PeopleSoft further extended its application with the use of BEA Jolt so it could offer Web-based self-service modules in its application suite. The number of tiers depends on what application infrastructure services the application is going to use. Application infrastructure services are required for an application to run, as opposed to management services that act as a value-added service for specific management functions in the computing environment. These basic application infrastructure services can include the following and more:
Gone are the days of developing a client-server application in which the application logic resided on the client with the requirement to connect, authenticate, and access data from a single database server. Gone are the days of a three-tier architecture in which a client accessed an application server (with the traditional application server acting mostly as a transaction-processing monitor), which, in turn, accessed the database over a LAN or a WAN. Today's service-driven, n-tier environments can incorporate a dozen-plus tiers in the application architecture, enough to adequately serve an enterprise's entire stakeholder community. The Upper Infrastructure Layer with the Service-Driven Architecture
The upper layer consists of portal and business services in the virtual layer; basic services in the application infrastructure layer; and management services in both the virtual application layer and the application infrastructure layer (as depicted in Figure 8). Business and portal services are built on top of the basic services. Basic services include mail, FTP, SNS, HTTP, LDAP, DHCP, DBMS, backup, storage, and so on. Business services include product configuration service, billing service, inventory service, shipping service, and others that might be offered via portal services, such as a portal service for purchasing computers online. Management services include all traditional IT management tools, such as problem management services (Remedy), change management services (Atria Clearcase), network management services (Tivoli TME), systems management services (SMC 3.0), and so on. The primary benefit of this architecture is that it offers a set of integrated services based on open standards technologies and protocols, thus addressing compatibility issues. Using techniques such as caching and replication, these services can scale to meet a large number of concurrent users. The basic services are generally clustered at the system level to address availability issues, especially as these basic services are shared by all the applications in the computing environment. One such basic service is the certificate management system, based on PKI, which offers security for e-business applications. In this service-driven architecture, a business is essentially made up of network-callable services, where each service is defined as to what it does, who consumes it, and which other services depend on it. Following is a brief description of the four distinct service groupings:
Management Services
Basic Infrastructure Services
Business Services
Portal Services The Service-Driven Architecture The basic services are typically the same in all enterprises and are built on standards such as HTTP, HTTPS, IMAP, POP, SMTP, LDAP, FTP, XML, and so on. These basic services form the vital application infrastructure on which the business services and portal services depend. The business services differ from one enterprise to the other. In many cases these services are built by the enterprise on top of the basic service tier; in other cases, they are purchased as off-the-shelf solutions (for example, commercially available J2EE technology-compliant applications). There may be hundreds of portal services in an enterprise and a few that aggregate these portal services into (master) portal services. (An employee portal might be an aggregate of several portals: mobile employee, management, professional service division, and so on.) This depends on the size of an enterprise and the extent to which it has migrated to an e-business model. Portal services are also available today as off-the-shelf products. Keeping in mind the fundamental differences between the four types of services, you can view this architecture as a hierarchical tree of network services offered via a few portal services. The four services may be combined to form one e-business application. Each service is meant to offer something unique in value not offered (other than for availability purposes) by other services in an e-business architecture. For example, there is only one directory server based on a standard protocol (LDAP) that offers directory information to other basic services (such as e-mail) and business services (such as customer contact application). If an application requires forewarning a customer that his or her bill payment is due in two business days, this application is going to tap into the e-mail service to automatically forewarn customers about the upcoming deadline. Viewing a business from this perspective reveals that a service-driven architecture naturally compels the enterprise toward a completely integrated e-business application that is modular and componentized. In essence, all the services in a network combined make up the e-business system in its entirety. This centralization of the network implies one physical network (one site -- or two sites when an entire e-business site is replicated geographically for availability purposes). Similarly, portal services that cater to the needs of the individual user community (for example, B2B, B2C, or B2E -- business-to-employee -- communities) are using a combination of basic and business services that actually overlap in many cases between each user population. While there are no clear distinctions between the services that form the required applications for B2C, B2B, or B2E, the distinctions are made possible virtually, via the concepts associated with portals. The Lower/Network Infrastructure Layer with the Network-Centric Architecture
The lower layer is dominated by hardware and related technologies such as routers, switches, gateways, servers, storage subsystems, and back-up devices. As Sun says, these combine to make the network a computer. The main driving force behind this architecture is the need to keep the client side as thin as possible but preserve the capability to run anytime, anywhere, from any device. All logic, including the presentation logic, is executed on the server side, leaving the client side ultra-thin. This point is made clear in Figures 9 and 10.
Adopting techniques such a virtual LANs, switching, dynamic bandwidth allocation, and storage consolidation, the network can scale to address capacity and performance demands. Dual network access points and redundant connectivity mean high availability for the network. Intranet and extranet merge into one network entity with multiple security mechanisms. Jini and Java technologies on all network devices address the network connectivity and compatibility issues. Secure communication to and from this network is made possible through techniques associated with virtual private networks and PKI. To summarize, in the service-driven architecture, the services that drive your architecture are housed primarily on the server side with support for thin clients. This represents a break away from the traditional client/server model of computing, where there were fat clients with many resources residing on both the client side and the server side. These systems used to be islands that met specific requirements but did not integrate with the rest of the systems in the enterprise. These systems were offered via the network: LANs for internal users, WANs for geographically distributed users, and VANs (value-added networks) for secure access to EDI-type applications between businesses. With the introduction of the Internet, applications deployed over a LAN were a good fit for deployment over an intranet; applications offered over a WAN were a good fit with the Internet; and applications offered over a VAN were a good fit with extranets (or VPN). Today the common interface for all these applications is the browser, built on standard protocols. The users of these systems have seamless access to all applications via portals that connect to the Internet. In many cases, applications with sensitive security requirements are accessible over the Internet via a VPN setup. The added advantage of this model is that it avoids costs associated with building private networks. The Server Layer with the Adaptive Compute Architecture
The server layer is where all compute resources are isolated, all the processors and memory that execute code. The central principle behind this architecture is the dynamic allocation of resources to system domains (a logical grouping of CPU/memory) based on demand. The dynamic system domain concept applies to all physical boxes in the network and can reside within a box or between boxes. The adaptive nature of the server resources makes it possible, for example, to take additional resources from domains in the network that may not be as active at the time and put them toward the database domain during peak periods. This adaptive nature of the computing infrastructure is depicted in Figures 11 and 12.
The primary techniques that make this possible are automatic dynamic reconfiguration and dynamic systems domain. With the combination of the UltraSPARC port design and the Solaris Operating Environment, a symmetric multiprocessing operating system, the server systems in the network can scale dynamically. Redundant and hot-swappable hardware components increase the uptime, adding to the adaptive nature of this system, and manageability is maintained through server consolidation techniques. The compatibility at the server layer is established by adhering to standards such as OS versions, patch levels, and common communication protocols. Additionally, box or domain tightening addresses security at this layer through modifications to specific configuration files such as the The next generation of Sun servers based on UltraSPARCIII take the adaptive nature of compute resources to new levels, with support for hot CPU upgrades, dynamic system domains across all servers, scalable shared memory, remote shared memory, dynamic dispatching of kernel and other OS patches, Solaris Resource Manager, Solaris Bandwidth Manager, and so on. These technologies combined offer the capability to assign compute resources on the fly to the applications as they need them. The Storage Infrastructure Layer with the Storage Network Architecture
The storage infrastructure layer holds all the data of the e-business and offers significant value to both the server and the network layer. The most significant point about this architecture is that it offers a shared storage model as opposed to a dedicated storage model. It isolates all storage devices in a "storage area network," or SAN, including network-attached storage, solid-state disks, backup devices, optical disks and so on (as shown in Figures 13 and 14).
The primary benefit of the storage-area network architecture is the manageability it offers for "server-free" and "LAN-free" backups and restores. It also makes simpler site-to-site replication for disaster recovery, due to the isolation of all data to the SAN. Networked RAID implementations on a SAN distribute I/O, not just between disks within storage subsystems, but also between disks and storage subsystems. The SAN architecture also makes it easier to add nodes in a cluster configuration, due to its "any-to-any" connectivity model. With the adherence to standards bodies, such as SNIA, and standard storage technologies such as Jiro, the SAN architecture significantly addresses compatibility issues. Data-level and application-level security can be maintained within the SAN via SAN-enabled logical volume management solutions. This storage-centric architecture is a natural fit with the service-driven, server-centric architecture and is built on the idea of storage consolidation. If an enterprise is based on a service-driven model, all services are essentially code that access application/user data and configuration information (metadata). In this case, metadata is information about the configuration of the services themselves as well as about the interdependencies between these services. Round-the-clock access to such data is critical to the success of the service-driven architecture, and the storage network architecture addresses this key requirement. In this storage-centric architecture, all services offered by an enterprise utilize storage resources from a central storage area network (SAN). When you combine the SAN with the service area network, the system truly demonstrates "The Network Is the Computer", because all processors and memory are divided between the servers that run services with access to data storage from the SAN. Portal services point to the many business services that are required by a particular user community -- with access based on security policies and profiles -- and business services use underlying basic services that access data from a central SAN (or two). This means that today's enterprise network works like a well-oiled machine to offer the services needed by employees, customers, suppliers, and partners.
Table 3: The "-ilities" and all infrastructure layers What is Strategic about this Technology Architecture Roadmap? The synergy of these combined architectures yields their strategic value in both the upper and the lower infrastructure layers. Component-based applications (distributed component architecture) makes it easier to tier them in the n-tier architecture, and a tiered approach exploits the basic shared services model in the service-driven architecture. Similarly, in the lower network layer, centralization governs all three layers: network, server, and storage. The adaptive server architecture fits easily into the shared storage model in the storage network architecture, because shared storage is the number one requirement to make this dynamic system domain technology work across all nodes in a network. The beauty of this framework is that it provides insight into the past, present, and future. Centralized architectures prevailed in the enterprise during the mainframe era; distributed architectures prevailed during the PC era; and a mixture of both -- centralized distribution architecture -- is going to rule the Internet era. The upper infrastructure layer is driving the concept of component, application, and service distribution, whereas the lower layer proposes centralization within one network (again, "The Network is the Computer") -- as demonstrated by a composite of a SAN and a LAN with server/storage consolidation. Every enterprise develops an e-business plan that addresses the direction an enterprise will take, both in the short and long run, to meet the needs of a digital economy. Sun's STAR (Strategic Technology Architecture Roadmap) focuses purely on the architecture piece, the framework upon which all e-business systems will be built. |
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||