Print-friendly Version

Configuring Sun Java System Identity Manager as a Service for the Solaris 10 OS

Scott Fehrman, August 2005

Overview

The Sun Java System Identity Manager provides a "service" that addresses the provisioning life cycle for corporate user identities. This provisioning service is delivered as a complete solution. Identity Manager implements the solution by leveraging many underlying technologies. Identity Manager requires, at a minimum, a JavaServer Pages (JSP) technology-based servlet container for its execution and a RDBMS for its repository. If the underlying "technologies" aren't available, then Identity Manager can't function. The use of Identity Manager is critical to customer operations. The monitoring of Identity Manager, and its dependencies, is a requirement when building a highly available solution.

This document explains how to configure the Identity Manager service in the Solaris 10 Operating System. The example configuration uses the Solaris 10 03/05 release, MySQL 4.0.15, Tomcat/Apache and Identity Manager 5.0 (sp3). The MySQL and Tomcat/Apache packages are provided as part of the Solaris 10 03/05 release.

From Processes to Services

Traditionally, applications and utilities have been implemented as independent processes (daemons). These processes were usually started via different "run levels" of the operating system. Within a given run level, the order in which processes were started relied upon how a start-up script was named. This technique of starting and stopping processes did not address many administrative issues:

• Establishing dependencies between applications and utilities
• Persistent enabling and disabling
• Monitoring and restarting of underlying processes
• Consistent management of state and logging information

The Solaris 10 OS introduced a facility which dramatically improves the management of applications and infrastructure utilities. The Service Management Facility (SMF) in the Solaris 10 OS provides a means to manage "services" instead of "processes". Within the SMF, a service knows:

• How to start itself
• How to stop itself
• What underlying processes it has to manage
• What services it depends upon
• What services depend on it

Service Management Facility

The Service Management Facility (SMF) is a complete framework for defining, deploying and managing services within the Solaris 10 OS. The SMF is started when the Solaris 10 OS is booted. During the booting process, the SMF Master Restarter (svc.startd) is initialized. The Master Restarter is responsible for starting and monitoring of all the SMF services. Services are stored in the SMF Repository. Within the Repository, services are stored in a hierarchical namespace (see Figure 1).

Figure 1: Service Management Facility Repository Namespace

It's recommended that new services be deployed under the appropriate branch of the namespace. Each service is defined by a SMF manifest. The manifest is an XML file that defines the service's methods, dependencies and other attributes. The service's methods typically include "start" and "stop". If a service requires another service to be "enabled", then they're defined as "dependencies". For more information related to SMF in the Solaris 10 OS, refer to the online documentation at http://docs.sun.com.

Prerequisites

The "services" created in this document require that the following installations have been completed:

• Solaris 10 (03/05) or higher
• Solaris 10 integrated MySQL database
• Solaris 10 integrated Apache/Tomcat (web server and JSP/Servlet container)
• Unbundled SJS Identity Manager 5.0 (sp3), using the above components

Architecture

Two services will be created. The first service, called mysql, will manage the MySQL database and will have a dependency on the local file system service. The mysql service will replace any existing start-up scripts in /etc/init.d and/or /etc/rc*. The second service will be called idmgr and it will control the Identity Manager web application. The idmgr service will actually be managing the Apache/Tomcat web server. The idmgr service will replace the existing /etc/init.d/apache and /etc/rc*/*apache scripts.

Notice: If the integrated Apache/Tomcat web server is already in use by other (non-Identity Manager) web sites, you might want to use an unbundled JSP/Servlet container such as the Sun Java System Web Server or the Sun Java System Application Server.

The two new services will be implemented per Figure 2 below.

Figure 2: Architecture of New SMF Services

The idmgr service will depend upon the mysql service being in the online state. The mysql service will depend upon the local file system service being in the online state. The idmgr service will be configured to restart if the mysql service is restarted for any reason.

Converting MySQL into a Service

Remove Legacy Start-Up Process

The integrated MySQL (RDBMS) Solaris package provides a maintenance script for starting, stopping and restarting. This script is located in /etc/sfw/mysql and is called mysql.server. The default Solaris 10 installation doesn't reference this script during the traditional start-up process. The system administrator may have copied (or linked) this script into the traditional start-up directories. You will need to remove any MySQL start-up scripts in /etc/rc*.d and /etc/init.d.

Use the following commands to check for any existing mysql start-up scripts:

# find /etc/rc* /etc/init.d | grep -i mysql
/etc/rc0.d/K01mysql
/etc/rc1.d/K01mysql
/etc/rc2.d/K01mysql
/etc/rc3.d/S99mysql
/etc/init.d/mysql

If the above command finds start-up scripts (as shown above), they need to be removed or relocated. Remove the scripts from the /etc/rc* directories:

# rm /etc/rc0.d/K01mysql
# rm /etc/rc1.d/K01mysql
# rm /etc/rc2.d/K01mysql
# rm /etc/rc3.d/S99mysql

Relocate the script in /etc/init.d to an alternate location:

# mv /etc/init.d/mysql /var/tmp

Create SMF mysql Manifest

SMF in the Solaris 10 OS has established a directory structure for storing SMF service manifests. The base directory for SMF manifests is /var/svc/manifest. For organizational purposes, as root, create a new directory called database under the application directory. Create a text file called mysql.xml and insert the XML manifest that defines the MySQL service (see below). The mysql manifest XML file needs to be copied into this directory.

# mkdir /var/svc/manifest/application/database
# vi mysql.xml (insert the manifest below)
# cp  mysql.xml /var/svc/manifest/application/database

The complete manifest for the mysql service is listed in the following example.

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<!--
    Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
    Use is subject to license terms.
    MySQL.xml : MySQL manifest, Scott Fehrman, Systems Engineer
    updated: 2005-09-16
-->

<service_bundle type='manifest' name='MySQL'>
<service name='application/database/mysql' type='service' version='1'>

   <single_instance />

   <dependency
      name='filesystem'
      grouping='require_all'
      restart_on='none'
      type='service'>
      <service_fmri value='svc:/system/filesystem/local' />
   </dependency>

   <exec_method 
      type='method' 
      name='start'
      exec='/etc/sfw/mysql/mysql.server start'
      timeout_seconds='120' />

   <exec_method 
      type='method' 
      name='stop'
      exec='/etc/sfw/mysql/mysql.server stop'
      timeout_seconds='120' />

   <instance name='default' enabled='false' />

   <stability value='Unstable' />

   <template>
      <common_name>
         <loctext xml:lang='C'>MySQL RDBMS 4.0.15</loctext>
      </common_name>
      <documentation>
         <manpage title='mysql' section='1' manpath='/usr/sfw/share/man' />
      </documentation>
   </template>

</service>
</service_bundle>

Import the Manifest into the Repository

Placing the XML file into the SMF directory does not mean the mysql service is ready for use. The manifest needs to be validated and imported into the Solaris 10 SMF Repository. Validate and import the XML file into the Repository with the svccfg (Service Configuration) command:

# svccfg validate /var/svc/manifest/application/database/mysql.xml
# svccfg import /var/svc/manifest/application/database/mysql.xml

Check the Service

The service will automatically start when the Solaris 10 OS boots if the service is enabled. The XML file contains the line <instance name='default' enabled='false' />, which means the service will not be enabled when it's imported into the Repository. Check the status of the service with the svcs command:

Status of the service:

# svcs mysql
STATE         STIME    FMRI
disabled      12:45:34 svc:/application/database/mysql:default

Converting Apache/Tomcat into the idmgr Service

Notice: The Solaris 10 OS contains two versions of Apache; "apache" and "apache2". The "apache" Solaris package uses legacy start-up and shut-down scripts. This example will convert the "apache" package into a SMF service. The "apache2" Solaris package is already implemented as a SMF service. This example will not involve the "apache2" service.

Remove Legacy Start-Up Process

The integrated Solaris package containing Apache/Tomcat (web server/JSP/Servlet container) has scripts for starting, stopping, and restarting itself. You will need to remove and/or relocate the Apache/Tomcat start-up scripts in /etc/rc*.d and /etc/init.d.

Use the following commands to check for any existing mysql start-up scripts:

# find /etc/rc* /etc/init.d | grep -i apache
/etc/rc0.d/K16apache 
/etc/rc1.d/K16apache 
/etc/rc2.d/K16apache
/etc/rc3.d/S50apache 
/etc/rcS.d/K16apache 
/etc/init.d/apache 

The scripts in the /etc/rc* directories need to be deleted:

# rm /etc/rc0.d/K16apache
# rm /etc/rc1.d/K16apache
# rm /etc/rc2.d/K16apache
# rm /etc/rc3.d/S50apache
# rm /etc/rcS.d/K16apache

The idmgr service will use the script in the /etc/init.d directory for starting and stopping the service. Use the following command to move and rename the script:

# mv /etc/init.d/apache /etc/apache/apache.sh

Create the SMF idmgr Manifest

SMF in the Solaris 10 OS has established a directory structure for storing SMF service manifests. The base directory for SMF manifests is /var/svc/manifest. For organizational purposes, as root, create a new directory called web under the application directory. Create a text file called idmgr.xml and insert the XML manifest that defines the Identity Manager service (see below). The idmgr manifest XML file needs to be copied into this directory.

# mkdir /var/svc/manifest/application/web
# vi idmgr.xml (insert the manifest below)
# cp idmgr.xml /var/svc/manifest/application/web 

The complete manifest for the idmgr service is listed in the following example.

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<!--
    Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
    Use is subject to license terms.
    IdMgrApache.xml : Identity Mgr manifest, Scott Fehrman, Systems Engineer
    updated: 2005-09-16
-->

<service_bundle type='manifest' name='Identity Manager'>
<service name='application/web/idmgr' type='service' version='1'>

   <single_instance />

   <dependency
      name='mysql'
      grouping='require_all'
      restart_on='restart'
      type='service'>
      <service_fmri value='svc:/application/database/mysql' />
   </dependency>

   <exec_method 
      type='method' 
      name='start'
      exec='/etc/apache/apache.sh start'
      timeout_seconds='120' />

   <exec_method 
      type='method' 
      name='stop'
      exec='/etc/apache/apache.sh stop'
      timeout_seconds='120' />

   <instance name='default' enabled='false' />

   <stability value='Unstable' />

   <template>
      <common_name>
         <loctext xml:lang='C'>SJS Identity Manager 5.0-sp3</loctext>
      </common_name>
      <documentation>
         <manpage title='apache' section='1' manpath='/usr/man' />
      </documentation>
   </template>

</service>
</service_bundle>

Import the Manifest into the Repository

Placing the XML file into the directory does not mean the idmgr service is ready for use. The manifest needs to be validated and imported into the Solaris 10 SMF Repository. Validate and import the XML file into the Repository with the svccfg (Service Configuration) command:

# svccfg validate /var/svc/manifest/application/web/idmgr.xml
# svccfg import /var/svc/manifest/application/web/idmgr.xml

Check the Service

The service will automatically start when the Solaris OS boots if the service is enabled. The XML file contains the line <instance name='default' enabled='false' /> , which means the service will not be enabled when it's imported into the Repository. Check the status of the service with the svcs command:

Status of the service:

# svcs idmgr 
STATE          STIME    FMRI
disabled       14:26:42 svc:/application/web/idmgr:default

List the services that idmgr depends upon:

# svcs -d idmgr
STATE          STIME    FMRI
disabled       12:45:34 svc:/application/database/mysql:default

Administering and Monitoring the Services

The new mysql and idmgr services are now available. Per their manifest files, they were imported into the SMF Repository disabled. Because the idmgr service has a dependency on the mysql service, the SMF will automatically handle the proper starting, stopping and restarting of the services. Disabled services will not be started when the Solaris OS boots. Enabled services will automatically start when the Solaris OS boots.

Enabling the Services

Because we have a dependency between these two services, there are few ways to manually enable the services. SMF in the Solaris OS will start "enabled" services in the proper order based on dependencies between the services.

Use Case 1:

Enable the mysql service first, then enable the idmgr service. Enabling the mysql service first means that when the idmgr starts, idmgr can reach the online state.

# svcs idmgr mysql
STATE          STIME    FMRI
disabled       14:39:51 svc:/application/database/mysql:default
disabled       14:41:30 svc:/application/web/idmgr:default
# svcadm enable mysql 
# svcs mysql 
STATE          STIME    FMRI 
online        14:51:16 svc:/application/database/mysql:default 
# svcadm enable idmgr 

# svcs idmgr 
STATE         STIME    FMRI 
online        14:57:54 svc:/application/web/idmgr:default

Use Case 2:

Enable idmgr service first, then enable the mysql service. Enabling the idmgr service first will cause it to enter the offline state because it's dependent upon a service that isn't online. The idmgr service will remain in the offline state until all problems are resolved and/or dependents are online. After the mysql service is enabled, the idmgr service will automatically transition to the online state.

# svcs idmgr mysql
STATE          STIME    FMRI
disabled       10:09:52 svc:/application/database/mysql:default
disabled       10:13:00 svc:/application/web/idmgr:default
# svcadm enable idmgr
# svcs idmgr mysql
STATE          STIME    FMRI
disabled       10:09:52 svc:/application/database/mysql:default
offline        10:14:09 svc:/application/web/idmgr:default
# svcadm enable mysql
# svcs idmgr mysql
STATE          STIME    FMRI
online         10:15:37 svc:/application/database/mysql:default
online         10:16:42 svc:/application/web/idmgr:default

Use Case 3:

Only enable the idmgr service. The SMF administration command, svcadm, supports an option that will recursively enable dependent services when a given service is enabled. Enabling the idmgr service with the -r option will cause its dependent mysql service to be enabled before the idmgr service is enabled.

# svcs idmgr mysql 

STATE          STIME    FMRI 
disabled       15:39:51 svc:/application/database/mysql:default 
disabled       15:41:30 svc:/application/web/idmgr:default 
# svcadm enable -r idmgr 
# svcs idmgr mysql 
STATE          STIME    FMRI 
online         15:44:28 svc:/application/database/mysql:default
online         15:44:28 svc:/application/web/idmgr:default

Monitoring the Services

The Solaris SMF svcs command has useful options for monitoring the services.

List a service's dependencies:

# svcs -d idmgr 
STATE          STIME    FMRI 
online         15:44:28 svc:/application/database/mysql:default 

List other services that depend on a given service:

# svcs -D mysql
STATE          STIME    FMRI 
online         15:44:28 svc:/application/web/idmgr:default

List the service's details:

# svcs -l idmgr mysql 
fmri        svc:/application/web/idmgr:default 
name        SJS Identity Manager 5.0-sp3 
enabled     true 
state       online 
next_state  none 
state_time  Fri  Mar 25 15:44:28 2005 
logfile      /var/svc/log/application-web-idmgr:default.log  
restarter    svc:/system/svc/restarter:default 
contract_id 101 
dependency   require_all/restart svc:/application/database/mysql (online)

fmri         svc:/application/database/mysql:default
name        MySQL RDBMS 4.0.15 
enabled     true 
state       online 
next_state  none 
state_time  Fri Mar 25 15:44:28 2005 
logfile     /var/svc/log/application-database-mysql:default.log 
restarter   svc:/system/svc/restarter:default 
contract_id 100 
dependency   require_all/none svc:/system/filesystem/local (online)

List the processes associated with a service:

# svcs -p idmgr mysql
STATE          STIME    FMRI
online         15:44:28 svc:/application/database/mysql:default
               15:44:28     1303 mysqld_safe
               15:44:28     1319 mysqld
online         15:44:28 svc:/application/web/idmgr:default
               15:44:28     1335 java
               15:44:29     1340 httpd
               15:44:30     1341 httpd
               15:44:30     1342 httpd
               15:44:30     1343 httpd
               15:44:30     1344 httpd
               15:44:30     1345 httpd

Stopping and Starting the Services

From time to time, services need to be restarted. The svcadm utility is used to restart a SMF service. Use the svcs -p <service> command to verify that the service was restarted. The service's processes should be different. You might need to wait after issuing the disable and enable commands before you re-display the service's processes; it could take a little time to stop and start them.

Restart the idmgr service:

# svcs -p idmgr 
STATE          STIME    FMRI
online         11:37:08 svc:/application/web/idmgr:default
               11:37:08     1282 java
               11:37:11     1290 httpd
               11:37:12     1291 httpd
               11:37:12     1292 httpd
               11:37:12     1293 httpd
               11:37:12     1294 httpd
               11:37:12     1295 httpd
               11:38:04     1355 httpd
               11:38:05     1357 httpd
               11:38:05     1358 httpd
# svcadm restart idmgr
# svcs -p idmgr
STATE          STIME    FMRI
online         12:00:19 svc:/application/web/idmgr:default
               12:00:19     1475 java
               12:00:20     1480 httpd
               12:00:21     1481 httpd
               12:00:21     1482 httpd
               12:00:21     1483 httpd
               12:00:21     1484 httpd
               12:00:21     1485 httpd

A service that has other services depending on it, can be configured to restart those dependents when the service is restarted. The mysql service has been configured to restart the idmgr service when it is restarted. You can verify that the services were restarted by displaying the processes associated with both services.

Restart the mysql service:

# svcs -p idmgr mysql

STATE          STIME    FMRI
online         11:37:07 svc:/application/database/mysql:default
               11:37:07     1252 mysqld_safe
               11:37:08     1287 mysqld
online         12:00:19 svc:/application/web/idmgr:default
               12:00:19     1475 java
               12:00:20     1480 httpd
               12:00:21     1481 httpd
               12:00:21     1482 httpd
               12:00:21     1483 httpd
               12:00:21     1484 httpd
               12:00:21     1485 httpd
# svcadm restart mysql
# svcs -p idmgr mysql
STATE          STIME    FMRI
online         12:07:49 svc:/application/database/mysql:default
               12:07:48     1516 mysqld_safe
               12:07:49     1541 mysqld
online         12:08:54 svc:/application/web/idmgr:default
               12:08:54     1563 java
               12:08:55     1568 httpd
               12:08:56     1569 httpd
               12:08:56     1570 httpd
               12:08:56     1571 httpd
               12:08:56     1572 httpd
               12:08:56     1573 httpd

Diagnosing Service Issues

Get a verbose explanation of the service's state. The author of the manifest might have included documentation information such as UNIX man pages. The documentation data is displayed when the -vx option is used.

# svcs -vx idmgr mysql

svc:/application/web/idmgr:default (SJS Identity Manager 5.0-sp3)
 State: online since Fri Mar 25 15:44:28 2005
   See: man -M /usr/man -s 1 apache
   See: /var/svc/log/application-web-idmgr:default.log
Impact: None.

svc:/application/database/mysql:default (MySQL RDBMS 4.0.15)
 State: online since Fri Mar 25 15:44:28 2005
   See: man -M /usr/sfw/share/man -s 1 mysql
   See: /var/svc/log/application-database-mysql:default.log
Impact: None.

Service log files: The Solaris SMF maintains a log file for each service. The log file can be reviewed when services don't enable.

# svcs -l idmgr mysql | grep logfile
logfile      /var/svc/log/application-web-idmgr:default.log
logfile      /var/svc/log/application-database-mysql:default.log
# tail /var/svc/log/application-web-idmgr:default.log
[ Mar 25 15:41:30 Disabled. ]
[ Mar 25 15:44:28 Enabled. ]
[ Mar 25 15:44:28 executing start method ("/etc/apache/apache.sh start") ]
httpd starting.
[ Mar 25 15:44:28 Method "start" exited with status 0 ]
[Fri Mar 25 15:44:28 2005] [warn] module mod_webapp.c is already added, 
skipping/usr/apache/bin/apachectl start: httpd started
# tail /var/svc/log/application-database-mysql:default.log
Wait for mysqld to exit050325 15:39:51  mysqld ended done
[ Mar 25 15:39:51 Method "stop" exited with status 0 ]
[ Mar 25 15:44:28 Enabled. ]
[ Mar 25 15:44:28 executing start method ("/etc/sfw/mysql/mysql.server start") ]
[ Mar 25 15:44:28 Method "start" exited with status 0 ]
Starting mysqld daemon with databases from /var/mysql

Accessing the Repository

SMF in the Solaris OS provides the svccfg command for administrating the Repository. It's beyond the scope of this document to completely cover the capabilities of the svccfg command. Please refer to the Solaris documentation for details.

Note: The svccfg command can be used to alter the state of any SMF service; use this utility with caution.

List the service's properties:

# svccfg -s mysql listprop 
general                   framework
general/entity_stability  astring  Unstable
general/single_instance   boolean  true
filesystem                dependency
filesystem/entities       fmri     svc:/system/filesystem/local
filesystem/grouping       astring  require_all
filesystem/restart_on     astring  none
filesystem/type           astring  service
script/grouping           astring  require_all
script/restart_on         astring  none
script/type               astring  path
start                     method
start/exec                astring  "/etc/sfw/mysql/mysql.server start"
start/timeout_seconds     count    120
start/type                astring  method
stop                      method
stop/exec                 astring  "/etc/sfw/mysql/mysql.server stop"
stop/timeout_seconds      count    120
stop/type                 astring  method
tm_common_name            template
tm_common_name/C          ustring  "MySQL RDBMS 4.0.15"
tm_man_mysql              template
tm_man_mysql/manpath      astring  /usr/sfw/share/man
tm_man_mysql/section      astring  1
tm_man_mysql/title        astring  mysql

Removing the Service

When a service is no longer needed, it needs to be removed from the Repository. The svccfg command is used to remove services.

Note: The svccfg command can be used to delete any SMF service; use this utility with caution.

Services need to be disabled:

# svcadm disable idmgr
# svcadm disable mysql
# svcs idmgr mysql 

STATE         STIME    FMRI 
disabled      17:24:23 svc:/application/database/mysql:default 
disabled      17:25:52 svc:/application/web/idmgr:default 

Delete the services from the Repository:

# svccfg delete idmgr 
# svccfg delete mysql 
# svcs idmgr mysql
svcs: Pattern 'idmgr' doesn't match any instances 
svcs: Pattern 'mysql' doesn't match any instances 
STATE         STIME    FMRI 

Summary

We've shown how to configure, install and manage two related services using the Service Management Facility in the Solaris 10 OS. The first service, "MySQL", is part of the standard Solaris 10 distribution. MySQL doesn't include a legacy startup (/etc/init.d) script or a SMF service. We created a service manifest for "MySQL" and deployed it to the SMF in the Solaris 10 OS. The second service, "Identity Manager", runs as an Apache/Tomcat web application. Apache/Tomcat is part of the standard Solaris 10 distribution. Apache/Tomcat (not Apache2) does include a legacy startup (/etc/init.d) script. We converted the legacy startup script into a SMF service.

About the Author

Scott Fehrman is a Systems Engineer in the Identity Management Practice at Sun Microsystems, Inc.

Unless otherwise licensed, code in all technical manuals herein (including articles, FAQs, samples) is provided under this License.