BigAdmin Submitted Article: Plex Mirroring for Business Continuity With Veritas Volume Manager and the Solaris 10 OS

Skip to Content Sun and Oracle Channel Sun How to Buy Log In United States [Change] English

BigAdmin System Administration Portal

Community-Submitted Article

Submit Content | Check BigAdmin Bucks

This content is submitted by a BigAdmin user. It has not been reviewed for technical accuracy by Sun Microsystems, though it may have been lightly edited to improve readability. If you find an error or would like to comment on the article, please contact the submitter or use the comment field at the bottom of the article. Community submissions may not follow Sun trademark guidelines. For information on Sun trademarks, please see http://www.sun.com/suntrademarks/.

Plex Mirroring for Business Continuity With Veritas Volume Manager and the Solaris 10 OS

Pradeep Baby, November 2009

Target Readers: Professionals interested in Veritas who have mid-level knowledge of the Solaris Operating System and Veritas technology.

Introduction

Applications, the network, the OS, database, and storage are the major components of a data center. In huge data centers the data redundancy and business continuity are attained by means of centralized storage with RAID configurations. But here I am interested in configuring and demonstrating a business continuity setup in a small environment using the Solaris OS and Veritas Volume Manager (VxVM) from Symantec. Professionals who have experience with the Solaris OS and Veritas should be able to understand the concepts quickly; I am not discussing the basic commands and concepts.

Brief Description

Here I am using Veritas Storage Foundation 4.1 version on the Solaris 10 OS. I have added three 10 GB disks to the Veritas disk group, keeping one disk as a spare for hot relocation. I created a mirrored volume with two other disks that are for hot relocation use in the same disk group. Here I did not use vxassist for creating the mirrored volumes, but used vxmake for creating the individual elements such as subdisks and plexes. So this article should provide an idea of how to handle the virtual entities of Veritas Volume Manager.

Summary of steps followed in the procedure:

Add the three disks to the disk group.
Make one disk as spare for hot relocation.
Create two different subdisks using the two vmdisks.
Attach the subdisks to two plexes.
Associate the plexes to a volume.
Create the file system using the raw device and mount it on the server.
Fail one of the mirrored disks from the Veritas volume.
Verify that Veritas replaces the failed disk automatically with the spare disk.
Replace the failed disk with another disk and check how Veritas reverts the configuration automatically.

Step-by-Step Procedure for Configuring the Mirror

1. Add the three disks to the disk group.

First, use the vxdisk command to see a one-line summary of all disks known to the system.


# vxdisk list
DEVICE  TYPE        DISK  GROUP STATUS
Disk_1  auto:none   -     -     online invalid
Disk_2  auto:none   -     -     online invalid
Disk_3  auto:none   -     -     online invalid

Add the three disks (Disk_1, Disk_2, and Disk_3) using the vxdiskadm menu-driven utility by creating a new disk group called pbn-test (pbn is the name I chose here).


# vxdiskadm

Executing vxdiskadm will open the menu-driven option for various Veritas tasks.

Select the first option "1" from the menu.
When prompted to select the disk to add, we can view all the available disks by selecting "list." Select the required disk, which in our case would be "Disk_1 Disk_2 Disk_3."
When asked for a disk group, provide the new disk group we need to create. In our case, this would be pbn-test.

For all other options, use the default settings as input. One such option is to include the disk in hot relocation use.

After we add the disk to the VxVM control, the disks are visible as follows:


# vxdisk list
DEVICE  TYPE          DISK       GROUP    STATUS
Disk_0  auto:none     -          -        online invalid
Disk_1  auto:cdsdisk  pbn-test01 pbn-test online
Disk_2  auto:cdsdisk  pbn-test02 pbn-test online
Disk_3  auto:cdsdisk  pbn-test03 pbn-test online

The default disk name pbn-test## has been selected from the disk group name pbn-test.

2. Make one disk as spare for hot relocation.

Assign one disk, for example pbn-test03, as a spare in the disk group pbn-test.


# vxedit -g pbn-test set spare=on  pbn-test03

Use the vxdisk command to see a summary of the disks.


# vxdisk list
DEVICE  TYPE          DISK       GROUP    STATUS
Disk_1  auto:cdsdisk  pbn-test01 pbn-test online
Disk_2  auto:cdsdisk  pbn-test02 pbn-test online
Disk_3  auto:cdsdisk  pbn-test03 pbn-test online spare

Make sure that all other disks are in hot relocation use. As we can see, pbn-test01 and pbn-test02 are already in hot relocation use, as we mentioned when using vxdiskadm. Maybe for future reference we can use these commands to turn on and turn off this facility.

Use the vxedit command to exclude a disk from hot use. Here, to exclude the disk pbn-test01 in the disk group pbn-test, by setting the nohotuse option to on, as follows:


# vxedit -g pbn-test set nohotuse=on  pbn-test01

Check the status of the disks again using the vxdisk command.


# vxdisk list
DEVICE  TYPE          DISK       GROUP    STATUS
Disk_1  auto:cdsdisk  pbn-test01 pbn-test online nohotuse
Disk_2  auto:cdsdisk  pbn-test02 pbn-test online
Disk_3  auto:cdsdisk  pbn-test03 pbn-test online spare

To make the disk pbn-test01 in the disk group pbn-test available for hot use, use the vxedit command to set nohotuse to off, which can be done as follows:


# vxedit -g pbn-test set nohotuse=off pbn-test01

Check the status of the disks.


# vxdisk list
DEVICE  TYPE          DISK       GROUP    STATUS
Disk_1  auto:cdsdisk  pbn-test01 pbn-test online
Disk_2  auto:cdsdisk  pbn-test02 pbn-test online
Disk_3  auto:cdsdisk  pbn-test03 pbn-test online spare

3. Create two different subdisks using the two vmdisks.

# vxprint -ht
Disk group: pbn-test

DG NAME   NCONFIG     NLOG     MINORS   GROUP-ID
ST NAME   STATE       DM_CNT   SPARE_CNT         APPVOL_CNT
DM NAME   DEVICE      TYPE     PRIVLEN  PUBLEN   STATE
RV NAME   RLINK_CNT   KSTATE   STATE    PRIMARY  DATAVOLS  SRL
RL NAME   RVG         KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
CO NAME   CACHEVOL    KSTATE   STATE
VT NAME   NVOLUME     KSTATE   STATE
V  NAME   RVG/VSET/CO KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME   VOLUME      KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME   PLEX        DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME   PLEX        VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME   PLEX        CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME   PARENTVOL   LOGVOL
SP NAME   SNAPVOL     DCO

dg pbn-test     default      default  54000    1250332486.37.f50318-11

dm pbn-test01   Disk_1       auto     2048     20967936 -
dm pbn-test02   Disk_2       auto     2048     20967936 -
dm pbn-test03   Disk_3       auto     2048     20967936 -

From the vxprint output, we can see the maximum size for the disk, so we are creating two different subdisks out of these. The size is shown in block size, and one block is 512 bytes.

We are making use of a complete disk to create the subdisks, so in this case, we are creating subdisk pt-sd1 and pt-sd2 using the vmdisks pbn-test01 and pbn-test02, with starting offset value 0 and LENGTH 20967936.


# vxmake -g pbn-test sd pt-sd1  pbn-test01,0,20967936
# vxmake -g pbn-test sd pt-sd2  pbn-test02,0,20967936

Run the vxprint command again:

# vxprint -ht
Disk group: pbn-test

DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
ST NAME         STATE        DM_CNT   SPARE_CNT         APPVOL_CNT
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
CO NAME         CACHEVOL     KSTATE   STATE
VT NAME         NVOLUME      KSTATE   STATE
V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME         PLEX         CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO

dg pbn-test     default      default  54000    1250332486.37.f50318-11

dm pbn-test01   Disk_1       auto     2048     20967936 -
dm pbn-test02   Disk_2       auto     2048     20967936 -
dm pbn-test03   Disk_3       auto     2048     20967936 SPARE

sd pt-sd1       -            pbn-test01 0      20967936 -         Disk_1   ENA
sd pt-sd2       -            pbn-test02 0      20967936 -         Disk_2   ENA

The output shows we have created two subdisks with the names pt-sd1 and pt-sd2.

4. Attach the subdisks to two plexes.

Like metadevices in Solaris Volume Manager, the file systems are mounted using volumes in Veritas, and these volumes are created by attaching plexes to volumes. These plexes are made of subdisks.

In this case, we are creating two plexes (pt-plex01 and pt-plex02) out of subdisks pt-sd1 and pt-sd2 using the following commands:


# vxmake -g pbn-test plex pt-plex01 sd=pt-sd1
# vxmake -g pbn-test plex pt-plex02 sd=pt-sd2

Run the vxprint command again:

# vxprint -ht
Disk group: pbn-test
:
:
:
dg pbn-test     default      default  54000    1250332486.37.f50318-11

dm pbn-test01   Disk_1       auto     2048     20967936 -
dm pbn-test02   Disk_2       auto     2048     20967936 -
dm pbn-test03   Disk_3       auto     2048     20967936 SPARE

pl pt-plex01    -            DISABLED -        20967936 CONCAT    -        RW
sd pt-sd1       pt-plex01    pbn-test01 0      20967936 0         Disk_1   ENA

pl pt-plex02    -            DISABLED -        20967936 CONCAT    -        RW
sd pt-sd2       pt-plex02    pbn-test02 0      20967936 0         Disk_2   ENA

5. Associate the plexes to a volume.

The volume name is pbn-vol1, and it is used for file system creation. The fsgen option is used.


# vxmake -g pbn-test  -U fsgen vol pbn-vol1 \
plex=pt-plex02,pt-plex01

Next we start the volume.


# vxvol -g pbn-test start pbn-vol1

At this point in time, if you check the task, you can see the synching is happening with the two plexes:


# sudo vxtask list
TASKID  PTID TYPE/STATE    PCT   PROGRESS
174RDWRBACK/R 45.33% 0/20967936/9504768 VOLSTART pbn-vol1
pbn-test

And the vxprint output looks like this:

dg pbn-test     default      default  54000    1250332486.37.f50318-11

dm pbn-test01   Disk_1       auto     2048     20967936 -
dm pbn-test02   Disk_2       auto     2048     20967936 -
dm pbn-test03   Disk_3       auto     2048     20967936 SPARE

v  pbn-vol1     -            ENABLED  ACTIVE   20967936 ROUND     -        fsgen
pl pt-plex01    pbn-vol1     ENABLED  ACTIVE   20967936 CONCAT    -        RW
sd pt-sd1       pt-plex01    pbn-test01 0      20967936 0         Disk_1   ENA
pl pt-plex02    pbn-vol1     ENABLED  ACTIVE   20967936 CONCAT    -        RW
sd pt-sd2       pt-plex02    pbn-test02 0      20967936 0         Disk_2   ENA

Here our volume pbn-vol1 is ready so we can create the file system and mount it.

6. Create the file system using the raw device and mount it on the server.


# mkfs -F vxfs /dev/vx/rdsk/pbn-test/pbn-vol1

Mount the file system on a mount point:


# mount -F vxfs /dev/vx/dsk/pbn-test/pbn-vol2 /pbn-mnt

So the file system is up and ready for use.

7. Fail one of the mirrored disks from the Veritas volume.

We fail one of the vmdisks for the purpose of demonstration. In this case, I removed the disk pbn-test02 from VxVM, and see what is going to happen!


# vxdisk list
DEVICE  TYPE          DISK       GROUP    STATUS
Disk_0  auto:none     -          -        online invalid
Disk_1  auto:cdsdisk  pbn-test01 pbn-test online
Disk_2  auto          -          -        error
Disk_3  auto:cdsdisk  pbn-test03 pbn-test online spare

We see error status for Disk_2 after failing the disk.

8. Verify that Veritas has reconfigured the failed disk.

Here we verify that Veritas has replaced the failed disk automatically with the spare disk.

On the vxtask list output, we got the following status:

TASKID  PTID TYPE/STATE    PCT   PROGRESS
VxVM vxtask WARNING V-5-1-2497 Unable to get disk group record: Record not in disk group
   342           PARENT/R  0.00% 1/0(1) VXRECOVER 0.0 0.0
   343   343     ATCOPY/R 00.10% 0/20967936/20480 PLXATT 0.1051 0.1048 1024.36
:
:
TASKID  PTID TYPE/STATE    PCT   PROGRESS
VxVM vxtask WARNING V-5-1-2497 Unable to get disk group record: Record not in disk group
   342           PARENT/R  0.00% 1/0(1) VXRECOVER 0.0 0.0
   343   343     ATCOPY/R 99.37% 0/20967936/20836352 PLXATT 0.1051 0.1048 1024.36
TASKID  PTID TYPE/STATE    PCT   PROGRESS

From vxtask output, we could see that the spare disk pbn-test03 has been automatically added to the pt-plex02 and the re-synching is happening.

And here is the output from vxprint:

dg pbn-test     default      default  54000    1250332486.37.f50318-11

dm pbn-test01   Disk_1       auto     2048     20967936 -
dm pbn-test02   -            -        -        -        NODEVICE
dm pbn-test03   Disk_3       auto     2048     20967936 SPARE

v  pbn-vol1     -            ENABLED  ACTIVE   20967936 ROUND     -        fsgen
pl pt-plex01    pbn-vol1     ENABLED  ACTIVE   20967936 CONCAT    -        RW
sd pt-sd1       pt-plex01    pbn-test01 0      20967936 0         Disk_1   ENA
pl pt-plex02    pbn-vol1     ENABLED  STALE    20967936 CONCAT    -        WO
sd pbn-test03-01 pt-plex02   pbn-test03 0      20967936 0         Disk_3   ENA

So here VxVM has automatically assigned the spare disk pbn-test03 to the volume pbn-vol1, removing the failed or removed disk pbn-test02.


# vxdisk list
DEVICE  TYPE          DISK       GROUP    STATUS
Disk_1  auto:cdsdisk  pbn-test01 pbn-test online
Disk_3  auto:cdsdisk  pbn-test03 pbn-test online spare
-       -             pbn-test02 pbn-test failed was:Disk_2

This shows Disk_2 failed.

9. Replace the failed disk with another disk and check how Veritas reverts the configuration automatically.

Here we check that Veritas automatically reconfigures the failed disk and makes the spare disk available for reuse.

I made Disk_2 available again for Veritas and reinitialized the disk:


# vxdisksetup  -i  Disk_2

Use the vxdisk command to see a summary of all disks.


# vxdisk list
DEVICE TYPE         DISK       GROUP    STATUS
Disk_1 auto:cdsdisk pbn-test01 pbn-test online
Disk_2 auto:cdsdisk -          -        online
Disk_3 auto:cdsdisk pbn-test03 pbn-test online spare
-      -            pbn-test02 pbn-test failed was:Disk_2

Using the vxdiskadm utility I replaced the failed disk with the new disk:


# vxdiskadm

From the menu, I selected the fifth option to replace a failed disk.

I used "list" when asked for selecting the disk, and from the list I selected the failed disk.

At this point, Veritas shows the disks that are available for replacement. You can select the disk to use as a replacement.

Select the default settings and proceed.


# vxdisk list
DEVICE TYPE         DISK       GROUP    STATUS
Disk_1 auto:cdsdisk pbn-test01 pbn-test online
Disk_2 auto:cdsdisk pbn-test02 pbn-test online
Disk_3 auto:cdsdisk pbn-test03 pbn-test online spare

Thus we can see that everything is reconfigured and Disk_3 is available for hot relocation again.

The information and links on this page have been provided by a BigAdmin user. The submitter is solely responsible for such information and links. Sun is not responsible for the availability of external sites or resources, and does not endorse and is not responsible or liable for any content, advertising, products, or other materials on or available from such sites or resources. Sun will not be responsible or liable, directly or indirectly, for any actual or alleged damage or loss caused by or in connection with use of or reliance on the information posted here, or goods or services available on or through any external site or resource.

Comments (latest comments first)

Discuss and comment on this resource in the BigAdmin Wiki

Unless otherwise licensed, code in all technical manuals herein (including articles, FAQs, samples) is provided under this License.

Search BigAdmin

» Search the HCL

BigAdmin Wiki
Discuss and comment on this resource in the wiki.