Linux Server

Migrating mpio SAN Storage to a Different Array for Linux

When changing SAN arrays, the data needs to migrated from the old array to the new array. After new drives are assigned to the linux server, follow these steps for migrating the data. You will need to get a copy of the inq command for Linux from EMC.

After the new disks from the new array have been added, make sure they are available. Make a note of the serial number (SID) in the right hand column.

(linuxserver)# /nfs/mts/bin/inq.linux -nodots -sid
Inquiry utility, Version V7.3-883 (Rev 1.0) (SIL Version V6.5.1.0 (Edit Level 883)
Copyright (C) by EMC Corporation, all rights reserved.
For help type inq -h.

———————————————————————————–
DEVICE :VEND ROD :REV :SER NUM :CAP(kb) :SYMM ID
———————————————————————————–
/dev/sda :EMC :SYMMETRIX :5773 :20010b0008 : 35354880 :000190300520
/dev/sdb :EMC :SYMMETRIX :5773 :20010b2008 : 35354880 :000190300520
/dev/sdc :EMC :SYMMETRIX :5671 :66001ab000 : 35354880 :000187721466
/dev/sdd :EMC :SYMMETRIX :5671 :66001af000 : 35354880 :000187721466
/dev/sde :EMC :SYMMETRIX :5773 :20010b0008 : 35354880 :000190300520
/dev/sdf :EMC :SYMMETRIX :5773 :20010b2008 : 35354880 :000190300520
/dev/sdg :EMC :SYMMETRIX :5671 :66001ab000 : 35354880 :000187721466
/dev/sdh :EMC :SYMMETRIX :5671 :66001af000 : 35354880 :000187721466
/dev/dm-0 :EMC :SYMMETRIX :5773 :20010b0008 : 35354880 :000190300520
/dev/dm-1 :EMC :SYMMETRIX :5773 :20010b2008 : 35354880 :000190300520
/dev/dm-2 :EMC :SYMMETRIX :5671 :66001ab000 : 35354880 :000187721466
/dev/dm-3 :EMC :SYMMETRIX :5671 :66001af000 : 35354880 :000187721466

After you make sure the drives are available, you need to determine which mpio devices go with which drives.

See what devices are currently active:

(linuxserver)# pvs
PV VG Fmt Attr PSize PFree
/dev/dm-0 uservg lvm2 a- 33.71G 22.71G
Read the the new multipath devices, and see which ones go with the active PV.
(linuxserver)# multipath –v2
(linuxserver)# multipath -ll
mpath2 (36006048000018772146653594d314142) dm-2 EMC,SYMMETRIX
[ round-robin 0 [prio=2][active]
_ 0:0:1:9 sdc 8:32 [active][ready]
_ 1:0:1:9 sdg 8:96 [active][ready]
mpath1 (360060480000190300520533031304232) dm-1 EMC,SYMMETRIX
[ round-robin 0 [prio=2][active]
_ 0:0:0:6 sdb 8:16 [active][ready]
_ 1:0:0:6 sdf 8:80 [active][ready]
mpath0 (360060480000190300520533031304230) dm-0 EMC,SYMMETRIX
[ round-robin 0 [prio=2][active]
_ 0:0:0:5 sda 8:0 [active][ready]
_ 1:0:0:5 sde 8:64 [active][ready]
mpath3 (36006048000018772146653594d314146) dm-3 EMC,SYMMETRIX
[ round-robin 0 [prio=2][active]
_ 0:0:1:10 sdd 8:48 [active][ready]
_ 1:0:1:10 sdh 8:112 [active][ready]

From this output, you can determine that /dev/dm-0 corresponds to multipath device mpath0

Now that you know which multipath device is currently active, you need to create a new PV for the inactive multipath device (on new array), which would be mpath2 in this case.

(linuxserver)# pvcreate /dev/mapper/mpath2
Physical volume “/dev/mapper/mpath2″ successfully created
(linuxserver)#

After you have created the multipath device, it needs to be added to the volume group (uservg).

(linuxserver)# vgextend uservg /dev/mapper/mpath2
Volume group “uservg” successfully extended
(linuxserver)#

Check and make sure that the multipath device was added correctly to the volume group (uservg).

(linuxserver)# pvs
PV VG Fmt Attr PSize PFree
/dev/dm-0 uservg lvm2 a- 33.69G 28.69G
/dev/dm-2 uservg lvm2 a- 33.69G 33.69G
(linuxserver)#

If everything looks good, move the old multipath device (mpath0) to the new multipath device (mpath2) with the pvmove command.

(linuxserver)# pvmove /dev/mapper/mpath0 /dev/mapper/mpath2
/dev/mapper/mpath0: Moved: 10.0%
/dev/mapper/mpath0: Moved: 20.0%
/dev/mapper/mpath0: Moved: 29.4%
/dev/mapper/mpath0: Moved: 39.4%
/dev/mapper/mpath0: Moved: 49.4%
/dev/mapper/mpath0: Moved: 59.4%
/dev/mapper/mpath0: Moved: 70.0%
/dev/mapper/mpath0: Moved: 80.0%
/dev/mapper/mpath0: Moved: 90.0%
/dev/mapper/mpath0: Moved: 100.0%
(linuxserver)#

After the move is complete, and you verify the data is good, remove the old multipath device from the volume group (uservg).

(linuxserver)# pvs
PV VG Fmt Attr PSize PFree
/dev/dm-0 uservg lvm2 a- 33.69G 33.69G
/dev/dm-2 uservg lvm2 a- 33.69G 28.69G
(linuxserver)# vgreduce uservg /dev/mapper/mpath0
Removed “/dev/dm-0″ from volume group “uservg”
(linuxserver)#

After the device is removed from the volume group, it can be removed from the system.

(linuxserver)# pvremove /dev/mapper/mpath0
Labels on physical volume “/dev/mapper/mpath0″ successfully wiped
(linuxserver)# pvs
PV VG Fmt Attr PSize PFree
/dev/dm-2 uservg lvm2 a- 33.69G 28.69G
(linuxserver)#

After the multipath device is removed, the old devices in the /dev tree can deleted, so that no errors are generated when the storage is removed. To remove the devices, you need to get a list of the /dev/sd* devices on the old array, with the inq command.

/nfs/mts/bin/linq.linux –nodots –sid_wwn

(linuxserver)# /nfs/mts/bin/inq.linux -nodots -sid_wwn 000190300520
Inquiry utility, Version V7.3-883 (Rev 1.0) (SIL Version V6.5.1.0 (Edit Level 883)
Copyright (C) by EMC Corporation, all rights reserved.
For help type inq -h.

——————————————————————–
Symmetrix DeviceSymm Serial # Device # WWN
——————————————————————–
/dev/sda 000190300520 010B0 60060480000190300520533031304230
/dev/sdb 000190300520 010B2 60060480000190300520533031304232
/dev/sde 000190300520 010B0 60060480000190300520533031304230
/dev/sdf 000190300520 010B2 60060480000190300520533031304232
/dev/dm-0 000190300520 010B0 60060480000190300520533031304230
/dev/dm-1 000190300520 010B2 60060480000190300520533031304232

To remove the devices just add a “1″ to the /sys/block//device/delete file. Only do this for the /dev/sd* devices, and not the /dev/dm-* devices.

(linuxserver)# ls -l /dev/sd*
brw-r—– 1 root disk 8, 0 Mar 9 09:59 /dev/sda
brw-r—– 1 root disk 8, 16 Mar 9 09:59 /dev/sdb
brw-r—– 1 root disk 8, 32 Mar 9 10:09 /dev/sdc
brw-r—– 1 root disk 8, 48 Mar 9 10:09 /dev/sdd
brw-r—– 1 root disk 8, 64 Mar 9 09:59 /dev/sde
brw-r—– 1 root disk 8, 80 Mar 9 09:59 /dev/sdf
brw-r—– 1 root disk 8, 96 Mar 9 10:09 /dev/sdg
brw-r—– 1 root disk 8, 112 Mar 9 10:09 /dev/sdh

(linuxserver)# echo 1 > /sys/block/sda/device/delete
(linuxserver)# echo 1 > /sys/block/sdb/device/delete
(linuxserver)# echo 1 > /sys/block/sde/device/delete
(linuxserver)# echo 1 > /sys/block/sdf/device/delete

(linuxserver)# ls -l /dev/sd*
brw-r—– 1 root disk 8, 32 Mar 9 10:09 /dev/sdc
brw-r—– 1 root disk 8, 48 Mar 9 10:09 /dev/sdd
brw-r—– 1 root disk 8, 96 Mar 9 10:09 /dev/sdg
brw-r—– 1 root disk 8, 112 Mar 9 10:09 /dev/sdh

The /dev tree is removed, and you should only see the new multipath devices now.

(linuxserver)# multipath -ll
mpath2 (36006048000018772146653594d314142) dm-2 EMC,SYMMETRIX
[ round-robin 0 [prio=2][active]
_ 0:0:1:9 sdc 8:32 [active][ready]
_ 1:0:1:9 sdg 8:96 [active][ready]
mpath3 (36006048000018772146653594d314146) dm-3 EMC,SYMMETRIX
[ round-robin 0 [prio=2][active]
_ 0:0:1:10 sdd 8:48 [active][ready]
_ 1:0:1:10 sdh 8:112 [active][ready]
(linuxserver)#

Remove the old storage from the server, and check the /var/log/messages file for any errors. Then you can reboot the server, and make sure everything comes up okay.

To re-add the drives, the following command can be used to rescan the SCSI bus, and re-register any devices.

echo “1″ > /sys/class/fc_host/host#/issue_lip (run on each host#)

(linuxserver)# ls /dev/sd*
/dev/sdc /dev/sdd /dev/sdg /dev/sdh
(linuxserver)# echo “1″ > /sys/class/fc_host/host0/issue_lip
(linuxserver)# echo “1″ > /sys/class/fc_host/host1/issue_lip
(linuxserver)# ls /dev/sd*
/dev/sda /dev/sdb /dev/sdc /dev/sdd/dev/sde /dev/sdf /dev/sdg /dev/sdh
(linuxserver)#