[Linux] How do I rescan the SCSI bus to add or remove a SCSI device without rebooting the computer?

How do I rescan the SCSI bus to add or remove a SCSI device without rebooting the computer?

Article ID: 3942 - Created on: Mar 12, 2006 6:00 PM - Last Modified:  Dec 16, 2009 3:03 PM

Issue

 

It is possible to add, remove, or re-size a SCSI device without rebooting a running system.  If this is done explicitly, one device at a time, then it is not necessary to stop all I/O to other storage devices on the system. If this is done by resetting or re-scanning an entire SCSI bus, then all storage I/O must be stopped and flushed before performing these operations.

 

This article is a brief summary of the information contained in the Online Storage Reconfiguration Guide. Refer to that document fo complete coverage of this topic.

 

 

Environment

 

Technical support for online storage reconfiguration is only provided on Red Hat Enterprise Linux 5. Limited tools for hot adding and removing storage are present in previous releases of Red Hat Enterprise Linux however they cannot be guaranteed to work correctly in all configurations.  Red Hat Enterprise Linux 5 includes many enhancements to udev, the low level device drivers, SCSI midlayer, and device-mapper multipath which enables comprehensive support for online storage reconfiguration.

 

This article, and the Online Storage Reconfiguration Guide, currently cover the FC and iSCSI transports. Future versions of this documentation will cover other SCSI transports, such as SAS and FCoE.

 

 

Resolution

 

Removing a Storage Device

 

Before removing access to the storage device itself, you may want to copy data from the device. When that is done, then you must stop and flush all I/O, and remove all operating system references to the device, as described below.  If this is a multipath device then you must do this for the multipath psuedo device, and each of the identifiers that represent a path to the device.

 

Removal of a storage device is not recommended when the system is under memory pressure, since the I/O flush will add to the load. To determine the level of memory pressure run the command:

 

vmstat 1 100

 

Device removal is not recommended if swapping is active (non-zero "si" and "so" columns in the vmstat output), and free memory is less than 5% of the total memory in more than 10 samples per 100.  (The total memory can be obtained with the "free" command.)

 

The general procedure for removing all access to a device is as follows:

 

      1. Close all users of the device. Copy data from the device, as needed.

 

      2. Use umount to unmount any file systems that mounted the device.

 

      3. Remove the device from any md and LVM volume that is using it. If the device is a member of an LVM Volume group, then it may be necessary to move data off the device using the pvmove command, then use the vgreduce command to remove the physical volume, and (optionally) pvremove to remove the LVM metadata from the disk.

 

      4. If you are removing a multipath device, run multipath -l and take note of all the paths to the device. When this has been done, remove the multipath device:

 

multipath  -f device

 

      5. Use the following command to flush any outstanding I/O to all paths to the device:

 

blockdev –flushbufs device

 

      This is particularly important for raw devices, where there is no umount or vgreduce operation to cause an I/O flush.

 

      6. Remove any reference to the device's path-based name, like /dev/sd or /dev/disk/by-path or the major:minor number, in applications, scripts, or utilities on the system.  This is important to ensure that a different device, when added in the future, will not be mistaken for the current device.

 

      7. The final step is to remove each path to the device from the SCSI subsystem.  The command to remove a path is:

 

echo 1 >  /sys/block/device-name/device/delete

 

Where device-name may be sde, for example.

 

Another variation of this operation is:

 

echo 1 >  /sys/class/scsi_device/h:c:t:l/device/delete

 

Where h is the HBA number, c is the channel on the HBA, t is the SCSI target ID, and l is the LUN.

 

You can determine the device-name and the h,c,t,l for a device from various commands, such as lsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*

 

If each of the steps above are followed, then a device can safely be removed from a running system. It is not necessary to stop I/O to other devices while this is done.

 

Other procedures, such as the physical removal of the device, followed by a rescan of the SCSI bus using rescan-scsi-bus or issue_lip to cause the operating system state to be updated to reflect the change, are not recommended. This will cause delays due to I/O timeouts, and devices may be removed unexpectedly. If it is necessary to perform a rescan of an interconnect, it must be done while I/O is paused. Refer to the Online Storage Reconfiguration Guide.

 

 

Adding a Storage Device or a Path

When adding a device, be aware that the path-based device name (the “sd” name, the major:minor number, and /dev/disk/by-path name, for example) that the system assigns to the new device may have been previously in use by a device that has since been removed. Ensure that all old references to the path-based device name have been removed. Otherwise the new device may be mistaken for the old device.

 

The first step is to physically enable access to the new storage device, or a new path to an existing device.  This is done with vendor-specific commands at the FC or iSCSI storage server. When you do this, take note of the LUN value for the new storage that will be presented to your host.

 

Next, make the operating system aware of the new storage device, or path to an existing device. The preferred command is:

 

echo "c t l" >  /sys/class/scsi_host/hostH/scan

 

where H is the HBA number, c is the channel on the HBA, t is the SCSI target ID, and l is the LUN.

 

You can determine the h,c,t by refering to another device that is already configured on the same path as the new device. This can be done with commands such as lsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*. This information, plus the LUN number of the new device, can be used as shown above to probe and configure that path to the new device.

 

After adding all the SCSI paths to the device, execute the multipath command, and check to see that the device has been properly configured. At this point, the device is available to be added to md, LVM, mkfs, or mount, for example.

 

If the steps above are followed, then a device can safely be added to a running system. It is not necessary to stop I/O to other devices while this is done.

 

Other procedures, involving  a rescan, or a reset, of the SCSI bus to cause the operating system to update its state to reflect the current device connectivity, are not recommended. Refer to the  Online Storage Reconfiguration Guide for more information.

 

Considerations for Virtual Guests

 

The procedures above also apply to hypervisors (i.e. dom0 in Red Hat Enterprise Linux 5 virtualization), but the procedures are different for dynamically altering the storage of running virtual guests. For more information about adding storage to virtual guests, see the Virtualization Guide.

 

Note for Hewlett-Packard SmartArray Controllers

 

Hewlett-Packard SmartArray controllers and other hardware that uses the cciss driver provide a different interface for manipulating SCSI devices.  Users of this hardware can find a similar guide here

Tags: reconfigure, scsi, rhel5, san, lun, storage, configuration, hot_plug