How to detect new hard disk attached without rebooting in Linux

Suppose you have added a new hard disk to your Linux OS running on any Virtual Environment which in my case is VMware workstation. Once added the new hard drive, generally the changes won't reflect unless you reboot the Guest OS.

But what if you are in no position to reboot the Gues Linux OS?

Solution:

In the below path you can find a list of host symlinks pointing to the iscsi device configured on your Linux box.

[root@varuntest ~]# ls -l /sys/class/scsi_host/
total 0
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host0 -> ../../devices/pci0000:00/0000:00:07.1/host0/scsi_host/host0
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host1 -> ../../devices/pci0000:00/0000:00:07.1/host1/scsi_host/host1
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host10 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host10/scsi_host/host10
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host11 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host11/scsi_host/host11
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host12 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host12/scsi_host/host12
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host13 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host13/scsi_host/host13
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host14 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host14/scsi_host/host14
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host15 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host15/scsi_host/host15
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host16 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host16/scsi_host/host16
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host17 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host17/scsi_host/host17
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host18 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host18/scsi_host/host18
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host19 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host19/scsi_host/host19
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host2 -> ../../devices/pci0000:00/0000:00:10.0/host2/scsi_host/host2
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host20 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host20/scsi_host/host20
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host21 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host21/scsi_host/host21
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host22 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host22/scsi_host/host22
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host23 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host23/scsi_host/host23
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host24 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host24/scsi_host/host24
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host25 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host25/scsi_host/host25
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host26 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host26/scsi_host/host26
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host27 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host27/scsi_host/host27
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host28 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host28/scsi_host/host28
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host29 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host29/scsi_host/host29
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host3 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host3/scsi_host/host3
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host30 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host30/scsi_host/host30
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host31 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host31/scsi_host/host31
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host32 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host32/scsi_host/host32
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host4 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host4/scsi_host/host4
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host5 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host5/scsi_host/host5
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host6 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host6/scsi_host/host6
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host7 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host7/scsi_host/host7
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host8 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host8/scsi_host/host8
lrwxrwxrwx. 1 root root 0 Jan 16 07:10 host9 -> ../../devices/pci0000:00/0000:00:11.0/0000:02:05.0/host9/scsi_host/host9

[root@varuntest ~]#

But to detect a new hard drive attached you need to first get your host bus number used which you can get by using below command.

# grep mpt /sys/class/scsi_host/host?/proc_name

You should get a output like below.

[root@varuntest ~]# grep mpt /sys/class/scsi_host/host?/proc_name

/sys/class/scsi_host/host2/proc_name:mptspi

[root@varuntest ~]#

So as you see your host2 is the relevant files where you need to reset the storage buffer values. Run the below command.

# echo "- - -" > /sys/class/scsi_host/host2/scan

Here "- - -" defines the three values stored inside host*/scan i.e. channel number, SCSI target ID, and LUN values. We are simply replacing the values with wild cards so that it can detect new changes attached to the Linux box. This procedure will add LUNs, but not remove them.

Once done verify if you can see the new hard drive which in my case worked very fine as see below.

[root@varuntest ~]# fdisk -l /dev/sde

Disk /dev/sde: 2147 MB, 2147483648 bytes

255 heads, 63 sectors/track, 261 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00000000

[root@varuntest ~]#

Linux Booting Process and Concepts.

There are 6 Stages of Linux Boot Process (Startup Sequence).

Press the power button on your system, and after few moments you see the Linux login prompt. 

Have you ever wondered what happens behind the scenes from the time you press the power button until the Linux login prompt appears?

The following are the 6 high level stages of a typical Linux boot process.

1. BIOS

The computer knows how to bring itself up, when you press the start button, because of the instructions that are fed to a program called as BIOS. BIOS stands for Basic Input Output System. The most important use of BIOS during the booting process is POST. POST stands for Power on Self Test. Its a series of tests conducted by the bios, which confirms the proper functioning of different hardware components attached to the computer. POST is very important thing to have before the Operating system is loaded. Just imagine if you have a faulty hard drive or faulty memory, sometimes these things can cause data loss. POST checks and confirms the integrity of the following hardware components.

*Timer IC's

*DMA controllers

*CPU

*Video ROM

 A full POST check will confirm the integrity of the following devices as well.

*Motherboard

*Keyboard

*Printer port

*Hard Drive etc

If you are doing a warm start (which means you did a reset of a running machine, most of the times reset button is the small one near the power button on the CPU), a full POST check will not be conducted by the BIOS. However if you are doing a Cold Start, which means you have applied the power now, it will conduct a full POST.

BIOS determines whether its a cold or warm start, by looking at a flag in a predefined memory location. Once the POST completes, the BIOS will inform you about any problems it found with the help of beep codes (through system speaker). Different number of beep codes have different meaning.

Once the POST check is completed successfully, BIOS will look CMOS settings to know what is the boot order. Boot order is nothing but a user defined order which tells where to look for the operating system. The order will be something like the below.

 *CD ROM

*HARD DISK

*USB

*Floppy DISK

 The above shown order means that the BIOS will look at CD ROM first to check whether an OS can be loaded from there, if it does not find a bootable disk in the CD ROM, it will look check whether a bootable OS is there in the hard disk, then USB and then Floppy disk. Let's assume that you don’t have a bootable CD in your CD ROM drive, then the BIOS will turn to HARD disk. 

Below in short what happening in the BIOS part for the Linux booting process.

• BIOS stands for Basic Input/Output System
• Performs some system integrity checks
• Searches, loads, and executes the boot loader program.
• It looks for boot loader in floppy, cd-rom, or hard drive. You can press a key (typically F12 of F2, but it depends on your system) during the BIOS startup to change the boot sequence.
• Once the boot loader program is detected and loaded into the memory, BIOS gives the control to it.
• So, in simple terms BIOS loads and executes the MBR boot loader.

2. MBR

BIOS is programmed to look at a permanent location on the hard disk to complete its task. This location is called a Boot sector. This is nothing but the first sector of your hard disk. This area is sometimes called as MBR (Master Boot Record). This is the location that contains the program that will help our computer to load the operating system. As soon as bios finds a valid MBR, it will load the entire content of MBR to RAM, and then further execution is done by the content of MBR.

This first sector of the hard disk is only of 512 bytes. Most of the operating system store only the first stage of their boot loader program in here. Only the first 440 bytes from the total of 512 bytes is used by the first stage boot loader, the remaining part is used to store partition table information. The remaining space is used for the partition table, ie: there can only be 4 primary partition in an MBR partitioning scheme.this remaining area also has the partition location information.

Below in short what happening in the MBR part for the Linux booting process.

• MBR stands for Master Boot Record.
• It is located in the 1st sector of the bootable disk. Typically /dev/hda, or /dev/sda
• MBR is less than 512 bytes in size. This has three components 1) primary boot loader info in 1st 446 bytes 2) partition table info in next 64 bytes 3) mbr validation check in last 2 bytes.
• It contains information about GRUB (or LILO in old systems).
• So, in simple terms MBR loads and executes the GRUB boot loader.

3. GRUB

The MBR contains the first stage of the grub, and partition table information.

The primary job of the stage 1 bootloader is to load the second stage boot loader. The second stage boot loader is the stage 2 grub, that actually does the job of loading the kernel and other initrd image files (we will come to that part in some time). GRUB (Grand Unified Boot Loader) is the combined name given to different stages of grub.

 Now there is a little surprise that am going to reveal. There are not two stages of grub. But there are three stages of grub in total. The three stages are mentioned below.

GRUB Stage 1

GRUB Stage 1.5

Grub Stage 2

Grub Stage 1.5 located in the MBR GAP (sector 1 to 63 before the beginning of the first partition) basically contains the drivers for reading file systems. So grub stage 1 will load grub stage 1.5 to the RAM, and will pass the control to it. Now grub stage 1.5 will load the file system drivers and once the file system drivers are loaded, it can now access /boot/grub/grub.conf file which contains other details about kernel path and initrd path etc.

Below in short what happening in the GRUB part for the Linux booting process.

• GRUB stands for Grand Unified Bootloader.
• If you have multiple kernel images installed on your system, you can choose which one to be executed.
• GRUB displays a splash screen, waits for few seconds, if you don’t enter anything, it loads the default kernel image as specified in the grub configuration file.
• GRUB has the knowledge of the filesystem (the older Linux loader LILO didn’t understand filesystem).
• Grub configuration file is /boot/grub/grub.conf (/etc/grub.conf is a link to this). 

4. Kernel

Similar to GRUB, kernel is also loaded in stages. A Linux kernel is responsible for handling Process management, Memory Management, Users, Inter process communication etc. Kernel is never used by a user. What the kernel does is to maintain a good environment for programs to run. Yeah we use kernel through different programs.

Kernel is a compressed image file. The location of this compressed kernel image is specified in the grub 2 configuration file. Its basically an executable bz Image file.

Now you need to have a lot of drivers and modules to access underlying hardware and other stuff. For example, if you have RAID configured on your / partition how will you mount it without knowing the programs, or think how will you include a kernel module or how to remove a kernel module. All these things required different set of programs and code. Including all these codes inside the kernel will make it a larger file image.

But our kernel image file needs to be smaller, that's the reason its compressed image file. So most of these drivers and tools along with a small similar root file system feel is given by initrd. Initrd is sometimes called as initial root file system. This is used by the kernel before the real root file system is mounted.  Initrd is available in the form of an images. 

Loading and unloading of kernel modules is done with the help of programs like insmod, and rmmod present in the initrd image.

Now as the kernel is loaded into the memory, the execution begins by checking the processor family and architecture. The kernel conducts a lot of hard ware specific operations and the first user space program it executes is /sbin/init.

Below in short what happening in the Kernel part for the Linux booting process. 

• Mounts the root file system as specified in the “root=” in grub.conf
• Kernel executes the /sbin/init program
• Since init was the 1st program to be executed by Linux Kernel, it has the process id (PID) of 1. Do a ‘ps -ef | grep init’ and check the pid.
• initrd stands for Initial RAM Disk.
• initrd is used by kernel as temporary root file system until kernel is booted and the real root file system is mounted. It also contains necessary drivers compiled inside, which helps it to access the hard drive partitions, and other hardware.

5. Init

As this is the first program executed by the kernel, it has got a process id number of 1. The process id number 1 for init was not kept intentionally, but it was  due to the fact that it is the first process executed by the kernel. Now as soon as the kernel executes the init process, it will look at the /etc/inittab configuration file to see the default run level.

There are different run level's in linux.

Run-Level Usage

0                 System Halt/Shut Down

1                 Single User Mode

2                 Multiuser Mode Without Networking

3                 Full Multiuser Mode

4                 Unused

5                 GUI/X11

6                 Reboot

The /etc/inittab file contains the default run level like the one shown below.

id:3:initdefault:

Now that means, we have a run level 3 as the default run level. Once this is identified, then run level specific programs are started by the kernel. 

This is the reason you have the following directories.

[root@varuntest grub]# cd /etc/rc.d/

[root@varuntest rc.d]# ls -al 

total 76

drwxr-xr-x.  10 root root  4096 Jan  3 05:57 .

drwxr-xr-x. 116 root root 12288 Jan 16 08:09 ..

drwxr-xr-x.   2 root root  4096 Jan  3 06:06 init.d

-rwxr-xr-x.   1 root root  2617 Oct 10  2013 rc

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc0.d

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc1.d

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc2.d

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc3.d

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc4.d

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc5.d

drwxr-xr-x.   2 root root  4096 Jan  3 06:15 rc6.d

-rwxr-xr-x.   1 root root   220 Oct 10  2013 rc.local

-rwxr-xr-x.   1 root root 19432 Oct 10  2013 rc.sysinit

[root@varuntest rc.d]# 

Below in short what happening in the INIT  part for the Linux booting process.

• Looks at the /etc/inittab file to decide the Linux run level.
• Following are the available run levels
  • 0 – halt
  • 1 – Single user mode
  • 2 – Multiuser, without NFS
  • 3 – Full multiuser mode
  • 4 – unused
  • 5 – X11
  • 6 – reboot
• Init identifies the default initlevel from /etc/inittab and uses that to load all appropriate program.
• Execute ‘grep initdefault /etc/inittab’ on your system to identify the default run level
• If you want to get into trouble, you can set the default run level to 0 or 6. Since you know what 0 and 6 means, probably you might not do that.
• Typically you would set the default run level to either 3 or 5.

6. Runlevel programs

The folders rc0.d, rc1.d, rc2.d etc contains run level specific programs that will be executed depending upon the default run level you have in your inittab configuration file.

Let's see what are the contents inside these run level folders.

[root@varuntest rc1.d]# ls

K01certmonger  K05atd        K15htcacheclean  K30postfix         K50snmptrapd   K73winbind    K75ntpdate    K83nfslock         K85messagebus   K88rsyslog      K92ip6tables  S02lvm2-monitor

K01smartd      K05wdaemon    K15httpd         K30spice-vdagentd  K60crond       K74acpid      K75quota_nld  K83rpcgssd         K87irqbalance   K88sssd         K92iptables   S13cpuspeed

K02oddjobd     K10cups       K16abrt-ccpp     K50dnsmasq         K60nfs         K74haldaemon  K76ypbind     K84NetworkManager  K87restorecond  K89portreserve  K95firstboot  S25blk-availability

K02rhsmcertd   K10psacct     K16abrtd         K50netconsole      K69rpcsvcgssd  K74ntpd       K80kdump      K84wpa_supplicant  K87rpcbind      K89rdisc        K99rngd       S26udev-post

K03rhnsd       K10saslauthd  K25sshd          K50snmpd           K72autofs      K75netfs      K83bluetooth  K85mdmonitor       K88auditd       K90network      S01sysstat    S99single

[root@varuntest rc1.d]#

If you see the files inside these run level specific folders, they either begin with S or they begin with K. The files are also numbered. Now files with an S at starting will be executed during the startup process, and files that begins with K, will be killed during shutdown process.

The number after either S or K is the sequence with which these will be executed.

Once the kernel has started all programs in your desired run level directory. You will get a login screen to log inside your booted system.

Below in short what happening in the RUN LEVELS  part for the Linux booting process.

• When the Linux system is booting up, you might see various services getting started. For example, it might say “starting sendmail …. OK”. Those are the runlevel programs, executed from the run level directory as defined by your run level.
• Depending on your default init level setting, the system will execute the programs from one of the following directories.
  • Run level 0 – /etc/rc.d/rc0.d/
  • Run level 1 – /etc/rc.d/rc1.d/
  • Run level 2 – /etc/rc.d/rc2.d/
  • Run level 3 – /etc/rc.d/rc3.d/
  • Run level 4 – /etc/rc.d/rc4.d/
  • Run level 5 – /etc/rc.d/rc5.d/
  • Run level 6 – /etc/rc.d/rc6.d/
• Please note that there are also symbolic links available for these directory under /etc directly. So, /etc/rc0.d is linked to /etc/rc.d/rc0.d.
• Under the /etc/rc.d/rc*.d/ directories, you would see programs that start with S and K.
• Programs starts with S are used during startup. S for startup.
• Programs starts with K are used during shutdown. K for kill.
• There are numbers right next to S and K in the program names. Those are the sequence number in which the programs should be started or killed.
• For example, S12syslog is to start the syslog deamon, which has the sequence number of 12. S80sendmail is to start the sendmail daemon, which has the sequence number of 80. So, syslog program will be started before sendmail.

There you have it. That is what happens during the Linux boot process.