Sunday, 5 March 2017

Linux YUM (Yellowdog Updater, Modified) for Package Management

YUM (Yellowdog Updater Modified) is an open source command-line as well as graphical based package management tool for RPM (RedHat Package Manager) based Linux systems. It allows users and system administrator to easily install, update, remove or search software packages on a systems. YUM uses numerous third party repositories to install packages automatically by resolving their dependencies issues.

YUM Repository Configuration File

We can install new software on Red Hat/CentOS Linux with "yum install packagename" command from console. Running this command first checks for existing YUM Repository configuration files in /etc/yum.repos.d/ directory. It reads each YUM Repository configuration file to get the information required to download and install new software, resolves software dependencies and installs the required RPM package files.

YUM Repository configuration files must: be located in /etc/yum.repos.d/ directory
have .repo extension, to be recognized by YUM. Available YUM Repository configuration file options are:

Repository ID - One word unique repository ID (example: [examplerepo])

Name - Human readable name of the repository (example: name=Example Repository)

Baseurl - URL to the repodata directory. You can use file://path if repository is located locally or ftp://link, http://link, https://link if repository is located remotely - HTTP Authentication available http://user:password@www.repo1.com/repo1 (example: baseurl=http://mirror.cisp.com/CentOS/6/os/i386/)

Enabled - Enable repository when performing updates and installs (example: enabled=1)

Gpgcheck - Enable/disable GPG signature checking (example: gpgcheck=1)
Gpgkey - URL to the GPG key (example: gpgkey=http://mirror.cisp.com/CentOS/6/os/i386/RPM-GPG-KEY-CentOS-6)

Exclude - List of the packages to exclude (example: exclude=httpd,mod_ssl)

Includepkgs - List of the packages to include (example: include=kernel)

Required YUM Repository configuration file options are:

Repository ID
Name
Baseurl
Enabled

Step 1: Create YUM Repository configuration file

Use your favorite console text editor and create a new YUM Repository configuration file with .repo extension in /etc/yum.repos.d/ directory. To create a new file with "vi editor" run the following command from console:

vi /etc/yum.repos.d/example.repo

Step 2: Insert YUM Repository options

Insert the desired YUM Repository options to the newly created YUM Repository configuration file and save changes.

Example YUM Repository Configuration file:

/etc/yum.repos.d/example.repo

[examplerepo]
name=Example Repository
baseurl=http://mirror.cisp.com/CentOS/6/os/i386/
enabled=1
gpgcheck=1

gpgkey=http://mirror.cisp.com/CentOS/6/os/i386/RPM-GPG-KEY-CentOS-6

1. Install a Package with YUM

To install a package called Firefox , just run the below command it will automatically find and install all required dependencies for Firefox.

#yum install firefox

The above command will ask confirmation before installing any package on your system. If you want to install packages automatically without asking any confirmation, use option -y as shown in below example.

#yum -y install firefox

2. Removing a Package with YUM

To remove a package completely with their all dependencies, just run the following command as shown below.


# yum remove firefox

3. Updating a Package using YUM

Let’s say you have outdated version of MySQL package and you want to update it to the latest stable version. Just run the following command it will automatically resolves all dependencies issues and install them.


# yum update mysql

4. List a Package using YUM

Use the list function to search for the specific package with name. For example to search for a package called openssh, use the command.


# yum list openssh

To make your search more accurate, define package name with their version, in case you know. For example to search for a specific version openssh-4.3p2 of the package, use the command.

# yum list openssh-4.3p2

5. Search for a Package using YUM

If you don’t remember the exact name of the package, then use search function to search all the available packages to match the name of the package you specified. For example, to search all the packages that matches the word .


# yum search vsftpd


6. Get Information of a Package using YUM

Say you would like to know information of a package before installing it. To get information of a package just issue the below command.

# yum info firefox

7. List all Available Packages using YUM

To list all the available packages in the Yum database, use the below command.

# yum list | less

8. List all Installed Packages using YUM

To list all the installed packages on a system, just issue below command, it will display all the installed packages.

# yum list installed | less

9. Yum Provides Function

Yum provides function is used to find which package a specific file belongs to. For example, if you would like to know the name of the package that has the /etc/httpd/conf/httpd.conf.

# yum provides /etc/httpd/conf/httpd.conf

10. Check for Available Updates using Yum

To find how many of installed packages on your system have updates available, to check use the following command.

# yum check-update

11. Update System using Yum

To keep your system up-to-date with all security and binary package updates, run the following command. It will install all latest patches and security updates to your system.

# yum update

12. List all available Group Packages

In Linux, number of packages are bundled to particular group. Instead of installing individual packages with yum, you can install particular group that will install all the related packages that belongs to the group. For example to list all the available groups, just issue following command.

# yum grouplist

13. Install a Group Packages

To install a particular package group, we use option as groupinstall. Fore example, to install “MySQL Database“, just execute the below command.


# yum groupinstall 'MySQL Database'

14. Update a Group Packages

To update any existing installed group packages, just run the following command as shown below.

# yum groupupdate 'DNS Name Server'

15. Remove a Group Packages

To delete or remove any existing installed group from the system, just use below command.

# yum groupremove 'DNS Name Server'

16. List Enabled Yum Repositories

To list all enabled Yum repositories in your system, use following option.

# yum repolist

16. List all Enabled and Disabled Yum Repositories

The following command will display all enabled and disabled yum repositories on the system.

# yum repolist all

17. Install a Package from Specific Repository

To install a particular package from a specific enabled or disabled repository, you must use –enablerepo option in your yum command. For example to Install PhpMyAdmin 3.5.2 package, just execute the command.

# yum --enablerepo=epel install phpmyadmin

18. Interactive Yum Shell

Yum utility provides a custom shell where you can execute multiple commands.

# yum shell

19. Clean Yum Cache

By default yum keeps all the repository enabled package data in /var/cache/yum/ with each sub-directory, to clean all cached files from enabled repository, you need to run the following command regularly to clean up all the cache and make sure that there is nothing unnecessary space is using. We don’t want to give the output of the below command, because we like to keep cached data as it is.

# yum clean all
20. View History of Yum

To view all the past transactions of yum command, just use the following command.

# yum history

List of Commands: check Check for problems in the rpmdb check-update Check for available package updates clean Remove cached data deplist List a package's dependencies distribution-synchronization Synchronize installed packages to the latest available versions downgrade downgrade a package erase Remove a package or packages from your system groupinfo Display details about a package group groupinstall Install the packages in a group on your system grouplist List available package groups groupremove Remove the packages in a group from your system help Display a helpful usage message history Display, or use, the transaction history info Display details about a package or group of packages install Install a package or packages on your system list List a package or groups of packages load-transaction load a saved transaction from filename makecache Generate the metadata cache provides Find what package provides the given value reinstall reinstall a package repolist Display the configured software repositories resolvedep Determine which package provides the given dependency search Search package details for the given string shell Run an interactive yum shell update Update a package or packages on your system upgrade Update packages taking obsoletes into account version Display a version for the machine and/or available repos.

Friday, 3 March 2017

How to Read Linux Top Command Output and Uses

When we execute top command on linux, it shows a lot of results, here i am trying to show you to how to read it row by row.

Result Row #1:


top - 19:49:16 up  1:43,  1 user,  load average: 0.00, 0.00, 0.00

Row 1 results shows about server up time from last reboot, currently logged in users and cpu load on server. The same output you can find using linux uptime command.

This first line indicates in order:
  • current time (19:49:16)
  • uptime of the machine (up 1:43)
  • users sessions logged in (1 users)
  • average load on the system (load average: 0.00, 0.00, 0.00) the 3 values refer to the last minute, five minutes and 15 minutes.

Result Row #2:

Tasks: 152 total,   1 running, 151 sleeping,   0 stopped,   0 zombie

Row 2 shows the number of process running on server and there state. The second row gives the following information:
  • Processes running in totals (152 total)
  • Processes running (1 running)
  • Processes sleeping (151 sleeping)
  • Processes stopped (0 stopped)
  • Processes waiting to be stoppati from the parent process (0 zombie)
Zombie process or defunct process is a process that has completed execution but still has an entry in the process table. This entry is still needed to allow the parent process to read its child’s exit status.

Result Row #3:

Cpu0  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

Row three shows the cpu utilization status on server, you can find here how much cpu is free and how much is utilizing by system. The third line indicates how the cpu is used. If you sum up all the percentages the total will be 100% of the cpu. Let’s see what these values indicate in order:
  • Percentage of the CPU for user processes (0.0%us)
  • Percentage of the CPU for system processes (0.3%sy)
  • Percentage of the CPU processes with priority upgrade nice (0.0%ni)
  • Percentage of the CPU not used (99,7%id)
  • Percentage of the CPU processes waiting for I/O operations(0.0%wa)
  • Percentage of the CPU serving hardware interrupts (0.0% hi — Hardware IRQ
  • Percentage of the CPU serving software interrupts (0.0% si — Software Interrupts
  • The amount of CPU ‘stolen’ from this virtual machine by the hypervisor for other tasks (such as running another virtual machine) this will be 0 on desktop and server without Virtual machine. (0.0%st — Steal Time)
Steal time is the time that a virtual CPU waits for a real CPU while the hypervisor is servicing another virtual processor.

Result Row #4:

Mem:   1907576k total,   336396k used,  1571180k free,    21424k buffers

Row 4 shows the memory utilization on server, you can find here how much memory is used, the same results you can find using free command. 
  • Total System Memory (1907576k total)
  • Current used Memory by system ( 336396k used)
  • Free Memory (1571180k free)
  • Total Memory used by Buffers (21424k buffers)

Result Row #5:

Swap:  2097144k total,        0k used,  2097144k free,   121764k cached

Row 4 shows the swap memory utilization on server, you can find here how much swap is being used, the same results you can find using free command.
  • Total Swap Memory in system (2097144k total).
  • Current  swap Memory used by system ( 0k used).
  • Free Swap Memory (2097144k free)
  • Total cached Memory in system (121764k cached).

Result Row #6 ( Running Processes ):


   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  SWAP COMMAND
   895      root      20   0     0    0    0 S  0.3  0.0   0:00.20    0 vmmemctl
   2403    root      20   0 15036 1272  948 R  0.3  0.1   0:02.78    0 top

In this steps you will see all running process on servers and there additional details about them like below and as last thing ordered by CPU usage (as default) there are the processes currently in use. Let’s see what information we can get in the different columns:
  • PID – l’ID of the process(895)
  • USER – The user that is the owner of the process (root)
  • PR – priority of the process (20)
  • NI – The “NICE” value of the process (0)
  • VIRT – virtual memory used by the process (0)
  • RES – physical memory used from the process (0)
  • SHR – shared memory of the process (0)
  • S – indicates the status of the process: S=sleep R=running Z=zombie (S)
  • %CPU – This is the percentage of CPU used by this process (0.3)
  • %MEM – This is the percentage of RAM used by the process (0.0)
  • TIME+ –This is the total time of activity of this process (0:00.20)
  • COMMAND – And this is the name of the process (vmmemctl)
If you want to sort output on basis of any fields, you can use following key combinations.
Press "SHIFT + F" and Select your choice below and press ENTER.

  A     ( PID = Process Id )
  b     (  PPID   = Parent Process Pid )
  d     (  UID= User Id )
  e     (  USER   = User Name )
  f     (  GROUP  = Group Name )
  g     (  TTY= Controlling Tty )
  h     (  PR = Priority )
  i     (  NI = Nice value )
  j     (  P  = Last used cpu (SMP) )
  k     (  %CPU  = CPU usage )
  l     (  TIME  = CPU Time )
  n     (  %MEM  = Memory usage (RES) )
  o     (  VIRT  = Virtual Image (kb) )
  p     (  SWAP  = Swapped size (kb) )
  q     (  RES= Resident size (kb) )
  r     (  CODE  = Code size  (kb) )
  s     (  DATA  = Data+Stack size (kb) )
  t     (  SHR= Shared Mem size (kb) )
  w     (  S  = Process Status )
  x     (  COMMAND= Command name/line )

Linux iostat Command to Report CPU Statistics and I/O Statistics

The  iostat  command  is used for monitoring system input/output device loading by observing the time the devices are  active  in  relation  to their average transfer rates. The iostat command generates reports that can be used to  change  system  configuration  to  better  balance  the input/output load between physical disks.

The iostat command generates two types of reports, the CPU  Utilization
report and the Device Utilization report.

       CPU Utilization Report
              The first report generated by the iostat command is the CPU Uti-
              lization Report. For multiprocessor systems, the CPU values  are
              global  averages  among all processors.  The report has the fol-
              lowing format:

              %user
                     Show the percentage  of  CPU  utilization  that  occurred
                     while executing at the user level (application).
              %nice
                     Show  the  percentage  of  CPU  utilization that occurred
                     while executing at the user level with nice priority.
              %sys
                     Show the percentage  of  CPU  utilization  that  occurred
                     while executing at the system level (kernel).
              %iowait
                     Show  the  percentage  of  time that the CPU or CPUs were
                     idle during which the system had an outstanding disk  I/O
                     request.
              %idle
                     Show  the  percentage  of  time that the CPU or CPUs were
                     idle and the system did not have an outstanding disk  I/O
                     request.

     Device Utilization Report

              The  second report generated by the iostat command is the Device
              Utilization Report. The device report provides statistics  on  a
              per  physical device or partition basis. Block devices for which
              statistics are to be displayed may be  entered  on  the  command
              line. Partitions may also be entered on the command line provid-
              ing that option -x is not used.  If no device nor  partition  is
              entered,  then statistics are displayed for every device used by
              the system, and providing that the kernel  maintains  statistics
              for  it.   If the ALL keyword is given on the command line, then
              statistics are displayed for every device defined by the system,
              including  those that have never been used.  The report may show
              the following fields, depending on the flags used:

              Device:
                     This column gives the device (or partition)  name,  which
                     is  displayed  as  hdiskn  with  2.2 kernels, for the nth
                     device. It is displayed as devm-n with 2.4 kernels, where
                     m  is the major number of the device, and n a distinctive
                     number.  With newer kernels, the device name as listed in
                     the /dev directory is displayed.

              tps
                     Indicate  the  number  of  transfers per second that were
                     issued to the device. A transfer is an I/O request to the
                     device.  Multiple logical requests can be combined into a
                     single I/O request to the device. A transfer is of  inde-
                     terminate size.

              Blk_read/s
                     Indicate the amount of data read from the drive expressed
                     in a number of blocks per second. Blocks  are  equivalent
                     to  sectors with 2.4 kernels and newer and therefore have
                     a size of 512 bytes. With older kernels, a  block  is  of
                     indeterminate size.

              Blk_wrtn/s
                     Indicate   the  amount  of  data  written  to  the  drive
                     expressed in a number of blocks per second.

              Blk_read
                     The total number of blocks read.

              Blk_wrtn
                     The total number of blocks written.

              kB_read/s
                     Indicate the amount of data read from the drive expressed
                     in  kilobytes  per  second. Data displayed are valid only
                     with kernels 2.4 and newer.

              kB_wrtn/s
                     Indicate  the  amount  of  data  written  to  the   drive
                     expressed  in  kilobytes  per  second. Data displayed are
                     valid only with kernels 2.4 and newer.

              kB_read
                     The total number of kilobytes read.  Data  displayed  are
                     valid only with kernels 2.4 and newer.

              kB_wrtn
                     The total number of kilobytes written. Data displayed are
                     valid only with kernels 2.4 and newer.

              rrqm/s
                     The number of read requests merged per second  that  were
                     issued to the device.

              wrqm/s
                     The  number of write requests merged per second that were
                     issued to the device.

              r/s
                     The number of read  requests  that  were  issued  to  the
                     device per second.

              w/s
                     The  number  of  write  requests  that were issued to the
                     device per second.

              rsec/s
                     The number of sectors read from the device per second.

              wsec/s
                     The number of sectors written to the device per second.

              rkB/s
                     The number of kilobytes read from the device per  second.

              wkB/s
                     The number of kilobytes written to the device per second.

              avgrq-sz
                     The average size (in sectors) of the requests  that  were
                     issued to the device.

              avgqu-sz
                     The average queue length of the requests that were issued
                     to the device.

              await
                     The average  time  (in  milliseconds)  for  I/O  requests
                     issued to the device to be served. This includes the time
                     spent by the requests in queue and the time spent servic-
                     ing them.

              svctm
                     The  average  service  time  (in  milliseconds)  for  I/O
                     requests that were issued to the device.

              %util
                     Percentage of CPU time during  which  I/O  requests  were
                     issued  to  the  device  (bandwidth  utilization  for the
                     device). Device saturation  occurs  when  this  value  is
                     close to 100%.

OPTIONS

       -c     The  -c  option  is exclusive of the -d option and displays only
              the CPU usage report.

       -d     The -d option is exclusive of the -c option  and  displays  only
              the device utilization report.

       -k     Display statistics in kilobytes per second instead of blocks per
              second.  Data displayed are valid  only  with  kernels  2.4  and
              newer.

       -p device | ALL
              The -p option is exclusive of the -x option and displays statis-
              tics for block devices and all their partitions that are used by
              the  system.   If  a device name is entered on the command line,
              then statistics for it and all  its  partitions  are  displayed.
              Last,  the ALL keyword indicates that statistics have to be dis-
              played for all the block devices and partitions defined  by  the
              system,  including  those  that have never been used.  Note that
              this option works only with post 2.5 kernels.

       -t     Print the time for each report displayed.

       -V     Print version number and usage then exit.

       -x     Display extended statistics.  This option is exclusive of the -p
              one,   and   works   with   post  2.5  kernels  since  it  needs
              /proc/diskstats file or a mounted sysfs to get  the  statistics.
              This  option may also work with older kernels (e.g. 2.4) only if
              extended statistics are available in /proc/partitions (the  ker-
              nel needs to be patched for that).

Linux Performance Measurements using vmstat

vmstat provides a number of values and will typically be called using two numerical parameters. vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.

  • Example: vmstat 1 5
    • 1 -> the values will be re-measured and reported every second
    • 5 -> the values will be reported five times and then the program will stop

The first line of the report will contain the average values since the last time the computer was rebooted. All other lines in the report will represent their respective current values. Vmstat does not need any special user rights. It can run as a normal user.

[root@varuntest ~]# vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 1590120  19624 110552    0    0  1903    87  448  664  8 18 23 51  0
 0  0      0 1590088  19624 110580    0    0     0     0   13   10  0  0 100  0  0
 0  0      0 1590088  19624 110580    0    0     0     0   15   16  0  0 100  0  0
 0  0      0 1588040  19624 110580    0    0     0     0   18   25  0  1 99  0  0
 0  0      0 1588040  19628 110588    0    0     4    88   24   24  0  1 94  5  0
[root@varuntest ~]#

FIELD DESCRIPTION FOR VM MODE

   Procs
       r: The number of processes waiting for run time.
       b: The number of processes in uninterruptible sleep.


   Memory
       swpd: the amount of virtual memory used.
       free: the amount of idle memory.
       buff: the amount of memory used as buffers.
       cache: the amount of memory used as cache.
       inact: the amount of inactive memory. (-a option)
       active: the amount of active memory. (-a option)


   Swap
       si: Amount of memory swapped in from disk (/s).
       so: Amount of memory swapped to disk (/s).


   IO
       bi: Blocks received from a block device (blocks/s).
       bo: Blocks sent to a block device (blocks/s).


   System
       in: The number of interrupts per second, including the clock.
       cs: The number of context switches per second.

   CPU
       These are percentages of total CPU time.
       us: Time spent running non-kernel code. (user time, including nice time)
       sy: Time spent running kernel code. (system time)
       id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
       wa: Time spent waiting for IO. Prior to Linux 2.5.41, shown as zero.

vmstat Options

The -a switch displays active/inactive memory, given a 2.5.41 kernel or
       better.

       The  -f  switch displays the number of forks since boot.  This includes
       the fork, vfork, and clone system calls, and is equivalent to the total
       number  of  tasks  created.  Each process is represented by one or more
       tasks, depending on thread usage.  This display does not repeat.

       The -m displays slabinfo.

       The -n switch causes the header to be displayed only once  rather  than
       periodically.

       The  -s  switch  displays  a table of various event counters and memory
       statistics. This display does not repeat.

       delay is the delay between updates in seconds.  If no delay  is  speci-
       fied, only one report is printed with the average values since boot.

       count  is the number of updates.  If no count is specified and delay is
       defined, count defaults to infinity.

       The -d reports disk statistics (2.5.70 or above required)

       The -p followed by some partition name for detailed statistics  (2.5.70
       or above required)

       The  -S  followed  by  k  or K or m or M switches outputs between 1000,
       1024, 1000000, or 1048576 bytes

       The -V switch results in displaying version information.