Automated Installation of CentOS 6.x And Kickstart File (Unattended Installation)

Hello everybody,

Today, I am going to show you how you can automate installation of CentOS version 6.0 or later without any user intervention. Just turn on computer and bingo! You can use the same process for RedHat too.

Advantage:
It’s totally automated installation for a mass of computers. Just imagine that you have 250 or more workstations or you have 100 servers in production. Of course, you don’t want to install OS one by one for each machine. Instead, you should use automated installation.

You may say that we use virtualization software such as KVM,  VMWare,  VirtualBox or etc... and we can clone servers easily. But this is not true in mass installation because of following reasons:

1.    In most virtualization software, you have to turn off the VM in order to clone it and it’s not possible in production environment
2.    If you clone a VM, you have to configure Mac address, ip address, hostname, all network setting of cloned VM and also customize other configurations manually
3.     You don’t have a menu with different flavour of OS to select to install. Automated installation has ability of creation a menu with different OS and changing the default selection. This gives you flexibility of installation for different OS, but clone cannot do this.

In KVM, you can use virt-install command with the same kickstart file, which I will explain later, but you are limited to using just KVM.

Automated Installation Process:

1. Setup TFTP and PXE server
2. Setup DHCP server
3. Setup FTP server
4. Setup Kickstart file

You can set up a server for TFTP/PXE, DHCP, and FTP (all of them). In my example, I set up one server for TFTP/PXE and one server for FTP.

Setup TFTP and PXE server:   

1.  Login as root:    su –
2. Install TFTP service:    yum install tftp-server
3.  Run  vi /etc/xinetd.d/tftp  command and change disable to 'no'         disable = no
4. Start xinetd service:      service xinetd start
5. Set xinetd service to start after booting server:   chkconfig xinetd on
6. Install syslinux, it’s a boot loader:   yum install syslinux
7. Copy the following files from syslinux directory to the TFTP directory:

            cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot/
            cp /usr/share/syslinux/menu.c32 /var/lib/tftpboot/
            cp /usr/share/syslinux/memdisk /var/lib/tftpboot/
            cp /usr/share/syslinux/mboot.c32 /var/lib/tftpboot/
            cp /usr/share/syslinux/chain.c32 /var/lib/tftpboot/

      8. Create the directory for your PXE menus:    mkdir /var/lib/tftpboot/pxelinux.cfg
      9. Create a directory for each pxeboot image:

            mkdir –p /var/lib/tftpboot/images/centos/x86_64/6.3
            mkdir –p /var/lib/tftpboot/images/centos/i386/6.3

     10. Download CentOS 6.x DVD1 from CentOS website
     11. Insert CentOS DVD or mount ISO file to /media directory
     12. Copy vmlinuz and initrd.img from /images/pxeboot/ directory on "DVD 1" to appropriate   release/arch directory like this:

cp /media/CentOS_6.3_Final/images/pxeboot/initrd.img /var/lib/tftpboot/images/centos/x86_64/6.3
cp /media/CentOS_6.3_Final/images/pxeboot/vmlinuz /var/lib/tftpboot/images/centos/x86_64/6.3

     13. Install DHCP server:    yum install dhcp
     14. Configure DHCP:       vi /etc/dhcp/dhcpd.conf
add the following lines to dhcpd.conf and change the ip addresses and domain name accordingly:

option domain-name      "taraghi.com";
option domain-name-servers      khosro.taraghi.com;
default-lease-time 600;
max-lease-time 7200;
authoritative;
#################The followings are mandatory to be able to boot from PXE ############
allow booting;
allow bootp;
option option-128 code 128 = string;
option option-129 code 129 = text;
next-server 10.0.0.150;
filename "/pxelinux.0";
######################################
subnet 10.0.0.0 netmask 255.255.255.0 {
        range dynamic-bootp 10.0.0.151 10.0.0.254;
        option broadcast-address 10.0.0.255;
        option routers 10.0.0.1;
}

As you can see here the ip address range is 10.0.0.151-10.0.0.254 and TFTP/PXE/DHCP server has static ip address of 10.0.0.150
     15. Restart DHCP service:     service dhcp restart
     16. Set dhcpd service to start after booting server:   chkconfig dhcpd on
     17. Adjust firewall setting, run the following commands:

          iptables -A INPUT –p udp --dport 67 -j ACCEPT
          iptables -A INPUT –p udp --dport 68 -j ACCEPT

     18. Creating a menu for OS selection and setting default OS selection after loading PXE:

          vi /var/lib/tftpboot/pxelinux.cfg/default

now, add the following lines to /var/lib/tftpboot/pxelinux.cfg/default

default menu.c32
prompt 0
timeout 30

MENU TITLE PXE Menu

LABEL CentsOS 6.3 x86_64
    MENU LABEL CentOS 6.3 x86_64
    KERNEL images/centos/x86_64/6.3/ vmlinuz
    APPEND initrd=images/centos/x86_64/6.3/initrd.img ks=ftp://10.0.0.153/pub/ks.cfg ramdisk_size=100000

If you have more images, you can also add those images above. Also, 10.0.0.153 is ftp server which contains kickstart file.

Setup FTP server:

1. Login to FTP server as root:     su- 
2. Install FTP server:     yum install vsftpd
3. Insert DVD1 installation of CentOS 6.3 or mount the ISO file to /media
4. Copy installation files to FTP public directory:

           cp -ar /media/CentOS_6.3_Final/. /var/ftp/pub/
          don’t forget dot “.”, it will copy hidden files as well

     5. Create an empty kickstart file in public directory:

            touch /var/ftp/pub/ks.cfg

     6. Set seliunx for /var/ftp/pub directory
            chcon -R -t public_content_t /var/ftp/
     7. Set up firewall:
            iptables -A INPUT -p tcp -m state --state NEW --dport 21 -j ACCEPT
     8. Save firewall setting:
           /etc/init.d/iptables save
     9. Start FTP service:    
          service vsftpd restart
    10. Set vsftpd to start after rebooting server:
          chkconfig vsftpd on

Setup Kickstart file:

    Edit ks.cfg:  vi  /var/ftp/pub/ks.cfg
    Add following lines to this file. I explain them with comments:

#It starts the installation process
Install
#configure a connection to a FTP server to locate installation files
url --url ftp://10.0.0.153/pub/
#setup language and keyboard
lang en_US.UTF-8
keyboard us
#Get network info from DHCP server
network --device eth0 bootproto dhcp
#setup encrypted root password, you can take out the encrypted password from /etc/shadow file
rootpw --iscrypted $6$NF6F/Yng442eA8oL$c/sHM
#setup firewall and open ssh port 22
firewall --service=ssh
#sets up the Shadow Password Suite
#(--enableshadow), the SHA 512 bit encryption algorithm for password encryption
#(--passalgo=sha512), and authentication with any existing fingerprint reader.
authconfig --enableshadow --passalgo=sha512 --enablefingerprint
#The selinux directive can be set to --enforcing, --permissive, or --disabled
selinux --enforcing
#setup timezone
timezone America/Toronto
#The default bootloader is GRUB. It should normally be installed on the Master
#Boot Record (MBR) of a hard drive. You can include a --driveorder switch to specify
#the drive with the bootloader and an --append switch to specify commands for
#the kernel.
bootloader --location=mbr --driveorder=sda --append=”crashkernel=auto rhgb quiet”
#Clear the Master Boot Record
zerombr yes
#This directive clears all volumes on the sda hard drive. If it hasn’t been used before,
#--initlabel initializes that drive.
clearpart --all --drives=sda --initlabel
#Changes are required in the partition (part) directives that follow.
part /boot --fstype=ext4 --size=500
part / --fstype=ext4 --size=27500
part swap --size=1000
part /home --fstype=ext4 --size=1000
#reboot machine
reboot
#skip answers to the First Boot process
firstboot --disable

%packages
              # This is the actual package install section. The
              # resolvedeps option allows you to make mistakes and
              # have anaconda sort it out for you, i.e. resolving
              # package dependencies.
@ Base
@ Development Tools
mc
wget
#If you want to switch to GUI mode, you have to install the following packages
@ basic-desktop
@ desktop-platform
@ x11
@ fonts
%end

%post
#Adding a user, in this case”khosro”
useradd -m khosro
#Set password for user “khosro”
echo Khosropass123 | passwd --stdin khosro
#expire the password and force the user to enter the new password after first login
passwd -e khosro
#Turn on the GUI mode, if you want to
sed -i 's/id:3:initdefault:/id:5:initdefault:/g' /etc/inittab

And that’s all. As soon as you turn on computer, you machine will go to Automated Installation mode without any user intervention.
Don't forget to send me your comments.

Hope, you enjoyed,
Khosro Taraghi

What Is Microscan and Delta Versioning

Hello everybody,

Some backup software use incremental or differential backups.So, they do a full backup at the first time, then they only store the changes for next backups.Some software are more intelligent and use a monitoring solution in file or block level.
Let's say an example. You created a text file with a word of "Hello" inside the text file and saved it yesterday. The file backed up during the daily backup process. Today, you open the file, change "Hello" to "Hello1" and save it with different name. As you see here, you just add one more character to file. The second file (new file) is a version of the first file and they are almost the same. The second file is actually a delta to the original. A delta is an offset of the original file.

Now, if we use the file-compare backup method, it will backup the entire new file because it has a different file name. If we use file-level hashing method, it will backup the entire new file again because the new file would be offset by the new data and a new hash would happen.
The best solution is delta versioning solution to de-duplicate this file.
It has two different methods:
1) Block-level delta vesioning
2) Sub-block-level delta versioning.

Block-level delta versioning:
This method is like snapshot that I explained before. This method monitors all updates on disk at block level and stores only the data that changed in relation to original data. This method really decreases the amount of data to send to Disaster Recovery site and it's really efficient.

Sub-block-level delta versioning (Microscan):
It's the same as the Block-level delta versioning but it's more efficient and it works in byte level and not block level. In figure, if we use our previous example, you add one character to the original file which means you need a single 512 bytes (one sector) to store the changes to the original file. Now, if you use an array replication software to send the file to Disaster Recovery site, it will send an entire 32K disk track because that's the minimum level of block definition for array-based solution. If we use a file level monitoring solution, we still need to send the entire 8K because the file system needs 8K to write the data.

Microscan monitors changes at a disk sector level (1 sector=512 bytes) and replicate only the 512-bytes changed delta. A block of data in Windows, for example, is 4KB (8 x 512). So, in Block-level delta versioning method, when a 512 bytes inside a 4KB block is changed, it will send entire 4KB block. But in Microscan or Sub-block-level method, only a 512 bytes changed data will replicate. Therefore, you save a lot of white space. Microscan is really the best efficient method when you are using monitoring solution in the data being updated.   
Hope you enjoyed.
Regards,
Khosro Taraghi

What Is Data De-Duplication and How It Works?

Hello all, 

Data De-Duplication means comparing objects(files and blocks) and removing all duplicated objects and  keep only unique objects. So, the results is a smaller group of data(files and blocks). For example,

 

If you look at the picture above, de-duplication has removed all duplicated data. The result is a smaller group of data. And, of course, there is a way to reproduce the original data. I’ll explain that.

What is the advantage of De-duplication?

1.  Reduced backup cost:  you can save a huge amount of data in terms of size when you save de-duplicated data to tape or sending backups to remote site using WAN or LAN.
2. Reduced WAN and LAN Bandwidth: Using de-duplicated data, you can save bandwidth and reduce the cost of using WAN if you send your data to a remote site. 
3. Reduced hardware cost: you need less tape and harddisk  
4. Increased efficiency of storage  

De-Duplicated methods:

1. File-based comparison and then compression:This method, which is an old method, uses operation system or application to compare files, for example, comparing the name, size, type, and date of modification. If all parameter matches, you can remove one of them. If you use a file-compression method, you can save more space too. In this method, the de-duplication ratio would be 2:1 or 3:1, which 50 % less data. This can be done through a script or operation system and it’s free.  
2. File level hashing:It’s like file-based method but it’s more intelligent. File level hashing creates a unique mathematical hash for files. Then, it will compare the hashes for new files with the originals one. If the hashes match each other, it means the files are the same and it can be removed. This method requires an index table to store the hashes and it can be referenced quickly for match. Usually, the indexes are stored in RAM and they are very quick and don’t slow down the process of hash look up. 
3. Block level hashing:  It’s the same concept as File level hashing but it works with the block of data. So, it’s independent of file system in OS or files themselves. So, the block of data means the way that data is stored on disk and it doesn’t care about the type of data. De-Duplication uses hash for blocks of data and compares every new block of data being stored through the de-duplication and it will remove the equal blocks.  
4. Sub-Block level hashing: It works like Block level hashing but it’s much more sufficient. This is the most common method that used in enterprise today. It divides or slices a block of data into a set of sub blocks with a specific size. For example, it gets a 64KB block of data and creates 4 segments of 16KB each. Then it creates a unique hash for each slice or chunk of data.

The hashes are stored in index hash table and it starts to compare the hashes. As you see in picture, the hashes for chunk number 1 and 3 are equal. So, it removes the duplicated chunk of data for segment 3 and it puts its hash as a pointer so the original data can be restored later.
 Please look at the picture below:

Segment 3 is removed and replaced with the hash as a pointer and now the original file takes less space. The original file can be rebuilt by replacing the hash for segment 3 with the data in segment 1.

       5.   Delta versioning: I will explain this method in my next blog post since it needs more explanations.

Hope you enjoyed.

Regards,
Khosro Taraghi