Sunday, 12 October 2014

Load balancers for web servers RHEL/CentOS 6/7

Load balancers distributes network traffic across backend servers and thus enables high availability.

This article explains how to perform software-based load balancing using Pound package. 


Hostname Running OS IP address HTML index pages
Poundserver CentOS 6.2 192.168.56.107 NA
webserver1 CentOS 6.5 192.168.56.119 webserver-node-1
webserver2 CentOS 6.5 192.168.56.120 Webserver-node-2


Below shown figure is self-explanatory architectural diagram of the pound server and backend webservers.


Below pre-requsites are to be applied on all the hosts and would be skipping as I would leave it as an exercise for the reader.

1. Ensure that the DNS is configured or add the entries in /etc/hosts file. 
2. Set the IP address 
3. Turn off the firewall
4. SElinux should be disabled.

Installation and configurations of webservers(webserver1 and webserver2).

The default apache web service will be installed on both the backend webservers. Since the installations and configurations of the apache web service is exactly the same for both the nodes, hence will record all steps in webserver1. The same steps are to be performed on webserver2.

Install Apache web service:

[root@webserver1]# yum install httpd
[root@webserver1]# 

[root@webserver1]# service httpd start
[root@webserver1]# 

[root@webserver1]# chkconfig httpd on
[root@webserver1]# 

Let's create an web page index.html at /var/www/html.
Repeat the same above steps for webserver2


[root@webserver1]# cat index.html
<html>
<body>
<font size="6">webserver-node-1 </font>
</body>
</html>
[root@pcmk-1 html]# cat index.html
<html>
<body>
<font size="6">webserver-node-1 </font>
</body>
</html>
[root@webserver1]# 

Installation and configurations of Pound server(Poundserver)

Pound package is not part of redhat, hence it has to be downloaded from EPEL.

[root@poundserver]# cd /var/tmp

[root@poundserver tmp]# rpm -ivh --aid --force epel-release-6-8.noarch.rpm
warning: epel-release-6-8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing...                ########################################### [100%]
   1:epel-release           ########################################### [100%]
[root@poundserver tmp]#

[root@poundserver tmp]#yum install Pound*
[root@poundserver tmp]#

You can view the default configuration file by using command below:
[root@poundserver]#cat /etc/pound.cfg

Make the changes to the Pound configuration file as below:

I would comment the section related to HTTPS as I don't need it now.


[root@poundserver]# cat /etc/pound.cfg
#
# Default pound.cfg
#
# Pound listens on port 80 for HTTP and port 443 for HTTPS
# and distributes requests to 2 backends running on localhost.
# see pound(8) for configuration directives.
# You can enable/disable backends with poundctl(8).
#

User "pound"
Group "pound"
Control "/var/lib/pound/pound.cfg"

ListenHTTP
    Address 192.168.56.107
    Port 80
End

#ListenHTTPS
#    Address 0.0.0.0
#    Port    443
#    Cert    "/etc/pki/tls/certs/pound.pem"
#End

Service
    BackEnd
        Address 192.168.56.119
        Port    80
    End
    
    BackEnd
        Address 192.168.56.120
        Port    80
    End
End
[root@poundserver]#


Now, start the Pound service:

[root@poundserver]#service pound start
Starting Pound: starting...
                                                          [  OK  ]
[root@poundserver]#

Observation:

Open a webbrowser and access URL http://192.168.56.107. It displays webservernode-1 
Refresh the webpage, it will flip from webserver1 and webserver2 back and forth.


       

We have now configured a system where the load on the web server is being balanced between two physical servers.

Thursday, 25 September 2014

Managing container - Docker command line #Redhat 7 / CentOS 7

In previous article I had written about the Linux Container archtecture and hereby, I am letting you know on how docker as a management interface could be used as a command line.

Ensure that the docker package is installed on your system else install.

#yum install docker


After you successfully installed the application, use the usual system ctl commands to start docker and to make it run automatically at boot time.

# systemctl start docker.service
# systemctl enable docker.service

Search for an existing image which you are looking for, the command searches the Docker.io Index which is currently the main public registry for sharing repository images where few of them are marked as trusted which means officially checked.

# docker search centos
NAME                                            DESCRIPTION                                     STARS     OFFICIAL   TRUSTED
centos                                          The official build of CentOS.                   463            [OK]       
#

To download a selected image from the remote registry to your local machine pull repository_name/image_name

# docker pull centos

Pulling repository centos
70214e5d0a90: Download complete 
5a1ebaa356ff: Download complete 
68eb857ffb51: Download complete 
511136ea3c5a: Download complete 
34e94e67e63a: Download complete 

Listing the images 

# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
centos              centos5             5a1ebaa356ff        2 weeks ago         484 MB
centos              centos7             70214e5d0a90        2 weeks ago         224 MB
centos              latest              70214e5d0a90        2 weeks ago         224 MB
centos              centos6             68eb857ffb51        2 weeks ago         212.7 MB

To remove one or more images from your system, 
#docker rmi image_name

Managing Containers

To creating a new container, replace the given container name and specify the image on top of which will the container run. The docker run command lets you say which command to run in a container. Once the container is running, you can manage [ start/stop/restart] accordingly. you could also remove if no longer needed. 

The command that you pass on the docker run will see to it that the command runs inside the container as its running environment, hence very little can be seen from the host system. 

# docker run --rm --name=dockertest centos ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
7: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
    link/ether 4a:69:d0:e6:b7:2e brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.3/16 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::4869:d0ff:fee6:b72e/64 scope link tentative 
       valid_lft forever preferred_lft forever

If you want to make a directory from the host available to the container, map network ports from the container to the host which can be done by docker run command line.

# mkdir -p /docker/centos7
# echo "Docker testing" >/docker/centos7/dockertest.txt
#
# docker run -d -p 8080:6000 --name="dockerwebserver" -w /docker -v /docker/centos7:/docker centos /bin/python -m SimpleHTTPServer 6000
acbc99f8b64b8f3472a6740a2d22ba10ab42ff2615c6adda08ca52236293e136
#
where,

Detach (-d) the container so it runs in the background
Map (-p) TCP port 6000 on the container to port 8080 on the host
Map Working directory(-w /docker) in the container when the command runs.
Map directory from the host (-v /docker/centos7) to the container
HTTPserver module(-m) with /bin/python command.
  
# netstat -tupln | grep 8080
tcp6       0      0 :::8080                 :::*                    LISTEN      2006/docker         

To execute commands inside of a running container, you need to connect to it through a command-line interface. T he docker attach command is not suitable for this, since it only lets you observe the standard output of the application currently running in the container. Instead, use the nsenter command to enter the container namespace. T his command requires the ID of the container as it appears on the host system.

# docker ps -a

CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                    NAMES
acbc99f8b64b        centos:centos7      /bin/python -m Simpl   11 minutes ago      Up 11 minutes       0.0.0.0:8080->6000/tcp   dockerwebserver     
#

# docker inspect -f {{.State.Pid}} acbc99f8b64b
4526
#


# nsenter -m -u -n -i -p -t 4526 /bin/sh
sh-4.2# cat /etc/centos-release 
CentOS Linux release 7.0.1406 (Core) 
sh-4.2# 

sh-4.2# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 13:35 ?        00:00:00 /bin/python -m SimpleHTTPServer 6000
root         6     0  0 13:49 ?        00:00:00 /bin/sh
root         9     6  0 13:50 ?        00:00:00 ps -ef
sh-4.2#
sh-4.2# uptime
13:50:33 up  2:16,  0 users,  load average: 0.00, 0.01, 0.05
sh-4.2# 
sh-4.2# 

sh-4.2# free -m
             total       used       free     shared    buffers     cached
Mem:           994        318        676          6          2        123
-/+ buffers/cache:        192        802
Swap:         1023          0       1023
sh-4.2#
sh-4.2# df -h
Filesystem                                                                                         Size  Used Avail Use% Mounted on
/dev/mapper/docker-253:3-3148627-acbc99f8b64b8f3472a6740a2d22ba10ab42ff2615c6adda08ca52236293e136  9.8G  280M  9.0G   3% /
tmpfs                                                                                              498M     0  498M   0% /dev
shm                                                                                                 64M     0   64M   0% /dev/shm
/dev/mapper/VolGroup-varlv                                                                         6.0G  1.5G  4.6G  25% /etc/hosts
/dev/mapper/VolGroup-rootlv                                                                        4.9G  102M  4.8G   3% /docker
tmpfs                                                                                              498M     0  498M   0% /proc/kcore
sh-4.2#
sh-4.2#
sh-4.2#exit

List/stop/start/restart/Monitoring the containers

# docker ps
CONTAINER ID        IMAGE               COMMAND                     CREATED             STATUS              PORTS               NAMES
acbc99f8b64b        centos:centos7      /bin/python -m Simpl   20 minutes ago      Up 20 minutes       0.0.0.0:8080->6000/tcp   dockerwebserver     
#

# docker stop dockerwebserver
dockerwebserver
#

# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

# docker start dockerwebserver
dockerwebserver

# docker ps
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                    NAMES
acbc99f8b64b        centos:centos7      /bin/python -m Simpl   22 minutes ago      Up 2 seconds        0.0.0.0:8080->6000/tcp   dockerwebserver     

# docker restart dockerwebserver
dockerwebserver

# docker ps
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                    NAMES
acbc99f8b64b        centos:centos7      /bin/python -m Simpl   22 minutes ago      Up 3 seconds        0.0.0.0:8080->6000/tcp   dockerwebserver     

Commit containers

To create a new image from changes made in the running container,syntax as below 
docker commit container_name [repository_name:tag] 


# docker commit dockerwebserver centos:dockerwebserver_V1.0
f7c53eac787979bccc347e08c1f7a6da96daf6a829c5f35165b2675763943cbc
#

# docker images
REPOSITORY          TAG                    IMAGE ID            CREATED             VIRTUAL SIZE
centos              dockerwebserver_V1.0   f7c53eac7879        10 seconds ago      224 MB
centos              centos5                5a1ebaa356ff        2 weeks ago         484 MB
centos              centos7                70214e5d0a90        2 weeks ago         224 MB
centos              latest                 70214e5d0a90        2 weeks ago         224 MB
centos              centos6                68eb857ffb51        2 weeks ago         212.7 MB

View containers

To view an overall information on how Docker is configured on your system


# docker info
Containers: 1
Images: 6
Storage Driver: devicemapper
 Pool Name: docker-253:3-3148627-pool
 Data file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
 Data Space Used: 1365.8 Mb
 Data Space Total: 102400.0 Mb
 Metadata Space Used: 1.4 Mb
 Metadata Space Total: 2048.0 Mb
Execution Driver: native-0.2
Kernel Version: 3.10.0-123.el7.x86_64
#

To display a detailed information about an image or a container
# docker inspect acbc99f8b64b
[{
    "ID": "acbc99f8b64b8f3472a6740a2d22ba10ab42ff2615c6adda08ca52236293e136",
    "Created": "2014-09-25T12:35:20.804560542Z",
    "Path": "/bin/python",

Archive containers/images

Archive files are useful for backing up or restoring containers and images. Note that you can not backup 
data volumes this way since they are external to containers. 
To export the contents of a container file system as an archive in tar compress format.

# docker ps 
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS                    NAMES
acbc99f8b64b        centos:centos7      /bin/python -m Simpl   46 minutes ago      Up 23 minutes       0.0.0.0:8080->6000/tcp   dockerwebserver     
#

# docker export dockerwebserver > centos-webserver-docker-7.0-x86_68
# tar -cvf centos-webserver-docker-7.0-x86_68.tar centos-webserver-docker-7.0-x86_68
# gzip centos-webserver-docker-7.0-x86_68.tar
# du -sh rhel-server-docker-7.0-21.4-x86_64.tar.gz 

Conversely, you can import an content from an URL or an tar archive as below
#docker import <source>

On conclusion, summarizing the docker command reference.

Docker commands Descriptions
docker version Find the version number currently installed
docker search <name> Search for existing images
docker pull repository_name/image_name Download a selected image from remote to your local machine
docker images Lists all locally installed image
docker push Copy image or repository to a remote location
docker rmi image_name Remove one or more images from your system
docker commit container_name create a new image from changes made in the running container
docker stop container_name stop the running container gracefully
docker kill container_name stop a container that is not responding
docker start container_name start a previously stopped container
docker restart container_name Restart a running container
docker rm container_name remove a container
docker ps Listing containers
docker info overall information on how Docker is configured on your system
docker top container_name dynamic view of processes currently running inside of a certain container
docker export container_name export the contents of a container file system as an archive in tar compress format

Thursday, 18 September 2014

Linux Containers Architecture RHEL 7

Initially thought to write about the docker command-line & configuration, later it was been advised by few of my colleagues that a brief introduction about Linux containers would be much helpful and hence I though to share below introductory article. 

Several components are needed for Linux Containers to function correctly, most of is them provided by the Linux kernel. Kernel namespaces ensure process isolation and cgroups are employed to control the system resources. SELinux is used to assure separation between the host and the container and also between the individual containers. Management interface forms a higher layer that interacts with the aforementioned kernel components and provides tools for construction and management of containers. 


Namespaces

The kernel provides process isolation by creating separate namespaces for containers. Namespaces allow you to create an abstraction of a particular global system resource and make it appear as a separated instance to processes within a namespace. Consequently, there are several containers which use same resource simultaneously without creating a conflict.

Control Groups (cgroups)


The kernel uses cgroups to group processes for the purpose of system resource management. Cgroups 
let you allocate CPU time, system memory, network bandwidth, or combinations of these among user-
defined groups of tasks.

SELinux

SELinux provides secure separation of containers by applying SELinux policy and labels.

Management Interface

RHEL 7 provides the Docker application as a main management tool for Linux 
Containers. Docker builds on the aforementioned kernel capabilities, adding several enhancement 
features, such as portability or version control. 

Containers

There are two general scenarios for using Linux containers in RHEL 7

1. Host Containers : a tool to carve out containers as lightweight application sandboxes, each runs the same user space as the host system, so all applications running in host container's share userspace and run time.



2. Image-based Containers : 


An application is packaged with individual run-time stack, which makes it 
independent from the host operating system. This way, you can run several instances of an application, 
each developed for a different platform. This is possible because the container run time and the 
application run time are deployed in the form of an image.



Image-based containers allow you to host multiple instances and versions of an application, with minimal overhead and increased flexibility. Such containers are not tied to the host-specific configuration, which makes them portable. These features are enabled by the Docker format for application packaging. 

docker command-line & configuration will be explained in next post.

Tuesday, 16 September 2014

NetworkManager on RHEL7

Objective: Introduction to NetworkManager using network tools on RHEL 7

Red Hat Enterprise Linux 7, the default networking service is provided by NetworkManager installed by default on RHEL, controls network and keeps up the network devices and their connections up and active.

users do not interact with NetworkManager system service directly, instead perform network configs using as below 

1. simple curses-based text user interface (TUI) for NetworkManager, nmtui
2. command line tool, nmcli, is provided to allow users and scripts to interact with NetworkManager
3. Graphical user interface tools are also available e.g control-center  provided by GNOME shell which incorporates a network settings tool.

Here, in this article I would discuss on using commnd line interface(nmcli) for controlling NetworkManager

In previous RHEL, default way to configure network was using network scripts(/etc/init.d/network) and any other installed script it calls. Although NetworkManager provides the default network service, it ensures that network scripts cooperate as well.

Red Hat Enterprise Linux 7, NetworkManager is started first, and /etc/init.d/network checks with NetworkManager to avoid tampering with NetworkManager's connections. NetworkManager is intended to be the primary application using sysconfig configuration files and /etc/init.d/network is intended to be secondary.


# systemctl status NetworkManager 
NetworkManager.service - Network Manager
   Loaded: loaded (/usr/lib/systemd/system/NetworkManager.service; enabled)
   Active: active (running) since Tue 2014-09-16 07:05:22 IST; 45min ago
 Main PID: 648 (NetworkManager)
   CGroup: /system.slice/NetworkManager.service
           └─648 /usr/sbin/NetworkManager --no-daemon

Below are summarized nmcli commands as reference.

NetworkManager Commands Description
nmcli general status show the overall status of NetworkManager
nmcli connection show show all connections
nmcli connection show --active show only currently active connections
nmcli device status show devices recognized by NetworkManager
nmcli con up/down id <interface> start/stop interface
nmcli dev connect/disconnect iface <inetrface> start/stop interface
nmcli -p connection up <profile> ifname <interface> bring up the new connection
nmcli -p con show <profile> view detailed information about the newly configured connection


Configure manual IP address on the interface using nmcli 

Method 1 : 

 # ifconfig enp0s8
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 08:00:27:82:69:0b  txqueuelen 1000  (Ethernet)

# nmcli connection add con-name enp0s8 ifname enp0s8 type ethernet ip4 192.168.56.131/24 gw4 192.168.56.1
Connection 'enp0s8' (952e2bfa-1dcb-4612-9bb4-df347b5a8a50) successfully added.

# nmcli device show enp0s8
GENERAL.DEVICE:                         enp0s8
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         08:00:27:82:69:0B
GENERAL.MTU:                            1500
GENERAL.STATE:                          30 (disconnected)
GENERAL.CONNECTION:                     --
GENERAL.CON-PATH:                       --
WIRED-PROPERTIES.CARRIER:               on

# nmcli device connect enp0s8
Device 'enp0s8' successfully activated with '952e2bfa-1dcb-4612-9bb4-df347b5a8a50'.

# ifconfig enp0s8
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.131  netmask 255.255.255.0  broadcast 192.168.56.255
        inet6 fe80::a00:27ff:fe82:690b  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:82:69:0b  txqueuelen 1000  (Ethernet)

Method 2 :

nmcli Interactive Connection Editor

# ifconfig enp0s8
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::a00:27ff:fe82:690b  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:82:69:0b  txqueuelen 1000  (Ethernet)

# nmcli connection edit id enp0s8

===| nmcli interactive connection editor |===

Editing existing '802-3-ethernet' connection: 'enp0s8'

Type 'help' or '?' for available commands.
Type 'describe [<setting>.<prop>]' for detailed property description.

You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, ipv4, ipv6, dcb
nmcli> help
------------------------------------------------------------------------------
---[ Main menu ]---
goto     [<setting> | <prop>]        :: go to a setting or property
remove   <setting>[.<prop>] | <prop> :: remove setting or reset property value
set      [<setting>.<prop> <value>]  :: set property value
describe [<setting>.<prop>]          :: describe property
print    [all]                       :: print the connection
verify   [all]                       :: verify the connection
save     [persistent|temporary]      :: save the connection
activate [<ifname>] [/<ap>|<nsp>]    :: activate the connection
back                                 :: go one level up (back)
help/?   [<command>]                 :: print this help
nmcli    <conf-option> <value>       :: nmcli configuration
quit                                 :: exit nmcli
------------------------------------------------------------------------------
nmcli> 

nmcli> goto ipv4
You may edit the following properties: method, dns, dns-search, addresses, address-labels, routes, ignore-auto-routes, ignore-auto-dns, dhcp-client-id, dhcp-send-hostname, dhcp-hostname, never-default, may-fail
nmcli ipv4> print
['ipv4' setting values]
ipv4.method:                            auto
ipv4.dns:                               
ipv4.dns-search:                        
ipv4.addresses:                         

nmcli ipv4> set ipv4.addresses 192.168.56.131/24 192.168.56.1
Do you also want to set 'ipv4.method' to 'manual'? [yes]: yes
nmcli ipv4> 

nmcli ipv4> print
['ipv4' setting values]
ipv4.method:                            manual
ipv4.dns:                               
ipv4.dns-search:                        
ipv4.addresses:                         { ip = 192.168.56.131/24, gw = 192.168.56.1 }

nmcli> save persistent
Connection 'enp0s8' (d0a11f5e-eb2b-4a7a-b225-71f8d07b5664) successfully updated.
nmcli> 
nmcli> activate enp0s8
Monitoring connection activation (press any key to continue)
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
nmcli> 

[root@localhost ~]# ifconfig enp0s8
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.131  netmask 255.255.255.0  broadcast 192.168.56.255

Method 3 : 

Configure IP using ifcfg files


To configure an interface with static network settings using ifcfg files, for an interface with the name 
eth0, create a file with name ifcfg-eth0 in the /etc/sysconfig/network-scripts/ directory as 
follows:


DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
NETMASK=255.255.255.0
IPADDR=192.168.56.131
USERCTL=no
HWADDR=08:00:27:49:f1:68  <--- NOTE: Specify this as this may influence the device naming procedure.

How does the interface rename happens to be ?


A rule in /usr/lib/udev/rules.d/60-net.rules instructs the udev helper utility, /lib/udev/rename_device, to look into all /etc/sysconfig/network-scripts/ifcfg-suffix files. If it finds an ifcfg file with a HWADDR entry matching the MAC address of an interface it renames the interface to the name given in the ifcfg file by the DEVICE directive.

I, conclude hereby that the reader would have an understanding on how to configure and interact with NetworkManager.