Table of Contents

Middleware

Pacemaker CLI Commands Cheet Sheat



RHEL7 was adopted Pacemaker.
RHEL5 or RHEL 6 was used RGMANAGER.



Knowlege

Formation

Formation Note
Pacemaker + Corosync recommend
Pacemaker + Heartbeat Versinon3

Component

Package Note
pacemaker Pacemaker is an Open Source, High Availability resource manager suitable for both small and large clusters.
corosync cluster.conf
pcs a full cluster lifecycle configuration shell and web based GUI.


Commands

Pacemaker (pcs)

Check

You must start pacemaker for pcs commands.

Commands Operation Note
pcs status
pcs status --full
crm_mon -1
Check Status
pcs config Check Configuration
pcs property list Check Cluster Properties
pcs cluster cib Check Configuration with XML
pcs status nodes Check Note Status

Operation

Commands Operation Note
pcs cluster start --all start cluster with two node start pacemaker and corosync
pcs cluster start SERVER1 start cluster with one node, SERVER1 start pacemaker and corosync
pcs cluster stop --all stop cluster stop pacemaker and corosync
pcs cluster stop SERVER1 stop cluster with one node, SERVER1 stop pacemaker and corosync
pcs cluster enalbe --all auto start
pcs cluster stop SERVER1 --force stop cluster on only one node
pcs cluster kill Forcibly stop cluster
pcs cluster standby SERVER1 change to standby Cluster SERVER1 to SERVER2
pcs cluster unstandby SERVER1 change to unstandby

Resource Operation

Commands Operation Note
pcs disalbe XXXX stop resource
pcs enable XXXX start resource

Configuration

Commands Operation Note
pcs cluster sync Sync corosync.conf
pcs cluster destroy Delete Configuration Deleted /var/lib/pacemaker/cib/cib.xml, too.
Deleted /etc/cluster/cluster.conf, too.
pcs cluster cib output.cib Output Configuration
pcs -f output.cib COMMAND Change Configuration
pcs cluster cib-push output.cib Input Configuration
RESOURCE GROUPS
Commands Operation
pcs resource group list The following command lists all currently configured resource groups
pcs resource group add group_name resource_id You create a resource group with the following command
If the group does not exist, this command creates the group. If the group exists, this command adds additional resources to the group.
pcs resource group remove group_name resource_id… You remove a resource from a group with the following command.
If there are no resources in the group, this command removes the group itself.
pcs resource show
pcs resource group list
pcs resource show RESCOURCE
pcs cluster corosync check node and totem
pcs stonith show RESOURCE
pcs constraint list check the resource order

Ref.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/High_Availability_Add-On_Reference/s1-resourcegroups-HAAR.html




How to create Pacemaker.

Installing

node1/2# yum install pacemaker corsync
node1/2# yum install pcs fence-agents-all

Start Service

node1/2# systemctl enable pcsd
node1/2# systemctl start pcsd


Configuration of Cluster

node1/2# echo redhat |passwd --stdin hacluster
one node
node1# pcs cluster auth -u hacluster -p redhat  server1 server2
node1# pcs cluster setup --start --name test-cluster --wait_for_all=0 server1 server2

--wait_for_all=0 : If it is 1node, start cluster.
node1# pcs status


Configration Pacemaker Poperty

Check Property
# pcs property list
Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: test-cluster
 dc-version: 1.1.16-a14efad
 default-resource-stickiness: INFINITY
 have-watchdog: false
 no-quorum-policy: ignore
 stonith-enabled: false
Disalbe STONISH (Shoot The Other Node In The Head)
# pcs property set stonith-enabled=false
Disable auto failback
# pcs property set default-resource-stickiness="INFINITY"
Do not manage the minimum number required

Even if the number of clusters has not reached the required minimum number, no special action is taken and management of all nodes is continued.

# pcs property set no-quorum-policy=ignore


Configuring Fencing (STONITH)

STONITH (Shoot The Other Node In The Head)

#pcs stonith list
#pcs stonith describe XXXXX
#pcs stonith list
#pcs stonith describe XXXXX
##                  unique name    fence agent   target node              device addr              options
# pcs stonith create fence_server1  fence_ilo4    pcmk_host_list="server2-ilo" ipaddr="xx.xx.xx.xx" action="reboot" login="admin" passwd="secret" delay=15 op monitor interval=60s
# pcs stonith create fence_server2  fence_ilo4    pcmk_host_list="server2-ilo" ipaddr="xx.xx.xx.xx" action="reboot" login="admin" passwd="secret" op monitor interval=60s

If you dont't use Fence, You can do action="off".
# pcs stonith show
 stonith-ilo4-server1    (stonith:fence_ilo4):    Started 
 stonith-ilo4-server2    (stonith:fence_ilo4):    Started 
Check stonith
#pcs property --all |grep -E "stonith-enabled|stonith-action"
 stonith-action: reboot
 stonith-enabled: true
Test fencing
# pcs stonith fence server2

-> server2 is rebooted.
Disable/Enable STONITH
# pcs resource disable RESOURCE_STONITH
# pcs resource enable RESOURCE_STONITH


Setting Resources

Failover

disable auto failover
It is failovered with manual operation or node down.
'migration-threshold' is retry number of resource restart.

pcs resource defaults resource-stickiness="INFINITY" migration-threshold="0"


VIP

set virtual IP address

pcs resource create ipfront ocf:heartbeat:IPaddr2 ip=192.168.40.xx cidr_netmask=24 nic="eth0" op monitor interval=10s
pcs resource create ipmaintenance ocf:heartbeat:IPaddr2 ip=192.168.50.xx cidr_netmask=24 nic="eth1" op monitor interval=10s


Process

systemd (systemctl)
pcs resource create httpd systemd:httpd --group zabbix-group
pcs resource create php-fpm systemd:php-fpm --group zabbix-group
pcs resource create zabbix-server systemd:zabbix-server --group zabbix-group
lsb (/etc/init.d/xxxx)

/etc/init.d/mysqld

pcs resource create mysql lsb:mysqld op monitor interval=60s
pcs resource update mysql op start interval=30s timeout=15 on-fail=ignore
pcs resource update mysql op stop interval=0s timeout=15 on-fail=ignore


Order of Resource

pcs constraint order frontip then maintenanceip
pcs constraint order maintenanceip then mysql

RESOURCE GROUPS

pcs resource group add mysqlgroup frontip maintenanceip mysql

Delete Resouce

# pcs resouce delete  XXXX

Cleanup Failure History cleanup

# pcs resource cleanup res_stonith_ipmilan01


Note

Backup/Restore

# pcs config backup /tmp/`date +%Y%m%d`_`uname -n`_pcs_config_backup
# pcs config restore /tmp/`date +%Y%m%d`_`uname -n`_pcs_config_backup.tar.bz2

CORROSYNC : corosync.conf

/etc/corosync/corosync.conf
/etc/corosync/corosync.conf.example

In AWS EC2, You must use UDP Unicast insted of UDP Multicast in corosync.
Sample Configuration : /etc/corosync/corosync.conf.example.udpu


Reference