This chapter explains how to plan the configuration of highly available services on your Linux FailSafe cluster. The major sections of this chapter are as follows: configuration planning overviewConfiguration planning involves making decisions about how you plan to use the Linux FailSafe cluster, and based on that, how the disks and interfaces must be set up to meet the needs of the highly available services you want the cluster to provide. Questions you must answer during the planning process are: What do you plan to use the nodes for? Your answers might include uses such as offering home directories for users, running particular applications, supporting an Oracle database, providing Netscape World Wide Web service, and providing file service. Which of these uses will be provided as a highly available service? To offer applications as highly available services that are not currently available as Linux Failsafe software options, a set of application monitoring shell scripts needs to be developed that provides switch over and switch back functionality. Developing these scripts is described in the Linux FailSafe Programmer's Guide. If you need assistance in this regard, contact SGI Global Services, which offers custom Linux FailSafe agent development and HA integration services. Which node will be the primary node for each highly available service? The primary node is the node that provides the service (exports the filesystem, is a Netscape server, provides the database, and so on) when the node is in an UP state. For each highly available service, how will the software and data be distributed on shared and non-shared disks? Each application has requirements and choices for placing its software on disks that are failed over (shared) or not failed over (non-shared). Are the shared disks going to be part of a RAID storage system or are they going to be disks in SCSI/Fibre channel disk storage that has mirroring such as the Linux Raid Tools implemented on them? For reliability, shared disks must be part of a RAID storage system or in SCSI/Fibre channel disk storage with mirroring on them. Will the shared disks be used as raw devices/volumes or as volumes with filesystems on them? Logical volumes, filesystems, and raw partitions are all supported by Linux Failsafe. The choice of volumes, filesystems, or raw devices depends on the application that is going to use the disk space. Which IP addresses will be used by clients of highly available services? Multiple interfaces may be required on each node because a node could be connected to more than one network or because there could be more than one interface to a single network. Which resources will be part of a resource group? All resources that are dependent on each other have to be in the resource group. What will be the failover domain of the resource group? The failover domain determines the list of nodes in the cluster where the resource group can reside. For example, a volume resource that is part of a resource group can reside only in nodes from which the disks composing the volume can be accessed. IP address planningHow many highly available IP addresses on each network interface will be available to clients of the highly available services? At least one highly available IP address must be available for each interface on each node that is used by clients of highly available services. Which IP addresses on primary nodes are going to be available to clients of the highly available services? For each highly available IP address that is available on a primary node to clients of highly available services, which interface on the other nodes will be assigned that IP address after a failover? Every highly available IP address used by a highly available service must be mapped to at least one interface in each node that can take over the resource group service. The highly available IP addresses are failed over from the interface in the primary node of the resource group to the interface in the replacement node. As an example of the configuration planning process, say that you have a two-node Linux FailSafe cluster that is a departmental server. You want to make four XFS filesystems available for NFS mounting and have two Netscape FastTrack servers, each serving a different set of documents. These applications will be highly available services. You decide to distribute the services across two nodes, so each node will be the primary node for two filesystems and one Netscape server. The filesystems and the document roots for the Netscape servers (on XFS filesystems) are each on their own striped LVM logical volume. The logical volumes are created from disks in a RAID storage system connected to both nodes. There are four resource groups: NFSgroup1 and NFSgroup2 are the NFS resource groups, and Webgroup1 and Webgroup2 are the Web resource groups. NFSgroup1 and Webgroup1 will have one node as the primary node. NFSgroup2 and Webgroup2 will have the other node as the primary node. Two networks are available on each node, eth0 and eth1. The eth0 interfaces in each node are connected to each other to form a private network. The following sections help you answer the configuration questions above, make additional configuration decisions required by Linux FailSafe, and collect the information you need to perform the configuration tasks described in , and . configuration planning disk disk configuration planningThe first subsection below describes the disk configuration issues that must be considered when planning a Linux FailSafe system. It explains the basic configurations of shared and non-shared disks and how they are reconfigured by Linux FailSafe after a failover. The second subsection explains how disk configurations are specified when you configure the Linux FailSafe system. For each disk in a Linux FailSafe cluster, you must choose whether to make it a shared disk, which enables it to be failed over, or a non-shared disk. Non-shared disks are not failed over. The nodes in a Linux FailSafe cluster must follow these requirements: The system disk must be a non-shared disk. The Linux FailSafe software, in particular the directories /var/run/failsafe and /var/lib/failsafe, must be on a non-shared disk. Choosing to make a disk shared or non-shared depends on the needs of the highly available services that use the disk. Each highly available service has requirements about the location of data associated with the service: Some data must be placed on non-shared disks Some data must be placed on shared disks Some data can be on shared or non-shared disks The figures in the remainder of this section show the basic disk configurations on Linux FailSafe clusters before failover. Each figure also shows the configuration after failover. The basic disk configurations are these: A non-shared disk on each node Multiple shared disks contained Web server and NFS file server documents In each of the before and after failover diagrams, just one or two disks are shown. In fact, many disks could be connected in the same way as each disk shown. Thus each disk shown can represent a set of disks. A Linux cluster can contain a combination of the basic disk configurations listed above. shows two nodes in a Linux FailSafe cluster, each of which has a non-shared disk with two resource groups. When non-shared disks are used by highly available applications, the data required by those applications must be duplicated on non-shared disks on both nodes. When a failover occurs, IP aliases fail over. The data that was originally available on the failed node is still available from the replacement node by using the IP alias to access it. The configuration in contains two resource groups, Group1 and Group2. Group1 contains resource 192.26.50.1 of IP_address resource type. Group2 contains resource 192.26.50.2 of IP_address resource type.

shows a two-node configuration with one resource group, Group1. Resource group Group1 has a failover domain of (xfs-ha1, xfs-ha2). Resource group Group1 contains three resources: resource 192.26.50.1 of resource type IP_address, resource /shared of resource type filesystem, and resource shared_vol of resource type volume. In this configuration, the resource group Group1 has a primary node, which is the node that accesses the disk prior to a failover. It is shown by a solid line connection. The backup node, which accesses the disk after a failover, is shown by a dotted line. Thus, the disk is shared between the nodes. In an active/backup configuration, all resource groups have the same primary node. The backup node does not run any highly available resource groups until a failover occurs.

, shows two shared disks in a two-node cluster with two resource groups, Group1 and Group2. Resource group Group1 contains the following resources: Resource 192.26.50.1 of type IP_address Resource shared1_vol of type volume Resource /shared1 of type filesystem Resource group Group1 has a failover domain of ( xfs-ha1, xfs-ha2). Resource group Group2 contains the following resources: Resource 192.26.50.2 of type IP_address Resource shared2_vol of type volume Resource /shared2 of type filesystem Resource group Group2 has a failover domain of ( xfs-ha2, xfs-ha1). In this configuration, each node serves as a primary node for one resource group. The solid line connections show the connection to the primary node prior to failover. The dotted lines show the connections to the backup nodes. After a failover, the surviving node has all the resource groups.

Other sections in this chapter provide more specific information about choosing between shared and non-shared disks for various types of data associated with each highly available service. There are no configuration parameters associated with non-shared disks. They are not specified when you configure a Linux FailSafe system. Only shared disks (actually, the logical volumes/partitions on shared disks) are specified at configuration. See the for details. configuration planning logical volume logical volumeconfiguration planning The first subsection below describes logical volume issues that must be considered when planning a Linux FailSafe system. The second subsection explains the aspects of the configuration that must be specified for a Linux FailSafe system. logical volume parameters configuration parameterslogical volumesConfiguration parameters for logical volumes list Owner of device filename (default value: root) Group of device filename (default value: sys) Mode of device filename (default value: 600) , lists a label and parameters for individual logical volumes. Resource Attribute volA volB volC Comments devname-owner root root root The owner of the device name. devname-group sys sys root The group of the device name. devname-mode 0600 0600 0600 The mode of the device name. configuration planning filesystem filesystemconfiguration planning The first subsection below describes filesystem issues that must be considered when planning a Linux FailSafe system. The second subsection gives an example of an XFS filesystem configuration on a Linux FailSafe system. The third subsection explains the aspects of the configuration that must be specified for a Linux FailSafe system. The Linux FailSafe software supports the automatic failover of filesystem including XFS, ext2fs, and reiserfs on shared disks. Shared disks must be either mirrored or RAID storage systems that are shared between the nodes in the two-node Linux FailSafe cluster. The following are special issues that you need to be aware of when you are working with filesystems on shared disks in a Linux FailSafe cluster: All filesystems to be failed over must be supported by Failsafe. For availability, filesystems to be failed over in a Linux FailSafe cluster should be created either on mirrored disks or using a system that supports hardware mirroring of the data such as a RAID storage system. Create the mount points for the filesystems on all nodes in the failover domain. When you set up the various highly available filesystems on each node, make sure that each filesystem uses a different mount point. Do not simultaneously mount filesystems on shared disks on more than one node. Doing so causes data corruption. Normally, Linux FailSafe performs all mounts of filesystems on shared disks. If you manually mount a filesystem on a shared disk, make sure that it is not being used by another node. Do not place filesystems on shared disks in the /etc/fstab file. Linux FailSafe mounts these filesystems only after making sure that another node does not have these filesystems mounted. The resource name of a resource of the filesystem resource type is the mount point of the filesystem. When clients are actively writing to a Linux FailSafe NFS filesystem during failover of filesystems, data corruption can occur unless filesystems are exported with the mode sync. This mode requires that local mounts of the filesystems use the sync mount mode as well. Using sync affects performance considerably. Continuing with the example configuration from the , say that volumes A and B have XFS filesystems on them: The filesystem on volume A is mounted at /sharedA with modes rw and noauto. Call it filesystem A. The filesystem on volume B is mounted at /sharedB with modes rw, noauto, and wsync. Call it filesystem B. filesystem configuration parameters configuration parametersfilesystem , lists a label and configuration parameters for each filesystem. Resource Attribute /sharedA /sharedB Comments monitoring-level 2 2 There are two types of monitoring: 1 – checks /etc/mtab file 2 – checks if the filesystem is mounted using stat command volume-name volA volB The label of the logical volume on which the filesystem was created. mode rw,noauto rw,noauto,wsync The mount options used to mount the filesystem. This is specified the same as the options for the mount command or other filesystems listed in /etc/fstab. See , for information about creating XFS filesystems. IP address planning configuration planningIP address IP addressconfiguration planning The first subsection below describes network interface and IP address issues that must be considered when planning a Linux FailSafe system. The second subsection gives an example of the configuration of network interfaces and IP addresses on a Linux FailSafe system. The third subsection explains the aspects of the configuration that must be specified for a Linux FailSafe configuration. Follow these guidelines when planning the configuration of the interfaces to the private network between nodes in a cluster that can be used as a control network between nodes (this information is used when you define the nodes): Each interface has one IP address. The IP addresses used on each node for the interfaces to the private network are on a different subnet from the IP addresses used for public networks. /etc/hosts file An IP name can be specified for each IP address in /etc/hosts. Choosing a naming convention for these IP addresses that identifies them with the private network can be helpful. For example, precede the hostname with priv- (for private), as in priv-xfs-ha1 and priv-xfs-ha2. Follow these guidelines when planning the configuration of the node interfaces in a cluster to one or more public networks: re-MACing dedicated backup interfaces requiredIf re-MACing is required, each interface to be failed over requires a dedicated backup interface on the other node (an interface that does not have a highly available IP address). Thus, for each IP address on an interface that requires re-MACing, there should be one interface in each node in the failover domain dedicated for the interface. Each interface has a primary IP address. The primary IP address does not fail over. The hostname of a node cannot be a highly available IP address. All IP addresses used by clients to access highly available services must be part of the resource group to which the HA service belongs. If re-MACing is required, all of the highly available IP addresses must have the same backup interface. Making good choices for highly available IP addresses is important; these are the “hostnames” that will be used by users of the highly available services, not the true hostnames of the nodes. Make a plan for publicizing the highly available IP addresses to the user community, since users of highly available services must use highly available IP addresses instead of the output of the hostname command. Do not configure highly available IP addresses in static Linux configuration files. re-MACing determining if requiredFollow the procedure below to determine whether re-MACing is required (see , for information about re-MACing). It requires the use of three nodes: node1, node2, and node3 . node1 and node2 can be nodes of a Linux FailSafe cluster, but they need not be. They must be on the same subnet. node3 is a third node. If you need to verify that a router accepts gratuitous ARP packets (which means that re-MACing is not required), node3 must be on the other side of the router from node1 and node2. Configure an IP address on one of the interfaces of node1: # /sbin/ifconfig interface  inet ip_address  netmask netmask up interface is the interface to be used access the node. ip_address is an IP address for node1. This IP address is used throughout this procedure. netmask is the netmask of the IP address. From node3, ping the IP address used in Step 1 : # ping -c 2 ip_address PING 190.0.2.1 (190.0.2.1): 56 data bytes 64 bytes from 190.0.2.1: icmp_seq=0 ttl=255 time=29 ms 64 bytes from 190.0.2.1: icmp_seq=1 ttl=255 time=1 ms ----190.0.2.1 PING Statistics---- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 1/1/1 ms Enter this command on node1 to shut down the interface you configured in Step1 : # /sbin/ifconfig interface  down On node2, enter this command to move the IP address to node2: # /sbin/ifconfig interface  inet ip_address  netmask netmask up From node3, ping the IP address: # ping -c 2 ip_address If the ping command fails, gratuitous ARP packets are not being accepted and re-MACing is needed to fail over the IP address. For this example, you are configuring an IP address of 192.26.50.1. This address has a network mask of 255.255.255.0, a broadcast address of 192.26.50.255, and it is configured on interface eth0. In this example, you are also configuring an IP address of 192.26.50.2. This address also has a network mask of 255.255.255.0, a broadcast address of 192.26.50.255, and it is configured on interface eth1. configuration parameters IP address , shows the Linux FailSafe configuration parameters you specify for these IP addresses. Resource Attribute Resource Name: 192.26.50.1 Resource Name: 192.26.50.2 network mask 255.255.255.0 255.255.255.0 broadcast address 192.26.50.255 192.26.50.255 interface eth0 eth1 You can configure your system so that an IP address will fail over to a second interface within the same host, for example from eth0 to eth1 on a single node. A configuration example that shows the steps you must follow for this configuration is provided in