We Tried Building an HA Cluster Using a Secondary IP on Azure (Linux)

July 29th, 2022

Machine translation is used partially for this article. See the Japanese version for the original article.

Last updated: November 1st, 2022

* The Windows environment of this article has been published here.
And then, in this article, the authentication process of the Azure CLI using the managed identity, which was performed only once in "3.1 Setting Managed identity", has been migrated to the script resource start/stop script so that it is performed every time.

Introduction

We tried to build an HA cluster on Microsoft Azure (hereinafter called “Azure”) using a secondary IP to switch HA cluster connection.

Using a secondary IP to switch HA cluster connection allows switching connection to server applications from within the same HA cluster.
As server applications and client applications can coexist on the Virtual Machines (hereinafter called "VM") that constituting an HA cluster, this makes it possible to build a client-server type system with fewer VMs.

1. What is Secondary IP?

Azure VM allows multiple private IP addresses (a primary IP address and secondary IP addresses) to be assigned to a single network interface (hereinafter called "NIC").

Azure VMs always have one or more NICs. And one or more IP configurations are assigned to the NICs.
There are two types of IP configurations: primary IP configuration and secondary IP configuration.
A NIC is always assigned one primary IP configuration, and by additionally assigning a secondary IP configuration, multiple IP addresses can be assigned to a VM.

This time, we will build an HA cluster using a secondary IP to switch HA cluster connection.

For traditional IP address switching using the Azure probe port resource, VMs constituting an HA cluster cannot switch connections to server applications from within the same HA cluster because they cannot access themselves by virtual IP address.
By assigning a secondary IP using Azure CLI, VMs can access own secondary IP address so it can switch connections to the server application from within the same HA cluster.

Please note that the switching secondary IP will take several minutes.
As the system failover time will be extended, please consider in advance whether the time required is acceptable for your system.
For a validation in advance, you can use the popup trial module of EXPRESSCLUSTER.

2. HA Cluster Configuration

Build an HA cluster using secondary IP.
The configuration is as follows:

Use Azure CLI to add/remove a secondary IP(e.g. 10.5.0.110) to/from the NIC of the VM constituting an HA cluster.
In addition, along with adding/removing a secondary IP to/from the NIC, assign/release a IP address on the OS using floating IP resource.

The procedure for adding/removing a secondary IP was referenced below:

[Reference]
popup

Configure IP addresses for an Azure network interface

3. HA Cluster Building Procedure

Build a "mirror disk type HA cluster" on Azure.
The configuration is as follows:

Settings of Resource group

■Name

TestGroup1

■Region for Resource group

West Central US

Settings of Virtual network

■Name

Vnet1

■Address space

10.5.0.0/24

■Subnet name

Vnet1-1

■Subnet address range

10.5.0.0/24

Settings of VM

■Hostname

Server1 (Active server) : server01
Server2 (Standby server) : server02

Settings of NIC

-Hostname(server01)

■NIC - IP configuration

ipconfig1 (primary) : 10.5.0.101
ipconfig2 (secondary) : 10.5.0.110 (when the following EXEC resource is activated)

-Hostname(server02)

■NIC - IP configuration

ipconfig1 (primary) : 10.5.0.102
ipconfig2 (secondary) : 10.5.0.110 (when the following EXEC resource is activated)

EXPRESSCLUSTER

-Failover group (failover)

■Floating IP resource

IP address : 10.5.0.110

■Mirror disk resource

Cluster partition : /dev/sdc1
Data partition : /dev/sdc2

■EXEC resource

For procedure for building an HA cluster on Azure, refer to the HA Cluster Configuration Guide.
This time, as a secondary IP is used for connection switching, no configuration related to load balancers or Azure DNS is required.

[Reference]
popup

Documentation - Setup Guides

Linux > Cloud > Microsoft Azure > EXPRESSCLUSTER X 5.0 HA Cluster Configuration Guide for Microsoft Azure (Linux)

3.1 Setting Managed identity

This time, we used a system-assigned managed identity for authentication when running the Azure CLI.
The system-assigned managed identity allows Azure resources to authenticate to cloud services without storing credentials in code.

Add right of the owner of the resource group (the resource group in where living the VMs constituting an HA cluster) to the VMs constituting an HA cluster so that they can run the Azure CLI to change the settings of VMs and NICs.

The procedure for setting a managed identity is as follows:

Enable system-assigned managed identities for each VM constituting an HA cluster. For each VM, execute the following operations:

1. By using Microsoft Azure portal (hereinafter called “Azure portal”), select the VM constituting an HA cluster (server01 or server02).
2. Click to [Identity] in the left panel.
3. Set the [Status] of [System assigned] to [On] and click the [Save].

Add the role of owner of the resource group to the VMs.

1. By using Azure portal, select the resource group (TestGroup1) created for the VMs constituting an HA cluster.
2. Click to [Access control (IAM)] in the left panel.
3. Click [Add] and then click [Add role assignment] from the menu that appears.
4. Under [Role], select [Owner] and click [Next].
5. Under [Members], Select [Managed identity] for [Assign access to]. And click [+ Select members] next to [Members].
6. Under [Select managed identities], select the following values and click [Select] at the bottom of the screen.
[Managed identity] : Virtual machine
[Select] : VMs constituting an HA cluster (server01, server02)
7. Click [Review + assign] twice to save the file.

[Reference]
popup

Configure managed identities for Azure resources on a VM using the Azure portal
popup

Assign a managed identity access to a resource by using the Azure portal
popup

Sign in with a managed identity

3.2 Building an HA Cluster

Register EXEC resource to add/remove a secondary IP.
Add the following steps to the steps "5.3 configuring the EXPRESSCLUSTER settings" in the HA Cluster Configuration Guide.

Add the EXEC resource and register scripts to add/remove a secondary IP to/from the NIC of each VM.
Set the dependency of the EXEC resource to no dependent resources (Depth 0).

* The scripts below are samples and are not guaranteed to work.
* Variable values should be set according to the environment.

Start Script (start.sh)

HostName1="server01"        <-- Hostname of server1
HostName2="server02"        <-- Hostname of server2
NicName1="server01535"      <-- NIC name of server1
NicName2="server02800"      <-- NIC name of server2
AZPath="/root/bin/az"       <-- Path of az command
ResourceGroup="TestGroup1"  <-- Resource group name
VnetName="Vnet1"            <-- Virtual network name
SecondaryIP="10.5.0.110"    <-- Secondary IP address
IPConfig="ipconfig2"        <-- IP configuration to assign a secondary IP address

CurrentHostName=`hostname`

if [ "${CurrentHostName}" = "${HostName1}" ]
then
    NicName="${NicName1}"
elif [ "${CurrentHostName}" = "${HostName2}" ]
then
    NicName="${NicName2}"
else
    exit 1
fi

"${AZPath}" login --identity
if [ "$?" != "0" ]
then
    echo "Failed to az login"
    exit 1
fi

#Check the usage status of secondary IP
CheckInUse=`"${AZPath}" network vnet check-ip-address -g "${ResourceGroup}" -n "${VnetName}" --ip-address "${SecondaryIP}" --output yaml | awk -F'[: ]' '$1=="available" {print $3}'`
if [ "${CheckInUse}" = "true" ]
then
    echo "Address not in use"
elif [ "${CheckInUse}" = "false" ]
then
    # Get nic-name
    NicInUse=`"${AZPath}" network vnet check-ip-address -g "${ResourceGroup}" -n "${VnetName}" --ip-address "${SecondaryIP}" --output yaml | awk -F'[:]' '$1=="inUseWithResource" {print $2}' | awk -F'[/]' '{print $NF}'`
    if [ "$?" != "0" ]
    then
        echo "Failed to get nic-name"
        "${AZPath}" logout
        exit 1
    fi
    if [ "${NicInUse}" = "${NicName}" ]
    then
        echo "Address already in use for ${CurrentHostName}"
        "${AZPath}" logout
        exit 0
    else
        # Delete secondary IP
        "${AZPath}" network nic ip-config delete -g "${ResourceGroup}" -n "${IPConfig}" --nic-name "${NicInUse}"
        if [ "$?" != "0" ]
        then
            echo "Failed to delete secondary IP"
            "${AZPath}" logout
            exit 1
        fi
    fi
else
    echo "Failed to check secondary IP"
    "${AZPath}" logout
    exit 1
fi

# Create secondary IP
"${AZPath}" network nic ip-config create -g "${ResourceGroup}" -n "${IPConfig}" --nic-name "${NicName}" --private-ip-address "${SecondaryIP}"

if [ "$?" != "0" ]
then
    echo "Failed to create secondary IP"
    "${AZPath}" logout
    exit 1
else
    echo "Success to create secondary IP"
    "${AZPath}" logout
    exit 0
fi

Stop Script (stop.sh)

HostName1="server01"        <-- Hostname of server1
HostName2="server02"        <-- Hostname of server2
NicName1="server01535"      <-- NIC name of server1
NicName2="server02800"      <-- NIC name of server2
AZPath="/root/bin/az"       <-- Path of az command
ResourceGroup="TestGroup1"  <-- Resource group name
IPConfig="ipconfig2"        <-- IP configuration to assign a secondary IP address

CurrentHostName=`hostname`
if [ "${CurrentHostName}" = "${HostName1}" ]
then
    NicName="${NicName1}"
elif [ "${CurrentHostName}" = "${HostName2}" ]
then
    NicName="${NicName2}"
else
    exit 1
fi

"${AZPath}" login --identity
if [ "$?" != "0" ]
then
    echo "Failed to az login"
    exit 1
fi

"${AZPath}" network nic ip-config delete --name "${IPConfig}" --nic-name "${NicName}" --resource-group "${ResourceGroup}"

if [ "$?" != "0" ]
then
    echo "Failed to delete secondary IP"
    "${AZPath}" logout
    exit 1
else
    echo "Success to delete secondary IP"
    "${AZPath}" logout
    exit 0
fi

Along with adding/removing a secondary IP, register a floating IP resource to assign/release a secondary IP on the OS.

Set the EXEC resource added above as Dependent Resource of the floating IP resource.

4. Checking the Operation

Check that you can connect to the secondary IP you have assigned to the server before and after the failover.

1. Start the failover group on the active server.
2. From the client machine, access the secondary IP (10.5.0.110) and check that you can connect to the active server.
3. From the active server, access the secondary IP (10.5.0.110) and check that you can connect to the active server.
4. By using Cluster WebUI, manually move the failover group from the active server to the standby server.
5. From the client machine, access the secondary IP (10.5.0.110) and check that you can connect to the standby server.
6. From the standby server, access the secondary IP (10.5.0.110) and check that you can connect to the standby server.

We have confirmed that we can connect to the HA cluster using a secondary IP.

Conclusion

This time, we have introduced the procedure for building an HA cluster using a secondary IP in the Linux environment.
If IP address distinction is necessary due to the requirements for applications to be HA clustered, etc., please refer to this procedure to build the HA cluster.

If you consider introducing the configuration described in this article, you can perform a validation with the popup trial module of EXPRESSCLUSTER. Please do not hesitate to contact us if you have any questions.

Contact

Free trial download

Displaying present location in the site.

We Tried Building an HA Cluster Using a Secondary IP on Azure (Linux)

Introduction

Contents

1. What is Secondary IP?

2. HA Cluster Configuration

3. HA Cluster Building Procedure

3.1 Setting Managed identity

3.2 Building an HA Cluster

4. Checking the Operation

Conclusion