February 14th, 2023

Machine translation is used partially for this article. See the Japanese version for the original article.

Introduction

This time, we will introduce points to consider and recommended settings when building a mirror disk type cluster.

Mirror disk type cluster can be used in both on-premise and cloud environments, but it is important to configure EXPRESSCLUSTER according to the specifications and characteristics of each environment.

For example, it is the performance of the hardware and network procured for on-premise environments, and the upper limits of performance, such as disks and network throughput and IOPS, etc., set for the resources used by the system for cloud environments.
Servers and EXPRESSCLUSTER must be set according to after considering the characteristics of each environment.

In this article, we introduce points to consider before actually building the cluster and recommended values, so be sure to read this when building a mirror disk type cluster.
In addition, the basic operation of mirror disk type cluster and the features and terms that should be understood are introduced in the following article, so please refer to it as well.

Contents

1. What is a Mirror Disk Type Cluster?

A mirror disk type cluster is a cluster configuration that enables access to the same data even after a failover by mirroring data between the local disks of two servers that build an HA cluster.

A mirror disk type cluster can basically be built as long as the cluster servers can be connected via a network, so it can be built even in an environment where it is difficult to prepare a shared disk. Some clouds provide services and features equivalent to shared disks, but they are basically available only within one data center (Availability Zone, Availability Domain, etc.). A mirror disk type cluster is more suitable for achieving an HA cluster configuration with higher availability in an environment that spans multiple data centers or regions.

2. Points to Consider and Recommended Settings When Building a Mirror Disk Type Cluster

The following are points to consider and recommended settings when building a mirror disk type cluster.
Points to consider include the performance impact of the cluster components, the disk configuration, the file system used, and EXPRESSCLUSTER settings related to mirroring.
Please refer to this article for considering and selecting the appropriate configuration and settings according to the environment in which the HA cluster is to be built.

2.1 Performance Check of the Environment for Building a Mirror Disk Type Cluster

The write performance of data mirrored by EXPRESSCLUSTER is affected not only by disk performance but also by CPU and network performance. Therefore, we recommend to check the performance of the disk and server when considering, and if possible, build an environment that simulates the business and actually verify the performance.
In a mirror disk type cluster, in addition to writing the data to be mirrored, writing the mirroring management information and transferring data over the network occur, which generally results in lower write performance than a single server configuration with the same spec.
Therefore, even if the performance of a single disk or server meets the performance requirements during business operation, it may become poor write performance when you use as a mirror disk type cluster. When selecting disks and servers, it is important to verify the actual performance.

When selecting disks and servers to be used in a cloud environment, be sure to check the information published by the cloud vendor, as the performance of various services may be predetermined. Taking disk services as an example, depending on the type and size of the disk, IOPS, which indicates the number of I/O per second, and the throughput, which indicates the amount of data transferred per second, may have a performance upper limit such as maximum IOPS or maximum throughput. Also, depending on the type of disk, there are types that guarantee a certain performance and types that are inexpensive and have low basic performance but have a burst (temporary performance increase) feature. For example, Amazon Web Services' Elastic Block Store and Microsoft Azure's Managed Disk provide the types shown in the table below. For more information on each disk type, please refer to the information published by the cloud vendor.

Similarly, for virtual machine services, performance upper limits may be set based on instance type and VM size. Some virtual machine types, upper limit values ​​for disk performance and network performance may be set, so performance may be suppressed even if the disk service performance does not reach its upper limit value.

In this way, it is necessary to consider the performance upper limits set for multiple services in a cloud environment. Therefore, in order to obtain consistent performance when running mirror disk type cluster, it is recommended to use either a guaranteed performance type service or a burstable type service that meets business requirements even at minimum performance.

2.2 Configuration of Disk to Create Mirror Partitions

In a mirror disk type cluster, mirror partitions (cluster partition and data partition) must be created.
A mirror partition can be created in the free area of the system disk or on a disk other than the system disk. Considering disk I/O performance and convenience of backup/restore, consider how to create a mirror partition from the following patterns.

For the configuration of the mirror partition, it is recommended to prepare a dedicated disk for the mirror partition and create the mirror partition from the following points of view.

2.3 Selecting the File System to be Used for the Data Partition

The file system for the data partition should be selected according to the system in consideration of business requirements, but if possible, it is recommended to select a file system that is the supported "feature of copying used area".

"Feature of copying used area" is a feature that can shorten the copy time by copying only the area where data exists on the mirror disk when performing a full copy of the mirror disk. In Linux, the supported file systems differ depending on the EXPRESSCLUSTER version. The file systems on which "feature of copying used area" can be used are as follows for Windows and Linux.

2.4 Selecting the Mode of Mirroring

When selecting a mode of mirroring, consider the write performance required by the business application, the disk I/O performance of the HA cluster construction environment, and the network speed, and select either synchronous or asynchronous mode. Please refer to the previous article for an overview of mirroring in synchronous or asynchronous mode.

First, consider whether it is possible to build an HA cluster in synchronous mode. The performance of writing to the mirror disk is affected by multiple factors, such as the disks and servers used to build the HA cluster, and the applications that run on the OS. If possible, measure performance using the trial module of EXPRESSCLUSTER in an evaluation environment that mimics the environment you plan to build to determine the server, disk, and network performance required to run in synchronous mode.
If, as a result of consideration and evaluation, the write performance does not meet the requirements in synchronous mode, we will consider building in asynchronous mode for the first time. Cases for considering asynchronous mode include remote configurations that tend to have poor network performance, and configurations that require very high write performance. However, when considering asynchronous mode, keep the following points in mind:

2.5 Recommended Settings for Each Feature Related to Mirroring in EXPRESSCLUSTER

2.5.1 Setting the Mirror Recovery I/O Size (Linux)

For Linux, set 64KB for the "Mirror Recovery I/O size".
If your EXPRESSCLUSTER version is X 4.2 or X 4.1, change the "Mirror Recovery I/O size" from the default value of 4KB to 64KB. There is no "Mirror Recovery I/O size" setting in X 4.0 or earlier, and the default value is 64KB in X 5.0 and X 4.3.

"Mirror recovery" is an operation/function that restores the normal state when a mirror disk error (data difference) occurs between cluster servers.
It is necessary to perform mirror recovery when the "mirror break status", such as returning the failed server to the cluster or restoring the backup data to the mirror disk. By setting the "Mirror Recovery I/O size" to 64KB, it is possible to reduce disk I/O compared to setting 4KB, which may reduce the time required for mirror recovery.

2.5.2 Setting Related to History File (Asynchronous Mode)

When selecting the asynchronous mode of mirroring, you should consider settings related to history files. See our previous article for an overview of history files . Recommended settings for history files are as follows:

Setting the history file store destination on a different disk from the system disk

It is recommended to set the history file store destination (folder or directory) on a different disk from the system disk.
Setting the history file store destination to a different disk from the system disk facilitates allocating sufficient free space. It is also expected to distribute writes to the history file by data that cannot be accumulated in the queue and writes to the system disk by the OS and business applications. Also, as the I/O to the data partition increases, the I/O to the history file store destination may also increase. In an environment that requires high I/O performance, it is recommended to prepare a dedicated disk for history file store destination. This makes it possible to distribute I/O to data partitions and history files.

Limiting history file size

For both Windows and Linux, it is recommended that the upper limit for the history file size be set at 100 GB.
By limiting the history file size, it is possible to prevent the disk from being overflowed with history files when the amount of data updates is too large to keep up with the data transfer process to the standby server.

Adjusting queue-related settings

Set a large queue size for Windows and a large number of queues for Linux so that mirroring write data is not stored as a history file as much as possible. The queue allocated in the memory can be processed faster than the history file, so adjusting the queue size can reduce writing to the history file. By default, a queue size is set 2048KB for Windows and a number of queues is set 2048 for Linux. Increasing a queue size or the number of queues can be expected to reduce the frequency with which write data is temporarily stored as history files.
However, if you set a large queue size or a large number of queues, the memory size required for cluster operations will also increase, so make sure there is enough free memory in advance. For the required memory size, please check the system requirements of EXPRESSCLUSTER.
Note that there are precautions when setting an unlimited number of queues or a value that is too large for Linux. If a failure such as mirror communication break occurs while a large amount of data is being written, a large amount of queue processing may occur at once, resulting in an extremely high load. Therefore, when setting a large number of queues, it is recommended to evaluate and adjust while applying an I/O load equivalent to the actual business.

Conclusion

This time, we introduced points to consider and recommended settings when building a mirror disk type cluster.
Since the environment in which a mirror disk type cluster is constructed differs depending on the system requirements, the points introduced this time may not necessarily be effective in all environments, but they can be used regardless of whether they are on-premises and cloud environments. Therefore, please refer to it when considering building a mirror disk type cluster.

If you consider introducing the configuration described in this article, you can perform a validation with the trial module of EXPRESSCLUSTER. Please do not hesitate to contact us if you have any questions.