Global Site
Displaying present location in the site.
March 11th, 2026
Machine translation is used partially for this article. See the Japanese version for the original article.
Introduction
EXPRESSCLUSTER X 5.3 features numerous enhancements, including easier setup and operation for cloud environments, improved usability, enhanced security and support for IoT/embedded OS, as well as compatibility with new OSes and platforms.
One of the enhancements is the improved backup functionality for mirror disks, which now allows securing a quiescent point to obtain backups without service (failover group) interruption. This article introduces these backup function enhancements.
Contents
1. About Backup with the Business Going on
In business services, ensuring reliable data backups is critically important. If backups are not properly created, restoring data may fail to maintain file system integrity, causing application failures and preventing services from starting. To prevent such issues, EXPRESSCLUSTER provides commands that support backup and restore operations in mirror disk and hybrid disk environments.
However, with conventional commands, it was necessary to stop the service (failover group) to ensure a quiescent point. Since some services require minimizing downtime as much as possible, there have been requests for backups that do not require stopping the failover group.
In EXPRESSCLUSTER X 5.3 for Linux, backup and restore support commands for mirror disk environments have been enhanced. The enhanced commands temporarily suspend write operations from applications to the mirror disk and then write data to the disk. This enables safe backup creation without stopping services. As a result, temporary response delays may occur from the service side, but backups can be created while the service itself continues to run.
This feature is designed to secure quiescent points for the mirror disk, so data on the system drive and similar components are not covered. Therefore, in actual operations, it is expected that this feature will be used to regularly back up the mirror disk with the business going on, while conventional backup and restore support commands will be used to perform full system backups that require business suspension. Both approaches should be utilized together.
For detailed information and important notes on backup operation with the business going on and restore operation, please refer to the following maintenance guide.
[Reference]
Documentation - Manuals
-
EXPRESSCLUSTER X 5.3 > EXPRESSCLUSTER X 5.3 for Linux > Maintenance Guide
→ 2. The system maintenance information
→ 2.22 How to back up a mirror disk with the business going on
→ 2.23 How to restore a mirror disk from its backup created with the business going on
2. Assumed HA Cluster Configuration
This time, we will perform backup and restore of the mirror disk in a 2-node mirror disk type HA cluster configuration during business operation. While executing backup and restore, we will use the backup and restore support commands enhanced in EXPRESSCLUSTER X 5.3.
The HA cluster configuration diagram is as follows.
The configuration of EXPRESSCLUSTER is as follows.
Please adapt the information as needed for your environment.
-
Servers
– Active Server : server1
– Standby Server : server2 -
Failover Group : failover1
– Floating IP Resource : fip1
– Mirror Disk Resource : md1
• Execute the initial mirror construction : disabled -
Monitor Resources
– Floating IP Monitor Resource : fipw1
– Mirror Connect Monitor Resource : mdnw1
– Mirror Disk Monitor Resource : mdw1
- *If [Execute the initial mirror construction] is enabled for the mirror disk resource, configuration changes will be required during the setup process. Therefore, in this HA cluster configuration, it is disabled in advance.
3. Backup with the Business Going on
This section explains the procedure for taking a backup of one server with the business going on.
The following diagram illustrates the backup procedure.
The procedure is the same regardless of whether the mirror disk to be backed up is on the active or standby server. If you need to back up the mirror disks for both the active and standby servers, first perform the backup for the active server according to this procedure, and then perform the same procedure for the standby server.
In this section, the mirror disk of the active server is the backup target.
3.1 Running the Pre-Backup Processing Command
On the active server (the server where the backup target mirror disk is running), run the following command to change the mirror disk from normal mode to backup mode.
# clpbackup.sh --pre --online
clpbackup.sh : Changing the setting of cluster services to Manual Startup.
clpbackup.sh : Suspending Cluster Service.
Suspend server1 : Command succeeded.
Suspend server2 : Command succeeded.
clpbackup.sh : Beginning backup-mode.
md1 has been frozen.
All backup flags have set to <ON>.
clpbackup.sh : Command succeeded.
While the server is set to backup mode by following this procedure, the following conditions will apply. This prevents recovery processes or failovers triggered by monitoring anomalies during backup, and ensures that the file system information remains consistent.
-
The cluster service enters a suspended state, temporarily halting all monitoring activities.
-
Write data cached in memory for the mirror disk is flushed to the mirror disk.
-
Write requests to the mirror disk are temporarily paused.
-
Automatic startup of the cluster service and failover groups is disabled.
3.2 Creating a Backup
You can create a backup of the mirror disk using features such as snapshots provided by physical storage, virtualization infrastructure, or cloud platforms.
3.3 Running the Post-Backup Processing Commands
After completing backup creation, such as by taking a snapshot, return the mirror disk from backup mode to normal mode.
# clpbackup.sh --post --online
clpbackup.sh : Changing the setting of cluster services to Auto Startup.
clpbackup.sh : Ending backup-mode.
All backup flags have set to <OFF>.
md1 has been unfrozen.
clpbackup.sh : Resuming Cluster Service.
Resume server1 : Command succeeded.
Resume server2 : Command succeeded.
clpbackup.sh : Command succeeded.
By returning the mirror disk to normal mode, the following state will be restored.
-
The cluster service and monitoring are resumed.
-
Mirror disks are writable.
-
Automatic startup of the cluster service and failover groups is enabled.
4. Restore
This section describes the procedure for restoring the mirror disks of both the active and standby servers using the same backup image. This procedure assumes that the backup image was created from the active server's mirror disk in "3. Backup with the Business Going on."
The following diagram illustrates the restore procedure.
4.1 Stopping the Failover Group
If a failover group including mirror disks is running, stop the failover group.
Run the following command on the active server.
# clpgrp -t failover1
Command succeeded.
4.2 Running the Pre-Restore Processing Command
Run the following command on both the active and standby servers.
# clprestore.sh --pre
clprestore.sh : Changing the setting of cluster services to Manual Startup.
clprestore.sh : Shutting down...
Command succeeded.
clprestore.sh : Command succeeded.
After running the command, the cluster service will be set to not start automatically, and the server will be shut down.
- *If the operating system cannot be started and the server needs to be rebuilt, rebuild the server first and then run the above command.
4.3 Mirror Disk Replacement
We will replace the mirror disk on both the active and standby servers.
Here, our approach is to simultaneously restore the same mirror disk image to both the active and standby servers. Therefore, for each server, we will create a replacement disk using the backup taken from the active server.
When restoring from a snapshot, create the disk volume—including both the cluster partition and the data partition—from the snapshot, and then replace the original disk volume with the new one.
4.4 Checking the Mirror Disk Settings
After completing the restoration process for both the active and standby servers, start all servers. Note that, even after server startup, the cluster service remains stopped because automatic startup is disabled.
After all servers have started, check the device names (such as /dev/sda) of the restored cluster partition and data partition. If the device names have changed, update the mirror disk resource settings accordingly. Connect to the Cluster WebUI, switch to [Configuration Mode], open the [Details] tab in the mirror disk resource properties, modify the device names as needed, and apply the changes.
If the device name setting is incorrect, there is a risk of failing to start the mirror disk resource or even corrupting the partition. Please pay close attention when configuring these settings.
When using LVM for mirror disk partitions on Red Hat Enterprise Linux 9, there may be cases where, after replacing a mirror disk, the device name of the mirror disk is displayed in the output of the lsblk command, but physical volumes and logical volumes are not displayed with the pvdisplay and lvdisplay commands. In such cases, running the following command will allow the system to recognize the new volume.
# lvmdevices --update --refresh
4.5 Running the Post-Restore Processing Command (Part 1)
Run the following command on both the active and standby servers. After running the command, all cluster partitions will be initialized, the automatic startup of the cluster service will be enabled, and the servers will reboot.
# clprestore.sh --post --skip-copy
When run, the following message will be displayed, and the server will reboot.
Mirror info will be set as default.
Initializing mirror disk of md1 failed.
Check if the Cluster Partition or Data Partition is OK.
Initializing mirror disk complete.
clprestore.sh : Changing the setting of cluster services to Auto Startup.
clprestore.sh : Rebooting...
Broadcast message from root@server1 (Thu 2025-06-11 13:58:34 JST):
The server will reboot now!
Reboot server1 : Command succeeded.
- *If [Execute the initial mirror construction] is enabled in the mirror disk resource settings, the command will result in an error.
In this case, please use the Cluster WebUI to disable [Execute the initial mirror construction], apply the settings, and then rerun the command.
If there are any stopped servers and a suspension message appears when applying the settings via Cluster WebUI due to this, enable the [Force apply settings] checkbox to forcibly proceed with applying the settings.
Additionally, if there are servers that are stopped and could not apply settings, be sure to apply the settings to those servers later to prevent inconsistencies. - *If the mirror agent is running, the command will result in an error during the cluster partition initialization process. In that case, run clprestore.sh --pre , then start the server, and run the clprestore.sh --post --skip-copy command again.
4.6 Checking the Status of the Mirror Disk
Once both the active and standby servers have started, check the status of each mirror disk using the Cluster WebUI or the clpmdstat command in EXPRESSCLUSTER. The status of the mirror disks on both the active and standby servers should be "Normal" (GREEN).
# clpmdstat --mirror md1
Mirror Status: Normal
md1 server1 server2
------------------------------------------------------------
Mirror Color GREEN GREEN
4.7 Running the Post-Restore Processing Command (Part 2)
Run the following command on both the active and standby servers. After running the command, the status of the mirror disk will change to normal mode, and automatic startup of cluster services and failover groups will be enabled.
# clprestore.sh --post --online --skip-copy
clprestore.sh : Changing the setting of cluster services to Auto Startup.
clprestore.sh : Ending backup-mode.
All backup flags have set to <OFF>.
clprestore.sh : Command succeeded.
If the failover group does not start automatically, start it manually.
Conclusion
This time, as one of the enhanced features of EXPRESSCLUSTER X 5.3, we introduced the commands and usage methods that support backup with the business going on, as well as restore. The ability to back up important data without stopping the production environment, as well as the streamlined backup procedures compared to previous versions, provide significant benefits. Knowing about this feature will be extremely useful when considering operational and maintenance procedures.
If you consider introducing the configuration described in this article, you can perform a validation with the trial module of EXPRESSCLUSTER. Please do not hesitate to contact us if you have any questions.