How to Test for Lock Table Consistency When Deploying High Availability for SAP ERP

Implementing high-availability capabilities for SAP ERP systems can be a complex undertaking. But the importance of high availability in keeping companies “open for business” can’t be overstated. Applications and servers will fail. Power outages will occur, and a natural disaster may strike at any time. 

This article provides technical guidance on two key steps in the process to implement high availability for SAP ERP systems: 

• How to test ASCS clusters for Enqueue lock table consistency
• How to create shared file systems for ASCS and Enqueue Replication Server instances 

Before undertaking these two tasks, first determine the version number of Standalone Enqueue Server Framework installed on your SAP clusters. If you are on version 1, it’s best to upgrade to version 2. If you initially installed your SAP system using SAP kernel 7.53 or later, version 2 of the Standalone Enqueue Server Framework is installed by default. 

To confirm the version number, view the  <sapcontrol GetProcessList webmethod> output for each SAP instance and follow these two steps: 

1. Switch to the SAP administrative user <sid>adm, where <sid> is the lower-case SAP System Identifier for the SAP installation:
   su - <sid>adm 
2. Call…  sapcontrol -nr <Instance #> -function GetProcessList 

Here are the process-name outputs you will see depending on which version of the Enqueue Server you are using for each ASCS instance:

• ENSAv1: enserver, EnqueueServer 
• ENSAv2: enq_server, Enqueue Server 2

 And here are the process-name outputs for each version of the Replication Server for each ERS instance: 

• ERSv1: enrepserver, EnqueueReplicator
• ERSv2: enq_replicator, Enqueue Replicator 2

The versions for the Enqueue Server and Replication Server must match. This is critical because processes from different versions are not compatible with each other.

Upgrading to Version 2 of the Standalone Server Framework

If you decide to upgrade to version 2 of the Standalone Enqueue Server Framework, which is the recommended approach, here are the basic high-level steps to follow:

1. Perform a rolling kernel switch to a version of the SAP kernel that supports version 2 of the Standalone Enqueue Server Framework (i.e., SAP kernel 7.51 or later).
2. If the ERS instance was previously installed locally on each cluster node, a new shared ERS file system may need to be created. The details of this step will depend on the file-sharing mechanisms in use.
3. Edit the ASCS and ERS instance profiles as documented by SAP to use the new binaries and instance parameters for Enqueue Server and Replication Server version 2.

You should then make any necessary vendor-specific changes within your high-availability software that are required to support the upgraded ASCS and ERS instances.

When completing the upgrade process, it is also important to follow SAP Note 2711036, which can be accessed at support.sap.com and pertains to using Standalone Enqueue Server 2 in a high-availability environment. The ENSAv2 documentation link within this note provides for a thorough explanation of the upgrade process. 

There may be additional configuration steps that must be performed within the high-availability software itself. So another important step is to follow any vendor-specific SAP Notes provided by your high-availability software provider.

Setting Up Shared File Systems for ASCS and ERS

Once you have completed the process to upgrade your SAP cluster to version 2 of the Enqueue Server Framework, you can then set up shared file systems for ASCS and ERS. A common approach is to use an NFS configuration in which the current host for each resource acts as the NFS server for the corresponding SAP instance file system. This allows the file system for each SAP instance to be accessible via the virtual IP associated with that instance.  

In this configuration, the Central Services (SCS/ASCS) resource host will also typically host the NFS share for the SAP Mount file system. One benefit of this configuration is that it only requires resources within the existing cluster, which can cut down on costs.

While this is the most common configuration, others are possible:

• A dedicated external NFS server or cluster to host all SAP-related shared file systems.
• Cloud-based file sharing solutions such as Amazon Web Services (AWS) Elastic File System (EFS) and Microsoft Azure Files. 

In these cases, the SAP file systems can be mounted on each cluster node using the <autofs> service, which may in turn be protected by your high-availability software if it offers this functionality. Alternatively, the NFS shares may be mounted as Network Attached Storage (NAS), and these mount points may also be protected by your high-availability software.  

The primary advantages of hosting the shared file systems on a server outside the cluster or with a cloud-based file sharing solution are improved failover times and enhanced availability of the file systems when servers within the cluster fail. The primary downside to this approach is the additional cost associated with either the additional servers or cloud service.

Testing ASCS/ERS Clusters for Enqueue Lock Table Consistency

After setting up the shared file systems for ASCS and ERS, you’re now ready to test the ASCS/ERS clusters for Enqueue lock table consistency. This will ensure data integrity and that failovers happen quickly and with minimal disruption.

 Here are the steps to follow:

1. Bring the ASCS resource in-service on the primary cluster node and bring the ERS resource in-service on a backup cluster node
2. Write a collection of dummy locks to the Enqueue Server in the ASCS instance
   • For ENSAv1, this requires the  <enqt>  utility with opcode 11
   • For ENSAv2, use the  <enq_admin>  utility with the  <--set_locks>  argument
     
     Replace <# Locks>  by the number of locks you want to write (e.g., 100) and  <Path to ASCS Profile>  by the full path to the ASCS instance profile. This is typically located at...
     
     /usr/sap/<SID>/SYS/profile/<SID>_ASCS<##>_<ASCS Virtual Host Name>  
     
     But the location can vary depending on how the SAP environment was installed:
     
     ENSAv1
     su - <sid>adm -c “enqt pf=<Path to ASCS Profile> 11 <# Locks>”  
     
     ENSAv2
     su - <sid>adm -c “enq_admin --set_locks=<# Locks>:X:DIAG::TAB:%u 
pf=<Path to ASCS Profile>” 
3. Verify that the correct number of locks was written to the Enqueue Server lock table. This can be done by looking at the  <locks_now>  field in the output of the following sapcontrol command on the cluster node where the ASCS resource is currently in-service:
   
   su - <sid>adm -c “sapcontrol -nr <ASCS Instance #> -function 
EnqGetStatistic” 
   
   
4. To test machine failover, power down the primary node currently hosting the ASCS instance. Alternatively, to test the resource hierarchy switchover, manually bring the ASCS resource in-service on a backup node.
5. Once the ASCS resource has successfully completed failover or switchover, check that the Enqueue lock table was successfully rebuilt by verifying that the value of the  <locks_now>  field is the same as in Step 3 when the following sapcontrol command is run on the cluster node that the ASCS resource is now in-service on:
   
   su - <sid>adm -c “sapcontrol -nr <ASCS Instance #> -function 
EnqGetStatistic” 
   
   
6. After the test and when appropriate, release the dummy locks written to the Enqueue Server lock table by using one of the following commands appropriate for the version of the Enqueue Server installed on the system:
   
   ENSAv1
   su - <sid>adm -c “enqt pf=<Path to ASCS Profile> 12 <# Locks>” 
   
   ENSAv2
   su - <sid>adm -c “enq_admin --release_locks=<# Locks>:X:DIAG::TAB:%u 
pf=<Path to ASCS Profile>” 

Once you verify that the Enqueue Server can successfully rebuild its lock table on switchover and failover, the ASCS/ERS cluster setup has been validated.

Ready for Any Type of Disaster

This technical guidance is meant to give you a sense of what you need to do in order to successfully implement high availability for your SAP ERP platform. Given the complexity, you may need to work with an outside consultant, depending on the internal resources you have on staff.  

If you do work with an outside consultant, the payoff is well worth the investment. By carefully following the upgrade and set-up documentation, and then testing the new configuration extensively, you can ensure your SAP ERP system — the heart of your business operations — continues to run smoothly, no matter what type of disaster may strike.

Further Reading

List of ERP Technologies: What is Right for Your Project?