Service Level Agreements and the Four Nines are Not Enough for High Availability in the Cloud

Service Level Agreements and the Four Nines are Not Enough for High Availability in the Cloud

When most people think of high availability, they set four nines (99.99%) or less than five minutes of downtime every month as the baseline. But according to Dave Bermingham, Senior Technical Evangelist at SIOS Technology, in this TFiR video interview, high availability is more than that.

Dave argues that counting on nines is really a measurement that you might be judged against, but really trying to guarantee a level of nines is almost impossible. Because there’s so many points in that availability chain that can be a single point of failure. Four nines is certainly a great number to be judged against and to strive for, but overall it doesn’t mean a lot to have just four nines for my database server.

Effective High Availability Covers a Complex Availability Chain

Even with Cloud SLAs (Service Level Agreements), one can’t be fully rest assured as most cloud providers offer four nines on compute, which is only one part of the availability chain (along with network, storage, and the hops between). Bermingham warns, “There’s a million points of failure. So, trying to think that my cloud provider offers four nines so I’m covered, you’re kind of fooling yourself there. You have to look at the big picture and do what you can to identify those points of failures, to minimize the potential points of failure and to have a recovery plan, should something happen.”

When considering High Availability/Disaster Recovery (HA/DR), Bermingham believes the thing that causes the most visible downtime is human error. Bermingham also suggests that authorization and access to the system should also be restricted to reduce the point of failure. “You should only give access to those who absolutely need access to it and you should also ensure that they are highly trained and that you have all the things in place to help minimize potential oops.”

SIOS offers a single solution to meet both high availability and disaster recovery needs across a wide variety of operating systems (Windows, Linux), platforms, and applications, including SAP, SAP HANA, MaxDB, SQL Server, Oracle, and other environments running in SAN-based, shared storage configurations or SANless, local data storage configurations.

Contact us for more information.

Reproduced with permission from SIOS

Retail

High Availability for Manufacturing

High Availability for Manufacturing

SIOS understand how deeply manufacturers rely on sophisticated applications, ERPs and databases to maintain production schedules, manage supply chains, control warehouse inventory, coordinate shipping, and perform a variety of other essential business operations. Downtime for these systems can cost thousands of dollars per hour and result in shipping delays, customer service issues, safety concerns, product loss, and productivity loss and supply chain disruption. SIOS clustering software provides reliable, cost-efficient high availability and disaster protection to deliver 99.99% uptime for critical applications, ERPs and databases running in Windows and Linux environments in on premises, cloud, hybrid cloud or multicloud environments.

20 Years of Experience in HA for Industry Manufacturing

For more than 20 years, SIOS has helped manufacturers deploy high availability for business critical applications. Our award-winning customer service team has in-depth experience with the application environments and requirements specific to the manufacturing industry.

Eliminate Service Disruptions

Unlike other solutions that only monitor VM or server operation, SIOS clustering software monitors the entire application environment – network, storage, operating system, server, and application to ensure business critical systems are operational and available. If an issue is detected, SIOS Protection Suite software automatically restarts the application, speeding recovery time and eliminating the need for more than 70% of failovers. If necessary, SIOS software automatically orchestrates failover in full compliance with application-specific best practices.

Cloud, Hybrid Cloud and Multicloud Environments

Migrate critical systems to Microsoft Azure, Amazon EC2 or Google Cloud Platform without sacrificing high availability or disaster recovery protection. Add SIOS DataKeeper software to your Windows Server Failover Clustering (WSFC) to create a SANless clustering environment without the cost and complexity of shared storage. Use SIOS Protection Suite for Linux for complete SAN or SANLess clustering protection in a Linux environment.

Application Awareness for Reliable, Error-Free Clustering

SIOS application recovery kits (ARKs) enable LifeKeeper to monitor the entire application stack – network, storage, OS, application, and server. They also orchestrate the failover in accordance with application-specific best practices for fast, reliable operation on the secondary node(s).

SAP S/4HANA Certified

Our products are SAP certified and feature our unique application recovery kits (ARK) for both SAP and HANA to ensure reliable failover in full compliance with SAP best practices. Powerful automation and entry validation speed enables error-free cluster configuration five times faster than Pacemaker-based solutions.

SIOS has clustering solutions for both Windows and Linux environments – enabling industry-leading HA configuration flexibility. Use SIOS products to protect SAP ERS, ASCS and other services on Windows and SAP and HANA on Linux in the same configuration.

Protect Critical Databases – SQL Server, Oracle, MaxDB

SIOS clustering software provides the HA protection you need to run business-critical databases wherever and however you run them – SQL Server, Oracle, MaxDB, MySQL in the cloud or hybrid cloud environments.

Reproduced with permission from SIOS

High Availability for Healthcare

Why SIOS HANA Multitarget Automation is a Bigger Deal Than you Think

Why SIOS HANA Multitarget Automation is a Bigger Deal Than you Think

Larry (not his real name) was a SIOS customer who had deployed a replication solution for high availability and disaster recovery (HA/DR) in the past. When he launched the PoC to test a two-node replication solution for Linux, using SIOS LifeKeeper and the DataKeeper replication, his top priority was protection of data integrity. Larry’s PoC test list included the standard items including: database start / stop, migrating the database to the backup node, maintenance activities, and server failover just to name a few.  Larry was adamant that the solution be capable of both fast server switchover,(i.e., the graceful migration), and fast failover (i.e., the sudden and forced migration), of applications, databases, storage and services from one server to another.  But, he was even more forceful and passionate that such activities should not cause data loss.

Protect Data Integrity By Avoiding Split Brain

In addition to these standard tests, Larry added specific tests to try to force a “split brain” scenario. Split brain is a condition that occurs when members of a cluster are unable to communicate with each other, but are in a running and operable state, and subsequently take ownership of common resources simultaneously. In effect, you have two bus drivers fighting for the steering wheel.  Due to its destructive nature, split brain, can cause data loss or data corruption and is best avoided through use of a mechanism to determine which node should remain active (driving the bus) and which node(s) should stop writing to disk.

While split brain scenarios are relatively uncommon in clusters that deploy the use of quorum and quorum plus witness capabilities, the difficulty of split brain resolution increases exponentially with every node added to the cluster configuration. In a multitarget configuration with three or more nodes, clustering software not only has to orchestrate a failover to the correct node, it has to automatically switch replication from the new primary node to the tertiary node to maintain DR protection while making sure to arbitrate properly between nodes. In other clustering solutions those complex actions have to be manually scripted and manually updated in the event of a failover and again to restore normal operation, and it only gets harder when a split brain occurs.

Due to the features and improvements in the SIOS LifeKeeper and SAP HANA Application Recovery Kit (ARK), Larry had difficulty introducing a split brain scenario. However, when he was able to finally contrive one, he benefited greatly from understanding the logic that the SIOS products used to protect his data. Larry realized the high level of sophistication designed into the data protection provided by SIOS clustering software. He selected SIOS LifeKeeper.

The SIOS HANA Multitarget Automation Difference

Scenarios like Larry’s are just one of nine reasons SIOS’ HANA multitarget automation is a bigger deal than you think. Here are all nine:

1. Enhanced Protection
   SIOS’ solution simplifies the protection of a HANA database resource in a multitarget scenario. Wizard-based options quickly detect the current configuration and precisely add the information to the LifeKeeper configuration. Error detection is both concise and informative to help users resolve any issues and subsequently save time.
2. Streamlined Administration
   Natalie(not her real name) was responsible for an HANA multinode configuration.  When servers failed or required maintenance, Natalie leveraged different scripts and tools to perform the required actions. This, however, was not scalable. After moving to SIOS LifeKeeper, Natalie and team had a simple UI to perform all core tasks such as stopping and restarting HANA and HANA system replication. Additionally, if a disaster strikes, the team can use the single, simplified SIOS UI instead of searching for the latest runbook, finding a copy of the right scripts, or calling Natalie at 2AM. .
3. Simplified Monitoring
   SIOS’ intuitive status reports in the UI provided the team with a quick way to determine the replication status.  Using a single tool, versus a collection of monitoring boards and homemade scripts, simplifies administration and saves time.
4. Automated Recovery
   Some HANA HSR solutions are capable of performing a failover of the HANA replication between those two nodes.  However, an administrator often has to re-register the replication after a system failover.  In the case of three or more nodes, will the administrator understand how to update the registration on the third or forth nodes?  Will they remember to use sync and async appropriately?  The SIOS solution, capable of handling three or even four nodes for multitarget replication, will seamlessly automate the registration of target nodes after a failure.
5. Flexibility and Scalability
   The ability to protect a HANA cluster in two, three, or four node combinations means that customers have the flexibility to dial up their level of both availability and disaster recovery. Two node customers, with quorum, are able to provide availability protection against a disaster and handle maintenance activities with near zero downtime leveraging HANA takeover with handshake feature.  Customers deploying three nodes can dial up additional disaster recovery functionality by deploying the third node with async replication in a different data center or region.  For added benefit, three node customers can deploy a fourth node, with storage quorum, to enable high availability and disaster recovery in the event of an entire data center loss.
6. Data Protection
   Let’s go back to Larry’s issue. He was running HANA on primary node A with multitarget replication to Nodes B and C. What happens when your manual efforts end in disaster?  Which node was the primary?  Were things in sync when node A crashed?  How do I avoid bringing up the wrong node?  In addition to adding support for three or more nodes in a multi-target HSR configuration, the new HANA ARK includes additional admin tools to help in the event of a disaster or unfortunate split brain event.

   The HANA_DATA_OUT_OF_SYNC_<tag> flag prevents users from accidentally restoring the database on the wrong system.  The HANA_LAST_OWNER_<tag> flag helps administrators know when an action was taken on the primary system while standby nodes were not in sync.  This flag tells the administrator that this node was the last owner and should be where replication is resumed.  HANA_DATA_CONSISTENCY_UNKNOWN_<tag> helps SIOS to automatically resolve and restore replication when all communications between standbys were temporarily lost and then restored.  When used with best practices, quorum deployment, and proper tuning, these tools allow administrators like Larry to avoid split brains and recover safely if and when they occur.

7. Reporting, Performance and Disaster Recovery
   Of course the true benefit for multi-target is in the extra nodes and the functionality that these nodes unlock. Using three nodes in the same data center can unlock the potential for more reporting via the logreplay_readaccess parameter, while still maintaining a node at a DR site.  In addition, SIOS’ support for different replication modes gives users the option to have sync nodes and async nodes for better performance across data centers (or regions).
8. Continuous Testing
   How often does your team test homemade scripts?  How often is your runbook reviewed with respect to configuration, administration, and 2 AM scenarios.  The HANA multi-target solution was not only continuously tested by SIOS engineers, QA, and Customer Experience experts, but the solution also continues to be tested and validated for HANA failover and recovery processes with each release and update.
9. Extensive Documentation
   Some time ago our team worked with a customer for cluster administration.  While his predecessor was very knowledgeable about their environment, staff promotions and reorganization had left many IT folks responsible for systems they knew little about. When asked about runbooks and documentation of their configuration, the customer was unable to find details from the previous team or previous administrators. In addition to rock solid automation, administration, monitoring, recovery, and data protection, the SIOS multi-target solution includes detailed, easy-to-use documentation about the implementation, operation, and management of a HANA multitarget system controlled by LifeKeeper.

Leveraging SIOS’ total solution means that customers can benefit from consistent, timely monitoring and detection, fast, reliable and efficient recovery, and a fully automated solution that guarantees high availability and disaster recovery protection. Contact us for more information on SAP HANA multitarget automation.

-By Cassius Rhue, VP Customer Experience

Reproduced with permission from SIOS