DataKeeper UI vs. Car Dashboards: A Guide to High Availability Monitoring

Besides sharing the enjoyment of using DataKeeper Cluster Edition with its high availability and disaster recovery capabilities, most of us have something else in common . . . we drive a car … electric, gasoline or a hybrid.  As of 2022, Forbes states that 92% of households owned at least one car.

And those millions of cars have a few things in common with DataKeeper . . .

A dashboard with status indicators.

In DataKeeper’s case, the user interface (DataKeeper.msc) has what are known as Mirror Definitions or Mirror Status.

In cars, indicators are  affectionately known as “dashboard lights” or for some of you Gen(eration) Xers may call them “idiot lights”

Let’s jump right in and talk about the similarities.

All cars have some level of combustion (fuel), electric (or not) and cooling abilities; thus the associated lights on the dashboard are usually in the colors of Red, Yellow and Green.  Like a car dashboard, the DataKeeper UI has the “traffic light” schema in the Console Tree.

As we mentioned, the possible scenarios in troubleshooting a car, e.g. battery, fuel problems, engine overheating, like that of your engine, DataKeeper troubleshooting too can be consolidated to the following areas of:

• Storage
• Network
• Other (Security, User Administration,etc.)

Referencing back to the “traffic light” identifiers in the Console Tree on the DataKeeper UI, let’s take a look “under the hood” to identify the state of the Mirror(s).

As a driver would take their car in for either to fix a problem or perform regular service, the dealership’s Technical Advisor or Service Technician, will plug in a OBD connector (On-board Diagnostic) to get a general idea of where the problem may be occurring (combustion, electrical or other)

Identify Mirror Status Colors to Diagnose DataKeeper Issues

As a user that’s supporting/driving DataKeeper, your first level of triage should be to identify those “color” changes of the mirror to confirm if Storage, Network or Other have been impacting users, performance, etc.

Those identifiers are affectionately called Mirror definitions. The mirror status, similar to the OBD, can be identified by launching a command called “emcmd . getmirrorvolinfo <drive letter>“.

Note:

To get to the “emcmd” commands, (which, by the way, stands for Extended Mirroring Command), we can launch an elevated (Administrator) command prompt as follows:

• “cd %extmirrbase%”
  • This is a shortcut to the installation path for where the utilities are located e.g. <root>\Program Files (x86)\SIOS\DataKeeper

The output will be displayed as “# hostname #”

The 1st integer represents the role of the Mirror, (1 = Source, 2 = Target)

As for the Last integer, there are 6 states a Mirror can obtain:

1 – Mirroring (Green)

2 – Mirror is Resyncing (Yellow)

3 – Mirror is Broken (Red)

4 – Mirror is Pause (Yellow)

5 – Resyc is Pending (Yellow)

With several variations – just as it pertains to your car or DataKeeper – one must obtain a starting point to identify the “pain points”

• Driver: Do you share the car with your family? Did someone leave a dome light on? Did the fuel type change from 94 octane to 87?
• DataKeeper Administrator:  Are you the sole Support person?  Do other departments have access to your cluster, e.g. Database Administrators, Infrastructure Personnel responsible for Storage, Network and other?

Looking under the Hood: Preventive Steps to Take Before You Scale your Clustered Resources

Storage

Do your homework. Infrastructure and Database Administrators want to scale their storage to meet growing demands. They are very knowledgeable about the tasks at hand but if performed incorrectly or in the wrong order, the Mirror colors in DataKeeper could be Red, Yellow or none. See our documentation and supporting video on How to Properly Resize Your Storage.

Network

Easily avoid full resynchs. Infrastructure and database admins may want to segment existing mirror(s) traffic to a different network without any downtime or loss of High Availability. Can you imagine not having HA for re-creating a new mirror and having to perform a FULL Resync for a couple of Terabytes or even a Petabyte of data?  See our documentation and supporting video on how this can be achieved with ease

Other (Security)

Understand password management. Password policies may change via prescribed timeframes in Active Directory. If this is unbeknownst to the Administrators, and if the SIOS DataKeeper Service is restarted, the password that has been changed for the SIOS DataKeeper Service account (Active Directory) is NOT propagated automatically. Therefore a manual update of the Service Account Password is required within the Service applet. (services.msc)  For proper usage of SIOS DataKeeper Service Accounts view our supporting documentation 

Managing DataKeeper Across Multiple Departments to Avoid Downtime

Your car can have multiple drivers, different Service Technicians, different road conditions and the like. DataKeeper is no different as there may be cross-functional departments that are responsible for Storage, Network, Security and other, that could adversely impact DataKeeper. There may be related clustered resources that have relationships/dependencies with DataKeeper.These departments perform tasks without the DataKeeper Administrator’s knowledge and just like the dashboard lights in your cars, your mirrors will display those “traffic light” colors; noticeably Yellow and Red.

Checklist for Monitoring DataKeeper Mirror Status and Infrastructure Changes

• Review “Traffic Light” colors
• Identify status of the Mirrors via emcmd . getmirrorvolinfo command
• If multiple departments are involved, ask about changes or scheduled events throughout the infrastructure albeit Storage, Network, Security or other
• Augment your initial triage via Event Viewer logs, ipconfig /all output, dkhealthcheck, just to name a few

Contact SIOS Support for DataKeeper Issues

If your car has gone “kaput” and left you on the side of the road, you’ll likely reach out to AAA and they’ll provide a tow truck.
If you have concerns about your clustered DataKeeper Volume Resource, contact SIOS Support. Got a question or a non-priority issue, email us at support@us.sios.com. If your issue is urgent, call us at  877-457-5113.

Reproduced with permission from SIOS