CloudCenter 4.8 has reached End of Life (EOL) as of November 14, 2018. See End of Support Notices for additional context.

Monitor and Recover PostgreSQL

Overview

Effective CloudCenter 4.9.1, you can monitor and recover the PostgreSQL database if running in HA mode. The utility runs on the CCM server and monitors the network and operational health of the PostgreSQL.

Prerequisites

Before configuring the monitor and recover utility in your CloudCenter HA environment, verify the following requirements:

  1. You have exchanged the SSH keys between the CCM and the PostgreSQL components. See Exchange SSH Keys between the CCM and PostgreSQL Components for additional details.

  2. Verify that you have installed the CloudCenter 4.9.1 package as this package includes the required scripts for this procedure. See Download Packages for additional details.

  3. Verify that you have installed the Python module Paramiko, if it is not already installed by the core-installer.

    The PostgreSQL Database VMs should preferably not be moved after installation to ensure a safe recovery (in case of environmental factors).

    Effective CloudCenter 4.9.1, Cisco provides an additional feature to monitor the PostgreSQL VM health and assist with the recovery process.


    To use this feature, follow this process:

    1. Ensure that the PostgreSQL HA setup is in a healthy state – before the feature is enabled.

    2. Restore the cluster to a healthy state (have both DB nodes in working state)

    3. Upgrade to CloudCenter 4.9.1

Exchange SSH Keys between the CCM and PostgreSQL Components

This procedure assumes that you have already performed the following tasks:

Verify mutual SSH access between the CCM_SA_PRIMARY server (and the CCM_SA_SECONDARY server) with the MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE servers by running the following commands.

#From the CCM_SA_PRIMARY:
ssh root@<MGMTPOSTGRES_MASTER/MGMTPOSTGRES_SLAVE>

#From the CCM_SA_SECONDARY:
ssh root@<MGMTPOSTGRES_MASTER/MGMTPOSTGRES_SLAVE>


Monitor PostgreSQL HA

Invoke the CCM wizard in the primary CCM to monitor both CCM servers.

  1. Invoke the CCM wizard as a root user on either the primary or secondary CCM server.

    /usr/local/cliqr/bin/ccm_config_wizard.sh
  2. Select Config_HA_MR to configure CCM DR.

    Wizard MenuFieldDescription

    Config_HA_MR

    Monitor_DB_HAThe monitor utility performs a health check of the DB at every 'n' interval, and records the status under /var/log. In case of any discrepancies, the monitor sends an email to the CCM admin to notify the problem
    Master Node Private IPRequired. The DNS (or IP address) of the master MGMTPOSTGRES server.
    Slave Node Private IPRequired. The DNS (or IP address) of the slave MGMTPOSTGRES server.
    VIP IP/EIP IP/DNS

    Required. The VIP/EIP for the master database or slave database. See Phase 1: Prepare Infrastructure > Cloud Nuances for additional context.

    When you configure the High Availability Properties, you would have configured the VIP/EIP address for the db_config_wizard. Similarly, you must provide the EIP/VIP address for the CCM_SA_PRIMARY and the CCM_SA_SECONDARY servers.

    Interval (mins)Optional. Default = 1 minute.
    The interval in minutes for the monitor utility to probe the database.
    SMTP SERVERRequired. The IP address or DNS for your SMTP mail server.
    SMTP PORTRequired. The open port for your SMTP mail server.
    SMTP SERVER USERRequired. The username for the person administering your SMTP mail server.
    SMTP SERVER PASSWORDRequired. The password for the person administering your SMTP mail server.
    TO EMAIL ADDRRequired. The email address of the CCM admin who should be receiving this email notification. The email notification is only sent if the monitor utility detects a unhealthy database.
  3. Select Yes to save your changes.

  4. Exit the CCM wizard.                                                                                          

Restore PostgreSQL HA

Invoke the CCM wizard in the primary CCM to restore the PostgreSQL database on both CCM servers.

  1. Invoke the CCM wizard as a root user on either the primary or secondary CCM server.

    /usr/local/cliqr/bin/ccm_config_wizard.sh
  2. Select Config_HA_MR to configure CCM DR. The default values are inherited from the monitor utility.

    Wizard MenuFieldDescription

    Config_HA_MR

    Restore_DB_HA

    In case of an issue the CCM admin, can run the second part of the utility to recover the database, resolve the issue, and restore the DB back to an operational state.

    Master Node Private IPRequired. The private IP address of the MGMTPOSTGRES_MASTER server.
    Slave Node Private IPRequired. The private IP address of the MGMTPOSTGRES_SLAVE server.
    VIP IP/EIP IP/DNS

    Required.The VIP/EIP for the master database and slave database. See Phase 1: Prepare Infrastructure > Cloud Nuances for additional context.

    When you configure the High Availability Properties, you would have configured the VIP/EIP address for the db_config_wizard. Similarly, you must provide the EIP/VIP address for the CCM_SA_PRIMARY and the CCM_SA_SECONDARY servers.

  3. Select Yes to save your changes.

  4. Exit the CCM wizard. 


  • No labels
© 2017-2019 Cisco Systems, Inc. All rights reserved