Install CCM HA Using Appliance

 

CCM HA installation is tested and verified for AWS, OpenStack, and VMware clouds.


To configure CCM in HA mode, you must use the following roles:

  • Database: MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE (and if required, MGMTPOSTGRES_VIP)

  • CCM: CCM_SA_PRIMARY and CCM_SA_SECONDARY

    Do not use the CCM or CCM_SA roles as those roles DO NOT allow you to configure high availability. See Virtual Appliance Overview and High Availability Best Practices for additional context.

  • Loadbalancer: CCM_LB

Unable to render {include} The included page could not be found.

Unable to render {include} The included page could not be found.

CCM_SA_PRIMARY/SECONDARY – Exchange CCM SSH Keys

To exchange the SSH keys between the CCM_SA_PRIMARY and CCM_SA_SECONDARY servers, follow this procedure using root permissions.

  1. On the CCM_SA_PRIMARY and the CCM_SA_SECONDARY instances, execute the following commands to generate a new SSH key on each instance. 

    ssh-keygen -t rsa
    cd ~/.ssh
    cat id_rsa.pub >> authorized_keys
    chmod 600 authorized_keys
  2. Copy the id_rsa.pub content from both the CCM instances and paste the content into the authorized_keys file.

  3. Verify mutual SSH access between the CCM_SA_PRIMARY and CCM_SA_SECONDARY by running the following command on each VM.

    ssh root@<CCM_SA_PRIMARY/CCM_SA_SECONDARY>

CCM_PRIMARY – Configure HA Wizard Properties

To configure high availability for CCM_SA_PRIMARY, follow this procedure.

  1. Invoke the CCM wizard as a root user (see Virtual Appliance Process > Cloud-Specific Setup Details for a sample setup).

    Wizard Path
    /usr/local/cliqr/bin/ccm_config_wizard.sh
  2. Configure the HA properties.

    Write this down for future reference!

    Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

    This section ONLY provides the HA details for EMPHASIS – In addition to this HA information, you must also configure the generic information in the remaining CCM details provided in the table identified by the Configure CCM Wizard Properties title.

    Wizard Menu

    Field

    Description

    DB – Configure Database



    DB IP or Hostname

    The VIP/EIP for the master database and slave database. See Phase 1: Prepare Infrastructure > Cloud Nuances for additional context.

    When you configure the MGMTPOSTGRES_MASTER – Configure High Availability Properties, you would have configured the VIP/EIP address for the db_config_wizard already. Similarly, you must provide the EIP/VIP address for the CCM_SA_PRIMARY and the CCM_SA_SECONDARY servers.

    DB Username
    and
    DB Password

    The following credentials are pre-populated:

    • Default username = cliqr (can be changed – manually change the password on MGMTPOSTGRES VMs or RDS and then update the username in the CCM through the database config wizard.

      Be sure to change the PostgresDB password and update the db.properties file to reflect the correct password.

    • Default password = cliqr (can be changed)

      Be sure to change the default password immediately after your first login. See PostgreSQL Password for additional context.

    Configure_HA


    Primary Node Private IPThe IP address of the primary CCM VM
    Secondary Node Private IP The IP address of the secondary CCM VM

    Mgmtserver DNS Name

    Use the DNS or IP of the CCM_LB – Used by the CCO VM to communicate with the CCM VM.

    Configure CCM Wizard Properties

    To configure the CCM wizard properties, follow this procedure.

    1. Invoke the CCM wizard as a root user (see Virtual Appliance Process > Cloud-Specific Setup Details for a sample setup).

      CCM Wizard Path
      /usr/local/cliqr/bin/ccm_config_wizard.sh
    2. Configure the server properties.

      Write this down for future reference!

      Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

      Wizard Menu

      Field

      Description

      Server_Info – Configure Server Info (Required)

      Mgmtserver DNS Name

      DNS (or IP address) of the CCM management server.

      If you are configuring the HA environment, enter the CCM_LB DNS (or IP) in this field.

      DB – Configure DatabaseIP or Hostname

      DNS or IP of the database.

      DB Username and Password

      The following credentials are pre-populated:

      Default username = cliqr (cannot be changed)

      Default password = cliqr (must be changed)

      Be sure to change the default password immediately after your first login. See PostgreSQL Password for additional context.

      ELK_Info –  Configure Log Collector InfoELK Host

      Specify the IP address for the Log Collector host.

      Elasticsearch PortDisplays 8881 by default.
      Kibana PortDisplays 8882 by default.
      ELK UserThe default ELK Username = logreader.
      ELK PasswordThe default ELK Password is re@d0nly (zero between d and n) (change this password after the initial login – see Download Log File for additional context).
      Host Identifier

      A Unique ID for the server – be sure to prefix the unique identifier with CCM_ for example, CCM_1
      If not set, the CloudCenter platform uses the CCM server date.

      The Host Identifier cannot contain capital letters for both CCM and CCO configurations.

      Host Identifier List

      Only applies to environments using the HA mode – provide a list of comma separated unique host identifiers for all Log Collector hosts in a HA setup = for example, CCM_1,CCM_2,myCCM.

      The Host Identifier List cannot contain capital letters for both CCM and CCO configurations.

      In an environment operating in HA mode, if you have two CCM instances with unique IDs configured as CCM_1,CCM_2 in their respective server.properties file, then this property should state CCM_1,CCM_2 in both CCM instances. Each CCM must be aware of the unique ID of the other CCM(s) when in HA mode.

      Custom Certs Menu
      Generate_CertsTo generate new certificates for CloudCenter components. See Certificate Authentication > Generate and Update the certs.zip File on the CCM for additional context.
      Update_CertsTo update certificates for CloudCenter components. See Certificate Authentication > Generate and Update the certs.zip File on the CCM for additional context.
    3. Exit the CCM configuration wizard.

    4. Select Yes, to restart the CCM server and corresponding CloudCenter services.

    You have successfully installed the CCM instance! You can now proceed to the next step:

    • Configure the Log Collector details in the CCM wizard's Configure Log Collector Info menu.

    • If you are installing the Enterprise Service Bus (ESB), do so at this point.

    Back to: CCM (Required)

  3. Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

    • Configuring CCM HA ...
    • Restart server (with the progress bar)
    • Configured CCM HA successfully
  4. Restart the secondary CCM server and corresponding CloudCenter services.
  5. Exit the CCM configuration wizard.

Back to: CCM HA

CCM_LB – HAProxy Installers

Use a plain clean OS image (such as CentOS7) to install a load balancer.

See CCM and Database Firewall Rules > CCM_LB Ports for the complete list of ports that need to be open for your deployment.

If you configure a load balancer for any CloudCenter component, be aware that the firewalId is enabled by default and you must explicitly disable it to ensure that the CloudCenter component(s) can communicate with the load balancer. See Firewall Rules Overview for additional context.

 Here is a sample configuration to load balance a CentOS7.x VM with HAProxy for the CCM.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Install HAProxy as the root user. 

    yum install -y haproxy
    
  3. Create .pem files for haproxy configuration for CCM_LB in the CCM Primary server.
    1. Run the following commands.

      sudo -i 
      cd /usr/local/cliqr/ssl/ccm
      cat ccm.crt ccm.key >> mgmtserver.pem 
      cat ca_root.crt ccm.key >> ca.pem

      You can name the mgmtserver and ca pem files as required for your environment, however, be sure to append them with the .pem extension.

       

    2. Place the mgmtserver.pem and ca.pem files created earlier to the CCM_LB server in the  /etc/haproxy location.


  4. Append the following details to the HAProxy config file.

    vi /etc/haproxy/haproxy.cfg        
                                                          
    # configuration to listen on 443 with SSL certs and loadbalance
    frontend https-in
        mode http
        log global
        bind *:443 ssl crt /etc/haproxy/mgmtserver.pem ca-file /etc/haproxy/ca.pem
        default_backend ccms
    
    # configuration to listen on 8443 with SSL certs and loadbalance
    frontend httpsalt-in
        mode tcp
        bind *:8443
        default_backend nodes
    
    backend ccms
        balance roundrobin
        mode    http
        log global
        option httplog
        cookie SVR insert preserve nocache
        server  ccm1 <CCM_SA_PRIMARY_IP>:443 check cookie ccm1 ssl verify none
        server  ccm2 <CCM_SA_SECONDARY_IP>:443 check cookie ccm2 ssl verify none
    
    backend nodes
        mode tcp
        balance roundrobin
        option ssl-hello-chk
        server  ccm1 <CCM_SA_PRIMARY_IP>:8443 check
        server  ccm2 <CCM_SA_SECONDARY_IP>:8443 check
    
  5. Start the HAProxy service and verify that the status response is active.

    systemctl start haproxy
    systemctl status haproxy
  6. At this point, you must use HTTPS to invoke the CCM server. For example:

    https://<CCM_LB_IP>

    The following option is an alternative step to the HTTPS step above

    Optional. To view the HA proxy status use the following configuration to access the ha_proxy from a web browser. These stats allow you to view the status of the nodes from a web browser and allows admins to drain/stop nodes without accessing the VMs directly.

    https://CCM_LB_IP:9000/haproxy_stats

    listen stats 0.0.0.0:9000 #Listen on all IP's on port 9000
     mode http
     balance
     timeout client 5000
     timeout connect 4000
     timeout server 30000
    
    #This is the virtual URL to access the stats page
     stats uri /haproxy_stats
    
    #Authentication realm. This can be set to anything. Escape space characters with a backslash.
     stats realm HAProxy\ Statistics
    
    #The user/pass you want to use. Change this password!
     stats auth admin:<password>
    
    #This allows you to take down and bring up back end servers.
     #This will produce an error on older versions of HAProxy.
     stats admin if TRUE

HA Environments

The timeout client and timeout server values must be changed from 1m (1minute) to 30m (30 minutes).

This change is required to ensure that the connection between the client and the server is open for a longer time to allow server push events (SSE).

In /etc/haproxy/haproxy.cfg

timeout client  1m #change to 30m
timeout server  1m #change to 30m

Or

Run the below commands in the HA Proxy loadbalancer of the CCM.

sed -i "s/timeout client\(\ \+\)1m/timeout client\130m/" /etc/haproxy/haproxy.cfg
sed -i "s/timeout server\(\ \+\)1m/timeout server\130m/" /etc/haproxy/haproxy.cfg


Back to: CCM (Required)

  • No labels
© 2017-2019 Cisco Systems, Inc. All rights reserved