CloudCenter 4.8 has reached End of Life (EOL) as of November 14, 2018. See End of Support Notices for additional context.

Install CCM HA Using Appliance

CCM HA installation is tested and verified for AWS, OpenStack, and VMware clouds.


To configure CCM in HA mode, you must use the following roles:

  • Database: MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE (and if required, MGMTPOSTGRES_VIP)

  • CCM: CCM_SA_PRIMARY and CCM_SA_SECONDARY

    Do not use the CCM or CCM_SA roles as those roles DO NOT allow you to configure high availability. See Virtual Appliance Overview and High Availability Best Practices for additional context.

  • Loadbalancer: CCM_LB


Proxy Settings

If you need a proxy server to connect to the internet, be sure to configure the Proxy setting for the Tomcat service on the CCM and CCO servers.

To connect to the Internet using a proxy server, follow this process:

  1. Modify your proxy environment to reflect your proxy settings.
  2. In the /usr/local/tomcat/bin/setenv.sh file, the CloudCenter platform requires you to add the following parameters to the existing JAVA_OPTS environment variable:

    REQUIRED for HTTP – Add the following parameters to the JAVA_OPTS environment variable
    -Dhttp.proxyHost=<proxy_hostname> -Dhttp.proxyPort=<port_number>

    or

    REQUIRED for HTTPS – Add the following parameters to the JAVA_OPTS environment variable
    -Dhttps.proxyHost=<proxy_hostname> -Dhttps.proxyPort=<port_number> 
    Sample /usr/local/tomcat/bin/setenv.sh with added parameters in JAVA_OPTS
    export LD_LIBRARY_PATH=/usr/local/apr/lib
    export CATALINA_PID=$CATALINA_HOME/catalina.pid
    export JPDA_ADDRESS=8000
    export JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1024m -Xmx2048m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -Dhttp.proxyHost=proxy.cisco.com -Dhttp.proxyPort=80"
    export CATALINA_OUT=/dev/null
  3. Additional parameters may be required if the CCM or CCO servers need to reach network resources, bypassing the proxy.

    -Dhttp.nonProxyHosts=<bypass_hostname>
    OPTIONAL – Sample /usr/local/tomcat/bin/setenv.sh bypassing the proxy
    export LD_LIBRARY_PATH=/usr/local/apr/lib
    export CATALINA_PID=$CATALINA_HOME/catalina.pid
    export JPDA_ADDRESS=8000
    export JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1024m -Xmx2048m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -Dhttp.proxyHost=proxy.cisco.com -Dhttp.proxyPort=80 -Dhttp.nonProxyHosts=10.1.1.1"
    export CATALINA_OUT=/dev/null
  4. Restart the Tomcat service.

    /etc/init.d/tomcat stop
    /etc/init.d/tomcat start
  5. Verify that the proxy settings are in effect by issuing the ps aux | grep java command.

    Output of the ps aux | grep java command
    cliqrus+ 25106 0.0 45.3 4368868 929000 ? Sl Mar02 54:15 /usr/lib/jvm/java-8-sun/bin/java -Djava.util.logging.config.file=/usr/local/tomcatgua/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1024m -Xmx2048m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -XX:MaxPermSize=512m -Djav.endorsed.dirs=/usr/local/tomcatgua/endorsed -classpath /usr/local/tomcatgua/bin/bootstrap.jar:/usr/local/tomcatgua/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcatgua -Dcatalina.home=/usr/local/tomcatgua -Djava.io.tmpdir=/usr/local/tomcatgua/temp -Dhttp.proxyHost=proxy.cisco.com -Dhttp.proxyPort=80 -Dhttp.nonProxyHosts=10.1.1.1 org.apache.catalina.startup.Bootstrap start

    Ensure that line -Dhttp.proxyHost=proxy.cisco.com -Dhttp.proxyPort=80 -Dhttp.nonProxyHosts=10.1.1.1 is present in the output.

Exchange MGMTPOSTGRES SSH Keys

To exchange the SSH keys between the MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE instances, follow this procedure.

  1. On the MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE instances, execute the following commands to generate a new SSH key on each instance. 

    ssh-keygen -t rsa
    cd ~/.ssh
    cat id_rsa.pub >> authorized_keys
    chmod 600 authorized_keys
  2. Copy the id_rsa.pub content from both MGMTPOSTGRES instances and paste the content into the authorized_keys file.

  3. Verify mutual SSH access between the MGMTPOSTGRES_MASTER and MGMTPOSTGRES_SLAVE by running the following command on each VM.

    ssh root@<MGMTPOSTGRES_MASTER/MGMTPOSTGRES_SLAVE>

Back to CCM (Required)

MGMTPOSTGRES_MASTER – Configure High Availability Properties

To configure high availability for MGMTPOSTGRES_MASTER, follow this procedure.

  1. Invoke the wizard as a root user (see Virtual Appliance Process > Cloud-Specific Setup Details for a sample setup).

    MGMTPOSTGRES Wizard Path
    /usr/local/cliqr/bin/db_config_wizard.sh
  2. Configure Postgres HA to ensure the PostgreSQL database HA and enter the information in each field as follows:

    Write this down for future reference!

    Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

    See Configure CCM Wizard Properties for other settings.

    Wizard Menu

    Field

    Description

    Configure_Postgres_HADB MasterThe hostname for the master database VM – not configurable.
    DB Master Private IPThe private IP address of the master database VM
    DB Slave HostnameThe hostname for the slave database VM
    DB Slave Private IPThe private IP address of the slave database VM

    VIP or EIP 

    The VIP/EIP IP for the database

    Use your mouse to select this option.

     AWS Cloud Nuances for EIP
    To setup PostgreSQL as an RDS service in the SA or HA modes, see Configuring HA for PostgreSQL Database on AWS. 

    Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

    • Configuring database for HA ...

    • Configuring database for replication

  3. Exit the configuration wizard.

  4. Go to the command line for each PostgreSQL server and enter the following command to review the status of the database and the HA connectivity:
    # pcs status

    1. Ensure that the PCSD Status for both database servers are Online.
    2. Ensure that the Daemon Status for Corosync, Pacemaker and the pcsd service are active/disabled.


Back to CCM (Required)

CCM_SA_PRIMARY/SECONDARY – Exchange CCM SSH Keys

To exchange the SSH keys between the CCM_SA_PRIMARY and CCM_SA_SECONDARY servers, follow this procedure using root permissions.

  1. On the CCM_SA_PRIMARY and the CCM_SA_SECONDARY instances, execute the following commands to generate a new SSH key on each instance. 

    ssh-keygen -t rsa
    cd ~/.ssh
    cat id_rsa.pub >> authorized_keys
    chmod 600 authorized_keys
  2. Copy the id_rsa.pub content from both the CCM instances and paste the content into the authorized_keys file.

  3. Verify mutual SSH access between the CCM_SA_PRIMARY and CCM_SA_SECONDARY by running the following command on each VM.

    ssh root@<CCM_SA_PRIMARY/CCM_SA_SECONDARY>


CCM_SA_PRIMARY – Configure HA Wizard Properties

To configure high availability for CCM_SA_PRIMARY, follow this procedure.

  1. Invoke the CCM wizard as a root user (see Virtual Appliance Process > Cloud-Specific Setup Details for a sample setup).

    CCM Wizard Path
    /usr/local/cliqr/bin/ccm_config_wizard.sh
  2. Configure the properties.

    Write this down for future reference!

    Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.

    Wizard Menu

    Field

    Description

    DB – Enter DB ParametersDB IP
    or Hostname

    The VIP/EIP for the master database and slave database. See Phase 1: Prepare Infrastructure > Cloud Nuances for additional context.

    When you configure the MGMTPOSTGRES_MASTER – Configure High Availability Properties, you would have configured the VIP/EIP address for the db_config_wizard already. Similarly, you must provide the EIP/VIP address for the CCM_SA_PRIMARY and the CCM_SA_SECONDARY server.

    DB Username
    and
    DB Password

    The following credentials are pre-populated:

    • Default username = cliqr (can be changed – manually change the password on MGMTPOSTGRES VMs or RDS and then update the username in the CCM through the database config wizard.

      Be sure to change the PostgresDB password and update the db.properties file to reflect the correct password.

    • Default password = cliqr (can be changed)

      Be sure to change the default password immediately after your first login. See PostgreSQL Password for additional context.

    Configure_HAPrimary Node Private IPThe IP address of the primary CCM VM
    Secondary Node Private IP The IP address of the secondary CCM VM

    Public DNS

    Use the DNS or IP of the CCM_LB – Used by the CCO VM to communicate with the CCM VM.

    Hazelcast IP

    Provide either Public or Private IP address of the both primary and secondary CCM VM separated by comma. If the two CCM VMs are configured on:

    • The same network, then use the Private IP address.
    •  Different networks, then use the Public IP address.

    External URL

    Use the CCM_LB_IP:

    • Optional for non-HA CCM scenarios
    • Required for HA CCM scenarios 

  3. Once the details are entered, the database server begins replication configuration between the database servers followed by HA configuration and finally presents the following status messages.

    • Configuring CCM HA ...
    • Restart server (with the progress bar)
    • Configured CCM HA successfully

    After configuring HA on the CCM primary server, the following message is displayed:
    Restarting secondary ccm tomcat failed due to cliqruser may not have sudo permission.

    At this point, you must manually restart tomcat on the secondary CCM server.

  4. Run the following command on the CCM secondary server to restart the Tomcat server.

    sudo -i 
    /etc/init.d/tomcat restart


  5. Exit the CCM configuration wizard.


Back to: CCM HA

CCM_LB


Use a plain clean OS image (such as CentOS7) to install a load balancer.

See CCM Firewall Rules > CCM_LB Ports for the complete list of ports that need to be open for your deployment.

If you configure a load balancer for any CloudCenter component, be aware that the firewalId is enabled by default and you must explicitly disable it to ensure that the CloudCenter component(s) can communicate with the load balancer. See Firewall Rules Overview for additional context.

 Here is a sample configuration to load balance a CentOS7.x VM with HAProxy for the CCM.

  1. SSH into the VM instance using the key pair that you used to launch the VM.
  2. Install HAProxy as the root user. 

    yum install -y haproxy
    
  3. Create .pem files for haproxy configuration for CCM_LB in the CCM Primary server.
    1. Run the following commands.

      sudo -i 
      cd /usr/local/tomcat/conf/ssl 
      cat mgmtserver.crt mgmtserver.key >> mgmtserver.pem 
      cat ca.crt mgmtserver.key >> ca.pem

      You can name the mgmtserver and ca pem files as required for your environment, however, be sure to append them with the .pem extension.

       

    2. Place the mgmtserver.pem and ca.pem files created earlier to the CCM_LB server in the  /etc/haproxy location.


  4. Append the following details to the HAProxy config file.

    vi /etc/haproxy/haproxy.cfg        
                                                          
    # configuration to listen on 443 with SSL certs and loadbalance
    frontend https-in
        mode http
        log global
        bind *:443 ssl crt /etc/haproxy/mgmtserver.pem ca-file /etc/haproxy/ca.pem
        default_backend ccms
    
    # configuration to listen on 8443 with SSL certs and loadbalance
    frontend httpsalt-in
        mode tcp
        bind *:8443
        default_backend nodes
    
    backend ccms
        balance roundrobin
        mode    http
        log global
        option httplog
        cookie SVR insert preserve nocache
        server  ccm1 <CCM_SA_PRIMARY_IP>:443 check cookie ccm1 ssl verify none
        server  ccm2 <CCM_SA_SECONDARY_IP>:443 check cookie ccm2 ssl verify none
    
    backend nodes
        mode tcp
        balance roundrobin
        option ssl-hello-chk
        server  ccm1 <CCM_SA_PRIMARY_IP>:8443 check
        server  ccm2 <CCM_SA_SECONDARY_IP>:8443 check
    
  5. Start the HAProxy service and verify that the status response is active.

    systemctl start haproxy
    systemctl status haproxy
  6. At this point, you must use HTTPS to invoke the CCM server. For example:

    https://myCCM_LB_server.com
    
    #or
    
    https://<CCM_LB_IP_Address>
  7. Optionally, to view the HAProxy stats use the following configuration to access the HAProxy from a web browser. These stats allow you to view the status of the nodes from a web browser and allows admins to drain/stop nodes without accessing the VMs directly.

    https://myCCM_LB_IP/haproxy_stats:9000

    listen stats 0.0.0.0:9000 #Listen on all IP's on port 9000
     mode http
     balance
     timeout client 5000
     timeout connect 4000
     timeout server 30000
    
    #This is the virtual URL to access the stats page
     stats uri /haproxy_stats
    
    #Authentication realm. This can be set to anything. Escape space characters with a backslash.
     stats realm HAProxy\ Statistics
    
    #The user/pass you want to use. Change this password!
     stats auth admin:<password>
    
    #This allows you to take down and bring up back end servers.
     #This will produce an error on older versions of HAProxy.
     stats admin if TRUE

 

Back to CCM (Required)

  • No labels
© 2017-2019 Cisco Systems, Inc. All rights reserved