// removed jquery ui css and js

Upgrade AMQP in HA Mode

Overview

Be sure to review Upgrade Overview before starting this procedure!

This section provides details on upgrading your AMQP in HA mode.

Currently, the CloudCenter platform does not support HA for the Guacamole component.

Prerequisites

Verify these requirements before you begin the upgrade process:

  • Review the information provided in the Upgrade Overview section and validate the following requirements for the release to which you are upgrading:

    • Is an upgrade path available?
    • Is the core_upgrade.bin file required?
  • See High Availability Best Practices for HA considerations.
  • For each AMQP instance that must be upgraded, verify the following prerequisites:

    • Ensure that a version file (/usr/local/osmosix/etc/version) exists in both AMQPs to be upgraded.

    • Verify that the version file contains the correct version number (for example, if your current CloudCenter release version is 4.7.2, ensure that the corresponding version value is 4.7.2).

    • See the corresponding release notes for release-specific information on the CloudCenter version to which you are upgrading. For example, the CloudCenter 4.8.0 Release Notes.

Download Packages

Download package files:

See Virtual Appliance Overview to understand the required components.

See Virtual Appliance Process to understand the process.

See Phase 4 Overview  to understand the various types of software download files.

  1. SSH into the VM instance designated for this component by using the key pair that you used to launch the VM.

    Along with the key pair, you may need to use your login credentials for sudo or root access based on your environment.

  2. Download the following required files for this component from software.cisco.com:

    • cco-installer.jar
    • conn_broker-response.xml
    • core_upgrade.bin

Upgrading AMQP Instances

To upgrade the Primary and Secondary AMQP instances, follow this process on each server.

  1. Login to the AMQP server and back up the data.

    cd $bakdir
    cp -r /usr/local/tomcatgua/webapps/access .
    cp -r /usr/local/tomcatgua/webapps/cliqr-connection-broker .
  2. Run the following commands on the AMQP server.

    sudo –i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local package store is setup:
    export CUSTOM_REPO=<http://local_package_store ip>
    
    ./core_upgrade.bin <ostype> <cloudtype> rabbit
    
    #After the above process completes, remove the core_upgrade.bin file
    rm core_upgrade.bin 
    cd /tmp
     Syntax

    <ostype>= centos6, centos7, rhel6, rhel7

    <cloudtype>= amazon, azureclassic, azurerm, azurepack, google, opsource, openstack, softlayer, vmware, and vcd

  3. Run the following commands from your download folder.

    java -jar cco-installer.jar conn_broker-response.xml

Post Upgrade Tasks

  1. Verify Your Upgrade – Ensure that the version file (/usr/local/osmosix/etc/version) reflects the new release.

    cat /usr/local/osmosix/etc/version
  2. Reboot the AMQP servers – Be aware of the following consequences if/when you reboot the AMQP server.

    Reboot AMQP

    Reboot AMQP VM

    If you change the AMQP server's host name, the local AMQP database is renamed and you must reboot the AMQP VM.

    • To reboot the AMQP VM, run the following commands as root:

      rm /usr/local/osmosix/etc/.RABBITINSTALLED
      /usr/local/osmosix/bin/rabbit_config.sh
      reboot
    • If you reboot the VM, be aware of the following details:
      • You may end up with a new host name and database name after the reboot.

      • Some clouds set the host name automatically for each new instance or reboot – RabbitMQ uses a preset host name to set the database name.

      • If a database user exists and a login is not associated, this user may not be able to log into the AMQP server.

        • Ensure that the required users (cliqr and cliqr_worker) are setup in your database. If you have additional users in your database, they will also be displayed when you run the rabbitmqctl command.

          rabbitmqctl list_users
          Listing users ...
          cliqr [administrator]
          cliqr_worker []
        • If you do not see these users in your database, run the following commands as root (to recreate the users in the AMQP configuration):

          rm /usr/local/osmosix/etc/.RABBITINSTALLED
          bash /usr/local/osmosix/bin/rabbit_config.sh

  3.  Configure the Properties in Primary AMQP

    This component is required for each cloud region (public cloud, private cloud, or datacenter).

    Dedicated Gucamole Setup

    This GUA config wizard step is not required if you have set up a Dedicated Guacamole Server (Optional) server – A self-signed Guacamole certificate is provided out-of-box for Appliance installations.

    AMQP – Configure CCM/CCO Properties for Guacamole Server

    Configure the following gua_config_wizard.sh procedure on both the AMQP PRIMARY server and the AMQP SECONDARY server.

      1. SSH into the GUA instance as a centos user.
      2. Run the following command:

        sudo -i
    1. Invoke the GUA wizard.

      GUA Wizard Path
      /usr/local/cliqr/bin/gua_config_wizard.sh
    2. Configure the CCM and CCO (once installed) properties.

      Write this down for future reference!

      Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.
    3. Configure the properties for the CCM and CCO (once installed) VMs:

      Wizard MenuFieldPossible IP Addresses

      CCM_Info

      CCM Host

      CCM_IP or  CCM_SA_IP or CCM_LB_IP

      CCO_InfoCCO Host (once installed)
      CCO_IP or  CCO_LB_IP
      Config_CertsCerts Zip PathProvide the path for the certs.zip file. The default path is /tmp/certs.zip.
    4. Verify your changes and Exit the GUA configuration wizard.

    5. Select Yes, to restart the Tomcat service for the changes to take effect.

      If you are installing the AMQP instance for the first time, then you may need to wait for a few minutes to ensure that all users are listed. You can verify that all users are listed by running the following commands:

      root> ls -alrt /usr/local/osmosix/etc/.RABBITINSTALLED
      -rw-r--r--. 1 root root o Feb 17 23:07 /user/local/osmosix/etc/.RABBITINSTALLED
      root> rabbitmqctl list_users
      Listing users ...
      cliqr   [administrator]
      cliqr_worker     []
      guest   [administrator]
      root>

      Reboot AMQP

      Reboot AMQP VM

      If you change the AMQP server's host name, the local AMQP database is renamed and you must reboot the AMQP VM.

      • To reboot the AMQP VM, run the following commands as root:

        rm /usr/local/osmosix/etc/.RABBITINSTALLED
        /usr/local/osmosix/bin/rabbit_config.sh
        reboot
      • If you reboot the VM, be aware of the following details:
        • You may end up with a new host name and database name after the reboot.

        • Some clouds set the host name automatically for each new instance or reboot – RabbitMQ uses a preset host name to set the database name.

        • If a database user exists and a login is not associated, this user may not be able to log into the AMQP server.

          • Ensure that the required users (cliqr and cliqr_worker) are setup in your database. If you have additional users in your database, they will also be displayed when you run the rabbitmqctl command.

            rabbitmqctl list_users
            Listing users ...
            cliqr [administrator]
            cliqr_worker []
          • If you do not see these users in your database, run the following commands as root (to recreate the users in the AMQP configuration):

            rm /usr/local/osmosix/etc/.RABBITINSTALLED
            bash /usr/local/osmosix/bin/rabbit_config.sh

    You have successfully configured the AMQP instance! Proceed to the CCO (Required) section.

  4.  Configure the Properties in Secondary AMQP

    This component is required for each cloud region (public cloud, private cloud, or datacenter).

    Dedicated Gucamole Setup

    This GUA config wizard step is not required if you have set up a Dedicated Guacamole Server (Optional) server – A self-signed Guacamole certificate is provided out-of-box for Appliance installations.

    AMQP – Configure CCM/CCO Properties for Guacamole Server

    Configure the following gua_config_wizard.sh procedure on both the AMQP PRIMARY server and the AMQP SECONDARY server.

      1. SSH into the GUA instance as a centos user.
      2. Run the following command:

        sudo -i
    1. Invoke the GUA wizard.

      GUA Wizard Path
      /usr/local/cliqr/bin/gua_config_wizard.sh
    2. Configure the CCM and CCO (once installed) properties.

      Write this down for future reference!

      Write down the Field details in a printed version of the Installation Approach > Your Notes section for later use.
    3. Configure the properties for the CCM and CCO (once installed) VMs:

      Wizard MenuFieldPossible IP Addresses

      CCM_Info

      CCM Host

      CCM_IP or  CCM_SA_IP or CCM_LB_IP

      CCO_InfoCCO Host (once installed)
      CCO_IP or  CCO_LB_IP
      Config_CertsCerts Zip PathProvide the path for the certs.zip file. The default path is /tmp/certs.zip.
    4. Verify your changes and Exit the GUA configuration wizard.

    5. Select Yes, to restart the Tomcat service for the changes to take effect.

      If you are installing the AMQP instance for the first time, then you may need to wait for a few minutes to ensure that all users are listed. You can verify that all users are listed by running the following commands:

      root> ls -alrt /usr/local/osmosix/etc/.RABBITINSTALLED
      -rw-r--r--. 1 root root o Feb 17 23:07 /user/local/osmosix/etc/.RABBITINSTALLED
      root> rabbitmqctl list_users
      Listing users ...
      cliqr   [administrator]
      cliqr_worker     []
      guest   [administrator]
      root>

      Reboot AMQP

      Reboot AMQP VM

      If you change the AMQP server's host name, the local AMQP database is renamed and you must reboot the AMQP VM.

      • To reboot the AMQP VM, run the following commands as root:

        rm /usr/local/osmosix/etc/.RABBITINSTALLED
        /usr/local/osmosix/bin/rabbit_config.sh
        reboot
      • If you reboot the VM, be aware of the following details:
        • You may end up with a new host name and database name after the reboot.

        • Some clouds set the host name automatically for each new instance or reboot – RabbitMQ uses a preset host name to set the database name.

        • If a database user exists and a login is not associated, this user may not be able to log into the AMQP server.

          • Ensure that the required users (cliqr and cliqr_worker) are setup in your database. If you have additional users in your database, they will also be displayed when you run the rabbitmqctl command.

            rabbitmqctl list_users
            Listing users ...
            cliqr [administrator]
            cliqr_worker []
          • If you do not see these users in your database, run the following commands as root (to recreate the users in the AMQP configuration):

            rm /usr/local/osmosix/etc/.RABBITINSTALLED
            bash /usr/local/osmosix/bin/rabbit_config.sh

    You have successfully configured the AMQP instance! Proceed to the CCO (Required) section.

  5.  Update the AMQP_LB Configuration

    AMQP_LB

    The AMQP load balancing can be done through HAProxy, NGiNX, Apache2, or a cloud that is natively available to services, like AWS Elastic Load Balancer (ELB). To configure the load balancer service and ensure AMQP load balancing, be sure to listen on port 5671 and balance the request at 443 on both the AMQP_PRIMARY and AMQP_SECONDARY servers.

    See AMQP Firewall Rules > AMQP_LB Ports for the complete list of ports that need to be open for your deployment.

    If you configure a load balancer for any CloudCenter component, be aware that the firewalId is enabled by default and you must explicitly disable it to ensure that the CloudCenter component(s) can communicate with the load balancer. See Firewall Rules Overview for additional context.

    The following load balancing configuration was performed on CentOS7.x VM with HAProxy for the AMQP VM.

    1. SSH into the VM instance using the key pair that you used to launch the VM.
    2. Install HAProxy as the root user.

      yum install –y haproxy
      
    3. Modify HAProxy config file as displayed in the following code block.

      vi /etc/haproxy/haproxy.cfg
      
      #configuration to listen on 5671 and loadbalance
      frontend amqps-in
          mode tcp
          log global
          bind *:5671
          default_backend amqps
      backend amqps
          mode tcp
          balance roundrobin
          option ssl-hello-chk
          server amqp1 <AMQP_PRIMARY>:5671 check
          server amqp2 <AMQP_SECONDARY>:5671 check
      
      #configuration to listen on 443 and loadbalance
      frontend gua-in
          mode tcp
          log global
          bind *:443
          default_backend guas
      backend guas
         mode tcp
         balance roundrobin
         option ssl-hello-chk
         server amqp1 <AMQP_PRIMARY>:443 check
         server amqp2 <AMQP_SECONDARY>:443 check backup
       
      #configuration to listen on 7788 and loadbalance
      frontend gua-wrk-in
          mode tcp
          log global
          bind *:7788
          default_backend gua-wrk
      backend gua-wrk
         mode tcp
         balance roundrobin
         server amqp1 <AMQP_PRIMARY>:7788 check
         server amqp2 <AMQP_SECONDARY>:7788 check backup
       
      #configuration to listen on 7789 and loadbalance
      frontend gua-rev-in
          mode tcp
          log global
          bind *:7789
          default_backend gua-rev
      backend gua-rev
         mode tcp
         balance roundrobin
         server amqp1 <AMQP_PRIMARY>:7789 check
         server amqp2 <AMQP_SECONDARY>:7789 check backup
    4. To bind to 5671 port you must disable SELinux – run the following command to disable SELinux.

      setenforce 0
      sed -i 's/=enforcing/=permissive/g' /etc/selinux/config*
      #This command ensures that SELINUX is disabled permanently and the changes are retained even in case of reboot 
    5. Start the HAProxy service and check the status, it should be active


      systemctl start haproxy
      systemctl status haproxy 
      
    6. Optionally, to view the HAProxy stats use the following configuration to access the HAProxy from a web browser. These stats allow you to view the status of the nodes from a web browser and allows admins to drain/stop nodes without accessing the VMs directly.

      https://myAMQP_LB_IP/haproxy_stats:9000

      listen stats 0.0.0.0:9000 #Listen on all IP's on port 9000
       mode http
       balance
       timeout client 5000
       timeout connect 4000
       timeout server 30000
      
      #This is the virtual URL to access the stats page
       stats uri /haproxy_stats
      
      #Authentication realm. This can be set to anything. Escape space characters with a backslash.
       stats realm HAProxy\ Statistics
      
      #The user/pass you want to use. Change this password!
       stats auth admin:<password>
      
      #This allows you to take down and bring up back end servers.
       #This will produce an error on older versions of HAProxy.
       stats admin if TRUE

    You have successfully configured the AMQP instance! Proceed to the CCO (Required) section.

  6. Upgrade the Monitor instances – See Monitor Upgrade  for additional context.
  • No labels
© 2017 Cisco Systems