Upgrade CCM

CCM Upgrade Overview

See Upgrade Overview for additional context.

The CloudCenter platform does not support the CCM and DB in the same instance and these components must be on separate instances.

The CCM server requires additional memory for the changes in the underlying architecture. See Hardware Requirements for details.

When upgrading, the current slave (even if it is dbmaster) will be renamed to Node2 and the current master (even if it is dbslave) will be renamed to Node1.

One Time Prerequisite

As a one-time task for all OS configurations, you must tighten the security configuration for the PosgreSQL and Load balancer VMs to ensure security compliance. See the Security Hardening Requirements section in the Upgrade Overview section for additional context.

Tagless Governance

If you are migrating your environment to ensure tagless governance, you must use the CloudCenter 4.10.0 ccm-response.xml file to upgrade to CloudCenter 4.10.0.

See Migrate to Tagless Governance for additional context.

CCM Non-HA Mode

Move MGMTPOSTGRES to a New Instance

To move MGMTPOSTGRES from the CCM instance to a new instance, follow this procedure.

  1. Create a new MGMTPOSTGRES instance. See CCM (Required) > CCM NON-HA for additional context.

  2. SSH into the instance.

  3. Download core_installer.bin in /tmp folder of MGMTPOSTGRES.

  4. Run the core installer to setup MGMTPOSTGRES.

    sudo -i
    cd /tmp
    chmod 755 core_installer.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_installer.bin <ostype> <cloud> mgmtpostgres

Configure Database Access to the CCM

To configure database access from the existing CCM server to the MGMTPOSTGRES server, follow this procedure.

  1. SSH into the MGMTPOSTGRES server.

  2. Invoke the db_config_wizard and configure the CCM IP to enable database access to the CCM.

    sudo -i
    /usr/local/cliqr/bin/db_config_wizard.sh
  3. Enter the CCM_IP in the CCM IP field to configure the PostgreSQL database access.

  4. Verify your changes and Exit the DB configuration wizard.

Convert CCM and Upgrade to CCM SA

To convert the CCM role and upgrade to the CCM SA role, follow this procedure.

  1. SSH into the CCM instance.

  2. Download the following files to the /tmp folder of CCM server.

    1. core_upgrade.bin

    2. ccm-installer.jar

    3. ccm-response.xml

  3. Run the core upgrade file to upgrade CCM system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> ccm_sa
  4. Edit the ccm-response.xml file.

    Response File Entry

    Value

    Notes

    <entry key="db_host" value=""/> 

    Replace localhost with IP of the PostgreSQL database. (MGMTPOSTGRES_IP)

    Required for the standalone CCM (CCM_SA) to connect to the remote database

    <entry key ="migrate_db" value ="true"


    Required to migrate DB in CCM to separate MGMTPOSTGRES to make CCM standalone

  5. Run the appliance installer to setup the CCM and DB as separate instance.

    java -jar ccm-installer.jar ccm-response.xml 

You have now completed the CCM upgrade and moved to the CCM_SA role.

CCM Non-HA Standalone

To upgrade the CCM and MGMTPOSTGRES components, follow this section.

Upgrade MGMTPOSTGRES

To run the core upgrade files, follow this procedure.

  1. SSH into the MGMTPOSTGRES instance.

  2. Download the core_upgrade.bin file in /tmp folder of MGMTPOSTGRES

  3. Run the core upgrade to upgrade MGMTPOSTGRES system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> mgmtpostgres

Upgrade CCM_SA

To run the upgrade and CCM_SA files, follow this procedure.

  1. SSH into the CCM_SA instance.

  2. Download the following files to the /tmp folder of CCM_SA:

    • core_upgrade.bin

    • ccm-installer.jar

    • ccm-response.xml

  3. Run the core upgrade file to upgrade CCM_SA system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> ccm_sa
    1. Edit the ccm-response.xml file.

      Response File Entry

      Value

      Notes

      <entry key="db_host" value=""/> 

      Replace localhost with IP of the PostgreSQL database. (MGMTPOSTGRES_IP)

      Required for the standalone CCM (CCM_SA) to connect to the remote database

    2. Run the appliance installer to upgrade CCM_SA.

      java -jar ccm-installer.jar ccm-response.xml 

    You have now completed the CCM_SA upgrade.

CCM HA

The MGMTPOSTGRES_NODE1 and MGMTPOSTGRES_NODE2 roles are not formal names for roles – they are merely references to Node1 and Node2 in the PostgreSQL cluster. Going forward, CloudCenter documentation will refer to these roles as MGMTPOSTGRES_NODE1 and MGMTPOSTGRES_NODE2 in keeping with the terms displayed in the output

If you are upgrading from CloudCenter 4.8.2x or 4.9.x, you do not need to exchange the SSH keys for CCM HA environments.

Upgrade MGMTPOSTGRES_NODE1 and MGMTPOSTGRES_NODE2

MGMTPOSTGRES upgrades required the DB slave node to be updated first – before the  DB master node.

Identify the Database Master and Slave Nodes

To identify the master and slave nodes, follow this procedure.

  1. SSH into both MGMTPOSTGRES instances.

  2. Run the following command on both instances.

    pcs status

    The following image reflects the sample out of this command:

    Cluster name: cliqrdbcluster
    Stack: corosync
    Current DC: dbslave (version 1.1.15-11.el7_3.4-e174ec8) - partition with quorum
    Last updated: Sat Dec 16 09:11:55 2017          Last change: Sat Dec 16 09:10:04 2017 by root via crm_attribute on dbslave
     
    2 nodes and 3 resources configured
     
    Online: [ dbmaster dbslave ]
     
    Full list of resources:
     
     Resource Group: VIPGroup
         PGMasterVIP (ocf::heartbeat:IPaddr2):       Started dbslave
     Master/Slave Set: mspostgresql [pgsql]
         Masters: [ dbslave ]
         Slaves: [ dbmaster ]
     
    Daemon Status:
      corosync: active/disabled
      pacemaker: active/enabled
      pcsd: active/enabled


    For example, in the above sample output:

    • The MGMTPOSTGRES_NODE1 is identified as dbmaster 

    • The MGMTPOSTGRES_NODE2 is identified as dbslave

Upgrade MGMTPOSTGRES_NODE1

To run the core upgrade files, follow this procedure.

  1. SSH into the MGMTPOSTGRES_NODE1 instance.

  2. Download the core_upgrade.bin file in /tmp folder of MGMTPOSTGRES_NODE1.

  3. Run the core upgrade to upgrade MGMTPOSTGRES_NODE2 system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> mgmtpostgres

Upgrade the MGMTPOSTGRES_NODE2

To run the core upgrade files, follow this procedure.

  1. SSH into the MGMTPOSTGRES_NODE2 instance.

  2. Download the core_upgrade.bin file in /tmp folder of MGMTPOSTGRES_NODE2.

  3. Run the core upgrade to upgrade MGMTPOSTGRES_NODE1 system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> mgmtpostgres

Check PCS Status after the Upgrade

To check the PCS status, follow this procedure.

  1. Ensure Both the NODE1 and NODE2 are online.
  2. PGMasterVIP should be started on any one of the nodes.
  3. Ensure that both master and slave is displayed in master/slave set.

    pcs status
  4. Ensure the following:

    1. Both instances (NODE1 and NODE2) are online.

    2. That PGMasterVIP is started on one of the instances.

    3. Both NODE1 and NODE2 are displayed in the cluster.

  5. See the following sample output:

    Cluster name: cliqrdbcluster
    Stack: corosync
    Current DC: node1 (version 1.1.15-11.el7_3.4-e174ec8) - partition with quorum
    Last updated: Sat Dec 19 17:25:55 2017          
    Last change: Sat Dec 19 14:10:04 2017 by root via crm_attribute on node1
     
    2 nodes and 3 resources configured
     
    Online: [ node1 node2 ]
     
    Full list of resources:
     
     Resource Group: VIPGroup
         PGMasterVIP (ocf::heartbeat:IPaddr2):       Started node1
     Master/Slave Set: mspostgresql [pgsql]
         Masters: [ node1 ]
         Slaves: [ node2 ]
     
    Daemon Status:
      corosync: active/disabled
      pacemaker: active/enabled
      pcsd: active/enabled

Upgrade the CCM_SA_PRIMARY and CCM_SA_SECONDARY

Exchange the SSH keys between the CCM_SA_PRIMARY and CCM_SA_SECONDARY to ensure mutual SSH authentication.

If you are upgrading from CloudCenter 4.8.2x or 4.9.x, you do not need to exchange the SSH keys for CCM HA environments.

Exchange SSH Keys

To exchange the SSH keys, follow this procedure.

  1. On the CCM_SA_PRIMARY – execute the following commands to generate a new SSH key.

    sudo -i
    
    # The ssh-keygen -t rsa command has two options. Use the defaults by pressing enter OR providing a custom name. DO NOT perform a COMBINATION of these two options!
    
    ssh-keygen -t rsa
    cd ~/.ssh
    cat id_rsa.pub >> authorized_keys
  2. On the CCM_SA_SECONDARY –  If the .ssh directory does not exist, create it using the following commands before copying the files.

    sudo -i
    mkdir -p ~/.ssh
    chmod 700 ~/.ssh
  3. Copy the id_rsa files (~/.ssh/id_rsa and ~/.ssh/id_rsa.pub) from the CCM_SA_PRIMARY to the same location on CCM_SA_SECONDARY.

  4. On the CCM_SA_SECONDARY, execute the following commands.

    sudo -i
    chmod 400 ~/.ssh/id_rsa*
    cat id_rsa.pub >> authorized_keys
  5. Verify SSH between CCM_SA_PRIMARY and CCM_SA_SECONDARY.

    sudo -i
    ssh root@<CCM_SA_PRIMARY/CCM_SA_SECONDARY>

    Accept the authentication message at the end of this process. For example:

    The authenticity of host 'example.com (10.0.1.17)' can't be established.
    ECDSA key fingerprint is SHA256:QzhsukVubu4zvV/c5g/eh68Rc1tY0/2FO/UncaclC0g.
    Are you sure you want to continue connecting (yes/no)?

    You must say Yes at this point.

Upgrade the CCM_SA_PRIMARY

To upgrade the CCM_SA_PRIMARY, follow this procedure.

  1. SSH into the CCM_SA_PRIMARY instance.

  2. Download the following files to the /tmp folder of CCM_SA_PRIMARY server.

    1. core_upgrade.bin

    2. ccm-installer.jar

    3. ccm-response.xml

  3. Run the core upgrade to upgrade the CCM_SA_PRIMARY system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> ccm_sa
  4. Edit the ccm-response.xml file.

    Response File Entry

    Value

    Notes

    <entry key="db_host" value=""/> 

    Use the VIP or elastic IP address

    Required.

  5. Run the appliance installer to setup the  CCM_SA_PRIMARY as separate instance.

    java -jar ccm-installer.jar ccm-response.xml 

You have now completed the CCM_SA_PRIMARY upgrade.

Upgrade the CCM_SA_SECONDARY

To upgrade the CCM_SA_SECONDARY, follow this procedure.

  1. SSH into the CCM_SA_SECONDARY instance.

  2. Download the following files to the /tmp folder of CCM_SA_SECONDARY server.

    1. core_upgrade.bin

    2. ccm-installer.jar

    3. ccm-response.xml

  3. Run the core upgrade to upgrade the CCM_SA_SECONDARY system components using the following commands.

    sudo -i
    cd /tmp
    chmod 755 core_upgrade.bin
    
    #Set the following only if a local store is setup
    export CUSTOM_REPO=<http://local_package_store IP>
    
    ./core_upgrade.bin <ostype> <cloud> ccm_sa
  4. Edit the ccm-response.xml file.

    Response File Entry

    Value

    Notes

    <entry key="db_host" value=""/> 

    Use the VIP or elastic IP address

    Required.

  5. Run the appliance installer to setup the  CCM_SA_SECONDARY as separate instance.

    java -jar ccm-installer.jar ccm-response.xml 

You have now completed the CCM_SA_SECONDARY upgrade.



                         

  • No labels
© 2017-2019 Cisco Systems, Inc. All rights reserved