Agent Troubleshooting

Node Not Reachable

Symptom

While the Deployment succeeded, after sometime the Agent was not in the Running state and the deployment shows the VM in red (Unreachable).

Applies to

All Clouds and Apps

Solution

  1. Verify the following files for additional details:
    • $AGENT_HOME/log/agent.log
    • $AGENT_HOME/log/nohup.err (Linux)
  2. Reboot the VM or manually start the Agent.

Agent and RabbitMQ communication issue

Symptom

A deployment failed due to Bootstrap Timeout.

Applies to

All clouds and applications.

Solution

To troubleshoot issues caused by a communication between the agent and RabbitMQ, do the following.

  1. Check if the broker host IP and broker host port were correctly specified in 
    user-data in the worker VM.

  2. Confirm that the RabbitMQ service is running:

    1. If you deployed Cloud Remote appliance for this region, from the command line of the Cloud Remote appliance enter:

      docker ps  | grep rabbitmq
    2. If you did not deployed Cloud Remote appliance for this region, ensure kubectl on your computer is configured to communicate with the Kubernetes cluster hosting the CloudCenter Suite and issue this command:

      kubectl get pod  | grep rabbitmq
  3. Ensure there are no firewall rules on the Worker VMs blocking outgoing RabbitMQ connections.

  4. Ensure there are no firewall rules blocking the RabbitMQ service, whether it is running in the CloudCenter Suite cluster or on the Cloud Remote appliance.

  5. Open the RabbitMQ console and confirm you can login. By default, the URL will use the following format:

    http://<RabbitMQ server IP>:15672

    If you are not using Cloud Remote, you can find the RabbitMQ server IP from the Worker AMQP IP Address field in the Region Connectivity section of the Regions / Details tab. If you are using Cloud Remote, use the the public IP address assigned to your Cloud Remote appliance when you launched it.

  6. Verify from the agent logs that a Node Bootstrap message was sent.

  7. Login to Kibana and verify from the logs that:

    1. a Node Bootstrap message was received  by the cloudcenter-cco service
    2. the cloudcenter-ccm-backend service received messages from the cloudcenter-cco service
  • No labels
© 2017-2019 Cisco Systems, Inc. All rights reserved