Nahuel Hernandez

Nahuel Hernandez

Another personal blog about IT, Automation, Cloud, DevOps and Stuff.

ChatOps on EKS using BotKube

ChatOps allows us to operate DevOps tools using chat clients. Implementing ChatOps with EKS is painless using BotKube. This tool lets us monitor and debug our K8S cluster. The app integrates with Slack, Teams, etc. So, for example, we could view our K8S deployments with a simple command on Slack or view logs in real-time. Also, give us recommendations for standard practice, such avoid using the latest containers image versions

3-Minute Read

BotKube

BotKube is a messaging bot for monitoring and debugging K8S clusters in real-time. Some features are:

  • Allow us to execute Kubectl commands on our chat clients.
  • By Default uses a ReadOnly Service Account, thus is only possible to execute read commands.
  • Can monitor any K8S resource, including Custom Resources (like certificate expiry or backup failure).
  • Pretty easy to configure, less than 5 minutes.
  • Let us do a quick debug using only our chat client mobile app.
  • Deploy on any K8S Flavors (not only EKS)

Table of contents

Requirements

  • AWS Account
  • EKS Cluster
  • Helm
  • Slack

Configure BotKube on Slack

We need to install the BotKube Slack Application to our Slack workspace, using the following Link

After installing it, we need to copy the Bot Access Token, we’ll use it in the next step

Now we need to add the BotKube user to a Slack channel, for example, #chatops-eks. To add, it is only to write @BotKube in the channel

Install BotKube on your EKS cluster

To install BotKube on EKS (or other K8S flavors), we use Helm, and it’s pretty easy. First, we need to configure some extra variables:

> export SLACK_API_BOT_TOKEN="xoxb-8064160117-xyz"
> export CLUSTER_NAME=eks-chatops
> export ALLOW_KUBECTL=true
> export SLACK_CHANNEL_NAME=chatops-eks

Now we can add the helm repo and deploy BotKube

> helm repo add botkube https://charts.botkube.io
> helm repo update
> helm install --version v0.13.0 botkube --namespace botkube --create-namespace \
--set communications.default-group.slack.enabled=true \
--set communications.default-group.slack.channels.default.name=${SLACK_CHANNEL_NAME} \
--set communications.default-group.slack.token=${SLACK_API_BOT_TOKEN} \
--set settings.clusterName=${CLUSTER_NAME} \
--set executors.kubectl-read-only.kubectl.enabled=${ALLOW_KUBECTL} \
botkube/botkube

Where:

  • SLACK_CHANNEL_NAME is the channel name where @BotKube is added
  • SLACK_API_BOT_TOKEN is the Token you received after installing BotKube app to your Slack workspace
  • CLUSTER_NAME is the cluster name set in the incoming messages
  • ALLOW_KUBECTL set true to allow kubectl command execution by BotKube on the cluster

BotKube will watch all and notify us for created/deleted resources and error events. Some examples:

BotKube Resources

Execute kubectl commands from Slack

First, we need to test the BotKube installation. Send @BotKube ping on your Slack Channel, it will show something like:

pong from cluster 'eks-chatops'
K8s Server Version: v1.23.7-eks-4721010
BotKube version: v0.13.0

Finally, we could execute kubectl commands using BotKube, for example @BotKube get pods. We could check all allowed commands with @BotKube commands list

BotKube Commands

Disable notifications for a specified resource

Sometimes, we have a resource we don’t want to notify us, such as a pipeline that creates a new pod and deletes it after the tasks. To avoid this, we could use Kubernetes Annotations. So, we only need to add the Annotation botkube.io/disable=true to our resource.

For example, I created a new Nginx pod and the BotKube notified me (also gave me an excellent recommendation to avoid latest images)

BotKube Advices

But, when i created the same pod with the annotation. BotKube didn’t say anything.

> kubectl run testing-botkube-annotation --image nginx --annotation botkube.io/disable=true

Conclusion

BotKube is a fantastic tool and pretty easy to install and configure. Sometimes we are far away from our laptops, and we need to check something in our cluster and even do some quick troubleshooting. In this last case, my recommendation is to install also metric server to have real-time info about CPU and memory.

From now, I’ll install BotKube in all my K8S Clusters.

References

Categories

Recent Posts

About

Over 15-year experience in the IT industry. Working in SysOps, DevOps and Architecture roles with mission-critical systems across a wide range of industries. Wide experience with AWS, Terraform, Kubernetes, Containers, CI/CD pipelines, and Linux. Always keeping up with the latest technologies. Passionate about automating the run of the mill. Big focus on problem-solving.