If you have an OpenShift (Origin or Enterprise) environment, you have likely deployed the EFK (elasticsearch, fluentd, and kibana) stack to help operators and developers easily view log files. Referred to as the Aggregated Container Logs in OSE, this stack is very useful and should be deployed in most situations. But log data tends to grow rapidly, especially in development environments with a large amount of projects and containers being deployed in a CI/CD pipeline, and cleanup is necessary; enter curator.
Curator allows operators to define how long elasticsearch indices should be retained. On a defined daily schedule it will purge any qualified indices from elasticsearch. Unfortunately, curator has only been added into the OpenShift Origin version of software, and is not yet included in the OpenShift Enterprise deployments (as of 3.2). Fortunately, version 3.1.1 and 3.2 of the EFK images do include admin credentials that can be used to authenticate a manually deployed curator template.
A few notes about these steps:
Commands are run with a cluster-admin authorization
The project we are using for logging is called ‘logging’
The logging-es deployment configuration specifies version 3.1.1 or 3.2
We are setting the defaults with environment variables within the yaml file, which can be done outside of this file as well
With the bulk of his career focused on infrastructure solution architecture and implementation, Shea now brings that knowledge into the development space. Working with software development and infrastructure teams, Shea focuses on improving technology processes through automation and collaboration tooling and culture changes. Being a technologist, Shea is happiest when found exploring emerging technologies and applying them to real-world customer challenges.
Outside of the office Shea can be found out with his friends enjoying a Jays game, being active, or hunting down a new craft brew.
Along with an eye on security processes and partnerships, Shea’s focus at Arctiq is in the following key areas:
Automation & self-service of infrastructure systems
Container platform deployment and training
CI/CD pipeline development with integrated security
DevOps culture improvements through process optimization and collaboration tooling