K8s Use Case

Kubernetes

Kubernetes a.k.a K8s is a portable, extensible and open-source container orchestration tool that can automatically scale, distribute and manage fault-tolerance on containers. It is available on various platforms and even cloud providers such as AWS, GCP, and Azure offer them as services.

Containers

Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions.

Containers have become popular because they provide extra benefits, such as:

• Agile application creation and deployment: Increased ease and efficiency of container image creation compared to VM image use.
• Continuous development, integration, and deployment: Provides for reliable and frequent container image build and deployment with quick and easy rollbacks (due to image immutability).
• Dev and Ops separation of concerns: Create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure.
• Observability not only surfaces OS-level information and metrics but also application health and other signals.
• Environmental consistency across development, testing, and production: Runs the same on a laptop as it does in the cloud.
• Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-premises, on major public clouds, and anywhere else.
• Application-centric management: Raises the level of abstraction from running an OS on virtual hardware to running an application on an OS using logical resources.
• Loosely coupled, distributed, elastic, liberated micro-services: Applications are broken into smaller, independent pieces and can be deployed and managed dynamically — not a monolithic stack running on one big single-purpose machine.
• Resource isolation: predictable application performance.
• Resource utilization: High efficiency and density.

Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn’t it be easier if this behaviour was handled by a system?

That’s how Kubernetes comes to the rescue! Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of scaling and failover for your application, provides deployment patterns, and more. For example, Kubernetes can easily manage a canary deployment for your system.

Kubernetes provides you with:

• Service discovery and load balancing Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.
• Storage orchestration Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.
• Automated rollouts and rollbacks You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.
• Automatic bin packing You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.
• Self-healing Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
• Secret and configuration management Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.

Deploying Kubernetes

Kubernetes offers a new way to deploy applications using containers. It creates an abstraction layer which can be manipulated with declarative rather than imperative programming. This way, it is much simpler to deploy and upgrade services over time.

There are many ways to deploy a tool. A Deployment, for example, is an upgrade from a replication controller that has mechanisms to perform rolling updates — updating a tool while keeping it available. Moreover, it is possible to configure Load Balancers, subnet, and even secrets through declarations.

Computing resources can occasionally remain idle; the main goal is to avoid excess, such as containing cloud environment costs. A good way to reduce idle time is to use namespaces as a form of virtual cluster inside your cluster. Each namespace is a completely isolated space inside Kubernetes, which means several environments can be created as necessary, such as production environments or staging environments. Services within a namespace will receive a DNS name such as <service-name>.<namespace-name>.svc.cluster.local. This means that services within the same namespace just need to make a request to another service using a service name.

K8s can be deployed in very different scenarios depending on the size of the company and its objectives:

• In-house: Organizations can transform their own data center into a K8s cluster. In this case, companies can take full advantage of their own resources.
• Cloud: The setup process is similar to an in-house deployment, but includes virtual machines on the cloud. This allows for the creation of a virtually infinite number of machines, depending on demand.
• Hybrid: An organization’s data center might perform well for most of the day, but sometimes a peak occurs that local computing resources cannot handle. In this case, a hybrid solution works well. When necessary, K8s will create virtual machines on the cloud to better distribute computing resources when on-premise servers are full.
• On-premise: Some cloud providers have their own K8s implementation embedded. In this case, there is no need to deploy and configure Kubernetes itself; an organization just needs to manage the service. Since deploying Kubernetes can be tricky, this is a good solution for companies that do not have a big IT team capable of handling cluster configuration and maintenance.
• Multicloud: This is the next level of a hybrid cloud solution. Computing resources are deployed among two or more cloud vendors. In this case, companies need to avoid vendor lock-in and minimize risk if something goes wrong.

Kubernetes is not the only container orchestrator available. Other popular tools on the market include Docker Swarm and Apache Mesos. Swarm is an open-source container orchestrator intended to be the “big brother” of Docker and Docker Compose. Swarm uses the same command line from Docker and is not very opinionated: organizations must decide which tools to use for nearly every feature needed on their cluster. Apache Mesos is another open-source orchestrator that manages other technologies in addition to managing containers. Apache Mesos calls itself a “data center operating system.” This is also the name of its commercial product, Mesosphere‘s Data Center Operating System (DC/OS). Apache Mesos is much less opinionated than K8s, allowing for the deployment of various types of applications besides containerized applications.

VSCO Use Case:

Challenge

After moving from Rackspace to AWS in 2015, VSCO began building Node.js and Go microservices in addition to running its PHP monolith. The team containerized the microservices using Docker, but “they were all in separate groups of EC2 instances that were dedicated per service,” says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: “That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the AWS EC2 instances.”

Solution

The team began exploring the idea of a scheduling system and looked at several solutions including Mesos and Swarm before deciding to go with Kubernetes. VSCO also uses gRPC and Envoy in their cloud-native stack.

Impact

Before deployments required “a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish,” says Senior Software Engineer Brendan Ryan. “We didn’t really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way.” There’s a faster onboarding process now. Before, the time to first deploy was two days’ hands-on setup time; now it’s two hours. By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Adds Gattu: “In man-hours, that’s one person versus a developer and a DevOps individual at the same time.” With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year. There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company’s EC2 bill. Ryan points to the company’s ability to go from managing one large monolithic application to 50+ microservices with “the same size developer team, more or less. And we’ve only been able to do that because we have increased trust in our tooling and a lot more flexibility, so we don’t need to employ a DevOps engineer to tune every service.” With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages.

After VSCO moved to AWS in 2015 and its user base passed the 30 million mark, the team quickly realized that set-up wouldn’t work anymore. Developers had started building some Node and Go microservices, which the team tried containerizing with Docker. But “they were all in separate groups of EC2 instances that were dedicated per service,” says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: “That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the EC2 instances.”

With a checklist that included ease of use and implementation, level of support, and whether it was open source, the team evaluated a few scheduling solutions, including Mesos and Swarm, before deciding to go with Kubernetes. “Kubernetes seemed to have the strongest open-source community around it,” says Lu. Plus, “We had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes.”