Category: Ketch

DevOps Challenge – Kubernetes Deployment: Ketch vs YAML

DevOps Challenge – Kubernetes Deployment: Ketch vs YAML

Ketch (https://www.theketch.io) is an open-source application delivery framework for Kubernetes.

YAML is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted. Customized YAML files are used by many for the complicated task of deploying to Kubernetes.

The Race

In this special event, we put Ketch head-to-head against the traditional/manual way of deploying to Kubernetes: creating and executing YAML files. Who will have the fastest Kubernetes Deployment?

Kubernetes Ketch vs YAML

The Competitors

Bruno Andrade, CEO and Founder of Shipa.io (https://www.shipa.io) is deploying with Ketch.

Bruno is CEO and Founder of Shipa. He has led distributed systems design and implementation at enterprises across different industries, defining methodologies, concepts and architecture and has dedicated the last few years to working on the Cloud Native space helping companies through a successful journey.

Ed Keen, Director of Cloud and DevOps at Burwood Group (https://www.burwood.com) is deploying with YAML.

Ed has broad experience across software development, operations, and cloud computing. He leads Burwood’s Google and Azure cloud practices, encompassing cloud enablement, application modernization, and migration. In complementary fashion, Ed leads Burwood’s DevOps tooling and practices to help Burwood customers evolve from traditional infrastructure to a software-defined approach to technology delivery. 

Jim Shilts, Developer Advocate at Shipa.io is the event moderator and race official.

Jim is a seasoned community builder and DevOps advocate. Jim founded North American DevOps Group (NADOG) in Southern California currently with over 30,000 members strong. Jim continues to focus on the importance of sharing and collaboration at an individual and a global community level.

Watch the race for the fastest Kubernetes Deployment below:

Download Ketch Today

Ketch fixes the last mile deployment, implements an application context, and removes the complexities of cloud-native application deployments

CI Pipeline and Deploying Applications to Kubernetes

Deploying Applications to Kubernetes from your CI Pipeline:

Now that we know the advantages of leveraging Ketch over other tools such as Helm to ease the deployment of our applications (BLOG: Helm vs. Ketch when Deploying Applications), a good next step is for us to understand how we can tie Ketch to our CI pipeline and have an automated deployment process. For this example, we will leverage Ketch, GitHub Actions, and a Kubernetes cluster on Google Kubernetes Engine (GKE).

GitHub Actions are event-driven, meaning that you can run a  series of commands after a specified event has occurred, which is excellent when automating your application deployment with Ketch as you can leverage Ketch across the different steps of your application deployment lifecycle.

For this example, we assume you already have:

Here is what we will deliver as part of this post:

Kubernetes Ketch CI Environment

GitHub: We will leverage the same podinfo application we used in our previous post. As explained before, we removed all code and files related to Helm Chart and Kustomize from the original podinfo application source code as Ketch won’t need those to deploy the application. The source code can be found here: https://github.com/brunoa19/podinfo.

GitHub Actions: We will leverage GitHub actions for:

  • Building our image based on the Dockerfile provided as part of the source code.
  • Pushing the built image to Google Container Registry
  • Creating a Ketch pool
  • Creating and deploying our podinfo application

GKE: Even though you can use Ketch with pretty much any available Kubernetes provider, for this example, we will use Google’s Kubernetes Engine.

GitHub Secrets

We will use the gcloud CLI to fetch the kubeconfig from our GKE cluster, where we will deploy our podinfo application. To store the information required to create the cluster config and deploy our application to our cluster, we will use GitHub Secrets.

Let’s create our secrets! To get started, go to the “Settings” tab in your project and click “Secrets” in the sidebar. Click on “New repository secret.

GitHub Secrets

You will need to create the following secrets:

  • GKE_PROJECT: The name of your project where your GKE cluster is located in your Google cloud account
  • GKE_SA_KEY: The service account used for the project with the Base64 encoded JSON service account key. More info available on https://github.com/GoogleCloudPlatform/github-actions/tree/docs/service-account-key/setup-gcloud#inputs 
  • KETCH_APP: The name of the application we want Ketch to use when deploying our podinfo application
  • KETCH_INGRESS_SP: The ingress service endpoint that should be used by Ketch when creating the pool so Ketch can automatically provision an endpoint for our podinfo application during the deployment
  • KETCH_POOL: The name of the pool we want Ketch to create to deploy our application. Not that Ketch will create a namespace in your cluster called ketch-poolname. If you want Ketch to use an existing namespace in your cluster, in addition to setting up this secret, you need to add the –namespace flag in the CI pipeline configuration.

Using the GitHub secrets in your workflow is relatively straightforward. Each secret gets added as an environment variable prefixed with “secrets”, which means we can easily use them when creating our config file.

Pipeline Settings

Pipeline Settings GitHub

In this example, we will use our actions when there are either push or pull requests on our “main” branch. Our pipeline config file is stored inside .github/workflows directory, using a file called gke.yml

While we are using the PROJECT_ID secret that we have set before, we are also pre-setting variables such as GKE_CLUSTER, GKE_ZONE, and DEPLOYMENT_NAME. If you want to further automate your pipeline, you can also set these as Secrets on GitHub instead of hardcoding it in the pipeline config file.

Setting Up gcloud

Setting Up gcloud

Using the steps above, we are:

  • Setting up the gcloud CLI
  • Configuring Docker to use the gcloud CLI tool as a credential helper for authentication
  • Getting the GKE credentials and kubeconfig so we can deploy to our GKE cluster

The steps above build our podinfo image using the Dockerfile present in the repository and, once built, storing the image in our Google Container Registry.

Setting Up Ketch and Deploying Our Application

Setting Up Ketch and Deploying to Kubernetes

Following the steps above, GitHub Actions will first setup Ketch by downloading Ketch’s v0.1.1 CLI and allowing it to be executed.

With Ketch’s CLI downloaded, our pipeline will then:

  • Create a Ketch pool (Creating a new or using an existing Kubernetes namespace) and assign an ingress service endpoint to it.
    • For this pool, we are assigning Istio as the ingress type, but you can also use Traefik instead, as described in Ketch’s documentation (https://learn.theketch.io/docs/pool-management#creating-pools )
    • We are assigning a cluster issuer to our Ketch pool. Later, if we want to create and assign HTTPS CNAMEs to our applications, Ketch will automatically generate the certificates and apply them to our CNAMEs. You can find more detailed information about CNAME management using Ketch here: https://learn.theketch.io/docs/cname-management.  
  • Create our podinfo application and assign it to the previously created Ketch pool.
  • Deploy our podinfo application pulling the image that we just stored on Google Container Registry as explained above
    • Note that during deployment, we are also passing a file called ketch.yaml, which is in our podinfo source code repository and is used to expose specific ports in Kubernetes.

Source Code

You can find the complete GitHub Action code used for this example here:

https://github.com/brunoa19/podinfo/blob/main/.github/workflows/gke.yml

In the next blog post, we will cover how you can easily automate the deployment of applications with multiple services using Ketch, GitHub Actions, and Kubernetes. As always, Ketch’s object is to allow you to continue focusing on your application code rather than Kubernetes objects, charts, and others.

Download Ketch Today!

Ketch fixes the last mile deployment, implements an application context, and removes the complexities of cloud-native application deployments

Ketch is an open-source application delivery framework created by Shipa that makes it extremely easy for developers to deploy and manage applications on Kubernetes using a simple command-line interface. No YAML is required!

Helm vs. Ketch when Deploying Applications

Kubernetes has become the de-facto standard for deploying microservices and containerized applications. Still, there is a learning curve for a developer to get familiar with Kubernetes concepts and objects, how to write and manage the required YAML files, etc. While there is undoubtedly value in learning these concepts and tasks, I believe there is even greater value in getting your applications deployed faster and spending more time on your application code than on infrastructure-related objects.

That’s where Ketch (https://www.theketch.io) comes in. Ketch focuses on easing application deployments for developers so they do not have to spend time creating Helm charts, Kustomize, YAML, etc, freeing them up to get their applications deployed and running quickly.

This blog post will cover the deployment of a sample application using Ketch vs. using Helm or Kustomize. For the sample application, we will use one called PodInfo, a Golang web application that showcases practices for running microservices in Kubernetes, developed by Stefan Prodan and found here: https://github.com/stefanprodan/podinfo.

** Note that we will not cover how you can install Ketch in this blog post. If you need more information on that, you can quickly get started with the instructions available on https://learn.theketch.io/docs/getting-started

For this example, you can find a pre-built Helm Chart and the Kustomize scripts, but assuming you had to build these for your application to get deployed, here is the structure you would have to create for your Helm charts:

For this example, you can find a pre-built Helm Chart and the Kustomize scripts, but assuming you had to build these for your application to get deployed, here is the structure you would have to create for your Helm charts:

This is the sample structure you would use for Kustomize:

This is the sample structure you would use for Kustomize:

To create these, you would have to have a good knowledge of crucial Kubernetes concepts, spend time building, and test it. Again, there is a great advantage in learning what each of those files do and what their roles are, but as a developer, it’s also a significant advantage if you can focus less on these objects and more on your application code; this increases the experience when developing and deploying applications on Kubernetes, and it also makes your application more portable and less dependent on infrastructure objects (which is what those YAML files are creating).

The complete folder structure for this project, including Kustomize, Helm charts, and others, can get pretty complex:

The complete folder structure for this project, including Kustomize, Helm charts, and others, can get pretty complex:

If you deployed the same application using Ketch, the project structure would be simpler and more focused on your application code than on the underlying cluster, objects, and charts. Since Ketch automatically creates the required objects for your application to run, you can remove both the Kustomize and charts folders since Ketch won’t need to leverage any of these to deploy your application.

Ketch for Kubernetes Architecture

Let’s see how we can quickly deploy the same application in an Amazon Elastic Kubernetes Service (EKS) cluster using Ketch and without the need to create and maintain Kubernetes related objects. For this example, we assume you already have an EKS cluster running and already installed Ketch (more details on how to get started with Ketch here: https://learn.theketch.io/docs/getting-started )

With Ketch installed, let’s start by adding a pool where we will deploy our application. Pools are created using the ketch pool add command, and once created, Ketch automatically configures a Kubernetes namespace, ingress controller parameters, and default templates for the application.

Kubernetes namespace, ingress controller parameters, and default templates for the application.

We can create our pool using the following command:

$ ketch pool add podinfo --ingress-service-endpoint ab2e08f4a7d944f2cab906edb996a8df-115895280.us-east-1.elb.amazonaws.com --ingress-type istio --cluster-issuer le

** Note that I am using a flag called –cluster-issuer when creating the pool, which will allow Ketch to later generate a certificate for HTTPS CNAMES. This is an optional flag in case you want Ketch to auto-generate certs for you; the name used should be the one you used when implementing your Kubernetes cluster-issuer.

** Also note that when creating a pool in Ketch, you will need to use the ingress service endpoint as part of the command and the ingress type. In our case, we are using Istio. You can find your ingress service endpoint by running the following command:

kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

With our pool created, we can now create our application, which will then be used to deploy our code. You can create the application by running the following command:

$ ketch app create podinfo --pool podinfo

Now that we have both our pool and application created, we can deploy our application. For the pod info, we will need to expose the following ports:

Ketch for Kubernetes PodInfo ports

To expose these ports, we will need to create a file called ketch.yaml, which will be used to deploy our application and dynamically provision these. Click here to download an example of the ketch.yaml file: https://github.com/brunoa19/podinfo/blob/main/ketch.yaml

With our ketch.yaml file in place, we can now deploy our podinfo application. To simplify the process, we can use the pre-built Docker image and the following command:

$ ketch app deploy podinfo -i stefanprodan/podinfo --ketch-yaml ketch.yaml

As part of the Kubernetes deployment process, Ketch will automatically create the endpoint for our podinfo application using the Istio ingress controller tied to our pool. You can find the application endpoint using the following command:

$ ketch app list

If we try accessing the address endpoint assigned by Ketch, we get the podinfo application UI:

PodInfo UI

If we try calling some API endpoints, we get this:

Ketch for Kubernetes API endpoints

And now, we have a much simpler and faster deployment of the podinfo application, and we shift the focus more and more to our application code rather than on Kubernetes related objects. 

Some might say, “but I have so much flexibility with Helm and wanted to customize some parameters, and Ketch can block me from being able to do that.” Not really! 

With the application deployed, Ketch can automatically generate a Helm chart for you to make any required customizations to your app, and speed up creating a Helm chart instead of doing it manually. You can do that by running the following command:

ketch app export podinfo -d helm

If we check the content of the folder, we can see that Ketch generated our application’s charts that we can leverage for any customization:

If we check the content of the folder, we can see that Ketch generated our application’s charts that we can leverage for any customization:

If you have control over your DNS and want to add a CNAME to the application deployed by Ketch, you can do that as well by using the following command:

ketch cname add podinfo-demo.theketch.io -a podinfo

With that, you can now access your podinfo using the CNAME defined.

With that, you can now access your podinfo using the CNAME defined.

In the next blog post, we will cover how to tie Ketch to your CI pipeline and have a complete workflow to deploy our podinfo application from Git all the way to our EKS clusters in a faster, dynamic, and more developer-friendly way.

Ketch Launch Announcement

Deploy cloud native applications to Kubernetes without a single YAML file

A Developer Friendly Approach to Deploying Faster on Kubernetes

The Pipeline: All things CI/CD & DevOps Podcast by The CD Foundation

On November 13th, 2020, Continuous Delivery Foundation (https://cd.foundation/) invited Shipa Founder, Bruno Andrade to join The Pipeline podcast to talk about the newly released open source project, Ketch (https://www.theketch.io/), for a developer-friendly approach to deploying on Kubernetes. Listen to the full conversation below.

Summary

What is Ketch?

Ketch is an application delivery framework that focuses on fixing the last mile deployment for applications on Kubernetes, helping to improve the developer experience. A lot of people talk about day-2 operations in regards to the delivery of apps, and how do you operate them later?

One challenge is that many developers get stuck on building Kubernetes objects, and simply trying to understand the platform. Shipa (https://www.shipa.io/) is abstracting the application delivery portion of its broader application management framework, and offering to the community as an open source project, Ketch.

Why Open Source?

The Shipa platform focuses on helping users build an enterprise framework for applications being delivered on Kubernetes across multiple clusters, multiple teams, etc.

Shipa saw an opportunity to abstract the application delivery component from Shipa, offer it as an open source project to the community, and help them to get further along in their journey and begin to mature to the point where they are ready for a broader application management framework for the enterprise.

Who Benefits?

Depending on the organization, many members of the team can benefit from using Ketch, developers and platform engineers. Developers can now focus on delivering quality applications and updates, and not on trying to understand how to build their applications, how to build the objects, what an ingress rule is, what a deployment set is, and so on.

As a platform engineer, Ketch can help you to implement boundaries and controls for your developers, create limits, manage ingress configurations, and so on. Take that load away from the developers and put the power into the hands of the platform engineers

Join the Ketch Community!

Getting started with Ketch only takes a few minutes

Shipa Open Sources Ketch

Deploy cloud native applications to Kubernetes without a single YAML file

Shipa, Corp., delivering a cloud native application management framework built to manage the full application lifecycle, today announced that it is open sourcing Ketch, Shipa’s deployment engine for Kubernetes, under Apache License Version 2.0. This open source release is available on GitHub and follows the general availability launch of Shipa’s full application management framework in October. Shipa is funded by Engineering Capital and Jump Capital; advisors include Google’s Kelsey Hightower, Mastercard’s Ken Owens, and Lyft’s Matt Klein

Ketch Architecture

Ketch makes it extremely easy to deploy and manage applications on Kubernetes using a simple command-line interface. No Kubernetes object YAML is required.​

Developers and DevOps teams interested in learning more about Ketch can get started here. Shipa’s deployment engine has already been highlighted as ITOps Times’ Open Source Project of the Week, which noted that the application-centric deployment engine “bridges the gap between continuous integration tools and the production environment, significantly reducing the need for custom Kubernetes scripts, Helm charts and YAML files.”

Using Ketch, application developers can manage the entire deployment process at the application level. Developers can stay focused on writing code and do not need any Kubernetes expertise to successfully deploy applications running on Kubernetes. As a result, teams can accelerate the time needed to adopt Kubernetes, while simultaneously increasing their pipeline’s resilience and reducing the compounding risk with each new deployment.

"Ketch is the perfect way to keep developers’ focus on code – not the infrastructure"​

“By open sourcing Ketch, the cloud native community can easily try and leverage this powerful cloud native application deployment engine,” said Bruno Andrade, Founder and CEO of Shipa. “Enterprises continue to struggle with the app deployment process on Kubernetes. Ketch is the perfect way to keep developers’ focus on code  not the infrastructure  as they scale with Kubernetes. Shipa’s vision is to manage the full application lifecycle, from code to post-deployment. With Ketch now available as a fully open source technology, there’s no faster and easier way for the community to get started.”

“Shipa is committed to open standards, application deployment and management, and the communities that support them,” said Henrik Rosendahl, COO, Shipa. “The future of application deployment is open. That’s why we are taking our significant investment toward improving and simplifying application deployment and offering it to the open source community – and committing engineers to support it.”

Shipa is a Silver member of the Cloud Native Computing Foundation and a General member within the Continuous Delivery Foundation.

“Companies like CD Foundation member Shipa are a great example of important products and services that help expand the CI/CD ecosystem and contribute to the vibrant ecosystem for both customers and community members to get involved in,” said Tracy Miranda, Continuous Delivery Foundation Executive Director. “CI/CD continues to expand outward beyond the software development industry, and open sourcing is an excellent path to broadening adoption and providing transparency and accessibility.”

Additional Information and Resources

About Shipa

Shipa delivers a cloud native application management framework built for the full application lifecycle. Using Shipa, organizations speed up cloud native application development by eliminating persistent workflow inefficiencies. For developers, Shipa provides an application-centric way to develop, deploy, and manage cloud native applications without requiring Kubernetes expertise. For platform and DevOps engineers, Shipa’s lightweight framework eliminates the need to develop custom scripts or manage a lengthy migration – while still providing centralized control over configurations. Shipa is headquartered in Santa Clara, California.