How Swiggy migrated its k8s workload to Graviton

Published in

Swiggy Bytes — Tech Blog

8 min readSep 4, 2023

Swiggy, India's leading food ordering and delivery platform, is always looking for ways to optimize its infrastructure and reduce costs without compromising on availability and performance. Recently, Swiggy turned to AWS Graviton, a custom-built processor designed for cloud-native applications, and found it to be a game-changer.

In this blog, we’ll take a closer look at how Graviton is helping Swiggy achieve its goals and how you too can benefit from this technology.

Why did we choose the Graviton machine for our K8s workload

At its core, Graviton is a custom-built processor designed for cloud-native applications. It is based on the Arm architecture and is optimized for performance, scalability, and cost-effectiveness. Graviton instances are available on Amazon EC2 and provide up to 40% better price-performance compared to x86-based instances. This makes it an ideal choice for workloads that require high performance at a lower cost.

Cost benefits: We run 10 Kubernetes clusters for serving our production and non-production workload that contains around 400+ EC2 instances(c5a.16xlarge, c5a.24xlarge, r5a.16xlarge.. etc.) which cost us approximately 13% of K8s worker ec2 on entire Infra cost. With Graviton, the cost was 10% cheaper as compared to current x86(c6a) machines.

Performance: Before the k8s graviton migration, we already migrated our HAProxy server on Graviton and we noticed significant changes in CPU utilization (around 60%) and desired capacity(around 55%) which resulted in 66% dollar savings.

CPU Utilisation After Graviton Migration

One-click Developer-friendly Migration:

We’ve created a pipeline where multiple teams can easily onboard their service on graviton with minimum change.

How can teams onboard their service on graviton?
We’ve created a pipeline their team needs to define whether they want to use the multiArch feature or not. If yes then they would need to make some fundamental changes on Dockerfile(like base image, MultiArch compatible version, etc.) Then our pipeline can create a MultiArch build and deploy it on Graviton worker nodes.
How to have controlled 650+ microservices on graviton?
We’ve created an Apollo Antipattern check that helps us block developer PR if they’re not migrating on Graviton.

Key considerations during Graviton migration:

For the Graviton migration journey, we’ve categorized the story task into the below points which can be considered as step to step process for graviton migrations.

Cluster compatibility:

We run multiple Addons on our k8s cluster like Istio-proxy, Tile38 sidecars and Promtail, Fluentd, node-exporter, etc. as daemonsets.
so our first primary goal was to upgrade all addons with MultiArch images to schedule pods on any kind of worker nodes. We’ve upgraded below K8s addons for MultiArch compatibility.

Istio Upgrade from v1.10.4 to v1.15.0
Node Exporter upgrade from v0.17.0 to v0.18.0
RBAC-proxy upgrade from v0.4 to v8.0

Build MultiArch Image:
We run around 750+ micro-services on production and creating MultiArch images for all the services was challenging. So, we created an automation pipeline where teams can easily onboard their service on our build platform by integration through webhook with some custom YAML files in their repository.
In the backend, we’ve used the docker buildx utility with AWS Codebuild and code pipeline.
Code snippet for Buildx steps:

docker run --privileged --rm tonistiigi/binfmt --install all
docker buildx create --use
docker buildx build --progress plain --platform linux/amd64,linux/arm64 -t <repo_name> -f Dockerfile --push .

Dockerfile Modification:

If the application is pulling a specific architecture image or specific architecture package, then we modify our Dockerfile for MultiArch CI compatibility.
Apart from this, cross-validate version compatibility with their GitHub releases like whether they publish arm binary or not.
For example grpc-health-probe v3.0.0 does not support arm binary but v3.0.1 supports arm binary.

Example: If the Dockerfile contains the below code, then Dockerfile modification is needed.

FROM ubuntu:18.04
RUN wget -qO /bin/grpc_health_probe <https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64>
RUN wget <https://github.com/prometheus/prometheus/releases/download/v2.37.1/prometheus-2.37.1.linux-amd64.tar.gz>

Modify DockerFile with below code snippet.

FROM ubuntu:18.04
ARG TARGETOS TARGETARCH
RUN wget -qO /bin/grpc_health_probe <https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-${TARGETARCH}>
RUN wget <https://github.com/prometheus/prometheus/releases/download/v2.37.1/prometheus-2.37.1.linux-${TARGETARCH}.tar.gz>

Push community release container on ECR:

For getting MultiArch images from ECR we found 2 ways:-

AWS ECR PULL THROUGH CACHE: It’s very easy to configure and get MultiArch Image but it has some limitations like you can pull images only from ECR publicand quay.io. It didn’t support DockerHub but AWS recently announced Docker Official images are available on ECR Public
Manual MultiArch Image Creation: There are some exceptions to MultiArch Images like rbac-proxy, opa, tensorflow-serving that you can’t find on ECR publicand quay.io For that, you’ll need to create manually a multiArch image(If you want to use a DockerHub image these steps are not required ⇒ we followed this because we only use private ECR images on our infra).
Example: pull both Arch images based on SHA digest and tag them based on architecture like rbac-proxy:v8.0-amd64and rbac-proxy:v8.0-arm64and push them on ECR. now you’ll need to use another docker command for merging them.

#pull and tag amd image
docker pull ${repo}:${AMD_DIGEST}
docker tag ${repo}:${AMD_DIGEST} ${repo}:${version}-amd64

#pull and tag arm image
docker pull ${repo}:${ARM_DIGEST}
docker tag ${repo}:${ARM_DIGEST} ${repo}:${version}-arm64

#create multiArch manifest
docker manifest create ${repo}:${version} ${repo}:${version}-arm64 ${repo}:${version}-amd64

ECR response after pushing MultiArch image on ECR.

Deployment patch using kubemod:

We’ve used kubemod operator for initial pod scheduling on graviton worker nodes. it’s easy to configure on a cluster as it has simple YAML configuration files.
Kubemod would be a very useful utility if you have a small workload on your cluster.
We’ve used our internal deployment API for deployment pipeline as Kubemod triggers based on event-driven processes and it can decrease scheduling performance if we have 1000+ rules.
example kubemod config:

apiVersion: api.kubemod.io/v1beta1
kind: ModRule
Metadata
name: demo-app
  namespace: kubemod-test
spec:
  executionTier: 0
  match:
  - matchFor: Any
    matchValue: Deployment
    select: $.kind
  - matchFor: Any
    matchValue: demo-app
    select: $.metadata.name
  patch:
  - op: add
    path: /spec/template/spec/affinity
    value: |-
      nodeAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - preference:
            matchExpressions:
            - key: nodegroup-type
              operator: In
              values:
              - graviton-nodes
          weight: 70
        - preference:
            matchExpressions:
            - key: nodegroup-type
              operator: In
              values:
              - amd-worker-nodes
          weight: 30
  - op: add
    path: /spec/template/spec/tolerations/0
    value: |-
      key: graviton-nodes
      effect: NoSchedule
  type: Patch

Technical Key Challenges during Migration:

[Application-specific issues and debugging]

Deprecating the previous k8s API:

Be cautious while deprecating the previous version of Istio because both( v1.10 and v1.15) versions of Istio use the same crds otherwise you’ll end up deleting all envoyfilters.networking.istio.io resources and Use canary mode for Istio upgrade.

Rollback Plan for ECR Images:

Don’t keep the same tags for your previous Vs MultiArch Image ⇒ otherwise you won’t have any backups for rollback.

Docker Buildx :

Buildx Image on local machine: Buildx can only store a single Architecture on your local machine with --loadflag. It won’t store the MultiArch build on local storage so each time you would need to store images either on ECR or Dockerhub.
Buildx build-in env: Buildx has some In-build env which you can use for your Dockerfile like BUILDPLATFORM, TARGETOS, TARGETARCH
--platform=$BUILDPLATFORM⇒ we can ignore this as sometime it don’t resolve(for most of the java application in our use-case)

MultiArch Kafka builds issue for GoLang applications:

If you’re using Golang and if it’s using. confluentic/confluent-kafka-go or edenhill/librdkafkapackage then kindly upgrade with v1.9.1 or higher for multi-arch support.
we use confluentic/confluent-kafka-go Library for Golang services but the build was failing while building MultiArch images and throwing errors like undefined:kafka.Stats, undefined:kafka.Message.Then we’ve Manually installed librdkafka inside our DockerFile and used CGO_ENABLED=1and -tags=musl,dynamicflags during go build ⇒ it internally uses librdkafka++.so. files for some internal mapping of Kafka variables.
muslis Required for Alpine image and dynamicdynamic is required for Debian Image

RUN LIBRDKAFKA_VERSION=1.9.2 && \\
curl -Lk -o /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz <https://github.com/edenhill/librdkafka/archive/v${LIBRDKAFKA_VERSION}.tar.gz> && \\
tar -xzf /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz -C /root && \\
cd /root/librdkafka-${LIBRDKAFKA_VERSION} && \\
./configure --prefix /usr --install-deps && make -j && make install && \\
rm -rf /root/librdkafka-{LIBRDKAFKA_VERSION}.tar.gz && \\
rm -rf /root/librdkafka-{LIBRDKAFKA_VERSION}
RUN CGO_ENABLED=1 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -v -tags=musl,dynamic -ldflags="-w -s" -o app cmd/main.go

Kafka Run issue for Java Service:
Update Kafka or any failure dependency from compile to implementation in build.gradlefile.
example: if you have Kafka dependency as below then file change is required

 // current kafka dependency
compile(“org.apache.kafka:kafka-clients:7.0.1-ccs”)

//update with below code snippet.
implementation ("org.apache.kafka:kafka-clients:7.0.1-ccs")

Benefits:

Reducing Costs with Graviton:

Swiggy was able to reduce its infrastructure costs by around 30%, by using Graviton instances. This was possible because Graviton instances offer a higher price-performance ratio compared to x86-based instances. Additionally, Graviton instances consume less power, which further reduces costs. Swiggy was also able to take advantage of Graviton’s support for multi-arch Docker images, which allowed them to run their services as container workloads, leading to further cost savings.

Improving Availability with Graviton:

Swiggy operates in a highly competitive market where availability and performance are critical to its success. Graviton has helped Swiggy improve availability by providing a more resilient infrastructure. Graviton instances are designed to be fault-tolerant, and Swiggy has been able to leverage this feature to ensure that their services are always available to their customers. Additionally, Graviton instances support the latest virtualization technologies, such as KVM, which helps to improve the reliability of their infrastructure.
MultiArch containers provided us flexibility towards scheduling our workload on different kinds of machines like x86 or arm64 machines for maintaining our 99.96% SLA at a cheaper price.

Less Pod count after graviton migration for GoLang service:

We’ve seen ~ 18–20% decrease in pod count during night and afternoon peak. Previously, we were running around 52 pods during night peak which has decreased to 42 pods after the graviton migration.

Less number of pods during service high peak post-Graviton Migration

Request/pod after graviton migration for GoLang service:

We’ve seen a 15% performance increase on request/pod. Previously, we were serving around 10.8k req/pod which increased 12.5k req/pod post-graviton migration.

Conclusion:

In conclusion, Swiggy’s experience with Graviton is a testament to the performance, scalability, and cost-effectiveness of this custom-built processor. By using Graviton instances and taking advantage of its support for multi-arch Docker images, Swiggy was able to reduce costs and improve availability without compromising on performance.

References: