Detailed Explanation of Pod Controller – Canary Release

Learning k8s from Scratch

Canary Release

The Deployment controller supports controlling the update process, such as “pause” or “resume” update operations.

For example, after a batch of new Pod resources is created, the update process is immediately paused. At this point, only a portion of the new version of the application exists, while the majority is still the old version. Then, a small portion of user requests is routed to the new version of the Pod application to continue observing whether it can run stably as expected. Once confirmed, the remaining Pod resources will continue to roll out updates; otherwise, the update operation will be rolled back immediately. This is known as a Canary Release (Canary Deployment).

# View rs
kubectl get rs -n dev
NAME                     DESIRED CURRENT READY  AGE
pc-deployment-545fbd7bb5   0        0      0    21m
pc-deployment-7d9bdf9475   3        3      3    37m
pc-deployment-9d4dbb5bd    0        0      0    32m
# Update the deployment version and configure to pause the deployment
kubectl set image deploy pc-deployment nginx=nginx:1.17.4-alpine -n dev &kubectl rollout pause deployment pc-deployment -n dev
deployment.apps/pc-deployment image updated
deployment.apps/pc-deployment paused
# Again, check rs, the old rs pods are not deleted, the new rs has been created and has 1 pod
kubectl get rs -n dev
NAME                     DESIRED CURRENT READY  AGE
pc-deployment-545fbd7bb5    0       0      0    22m
pc-deployment-5bc448d66     1       1      1    26s
pc-deployment-7d9bdf9475    3       3      3    38m
pc-deployment-9d4dbb5bd     0       0      0    34m
# Observe the update status, updating 3, already updated 1
kubectl rollout status deploy pc-deployment -n dev
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated.
# Monitor the update process, you can see that a new resource has been added, but an old resource has not been deleted as expected, because the pause command was used
kubectl get rs -n dev -o wide
NAME                      DESIRED CURRENT READY  AGE  CONTAINERS        IMAGES                  SELECTOR
pc-deployment-545fbd7bb5     0       0      0    24m     nginx   nginx:1.17.3-alpine    app=nginx-pod,pod-template-hash=545fbd7bb5
pc-deployment-5bc448d66      1       1      1    2m30s   nginx   nginx:1.17.4-alpine    app=nginx-pod,pod-template-hash=5bc448d66
pc-deployment-7d9bdf9475     3       3      3    40m     nginx   nginx:1.17.1-alpine    app=nginx-pod,pod-template-hash=7d9bdf9475
pc-deployment-9d4dbb5bd      0       0      0    36m     nginx   nginx:1.17.2-alpine    app=nginx-pod,pod-template-hash=9d4dbb5bd
# Here is a new pod, the old 3 pods are still there
kubectl get pods -n dev
NAME                           READY  STATUS   RESTARTS  AGE
pc-deployment-5bc448d66-l96hr  1/1    Running    0       3m32s
pc-deployment-7d9bdf9475-pnvhs 1/1    Running    0       22m
pc-deployment-7d9bdf9475-t8hzz 1/1    Running    0       22m
pc-deployment-7d9bdf9475-th6w4 1/1    Running    0       22m
# Ensure the updated pod is fine, continue updating
kubectl rollout resume deploy pc-deployment -n dev
deployment.apps/pc-deployment resumed
# Observe the update status, updating 3 completed
kubectl rollout status deploy pc-deployment -n dev
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated.
Waiting for deployment spec update to be observed.
Waiting for deployment spec update to be observed.
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated.
Waiting for deployment "pc-deployment" rollout to finish: 1 out of 3 new replicas have been updated.
Waiting for deployment "pc-deployment" rollout to finish: 2 out of 3 new replicas have been updated.
Waiting for deployment "pc-deployment" rollout to finish: 2 out of 3 new replicas have been updated.
Waiting for deployment "pc-deployment" rollout to finish: 2 out of 3 new replicas have been updated.
Waiting for deployment "pc-deployment" rollout to finish: 1 old replicas are pending termination.
Waiting for deployment "pc-deployment" rollout to finish: 1 old replicas are pending termination.
deployment "pc-deployment" successfully rolled out
# View the final update status
kubectl get rs -n dev -o wide
NAME                     DESIRED  CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-deployment-545fbd7bb5   0         0      0 28m nginx nginx:1.17.3-alpine app=nginx-pod,pod-template-hash=545fbd7bb5
pc-deployment-5bc448d66    3         3      3 6m9s nginx nginx:1.17.4-alpine app=nginx-pod,pod-template-hash=5bc448d66
pc-deployment-7d9bdf9475   0         0      0 44m nginx nginx:1.17.1-alpine app=nginx-pod,pod-template-hash=7d9bdf9475
pc-deployment-9d4dbb5bd    0         0      0 39m nginx nginx:1.17.2-alpine app=nginx-pod,pod-template-hash=9d4dbb5bd
kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pc-deployment-5bc448d66-b2xpc 1/1 Running 0 83s
pc-deployment-5bc448d66-l96hr 1/1 Running 0 6m25s
pc-deployment-5bc448d66-r2lz5 1/1 Running 0 84s

Summary:

Canary Release is to pause during the update process, allowing a portion of users to test the new version pod. If the test is successful, the remaining pods will continue to update, while the old version pods remain in service until the update is complete.

Delete Deployment

# Delete the deployment, its rs and pods will also be deleted
kubectl delete -f pc-deployment.yaml

Leave a Comment