Continuing from the previous study on Controllers, this section will focus on the practical aspects of Job and CronJob.
First, consider the following question:
-
Can we run task processes directly through Pods?
If we do this, how do we solve the following issues?
-
How to ensure that the processes within the Pod terminate correctly?
-
If a process fails, how do we retry?
-
How to manage multiple tasks with dependencies between them?
-
How to run tasks in parallel and manage their queue size?
A Job is used to manage tasks as a controller.
What can a Job help us with?
-
Creating one or more Pods to ensure that a specified number of Pods can successfully run to completion
-
Tracking the status of Pods and retrying failed Pods in a timely manner based on configuration
-
Determining dependencies to ensure that the next task runs only after the previous task has completed
-
Controlling task parallelism and ensuring the Pod queue size based on configuration
Let’s look at an orchestration example:
# cat job.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pi
spec:
template:
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
backoffLimit: 4Note the two points in the orchestration file:
-
restartPolicy: Restart policy
-
backoffLimit: Retry limit
Creating a Job:
# kubectl create -f job.yaml
job.batch/pi created
# kubectl get job
NAME COMPLETIONS DURATION AGE
pi 0/1 2m34s 2m34sParameter meanings:
NAME: Job name
COMPLETIONS: Number of completed Pods
DURATION: Actual runtime of the Job
AGE: Duration since the Job was created
Viewing Pods:
# kubectl get pod
NAME READY STATUS RESTARTS AGE
pi-h8j6g 1/1 Running 0 15sThe naming format for Pods is:
${job-name}-${random-suffix}
Viewing Pod configuration information:
# kubectl get pod pi-h8j6g -o yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2022-06-19T08:19:38Z"
generateName: pi-
labels:
controller-uid: e06e2be9-bbd9-4db9-b963-beebb1379dcc
job-name: pi
name: pi-h8j6g
namespace: default
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: pi
uid: e06e2be9-bbd9-4db9-b963-beebb1379dcc
resourceVersion: "1122231"
uid: bfdf6b5d-ceef-4788-87dd-9efd8dad63d1
spec:
containers:
- command:
- perl
- -Mbignum=bpi
- -wle
- print bpi(2000)
image: perl
imagePullPolicy: Always
name: pi
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-6h9zr
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: k8snode
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-6h9zr
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2022-06-19T08:19:38Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2022-06-19T08:20:05Z"
message: 'containers with unready status: [pi]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2022-06-19T08:20:05Z"
message: 'containers with unready status: [pi]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2022-06-19T08:19:38Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://91158655db8e050244e42e09aa7d96dab59c0c2d2241a1d9eb314b616b801ede
image: docker.io/library/perl:latest
imageID: docker.io/library/perl@sha256:d7fd57000721562918b96a687a216eddceb450436de4b89c2960fd7c5f2e34a7
lastState: {}
name: pi
ready: false
restartCount: 0
started: false
state:
terminated:
containerID: containerd://91158655db8e050244e42e09aa7d96dab59c0c2d2241a1d9eb314b616b801ede
exitCode: 255
finishedAt: "2022-06-19T08:20:05Z"
reason: Error
startedAt: "2022-06-19T08:19:42Z"
hostIP: 192.168.213.152
phase: Failed
podIP: 192.168.1.56
podIPs:
- ip: 192.168.1.56
qosClass: BestEffort
startTime: "2022-06-19T08:19:38Z"How to implement parallel Job execution?
# cat job-parallelism.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: paral
spec:
completions: 8
parallelism: 2
template:
spec:
containers:
- name: paral
image: centos
command: ["/bin/bash"]
args: ["-c", "sleep 30; date"]
restartPolicy: OnFailureNote the two parameters:
-
completions: Represents the number of times this Pod queue will execute
-
parallelism: Represents the number of parallel executions
Creating a parallel Job:
# kubectl create -f job-parallelism.yaml
job.batch/paral created
# kubectl get job paral
NAME COMPLETIONS DURATION AGE
paral 0/8 105s 105s
# kubectl get pods
NAME READY STATUS RESTARTS AGE
paral-2lklq 0/1 Completed 0 3m51s
paral-48d2t 0/1 Completed 0 5m8s
paral-7bfg5 0/1 Completed 0 7m2s
paral-hvvlh 0/1 Completed 0 4m31s
paral-ld7nl 0/1 Completed 0 7m2s
paral-mkq8n 0/1 Completed 0 3m53s
paral-nfrf2 0/1 Completed 0 4m30s
paral-vtj59 0/1 Completed 0 5m10sFrom the Pod results, we can see that based on AGE, two Pods were almost simultaneously created at the same time.
Now let’s look at how to implement scheduled tasks with CronJob:
# cat cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox
args:
- /bin/sh
- -c
- date; echo Hello from the k8s
restartPolicy: OnFailure
startingDeadlineSeconds: 10
concurrencyPolicy: Allow
successfulJobsHistoryLimit: 3Note the following parameters:
-
schedule: Follows the same format as crontab time
-
startingDeadlineSeconds: Maximum time to start the Job
-
concurrencyPolicy: Whether to allow parallel execution
-
successfulJobsHistoryLimit: Number of historical Jobs allowed to be retained
Creating a CronJob:
# kubectl create -f cronjob.yaml
cronjob.batch/hello created
# kubectl get cronjob
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
hello */1 * * * * False 0 44s 7m17sSummary of Job management modes:
-
The Job Controller is responsible for creating Pods based on configuration
-
The Job Controller tracks the Job status and retries Pods or continues to create them based on configuration
-
The Job Controller automatically adds labels to track corresponding Pods and creates Pods in parallel or serially based on configuration
If you find this article useful, please follow our WeChat public account – Full Stack Career
Click below to view for promotions and salary increases👇