How do I automatically remove completed Kubernetes jobs?

Question

Is there a way to automatically remove completed jobs besides making a cronjob to clean up completed jobs?

The K8s Job Documentation states that the intended behavior of completed jobs is for them to remain in a completed state until manually deleted. Because I am running thousands of jobs a day via k8s cronjobs and I don't want to keep completed jobs around.

Answer 1

You can now set history limits, or disable history altogether, so that failed or successful jobs are not kept around indefinitely. See my answer here. Documentation is here.

Answer 2

This is possible from version 1.12 Alpha with ttlSecondsAfterFinished. An example from Clean Up Finished Jobs Automatically:

apiVersion: batch/v1
kind: Job
metadata:
  name: pi-with-ttl
spec:
  ttlSecondsAfterFinished: 100
  template:
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never

Answer 3

I've found the below to work

To remove failed jobs:

kubectl delete job $(kubectl get jobs | awk '$3 ~ 0' | awk '{print $1}')

To remove completed jobs:

kubectl delete job $(kubectl get jobs | awk '$3 ~ 1' | awk '{print $1}')

Answer 4

As stated in the documentation "It is up to the user to delete old jobs", see http://kubernetes.io/docs/user-guide/jobs/#job-termination-and-cleanup

I would run a pod to do this cleanup based on job name and certain conditions, thus letting kubernetes at least take care of the availability of your process here. You could run a recurring job for this (assuming you run kubernetes 1.5).

Answer 5

I recently built a kubernetes-operator to do this task.

After deploy it will monitor selected namespace and delete completed jobs/pods if they completed without errors/restarts.

https://github.com/lwolf/kube-cleanup-operator

Answer 6

Using jsonpath:

kubectl delete job $(kubectl get job -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')

Answer 7

A simple way to delete them by running a cron job:

kubectl get jobs --all-namespaces | sed '1d' | awk '{ print $2, "--namespace", $1 }' | while read line; do kubectl delete jobs $line; done

Answer 8

i'm using wernight/kubectl's kubectl image

scheduled a cron deleting anything that is

• completed
• 2 - 9 days old (so I have 2 days to review any failed jobs)

it runs every 30mins so i'm not accounting for jobs that are 10+ days old

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cleanup
spec:
  schedule: "*/30 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: kubectl-runner
            image: wernight/kubectl
            command: ["sh", "-c", "kubectl get jobs | awk '$4 ~ /[2-9]d$/ || $3 ~ 1' | awk '{print $1}' | xargs kubectl delete job"]
          restartPolicy: Never