Clean images from your Docker registry

10 February 2018

If you have a free Docker Hub account probably you aren’t worry about how many images you have in it. In fact, probably you want to have all the history in case somebody are stell using some old version. But when you use it for a private repository it’s different because you pay in base number of images you have uploaded. So if you want to have the invoice under control you need to review your repository periodically in order to remove unnecessary images.

With this script we’ll have a report with the information of all images uploaded to Docker Hub. This script can be scheduled and it can send to you a report to know wich image can be remove using your own criteria (in this case we want to maintain only the last 4). If you want it can delete the tags for you.

Dependencies

Docker Hub has a secure REST service oriented to manage your repos, images, tags and so on. In this script we’ll use HttpBuilder-NG (https://http-builder-ng.github.io/http-builder-ng/asciidoc/html5) as the tool to do the requests.

Also to generate the report we’ll use Groovy’s markup capacities using MarkupBuilder for it (http://docs.groovy-lang.org/docs/groovy-latest/html/api/groovy/xml/MarkupBuilder.html)

@Grapes([
    @Grab(group='org.asciidoctor', module='asciidoctorj', version='1.5.6'),
    @Grab(group='io.github.http-builder-ng', module='http-builder-ng-core', version='1.0.3'),
    @Grab(group='org.slf4j', module='slf4j-simple', version='1.7.25', scope='test')
])
import static groovyx.net.http.HttpBuilder.configure
import static groovyx.net.http.ContentTypes.JSON
import groovy.json.*
import groovy.xml.MarkupBuilder

import org.asciidoctor.OptionsBuilder
import org.asciidoctor.AttributesBuilder
import org.asciidoctor.SafeMode
import org.asciidoctor.Asciidoctor.Factory

Configuration and arguments

REPORT=false //(1)
ASCIIDOC=true
DELETE=false //(2)
TRESHOLD=4  //(3)

user=args[0]        //(4)
password=args[1]    //(5)
namespace=args[2]   //(6)

If we want the report
If we want to delete automatically the images
Number of images to maintain
First argument is the username
Second argument the password
The namespace to inspect (usually is identical to the username)

Authorization

token = configure {  //(1)
    request.uri='https://hub.docker.com/'
    request.contentType=JSON[0]
    request.accept=['application/json']
}.post { //(2)
    request.uri.path='/v2/users/login'
    request.body=[username:user,password:password]
}.token //(3)

dockerHub = configure {   //(4)
    request.uri='https://hub.docker.com/'
    request.contentType=JSON[0]
    request.accept=['application/json']
    request.headers['Authorization']="JWT $token"
}

Configuration
send a JSON with GET method
If the answer is valid we can work directly with the return value
As the rest of calls are similar we can reuse the configuration

Repositories

repos=dockerHub.get { //(1)
    request.uri.path="/v2/repositories/$namespace"
    request.uri.query=[page_size:200]
}.results.collect{ repo->   //(2)
    [name:repo.name,lastUpdate:Date.parse("yyyy-MM-dd'T'HH:mm:ss",repo.last_updated)]
}.sort{ a,b->   //(3)
    a.name <=> b.name
}

We’ll create a lis of map with the results returned
Build a map only with the name and the last updated
Sort repos by name

Report

If we want the report, the script will write an HTML file ( i.e. Docker Report ) that you can send, for example, by email. The script use MarkupBuilder iterating accross the repos list and for every repo it iterate across every tag. It dump all tags marked as candidates to remove in a different section (in this case a H3 html tag)

    writer=new StringWriter()
    html=new MarkupBuilder(writer)
    html.html {
        head {
            title "Docker Hub Images reporting"
        }
        body(id: "main") {
            h1 id: "namespace",  "Repository $namespace"
            repos.each{ repo->
                div {
                    h2 "$repo.name (Last update:${repo.lastUpdate.format('yyyy-MM-dd HH:mm')})"
                    repo.tags.findAll{!it.toRemove}.each{tag->
                        p "$tag.name (Last update:${tag.lastUpdate.format('yyyy-MM-dd HH:mm')})"
                    }
                    if(repo.tags.find{it.toRemove} )
                       h3 "Candidates to remove"
                    repo.tags.findAll{it.toRemove}.each{tag->
                        p "$tag.name (Last update:${tag.lastUpdate.format('yyyy-MM-dd HH:mm')})"
                    }

                }
            }
        }
    }
    new File('docker_report.html').newWriter().withWriter { w -> w << writer.toString() }

Asciidoctor

Another posibility for the report is to use Asciidoctor (see this link as an example Docker Report en Asciidoctor ).

We create a file with a dynamic content using the asciidoctor sintax, iterating between repos and tags. If the tag has candidates to be removed we create a subsection who help to identify them.

    file = new File("/tmp/asciidoc_docker_report.adoc") //(1)
    file.newWriter().withPrintWriter { mainWriter ->
        mainWriter.println "= Docker Hub status"
        mainWriter.println "Jorge Aguilera <jorge.aguilera@puravida-software.com>"
        mainWriter.println new Date().format('yyyy-MM-dd')
        mainWriter.println ":icons: font"
        mainWriter.println ":toc: left"
        mainWriter.println ""
        mainWriter.println """
[abstract]
Report status of repository *${namespace}* at *${new Date().format('yyyy-MM-dd HH:mm')}*
        """

        repos.each { repo ->    //(2)
            mainWriter.println "\n== ${repo.name}"
            mainWriter.println "Last update: ${repo.lastUpdate.format('yyyy-MM-dd HH:mm')}"
            mainWriter.println ""

            repo.tags.findAll { !it.toRemove }.each { tag ->
                mainWriter.println "- ${tag.name} ${tag.lastUpdate.format('yyyy-MM-dd HH:mm')}"
            }
            if (repo.tags.find { it.toRemove }) {
                mainWriter.println "\n=== Candidates to remove"
            }
            repo.tags.findAll { it.toRemove }.each { tag ->
                mainWriter.println "- ${tag.name} ${tag.lastUpdate.format('yyyy-MM-dd HH:mm')}"
            }
        }
    }

    asciidoctor = Factory.create();
    attributes = AttributesBuilder.attributes().
            sectionNumbers(true).
            sourceHighlighter("coderay").
            get()
    options = OptionsBuilder.options().
            backend('html').
            attributes(attributes).
            safe(SafeMode.UNSAFE).
            get()
    asciidoctor.convertFile(file, options)  //(3)

New file with a custom header
Every repo has his own section
Call to Asciidoctor using the html backend

Clean

If you want the script can remove the obsoletes images calling the DELETE Rest method

    repos.each{ repo->
        repo.tags.findAll{it.toRemove}.each{ tag->
            dockerHub.delete {
                request.uri.path="/v2/repositories/$namespace/$repo.name/tags/$tag.name"
            }
        }
    }

Script

//tag::dependencies[]
@Grapes([
    @Grab(group='org.asciidoctor', module='asciidoctorj', version='1.5.6'),
    @Grab(group='io.github.http-builder-ng', module='http-builder-ng-core', version='1.0.3'),
    @Grab(group='org.slf4j', module='slf4j-simple', version='1.7.25', scope='test')
])
import static groovyx.net.http.HttpBuilder.configure
import static groovyx.net.http.ContentTypes.JSON
import groovy.json.*
import groovy.xml.MarkupBuilder

import org.asciidoctor.OptionsBuilder
import org.asciidoctor.AttributesBuilder
import org.asciidoctor.SafeMode
import org.asciidoctor.Asciidoctor.Factory
//end::dependencies[]

//tag::config[]
REPORT=false //(1)
ASCIIDOC=true
DELETE=false //(2)
TRESHOLD=4  //(3)

user=args[0]        //(4)
password=args[1]    //(5)
namespace=args[2]   //(6)
//end::config[]

//tag::auth[]
token = configure {  //(1)
    request.uri='https://hub.docker.com/'
    request.contentType=JSON[0]
    request.accept=['application/json']
}.post { //(2)
    request.uri.path='/v2/users/login'
    request.body=[username:user,password:password]
}.token //(3)

dockerHub = configure {   //(4)
    request.uri='https://hub.docker.com/'
    request.contentType=JSON[0]
    request.accept=['application/json']
    request.headers['Authorization']="JWT $token"
}
//end::auth[]


//tag::repos[]
repos=dockerHub.get { //(1)
    request.uri.path="/v2/repositories/$namespace"
    request.uri.query=[page_size:200]
}.results.collect{ repo->   //(2)
    [name:repo.name,lastUpdate:Date.parse("yyyy-MM-dd'T'HH:mm:ss",repo.last_updated)]
}.sort{ a,b->   //(3)
    a.name <=> b.name
}
//end::repos[]

//tag::tags[]
repos.each{ repo->
    repo.tags =dockerHub.get {  //(1)
        request.uri.path="/v2/repositories/$namespace/$repo.name/tags"
        request.uri.query=[page_size:200]
    }.results.collect{ tag->    //(2)
        [id:tag.id,name:tag.name,lastUpdate:Date.parse("yyyy-MM-dd'T'HH:mm:ss",tag.last_updated)]
    }.sort{ a,b->               //(3)
        b.lastUpdate<=>a.lastUpdate
    }
    if(repo.tags.size()>=TRESHOLD){
        repo.tags.takeRight(repo.tags.size()-TRESHOLD).each{ it.toRemove=true } //(4)
    }
}
//end::tags[]

if( REPORT ){
//tag::report[]
    writer=new StringWriter()
    html=new MarkupBuilder(writer)
    html.html {
        head {
            title "Docker Hub Images reporting"
        }
        body(id: "main") {
            h1 id: "namespace",  "Repository $namespace"
            repos.each{ repo->
                div {
                    h2 "$repo.name (Last update:${repo.lastUpdate.format('yyyy-MM-dd HH:mm')})"
                    repo.tags.findAll{!it.toRemove}.each{tag->
                        p "$tag.name (Last update:${tag.lastUpdate.format('yyyy-MM-dd HH:mm')})"
                    }
                    if(repo.tags.find{it.toRemove} )
                       h3 "Candidates to remove"
                    repo.tags.findAll{it.toRemove}.each{tag->
                        p "$tag.name (Last update:${tag.lastUpdate.format('yyyy-MM-dd HH:mm')})"
                    }

                }
            }
        }
    }
    new File('docker_report.html').newWriter().withWriter { w -> w << writer.toString() }
//end::report[]
}

if(ASCIIDOC){
    //tag::asciidoctor[]
    file = new File("/tmp/asciidoc_docker_report.adoc") //(1)
    file.newWriter().withPrintWriter { mainWriter ->
        mainWriter.println "= Docker Hub status"
        mainWriter.println "Jorge Aguilera <jorge.aguilera@puravida-software.com>"
        mainWriter.println new Date().format('yyyy-MM-dd')
        mainWriter.println ":icons: font"
        mainWriter.println ":toc: left"
        mainWriter.println ""
        mainWriter.println """
[abstract]
Report status of repository *${namespace}* at *${new Date().format('yyyy-MM-dd HH:mm')}*
        """

        repos.each { repo ->    //(2)
            mainWriter.println "\n== ${repo.name}"
            mainWriter.println "Last update: ${repo.lastUpdate.format('yyyy-MM-dd HH:mm')}"
            mainWriter.println ""

            repo.tags.findAll { !it.toRemove }.each { tag ->
                mainWriter.println "- ${tag.name} ${tag.lastUpdate.format('yyyy-MM-dd HH:mm')}"
            }
            if (repo.tags.find { it.toRemove }) {
                mainWriter.println "\n=== Candidates to remove"
            }
            repo.tags.findAll { it.toRemove }.each { tag ->
                mainWriter.println "- ${tag.name} ${tag.lastUpdate.format('yyyy-MM-dd HH:mm')}"
            }
        }
    }

    asciidoctor = Factory.create();
    attributes = AttributesBuilder.attributes().
            sectionNumbers(true).
            sourceHighlighter("coderay").
            get()
    options = OptionsBuilder.options().
            backend('html').
            attributes(attributes).
            safe(SafeMode.UNSAFE).
            get()
    asciidoctor.convertFile(file, options)  //(3)
    //end::asciidoctor[]
}

if( DELETE ){
//tag::delete[]
    repos.each{ repo->
        repo.tags.findAll{it.toRemove}.each{ tag->
            dockerHub.delete {
                request.uri.path="/v2/repositories/$namespace/$repo.name/tags/$tag.name"
            }
        }
    }
//end::delete[]
}

Fullmenu null