Nginx, docker push and 404 errors
Recently after trying some more elaborate Docker image builds, I discovered I had an issue with my Docker Registry setup (as described in Deploying a secure public Docker registry with Nginx and Traefik), which was manifesting itself as random
404 page not found errors:
$ docker --debug push my.private.registry/cant-think-of-a-container-name:latest The push refers to a repository [my.private.registry/cant-think-of-a-container-name] 5ae877b9fd02: Pushing 3.164MB 34be67ece980: Layer already exists 9411f02cc27c: Pushing 1.092MB/296.6MB 5df38987acef: Pushing 290MB/290MB 683c7228321d: Layer already exists e6562eb04a92: Pushing 122.9MB/122.9MB 596280599f68: Pushing 44.64MB/44.64MB 5d6cbe0dbcf9: Pushing 123.4MB/123.4MB Error: Status 404 trying to push repository cant-think-of-a-container-name: "404 page not found\n"
If you check the docker push logs (in debug mode), the
404 page not found error actually relates to an attempt to access something in the
/v1/... path on the registry - which is odd because the Docker registry protocol has been
v2 for, like, ever now.
So whats going on?
Looking at the logs from Nginx, it becomes quite clear - Nginx itself is throwing a
http 413 error because the PUT and POST bodies being sent by
docker push are well in excess of the size allowed as "valid" request bodies by my Nginx setup.
docker push doesn't really understand the
http 413 error, and tries to fall back to the
v1 protocol in desperation - personally, I'd have called it a day and gone down the pub, but docker didn't, poor little thing.
404 page not found error as shown by
docker push is actually the last error in a chain, and that chain is caused by the first
So how do you solve the
http 413 error?
Quite simple, bump up the size Nginx allows for request bodies:
This can go in the
location configuration blocks, depending on how wide you want it applied.