Fixing Docker Hub rate limit issue with ECR source image

Pulling an image from a Docker Hub repository is limited.

• 100 pull requests per six hours for anonymous users
• 200 pull requests per six hours for authenticated users

We are running an ECS Fargate container and hosting our docker image on AWS ECR. We have also set up a CICD on AWS with a CodeBuild project, which uses our Dockerfile so we can do a proper deployment. But very often we have received this error:

Step 1/7 : FROM node:12

toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

We have reached the limit, so the build phase fails.

Possible solutions

The first potential solution would be to make a Docker Hub account.

The change is in the buildspec.yml

pre_build:
commands:
- echo Logging in to Amazon ECR…
- $(aws ecr get-login — no-include-email — region $AWS_DEFAULT_REGION)

Instead of this, the build script should contain the docker login command.

pre_build:
commands:
- echo Logging in to Docker Hub…
- echo $DOCKERHUB_PASSWORD | docker login — username $DOCKERHUB_USERNAME — password-stdin

Furthermore, credentials should be kept secret in some AWS services such as Parameter Store or Secrets Manager. An additional IAM role should be created to access these secrets for the needed permissions from the Codebuild project itself.

This could do the trick, probably if you do not have an intense workload and don’t do deployments that often. Pulling, building the image, and pushing it to ECR will stay the same. We are just now authenticated users, expecting that this will solve the problem.

But we decided to go with another solution. We have a Nodejs application, so that is the image we want to build from, available from the Docker hub. Instead of pulling that image always within the pipeline when we want to do a deployment, we added a new Dockerfile under the source folder in our repository which contains the following code:

# Specify our base image
FROM node:12
# Create app directory
WORKDIR /app
EXPOSE 8080

And we build this image in a new ECR, as a “source”. The main Dockerfile looks like this:

# Define variables to build the source docker image
ARG account_id
ARG region
# Source image
FROM $account_id.dkr.ecr.$region.amazonaws.com/source:latest
ADD . /app
RUN npm i
CMD [“node”, “server.js”]

As you can see, the source image is not the Nodejs image anymore. It is the image URI of the source ECR repository. Adding this line, and deploying the source ECR will solve the problem with pull limits because we will refer to an image that has already been build, whenever we do deployments.

We define these two arguments: account_id and region, so the value of the image URI is not hardcoded.

The buildspec.yml is slightly transformed and it looks like this:

The main difference here is in the build command:

- docker build -t $IMAGE_REPO_NAME — build-arg account_id=$AWS_ACCOUNT_ID — build-arg region=$AWS_DEFAULT_REGION .

In this step we are setting build-time variables, referring to the ARG in the Dockerfile.

Additionally, we can set up a CICD for the deployment of the source image, only if the Nodejs version has been changed.

Co-author Stojce Slavkovski