Install node_modules inside Docker container and synchronize them with host

Question

I have the problem with installing node_modules inside the Docker container and synchronize them with the host. My Docker\'s version is 18.03.1-ce, build

Answer 1

My workaround is to install dependencies when the container is starting instead of during build-time.

Dockerfile:

# We're using a multi-stage build so that we can install dependencies during build-time only for production.

# dev-stage
FROM node:14-alpine AS dev-stage
WORKDIR /usr/src/app
COPY package.json ./
COPY . .
# `yarn install` will run every time we start the container. We're using yarn because it's much faster than npm when there's nothing new to install
CMD ["sh", "-c", "yarn install && yarn run start"]

# production-stage
FROM node:14-alpine AS production-stage
WORKDIR /usr/src/app
COPY package.json ./
RUN yarn install
COPY . .

.dockerignore

Add node_modules to .dockerignore to prevent it from being copied when the Dockerfile runs COPY . .. We use volumes to bring in node_modules.

**/node_modules

docker-compose.yml

node_app:
    container_name: node_app
    build:
        context: ./node_app
        target: dev-stage # `production-stage` for production
    volumes:
        # For development:
        #   If node_modules already exists on the host, they will be copied
        #   into the container here. Since `yarn install` runs after the
        #   container starts, this volume won't override the node_modules.
        - ./node_app:/usr/src/app
        # For production:
        #   
        - ./node_app:/usr/src/app
        - /usr/src/app/node_modules

Answer 2

Binding your host node_modules folder with your container node_modules is not a good practice as you mention. I have seen the solution of creating an internal volume for this folder quite often. Not doing so will cause problems during the building stage.

I ran into this problem when I was trying to build a docker development environment for an angular app, that shows tslib errors when I was editing the files within my host folder cause my host's node_modules folder was empty (as expected).

The cheap solution that helps me, in this case, was to use the Visual Studio Code Extension called "Remote-Containers".

This extension will allow you to attach your Visual Studio Code to your container and edit transparently your files within your container folders. To do so, it will install an internal vscode server within your development container. For more information check this link.

Ensure, however, that your volumes are still created in your docker-compose.yml file.

I hope it helps :D!

Answer 3

At first, I would like to thank David Maze and trust512 for posting their answers. Unfortunately, they didn't help me to solve my problem.

I would like to post my answer to this question.

My docker-compose.yml:

---
# Define Docker Compose version.
version: "3"

# Define all the containers.
services:
  # Frontend Container.
  frontend:
    build: ./app/frontend
    volumes:
      - ./app/frontend:/usr/src/app
    ports:
     - 3000:3000
    environment:
      NODE_ENV: development
    command: /usr/src/app/entrypoint.sh

My Dockerfile:

# Set the base image.
FROM node:10

# Create and define the node_modules's cache directory.
RUN mkdir /usr/src/cache
WORKDIR /usr/src/cache

# Install the application's dependencies into the node_modules's cache directory.
COPY package.json ./
COPY package-lock.json ./
RUN npm install

# Create and define the application's working directory.
RUN mkdir /usr/src/app
WORKDIR /usr/src/app

And last but not least entrypoint.sh:

#!/bin/bash

cp -r /usr/src/cache/node_modules/. /usr/src/app/node_modules/
exec npm start

The trickiest part here is to install the node_modules into the node_module's cache directory (/usr/src/cache) which is defined in our Dockerfile. After that, entrypoint.sh will move the node_modules from the cache directory (/usr/src/cache) to our application directory (/usr/src/app). Thanks to this the entire node_modules directory will appear on our host machine.

Looking at my question above I wanted:

• to install node_modules automatically instead of manually
• to install node_modules inside the Docker container instead of the host
• to have node_modules synchronized with the host (if I install some new package inside the Docker container, it should be synchronized with the host automatically without any manual actions

The first thing is done: node_modules are installed automatically. The second thing is done too: node_modules are installed inside the Docker container (so, there will be no cross-platform issues). And the third thing is done too: node_modules that were installed inside the Docker container will be visible on our host machine and they will be synchronized! If we install some new package inside the Docker container, it will be synchronized with our host machine at once.

The important thing to note: truly speaking, the new package installed inside the Docker container, will appear in /usr/src/app/node_modules. As this directory is synchronized with our host machine, this new package will appear on our host machine's node_modules directory too. But the /usr/src/cache/node_modules will have the old build at this point (without this new package). Anyway, it is not a problem for us. During next docker-compose up --build (--build is required) the Docker will re-install the node_modules (because package.json was changed) and the entrypoint.sh file will move them to our /usr/src/app/node_modules.

You should take into account one more important thing. If you git pull the code from the remote repository or git checkout your-teammate-branch when Docker is running, there may be some new packages added to the package.json file. In this case, you should stop the Docker with CTRL + C and up it again with docker-compose up --build (--build is required). If your containers are running as a daemon, you should just execute docker-compose stop to stop the containers and up it again with docker-compose up --build (--build is required).

If you have any questions, please let me know in the comments.

Hope this helps.

Answer 4

Thanks Vladyslav Turak for answer with entrypoint.sh where we copy node_modules from container to host.

I implemented the similar thing but I run into the issue with husky, @commitlint, tslint npm packages.
I can't push anything into repository.
Reason: I copied node_modules from Linux to Windows. In my case <5% of files are different (.bin and most of package.json) and 95% are the same. example: image with diff

So I returned to solution with npm install of node_modules for Windows first (for IDE and debugging). And Docker image will contain Linux version of node_modules.