Activate and switch Anaconda environment in Dockerfile during build

Question

I\'ve been trying for hours and can\'t figure out how to activate and switch anaconda environments in a Dockerfile during the build process

Here\'s the initial code:

Answer 1

You've got way too many RUN commands in your Dockerfile. It's not just that each RUN creates a new layer in the image. It's also that each RUN command starts a new shell, and conda activate applies only to the current shell.

You should combine logical groups of actions into a single RUN command. Use && to combine commands, and \ to break lines for readability:

RUN conda activate <myenv> \
 && conda install <whatever> \
 && ...

Keep in mind: at the end of that RUN command, the shell will be gone. So if you want to do something else to that conda environment afterwards, you've got to run conda activate again, or else use -n <myenv> to put something into an environment without activating it first.

When you start a container from the image, you will also have to call conda activate inside the container.

Answer 2

I haven't tested it with the nvidia image, but multi-stage Docker builds should help you out, which will probably look something like:

# get Miniconda docker image to get a installed conda env; WARNING: That image is Debian based
FROM continuumio/miniconda3:latest AS miniconda


# your Docker commands
FROM nvidia/cuda:10.1-cudnn7-devel-ubuntu16.04

# Set user
ENV SETUSER myuser

RUN useradd -m $SETUSER
USER $SETUSER
WORKDIR /home/$SETUSER


# Miniconda: get necessary files from build
COPY --from=miniconda /opt/conda /opt/conda
# Set correct permissions
RUN chown -R $SETUSER: /opt/conda
#   New terminals will have conda active
# If nvidia's Docker image has no .bashrc
# COPY --from=miniconda /home/$SETUSER/.bashrc
# else
RUN echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
    echo "conda activate base" >> ~/.bashrc

# switch shell sh (default in Linux) to bash
SHELL ["/bin/bash", "-c"]

# give bash access to Anaconda, then normal anaconda commands, e.g. (-q: quiet, -y: answer yes)
RUN source /home/$SETUSER/.bashrc \
 && conda create -q --name testy \
 && conda activate testy \
 && conda install -y your_package

Inspiration from this GitHub issue: https://github.com/ContinuumIO/docker-images/issues/89

Answer 3

Assuming you want to install the conda environment and run something in it, this approach uses ENV PATH to launch python indirectly in that conda environment. One can be left wondering if this approach really activates the environment, but so long as the subsequent commands work, and indeed they do, it may not matter.

FROM continuumio/miniconda3:latest
WORKDIR myappdir
COPY environment.yml .
RUN set -x && \
#   apt-get update && apt-get -y install gcc && \
    conda install -n base -c defaults conda=4.* && \
    conda env create -n condaenv  # Installs environment.yml && \
    conda clean -a
COPY myapppkg myapppkg
ENV PATH /opt/conda/envs/condaenv/bin:$PATH
ENTRYPOINT ["python", "-m", "myapppkg"]

I advise against using conda run while it is experimental due to a history of severe bugs such as this one affecting it. Although this particular bug is now fixed, its continued "experimental" nature as shown by conda run -h means that it can again break upstream, limiting the trust that one can place in it.

For reference:

• List of Miniconda Docker tags
• List of conda updates by channel: defaults, conda-forge