Using conda install within a python script

Question

According to this answer you can import pip from within a Python script and use it to install a module. Is it possible to do this with conda install?

The conda documentation only shows examples from the command line but I'm looking for code that can be executed from within a Python script.

Yes, I could execute shell commands from within the script but I am trying to avoid this as it is basically assuming that conda cannot be imported and its functions called.

Answer 1

I was looking at the latest Conda Python API and noticed that there are actually only 2 public modules with “very long-term stability”:

1. conda.cli.python_api
2. conda.api

For your question, I would work with the first:

NOTE: run_command() below will always add a -y/--yes option (i.e. it will not ask for confirmation)

import conda.cli.python_api as Conda
import sys

###################################################################################################
# The below is roughly equivalent to:
#   conda install -y 'args-go-here' 'no-whitespace-splitting-occurs' 'square-brackets-optional'

(stdout_str, stderr_str, return_code_int) = Conda.run_command(
    Conda.Commands.INSTALL, # alternatively, you can just say "install"
                            # ...it's probably safer long-term to use the Commands class though
                            # Commands include:
                            #  CLEAN,CONFIG,CREATE,INFO,INSTALL,HELP,LIST,REMOVE,SEARCH,UPDATE,RUN
    [ 'args-go-here', 'no-whitespace-splitting-occurs', 'square-brackets-optional' ],
    use_exception_handler=True,  # Defaults to False, use that if you want to handle your own exceptions
    stdout=sys.stdout, # Defaults to being returned as a str (stdout_str)
    stderr=sys.stderr, # Also defaults to being returned as str (stderr_str)
    search_path=Conda.SEARCH_PATH  # this is the default; adding only for illustrative purposes
)
###################################################################################################

---

The nice thing about using the above is that it solves a problem that occurs (mentioned in the comments above) when using conda.cli.main():

...conda tried to interpret the comand line arguments instead of the arguments of conda.cli.main(), so using conda.cli.main() like this might not work for some things.

---

The other question in the comments above was:

How [to install a package] when the channel is not the default?

import conda.cli.python_api as Conda
import sys

###################################################################################################
# Either:
#   conda install -y -c <CHANNEL> <PACKAGE>
# Or (>= conda 4.6)
#   conda install -y <CHANNEL>::<PACKAGE>

(stdout_str, stderr_str, return_code_int) = Conda.run_command(
    Conda.Commands.INSTALL,
    '-c', '<CHANNEL>',
    '<PACKAGE>'
    use_exception_handler=True, stdout=sys.stdout, stderr=sys.stderr
)
###################################################################################################

Answer 2

You can use conda.cli.main. For example, this installs numpy:

import conda.cli

conda.cli.main('conda', 'install',  '-y', 'numpy')

Use the -y argument to avoid interactive questions:

-y, --yes Do not ask for confirmation.

Answer 3

Having worked with conda from Python scripts for a while now, I think calling conda with the subprocess module works the best overall. In Python 3.7+, you could do something like this:

import json
from subprocess import run


def conda_list(environment):
    proc = run(["conda", "list", "--json", "--name", environment],
               text=True, capture_output=True)
    return json.loads(proc.stdout)


def conda_install(environment, *package):
    proc = run(["conda", "install", "--quiet", "--name", environment] + packages,
               text=True, capture_output=True)
    return json.loads(proc.stdout)

As I pointed out in a comment, conda.cli.main() was not intended for external use. It parses sys.argv directly, so if you try to use it in your own script with your own command line arguments, they will get fed to conda.cli.main() as well.

@YenForYang's answer suggesting conda.cli.python_api is better because this is a publicly documented API for calling conda commands. However, I have that it still has rough edges. conda builds up internal state as it executes a command (e.g. caches). The way conda is usually used and usually tested is as a command line program. In that case, this internal state is discarded at the end of the conda command. With conda.cli.python_api, you can execute several conda commands within a single process. In this case, the internal state carries over and can sometimes lead to unexpected results (e.g. the cache becomes outdated as commands are performed). Of course, it should be possible for conda to handle this internal state directly. My point is just that using conda this way is not the main focus of the developers. If you want the most reliable method, use conda the way the developers intend it to be used -- as its own process.

conda is a fairly slow command, so I don't think one should worry about the performance impact of calling a subprocess. As I noted in another comment, pip is a similar tool to conda and explicitly states in its documentation that it should be called as a subprocess, not imported into Python.

Answer 4

Try this:

!conda install xyzpackage

Please remember this has to be done within the Python script not the OS prompt.

Or else you could try the following:

import sys from conda.cli import main

sys.exit(main())

try:
    import conda
    from conda.cli import main
    sys.argv = ['conda'] + list(args)
    main()