Installing Anaconda packages more quickly using Mamba¶
The ITSR support team often receive tickets from Anaconda users concerned that creating environments and installing packages is taking quite a long time. We recently installed Miniconda as a module on Apocrita, which enables users to install packages using the Mamba libsolver.
What is Mamba?¶
Anaconda has traditionally used conda
as its package manager, but over time it
has become quite slow at resolving dependencies when installing packages.
mamba
is a re-implementation of conda
written in C++ offering the following
benefits:
- parallel downloading of repository data and package files using multi-threading
libsolv
for much faster dependency solving, a state of the art library used in the RPM package manager of Red Hat, Fedora and OpenSUSE- core parts of
mamba
are implemented in C++ for maximum efficiency
At the same time, mamba
utilises the same command line parser, package
installation code and transaction verification routines as conda
to stay as
compatible as possible.
Why Mamba?¶
Put simply, in a lot of cases mamba
is a lot faster at installing packages
when compared to conda
:
To give another example, recently some Apocrita users were attempting to install
the busco package using Anaconda and the conda
package manager, and after two hours it still hadn't resolved all the
required dependencies.
By using mamba
instead, busco
takes less than five minutes to install.
Installing packages with Mamba¶
So, to use mamba
to install busco
:
1) Load the miniconda
module:
module load miniconda
2) Create a new conda env (I've called mine busco
but it can be called
anything you like):
mamba create --quiet --yes --name busco
3) Activate the environment:
mamba activate busco
4) Install busco
(version 5.1.1) using mamba
:
mamba install -c conda-forge -c bioconda busco=5.1.1
mamba
will now rapidly pull all required dependencies and ask you to confirm
the package installation list (which you can do by typing y
and pressing
Enter
). The installation should then complete within just a few minutes, and
busco
will be available in your env:
(busco) $ busco -v
BUSCO 5.1.1
(busco) $ busco -h
usage: busco -i [SEQUENCE_FILE] -l [LINEAGE] -o [OUTPUT_NAME] -m [MODE] [OTHER OPTIONS]
...
Can I use Mamba with the Anaconda modules on Apocrita?¶
Sometimes the miniconda
Apocrita module doesn't offer users all of the things
the anaconda3
module does. Using mamba
with anaconda3
is possible, but
only if you use the most recent anaconda3/2022.05
module (mamba
isn't
available in older versions). You can't run mamba
commands directly using this
module, but instead you have to add --experimental-solver=libmamba
to your
conda install
commands. Future versions of Anaconda may integrate mamba
so
that it can be used directly - we'll update this blog page if/when Anaconda
natively supports mamba
without the --experimental-solver=libmamba
extra
switch.
So, once again using the busco
package as an example:
1) Load the anaconda3/2022.05
module:
module load anaconda3/2022.05
2) Create a new conda env (I've called mine busco
but it can be called
anything you like):
conda create --quiet --yes --name busco
3) Activate the environment:
conda activate busco
4) Install busco
(version 5.1.1) using conda
and the
--experimental-solver=libmamba
flag:
conda install -c conda-forge -c bioconda busco=5.1.1 --experimental-solver=libmamba
mamba
will now rapidly pull all required dependencies and ask you to confirm
the package installation list (which you can do by typing y
and pressing
Enter
). The installation should then complete within just a few minutes, and
busco
will be available in your env:
(busco) $ busco -v
BUSCO 5.1.1
(busco) $ busco -h
usage: busco -i [SEQUENCE_FILE] -l [LINEAGE] -o [OUTPUT_NAME] -m [MODE] [OTHER OPTIONS]
...
You can find more information about mamba
on the project's
official docs page, as well as on the
QMUL Miniconda
docs page.
We hope you find these tips useful. As usual, you can ask a question on our Slack channel (QMUL users only), or by sending an email to its-research-support@qmul.ac.uk which is handled directly by staff with relevant expertise.
Title image: Generated using Stable Diffusion
Comparison video: Taken from the
Anaconda Blog