Case Status
Log In

Wiki

Options

 
Managing software with conda
  • RSS Feed

Last modified on 21-08-2017 09:11 by User.

Managing software with conda

Downloading and installing conda is very simple, you just download and run the installer:

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh

You can use follow the installers suggestions about where to install and the let it add conda to your .bashrc.

Now that we have conda available we also need a few settings. The first is necessary and tells conda to trust our proxy server, the rest adds some recommended "channels" which will make a whole bunch of useful packages available to install.

conda config --set ssl_verify /com/etc/ssl-proxy-cert.pem
conda config --add channels r # this line is only for compatibility and can probably be skipped
conda config --add channels defaults
conda config --add channels conda-forge
conda config --add channels bioconda

The clever thing about conda is that it allows you to use separate environments for separate projects. If you have a project where you've installed a bunch packages into your python or R there is no reason for those to accidentally seep in to your next project. If you want to try different versions of some package you can just create separate environments for them instead of installing and uninstalling multiple times. With separate environments you force yourself to make the dependencies for each project explicit which in turn makes it easier for collaborators to run your code and improves reproducibility.

Here is how the usage might look if we want to create a new environment with the newest version of pysam:

[user@fe1 ~]$ conda create --name amazing-project pysam
Fetching package metadata ...............
Solving package specifications: .

Package plan for installation in environment /home/user/miniconda3/envs/amazing-project:

The following NEW packages will be INSTALLED:

    bcftools:   1.3.1-1       bioconda
    curl:       7.45.0-2      bioconda
    htslib:     1.3.2-0       bioconda
    libgcc:     5.2.0-0
    openssl:    1.0.2k-0
    pip:        9.0.1-py35_1
    pysam:      0.10.0-py35_1 bioconda
    python:     3.5.2-0
    readline:   6.2-2
    samtools:   1.3.1-5       bioconda
    setuptools: 27.2.0-py35_0
    sqlite:     3.13.0-0
    tk:         8.5.18-0
    wheel:      0.29.0-py35_0
    xz:         5.2.2-1
    zlib:       1.2.8-3

Proceed ([y]/n)?

libgcc-5.2.0-0 100% |#############################################| Time: 0:00:00   7.78 MB/s
openssl-1.0.2k 100% |#############################################| Time: 0:00:00  15.10 MB/s
readline-6.2-2 100% |#############################################| Time: 0:00:00  11.57 MB/s
sqlite-3.13.0- 100% |#############################################| Time: 0:00:00  15.26 MB/s
tk-8.5.18-0.ta 100% |#############################################| Time: 0:00:00  11.79 MB/s
xz-5.2.2-1.tar 100% |#############################################| Time: 0:00:00  13.30 MB/s
zlib-1.2.8-3.t 100% |#############################################| Time: 0:00:00  13.83 MB/s
bcftools-1.3.1 100% |#############################################| Time: 0:00:01 920.51 kB/s
curl-7.45.0-2. 100% |#############################################| Time: 0:00:00   3.19 MB/s
python-3.5.2-0 100% |#############################################| Time: 0:00:01  16.34 MB/s
htslib-1.3.2-0 100% |#############################################| Time: 0:00:00   1.56 MB/s
samtools-1.3.1 100% |#############################################| Time: 0:00:00   2.11 MB/s
setuptools-27. 100% |#############################################| Time: 0:00:00  10.32 MB/s
wheel-0.29.0-p 100% |#############################################| Time: 0:00:00  18.17 MB/s
pip-9.0.1-py35 100% |#############################################| Time: 0:00:00  14.98 MB/s
pysam-0.10.0-p 100% |#############################################| Time: 0:00:00   6.23 MB/s
#
# To activate this environment, use:
# > source activate amazing-project
#
# To deactivate this environment, use:
# > source deactivate amazing-project
#

[user@fe1 ~]$ source activate amazing-project
(amazing-project) [user@fe1 ~]$ python -c 'import pysam; print(pysam.__version__)'
0.10.0

This gives us a clean environment with just the minimal number of packages necessary to support pysam. We used an installer that prefers python3 so it will always try to use the newest python that it can - if you change the 3's to 2's in the bit about the installer then it will prefer python2.7 instead. The newest version of python that pysam is prepared for is 3.5 but lets say you absolutely needed it for python 3.6. Luckily it is available as a pip package for 3.6 so we can just delete our environment(or use a new name) and create a new one with python3.6 where we install pysam through pip instead:

(amazing-project) [user@fe1 ~]$ source deactivate amazing-project
[user@fe1 ~]$ conda remove --name amazing-project --all
...
[user@fe1 ~]$ conda create --name amazing-project python=3.6
...
[user@fe1 ~]$ source activate amazing-project
(amazing-project) [user@fe1 ~]$ pip install --cert /com/etc/ssl-proxy-cert.pem pysam
Collecting pysam
  Downloading pysam-0.10.0-cp36-cp36m-manylinux1_x86_64.whl (7.4MB)
    100% |################################| 7.4MB 140kB/s
Installing collected packages: pysam
Successfully installed pysam-0.10.0
(amazing-project) [user@fe1 ~]$ python -c 'import pysam; print(pysam.__version__);'
0.10.0

Since pip doesn't know about conda we have to tell it about the proxy manually with the --cert option.

Conda can also install R and a bunch of R packages and even a bunch of stuff from bioconductor and this is generally better that just dumping all packages where they are globally available for all your own R sessions and especially better than downloading the newest version from bioconductor every time you want to load a package.

Searching for packages is best done here: https://anaconda.org/search?q=ggplot2