Installing packages using pip and virtual environments¶
This guide discusses how to install packages using pip and a virtual environment manager: either venv for Python 3 or virtualenv for Python 2. These are the lowest-level tools for managing Python packages and are recommended if higher-level tools do not suit your needs.
This doc uses the term package to refer to a Distribution Package which is different from an Import Package that which is used to import modules in your Python source code.
Installing pip¶
pip is the reference Python package manager. It’s used to install and update packages. You’ll need to make sure you have the latest version of pip installed.
Debian and most other distributions include a python-pip package; if you want to use the Linux distribution-provided versions of pip, see Installing pip/setuptools/wheel with Linux Package Managers .
You can also install pip yourself to ensure you have the latest version. It’s recommended to use the system pip to bootstrap a user installation of pip:
Afterwards, you should have the latest version of pip installed in your user site:
The Python installers for Windows include pip. You can make sure that pip is up-to-date by running:
Afterwards, you should have the latest version of pip:
Installing virtualenv¶
If you are using Python 3.3 or newer, the venv module is the preferred way to create and manage virtual environments. venv is included in the Python standard library and requires no additional installation. If you are using venv, you may skip this section.
virtualenv is used to manage Python packages for different projects. Using virtualenv allows you to avoid installing Python packages globally which could break system tools or other projects. You can install virtualenv using pip.
Creating a virtual environment¶
venv (for Python 3) and virtualenv (for Python 2) allow you to manage separate package installations for different projects. They essentially allow you to create a “virtual” isolated Python installation and install packages into that virtual installation. When you switch projects, you can simply create a new virtual environment and not have to worry about breaking the packages installed in the other environments. It is always recommended to use a virtual environment while developing Python applications.
To create a virtual environment, go to your project’s directory and run venv. If you are using Python 2, replace venv with virtualenv in the below commands.
The second argument is the location to create the virtual environment. Generally, you can just create this in your project and call it env .
venv will create a virtual Python installation in the env folder.
You should exclude your virtual environment directory from your version control system using .gitignore or similar.
Activating a virtual environment¶
Before you can start installing or using packages in your virtual environment you’ll need to activate it. Activating a virtual environment will put the virtual environment-specific python and pip executables into your shell’s PATH .
You can confirm you’re in the virtual environment by checking the location of your Python interpreter:
Python pip utils
P ip is the tool to use when it comes to managing Python packages. Here are some useful pip commands you should know :
Install a package
The command above works, but the best practice is to always specify a version when installing a package to control the version installed and list it in the requirements.txt file(cf. section below)
Delete a package
How to upgrade a specific package
Sometimes you may have to downgrade or upgrade a specific package to a given version due to restrictions. In this case, use the following command :
Use venv
This is not really a pip command utils, but more like a best practice. For some projects, you will have to install packages that could upgrade or downgrade other packages that are already being used on other projects. Thus, we need to be careful because our installations could severely impact the other projects.
The best practice is to use a virtual environment (venv) for each project, or at least each domain. The venv will allow you to install any packages you want without updating the other packages already installed on your machine. To do so, create a venv using the following command :
Each time you will need to install a package, just activate the venv using :
To make sure you are using your venv, check the location of the Python interpreter using :
Once you installed the package, you can get out of your env using :
(Don’t forget to run your project using your venv to get all your packages)
Get the versions of all installed python packages
The second best practice is to save the version of every package before installing some other packages in case the installation fails or impact some running services.
If you have tons of packages, you can use grep to print only the version of the searched package :
Use requirements.txt file
Another best practice when installing packages on a project is to use requirements.txt file to store the list of the needed packages to run the project. You can build this file by listing one by one all the packages you installed since you started working on your projects. Call the file requirements.txt , it should look like this :
This file is very useful when it comes to tracking your packages, starting an env from scratch or even when you a many to work on a single projects. This will help other team members to understand which packages are needed and which version they should install.
You can use this file to install all the packages mentioned inside :
Use pip freeze to create your requirements
If you didn’t create a requirements file for your project, you will spend hours to recover all the Python packages you use and what version you uses. A simple trick is to use pip freeze to create your requirements.txt file using the following command :
This is helpful, but has major flaws : first being the fact that you could introduce in your requirements.txt file packages that you installed in the past but that you are not using any more.
Bonus tip : use pipreqs to create the requirements file
This package has been built to help developers automatically build the requirements.txt file from a project. It will automatically generate the requirements file and list all the packages you are using in your project.
Thank you for reading the article. There are many useful pip commands, I tried to be short and clear. Feel free to ask me if you have any question. Don’t hesitate to look at pip’s documentation, mentioned below.
pip install#
PyPI (and other indexes) using requirement specifiers.
VCS project urls.
Local project directories.
Local or remote source archives.
pip also supports installing from “requirements files”, which provide an easy way to specify a whole environment to be installed.
Overview#
pip install has several stages:
Identify the base requirements. The user supplied arguments are processed here.
Resolve dependencies. What will be installed is determined here.
Build wheels. All the dependencies that can be are built into wheels.
Install the packages (and uninstall anything being upgraded/replaced).
Note that pip install prefers to leave the installed version as-is unless —upgrade is specified.
Argument Handling#
When looking at the items to be installed, pip checks what type of item each is, in the following order:
Project or archive URL.
Local directory (which must contain a setup.py , or pip will report an error).
Local file (a sdist or wheel format archive, following the naming conventions for those formats).
A requirement, as specified in PEP 440.
Each item identified is added to the set of requirements to be satisfied by the install.
Working Out the Name and Version#
For each candidate item, pip needs to know the project name and version. For wheels (identified by the .whl file extension) this can be obtained from the filename, as per the Wheel spec. For local directories, or explicitly specified sdist files, the setup.py egg_info command is used to determine the project metadata. For sdists located via an index, the filename is parsed for the name and project version (this is in theory slightly less reliable than using the egg_info command, but avoids downloading and processing unnecessary numbers of files).
Any URL may use the #egg=name syntax (see VCS Support ) to explicitly state the project name.
Satisfying Requirements#
Once pip has the set of requirements to satisfy, it chooses which version of each requirement to install using the simple rule that the latest version that satisfies the given constraints will be installed (but see here for an exception regarding pre-release versions). Where more than one source of the chosen version is available, it is assumed that any source is acceptable (as otherwise the versions would differ).
Obtaining information about what was installed#
The install command has a —report option that will generate a JSON report of what pip has installed. In combination with the —dry-run and —ignore-installed it can be used to resolve a set of requirements without actually installing them.
The report can be written to a file, or to standard output (using —report — in combination with —quiet ).
The format of the JSON report is described in Installation Report .
Installation Order#
This section is only about installation order of runtime dependencies, and does not apply to build dependencies (those are specified using PEP 518).
As of v6.1.0, pip installs dependencies before their dependents, i.e. in “topological order.” This is the only commitment pip currently makes related to order. While it may be coincidentally true that pip will install things in the order of the install arguments or in the order of the items in a requirements file, this is not a promise.
In the event of a dependency cycle (aka “circular dependency”), the current implementation (which might possibly change later) has it such that the first encountered member of the cycle is installed last.
For instance, if quux depends on foo which depends on bar which depends on baz, which depends on foo:
Prior to v6.1.0, pip made no commitments about install order.
The decision to install topologically is based on the principle that installations should proceed in a way that leaves the environment usable at each step. This has two main practical benefits:
Concurrent use of the environment during the install is more likely to work.
A failed install is less likely to leave a broken environment. Although pip would like to support failure rollbacks eventually, in the mean time, this is an improvement.
Although the new install order is not intended to replace (and does not replace) the use of setup_requires to declare build dependencies, it may help certain projects install from sdist (that might previously fail) that fit the following profile:
They have build dependencies that are also declared as install dependencies using install_requires .
python setup.py egg_info works without their build dependencies being installed.
For whatever reason, they don’t or won’t declare their build dependencies using setup_requires .
Requirements File Format
This section has been moved to Requirements File Format .
This section has been moved to Requirement Specifiers .
Pre-release Versions#
Starting with v1.4, pip will only install stable versions as specified by pre-releases by default. If a version cannot be parsed as a compliant PEP 440 version then it is assumed to be a pre-release.
If a Requirement specifier includes a pre-release or development version (e.g. >=0.0.dev0 ) then pip will allow pre-release and development versions for that requirement. This does not include the != flag.
The pip install command also supports a —pre flag that enables installation of pre-releases and development releases.
This is now covered in VCS Support .
Finding Packages#
pip searches for packages on PyPI using the HTTP simple interface, which is documented here and there.
pip offers a number of package index options for modifying how packages are found.
pip looks for packages in a number of places: on PyPI (if not disabled via —no-index ), in the local filesystem, and in any additional repositories specified via —find-links or —index-url . There is no ordering in the locations that are searched. Rather they are all checked, and the “best” match for the requirements (in terms of version number — see PEP 440 for details) is selected.
SSL Certificate Verification
This is now covered in HTTPS Certificates .
This is now covered in Caching .
This is now covered in Caching .
Hash checking mode
This is now covered in Secure installs .
Local Project Installs
Build System Interface
Options#
-r , —requirement <file> #
Install from the given requirements file. This option can be used multiple times.
-c , —constraint <file> #
Constrain versions using the given constraints file. This option can be used multiple times.
Don’t install package dependencies.
Include pre-release and development versions. By default, pip only finds stable versions.
-e , —editable <path/url> #
Install a project in editable mode (i.e. setuptools “develop mode”) from a local project path or a VCS url.
Don’t actually install anything, just print what would be. Can be used in combination with —ignore-installed to ‘resolve’ the requirements.
Install packages into <dir>. By default this will not replace existing files/folders in <dir>. Use —upgrade to replace existing packages in <dir> with new versions.
Only use wheels compatible with <platform>. Defaults to the platform of the running system. Use this option multiple times to specify multiple platforms supported by the target interpreter.
The Python interpreter version to use for wheel and “Requires-Python” compatibility checks. Defaults to a version derived from the running interpreter. The version can be specified using up to three dot-separated integers (e.g. “3” for 3.0.0, “3.7” for 3.7.0, or “3.7.3”). A major-minor version can also be given as a string without dots (e.g. “37” for 3.7.0).
Only use wheels compatible with Python implementation <implementation>, e.g. ‘pp’, ‘jy’, ‘cp’, or ‘ip’. If not specified, then the current interpreter implementation is used. Use ‘py’ to force implementation-agnostic wheels.
Only use wheels compatible with Python abi <abi>, e.g. ‘pypy_41’. If not specified, then the current interpreter abi tag is used. Use this option multiple times to specify multiple abis supported by the target interpreter. Generally you will need to specify —implementation, —platform, and —python-version when using this option.
Install to the Python user install directory for your platform. Typically
/.local/, or %APPDATA%Python on Windows. (See the Python documentation for site.USER_BASE for full details.)
Install everything relative to this alternate root directory.
Installation prefix where lib, bin and other top-level folders are placed. Note that the resulting installation may contain scripts and other resources which reference the Python interpreter of pip, and not that of —prefix . See also the —python option if the intention is to install packages into another (possibly pip-free) environment.
Directory to check out editable projects into. The default in a virtualenv is “<venv path>/src”. The default for global installs is “<current dir>/src”.
Upgrade all specified packages to the newest available version. The handling of dependencies depends on the upgrade-strategy used.
Determines how dependency upgrading should be handled [default: only-if-needed]. “eager” — dependencies are upgraded regardless of whether the currently installed version satisfies the requirements of the upgraded package(s). “only-if-needed” — are upgraded only when they do not satisfy the requirements of the upgraded package(s).
Reinstall all packages even if they are already up-to-date.
Ignore the installed packages, overwriting them. This can break your system if the existing package is of a different version or was installed with a different package manager!
Ignore the Requires-Python information.
Disable isolation when building a modern source distribution. Build dependencies specified by PEP 518 must be already installed if this option is used.
Use PEP 517 for building source distributions (use —no-use-pep517 to force legacy behaviour).
Check the build dependencies when PEP517 is used.
Allow pip to modify an EXTERNALLY-MANAGED Python installation
-C , —config-settings <settings> #
Configuration settings to be passed to the PEP 517 build backend. Settings take the form KEY=VALUE. Use multiple —config-settings options to pass multiple keys to the backend.
Extra global options to be supplied to the setup.py call before the install or bdist_wheel command.
Compile Python source files to bytecode
Do not compile Python source files to bytecode
Do not warn when installing scripts outside PATH
Do not warn about broken dependencies
Do not use binary packages. Can be supplied multiple times, and each time adds to the existing value. Accepts either “:all:” to disable all binary packages, “:none:” to empty the set (notice the colons), or one or more package names with commas between them (no colons). Note that some packages are tricky to compile and may fail to install when this option is used on them.
Do not use source packages. Can be supplied multiple times, and each time adds to the existing value. Accepts either “:all:” to disable all source packages, “:none:” to empty the set, or one or more package names with commas between them. Packages without binary distributions will fail to install when this option is used on them.
Prefer older binary packages over newer source packages.
Require a hash to check each requirement against, for repeatable installs. This option is implied when any package in a requirements file has a —hash option.
Specify whether the progress bar should be used [on, off] (default: on)
Action if pip is run as a root user. By default, a warning message is shown.
Generate a JSON file describing what pip did to install the provided requirements. Can be used in combination with —dry-run and —ignore-installed to ‘resolve’ the requirements. When — is used as file name it writes to stdout. When writing to stdout, please combine with the —quiet option to avoid mixing pip logging output with JSON output.
Don’t clean up build directories.
Base URL of the Python Package Index (default https://pypi.org/simple). This should point to a repository compliant with PEP 503 (the simple repository API) or a local directory laid out in the same format.
Extra URLs of package indexes to use in addition to —index-url. Should follow the same rules as —index-url.
Ignore package index (only looking at —find-links URLs instead).
If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files. If a local path or file:// URL that’s a directory, then look for archives in the directory listing. Links to VCS project URLs are not supported.
Examples#
Install SomePackage and its dependencies from PyPI using Requirement Specifiers
Installing specific package version with pip
I am trying to install version 1.2.2 of MySQL_python , using a fresh virtualenv created with the —no-site-packages option. The current version shown in PyPi is 1.2.3. Is there a way to install the older version? I have tried:
However, when installed, it still shows MySQL_python-1.2.3-py2.6.egg-info in the site packages. Is this a problem specific to this package, or am I doing something wrong?
12 Answers 12
TL;DR:
Update as of 2022-12-28:
pip install —force-reinstall -v
For example: pip install —force-reinstall -v "MySQL_python==1.2.2"
What these options mean:
- —force-reinstall is an option to reinstall all packages even if they are already up-to-date.
- -v is for verbose. You can combine for even more verbosity (i.e. -vv ) up to 3 times (e.g. —force-reinstall -vvv ).
Thanks to @Peter for highlighting this (and it seems that the context of the question has broadened given the time when the question was first asked!), the documentation for Python discusses a caveat with using -I , in that it can break your installation if it was installed with a different package manager or if if your package is/was a different version.
- pip install -Iv (i.e. pip install -Iv MySQL_python==1.2.2 )
What these options mean:
- -I stands for —ignore-installed which will ignore the installed packages, overwriting them.
- -v is for verbose. You can combine for even more verbosity (i.e. -vv ) up to 3 times (e.g. -Ivvv ).
For more information, see pip install —help
First, I see two issues with what you’re trying to do. Since you already have an installed version, you should either uninstall the current existing driver or use pip install -I MySQL_python==1.2.2
However, you’ll soon find out that this doesn’t work. If you look at pip’s installation log, or if you do a pip install -Iv MySQL_python==1.2.2 you’ll find that the PyPI URL link does not work for MySQL_python v1.2.2. You can verify this here: http://pypi.python.org/pypi/MySQL-python/1.2.2
The download link 404s and the fallback URL links are re-directing infinitely due to sourceforge.net’s recent upgrade and PyPI’s stale URL.
So to properly install the driver, you can follow these steps:
You can even use a version range with pip install command. Something like this:
And if the package is already installed and you want to downgrade it add —force-reinstall like this:
One way, as suggested in this post, is to mention version in pip as:
i.e. Use == and mention the version number to install only that version. -I, —ignore-installed ignores already installed packages.
To install a specific python package version whether it is the first time, an upgrade or a downgrade use:
MySQL_python version 1.2.2 is not available so I used a different version. To view all available package versions from an index exclude the version:
I believe that if you already have a package it installed, pip will not overwrite it with another version. Use -I to ignore previous versions.
Sometimes, the previously installed version is cached.
It returns the followings:
Requirement already satisfied: pillow==5.2.0 in /home/ubuntu/anaconda3/lib/python3.6/site-packages (5.2.0)
We can use —no-cache-dir together with -I to overwrite this
Since this appeared to be a breaking change introduced in version 10 of pip, I downgraded to a compatible version:
This command tells pip to install a version of the module lower than version 10. Do this in a virutalenv so you don’t screw up your site installation of Python.
This below command worked for me
Python version — 2.7
command — $ pip install ‘python-jenkins>=1.1.1’
I recently ran into an issue when using pip ‘s -I flag that I wanted to document somewhere:
-I will not uninstall the existing package before proceeding; it will just install it on top of the old one. This means that any files that should be deleted between versions will instead be left in place. This can cause weird behavior if those files share names with other installed modules.
For example, let’s say there’s a package named package . In one of package s files, they use import datetime . Now, in package@2.0.0 , this points to the standard library datetime module, but in package@3.0.0 , they added a local datetime.py as a replacement for the standard library version (for whatever reason).
Now lets say I run pip install package==3.0.0 , but then later realize that I actually wanted version 2.0.0 . If I now run pip install -I package==2.0.0 , the old datetime.py file will not be removed, so any calls to import datetime will import the wrong module.
In my case, this manifested with strange syntax errors because the newer version of the package added a file that was only compatible with Python 3, and when I downgraded package versions to support Python 2, I continued importing the Python-3-only module.
Based on this, I would argue that uninstalling the old package is always preferable to using -I when updating installed package versions.