Pip — The complete reference guide
pip is the package manager for python.pip can be used to install python libraries from PyPI. In other words, pip is the npm of python . We use pip to install and manage python libraries that are not part of the python standard library.
Motivation:
Any python developer predeminantly uses pip day in and out.However, the usage is limited to just install and occasionally upgrade.
This article is a consolidated non-exhaustive list of all the pip commands every python developer should know. This could be a reference for any future pip commands.
Commands:
Install — plain install:
The install command is used to install the specified library from the pypi package. I am not giving the output of pip commands here as they are well known.
Install — per-user:
This may come in handy in the following scenarios
- You are not using a virtual environment
- You are using a shared PC
- You don’t have admin previleges to install your packages at system level
pip packages are installed to the system directories by default.In case of linux it is(/usr/local/lib/python3.8/dist-packages). When installed with —user flag, it is installed for a specific user.
Here is the code snippet to get sitepackages and usersitepackages.
Install — with proxy:
If your system is located behind a proxy and you are unable to download from PyPI directly, you could use a proxy provided by your organization.
Install — upgrade:
This command upgrades the existing pip library.
Install — from a different host:
Install from a different host apart from PyPI package.
Install — from requirements.txt:
Install from requirements.txt file.
Install — editable mode:
Install a local package in editable mode. This is useful if you have a local package and you would like to use it. The local package is still under development and any changes to the package will reflect.
Install —to a target location:
Install package to a specific location
Installs matplotlib to the current directory. This could be useful if you are uploading your project to AWS Lambda .
Install — from a local directory:
Install from a local directory and do not scan PyPI.
Here, the find links points to current directory as it has the wheel file for certifi library.The —no-index flag specifies not to look in the PyPI package.
Install — from VCS:
The libraries can be installed directly from a version control system like git.This is useful for huge organizations who have their projects as dependencies. If the access to the repository is restricted, this can be installed directly from git.
Uninstall
To uninstall a library..
Version:
Check pip version
Get information about an installed package. This includes the version, the location where the package is installed and its dependencies.
Freeze:
Predominantly used as an input to requirements.txt file.
This could be very helpful if you are using a virtualenv and want to copy all your dependencies to requirements.txt .
Lists all the installed packages. If executed inside a virtualenv, it will list all the packages installed in the virtual env.
Download:
Download the python libraries but don’t install them
The download command downloads the specified library’s wheel files in addition to their dependecies. The above command should give the following output.By default, the files are downloaded to the current directory.
Search:
pip search searches for python packages. The default location it searches is the PyPI package (https://pypi.org/pypi).
This does not work now.
However, this could be useful if you have libraries hosted internally in your organization like artifcatory .
The default URL could be overridden by the -i flag.
Check:
Checks compatibility for installed packages
Cache:
pip usually stores all the libraries installed in cache.
The wheel files are cached inside dir directory. Whenever a pip install is triggered, it checks the local cache and then connects to PyPI.
The idea behind the pip cache is simple ,when you install a Python package using pip for the first time ,it gets saved on the cache .If you try to download/install the same version of the package on a second time ,pip will just use the local cached copy
Please note that this will not work in a virtual environment as there may not be a need to cache libraries .
Disable cache :
Install package without saving to cache.
Displays help for pip commands.
Verbose:
Run pip in verbose mode for additional information.
This produces a bunch of output with overwhelming information.
Disable pip version check:
Disables the periodic check to PyPI if a new version of pip is available.
I would recommend not disabling this as you may miss important updates from PyPI.
User Guide#
pip is a command line program. When you install pip, a pip command is added to your system, which can be run from the command prompt as follows:
python -m pip executes pip using the Python interpreter you specified as python. So /usr/bin/python3.7 -m pip means you are executing pip for your interpreter located at /usr/bin/python3.7 .
py -m pip executes pip using the latest Python interpreter you have installed. For more details, read the Python Windows launcher docs.
Installing Packages#
pip supports installing from PyPI, version control, local projects, and directly from distribution files.
The most common scenario is to install from PyPI using Requirement Specifiers
For more information and examples, see the pip install reference.
Basic Authentication Credentials
This is now covered in Authentication .
This is now covered in Authentication .
This is now covered in Authentication .
Using a Proxy Server#
When installing packages from PyPI, pip requires internet access, which in many corporate environments requires an outbound HTTP proxy server.
pip can be configured to connect through a proxy server in various ways:
using the —proxy command-line option to specify a proxy in the form scheme://[user:passwd@]proxy.server:port
by setting the standard environment-variables http_proxy , https_proxy and no_proxy .
using the environment variable PIP_USER_AGENT_USER_DATA to include a JSON-encoded string in the user-agent variable used in pip’s requests.
Requirements Files#
“Requirements files” are files containing a list of items to be installed using pip install like so:
Details on the format of the files are here: Requirements File Format .
Logically, a Requirements file is just a list of pip install arguments placed in a file. Note that you should not rely on the items in the file being installed by pip in any particular order.
Requirements files can also be served via a URL, e.g. http://example.com/requirements.txt besides as local files, so that they can be stored and served in a centralized place.
In practice, there are 4 common uses of Requirements files:
Requirements files are used to hold the result from pip freeze for the purpose of achieving Repeatable Installs . In this case, your requirement file contains a pinned version of everything that was installed when pip freeze was run.
Requirements files are used to force pip to properly resolve dependencies. pip 20.2 and earlier doesn’t have true dependency resolution, but instead simply uses the first specification it finds for a project. E.g. if pkg1 requires pkg3>=1.0 and pkg2 requires pkg3>=1.0,<=2.0 , and if pkg1 is resolved first, pip will only use pkg3>=1.0 , and could easily end up installing a version of pkg3 that conflicts with the needs of pkg2 . To solve this problem, you can place pkg3>=1.0,<=2.0 (i.e. the correct specification) into your requirements file directly along with the other top level requirements. Like so:
Requirements files are used to force pip to install an alternate version of a sub-dependency. For example, suppose ProjectA in your requirements file requires ProjectB , but the latest version (v1.3) has a bug, you can force pip to accept earlier versions like so:
Requirements files are used to override a dependency with a local patch that lives in version control. For example, suppose a dependency SomeDependency from PyPI has a bug, and you can’t wait for an upstream fix. You could clone/copy the src, make the fix, and place it in VCS with the tag sometag . You’d reference it in your requirements file with a line like so:
If SomeDependency was previously a top-level requirement in your requirements file, then replace that line with the new line. If SomeDependency is a sub-dependency, then add the new line.
It’s important to be clear that pip determines package dependencies using install_requires metadata, not by discovering requirements.txt files embedded in projects.
Constraints Files#
Constraints files are requirements files that only control which version of a requirement is installed, not whether it is installed or not. Their syntax and contents is a subset of Requirements Files , with several kinds of syntax not allowed: constraints must have a name, they cannot be editable, and they cannot specify extras. In terms of semantics, there is one key difference: Including a package in a constraints file does not trigger installation of the package.
Use a constraints file like so:
Constraints files are used for exactly the same reason as requirements files when you don’t know exactly what things you want to install. For instance, say that the “helloworld” package doesn’t work in your environment, so you have a local patched version. Some things you install depend on “helloworld”, and some don’t.
One way to ensure that the patched version is used consistently is to manually audit the dependencies of everything you install, and if “helloworld” is present, write a requirements file to use when installing that thing.
Constraints files offer a better way: write a single constraints file for your organisation and use that everywhere. If the thing being installed requires “helloworld” to be installed, your fixed version specified in your constraints file will be used.
Constraints file support was added in pip 7.1. In Changes to the pip dependency resolver in 20.3 (2020) we did a fairly comprehensive overhaul, removing several undocumented and unsupported quirks from the previous implementation, and stripped constraints files down to being purely a way to specify global (version) limits for packages.
Same as requirements files, constraints files can also be served via a URL, e.g. http://example.com/constraints.txt, so that your organization can store and serve them in a centralized place.
Installing from Wheels#
“Wheel” is a built, archive format that can greatly speed installation compared to building and installing from source archives. For more information, see the Wheel docs , PEP 427, and PEP 425.
pip prefers Wheels where they are available. To disable this, use the —no-binary flag for pip install .
If no satisfactory wheels are found, pip will default to finding source archives.
To install directly from a wheel archive:
To include optional dependencies provided in the provides_extras metadata in the wheel, you must add quotes around the install target name:
In the future, the path[extras] syntax may become deprecated. It is recommended to use PEP 508 syntax wherever possible.
For the cases where wheels are not available, pip offers pip wheel as a convenience, to build wheels for all your requirements and dependencies.
pip wheel requires the wheel package to be installed, which provides the “bdist_wheel” setuptools extension that it uses.
To build wheels for your requirements and all their dependencies to a local directory:
And then to install those requirements just using your local directory of wheels (and not from PyPI):
Uninstalling Packages#
pip is able to uninstall most packages like so:
pip also performs an automatic uninstall of an old version of a package before upgrading to a newer version.
For more information and examples, see the pip uninstall reference.
Listing Packages#
To list installed packages:
To list outdated packages, and show the latest version available:
To show details about an installed package:
For more information and examples, see the pip list and pip show reference pages.
Searching for Packages#
pip can search PyPI for packages using the pip search command:
The query will be used to search the names and summaries of all packages.
For more information and examples, see the pip search reference.
This is now covered in Configuration .
This is now covered in Configuration .
This is now covered in Configuration .
This is now covered in Configuration .
Command Completion#
pip comes with support for command line completion in bash, zsh and fish.
To setup for bash:
To setup for zsh:
To setup for fish:
To setup for powershell:
Alternatively, you can use the result of the completion command directly with the eval function of your shell, e.g. by adding the following to your startup file:
Installing from local packages#
In some cases, you may want to install from local packages only, with no traffic to PyPI.
First, download the archives that fulfill your requirements:
Note that pip download will look in your wheel cache first, before trying to download from PyPI. If you’ve never installed your requirements before, you won’t have a wheel cache for those items. In that case, if some of your requirements don’t come as wheels from PyPI, and you want wheels, then run this instead:
Then, to install from local only, you’ll be using —find-links and —no-index like so:
“Only if needed” Recursive Upgrade#
pip install —upgrade now has a —upgrade-strategy option which controls how pip handles upgrading of dependencies. There are 2 upgrade strategies supported:
eager : upgrades all dependencies regardless of whether they still satisfy the new parent requirements
only-if-needed : upgrades a dependency only if it does not satisfy the new parent requirements
The default strategy is only-if-needed . This was changed in pip 10.0 due to the breaking nature of eager when upgrading conflicting dependencies.
It is important to note that —upgrade affects direct requirements (e.g. those specified on the command-line or via a requirements file) while —upgrade-strategy affects indirect requirements (dependencies of direct requirements).
As an example, say SomePackage has a dependency, SomeDependency , and both of them are already installed but are not the latest available versions:
pip install SomePackage : will not upgrade the existing SomePackage or SomeDependency .
pip install —upgrade SomePackage : will upgrade SomePackage , but not SomeDependency (unless a minimum requirement is not met).
pip install —upgrade SomePackage —upgrade-strategy=eager : upgrades both SomePackage and SomeDependency .
As an historic note, an earlier “fix” for getting the only-if-needed behaviour was:
A proposal for an upgrade-all command is being considered as a safer alternative to the behaviour of eager upgrading.
User Installs#
With Python 2.6 came the “user scheme” for installation, which means that all Python distributions support an alternative install location that is specific to a user. The default location for each OS is explained in the python documentation for the site.USER_BASE variable. This mode of installation can be turned on by specifying the —user option to pip install .
Moreover, the “user scheme” can be customized by setting the PYTHONUSERBASE environment variable, which updates the value of site.USER_BASE .
To install “SomePackage” into an environment with site.USER_BASE customized to ‘/myappenv’, do the following:
pip install —user follows four rules:
When globally installed packages are on the python path, and they conflict with the installation requirements, they are ignored, and not uninstalled.
When globally installed packages are on the python path, and they satisfy the installation requirements, pip does nothing, and reports that requirement is satisfied (similar to how global packages can satisfy requirements when installing packages in a —system-site-packages virtualenv).
pip will not perform a —user install in a —no-site-packages virtualenv (i.e. the default kind of virtualenv), due to the user site not being on the python path. The installation would be pointless.
In a —system-site-packages virtualenv, pip will not install a package that conflicts with a package in the virtualenv site-packages. The —user installation would lack sys.path precedence and be pointless.
To make the rules clearer, here are some examples:
From within a —no-site-packages virtualenv (i.e. the default kind):
From within a —system-site-packages virtualenv where SomePackage==0.3 is already installed in the virtualenv:
From within a real python, where SomePackage is not installed globally:
From within a real python, where SomePackage is installed globally, but is not the latest version:
From within a real python, where SomePackage is installed globally, and is the latest version:
This is now covered in Repeatable Installs .
Fixing conflicting dependencies
Using pip from your program#
As noted previously, pip is a command line program. While it is implemented in Python, and so is available from your Python code via import pip , you must not use pip’s internal APIs in this way. There are a number of reasons for this:
The pip code assumes that it is in sole control of the global state of the program. pip manages things like the logging system configuration, or the values of the standard IO streams, without considering the possibility that user code might be affected.
pip’s code is not thread safe. If you were to run pip in a thread, there is no guarantee that either your code or pip’s would work as you expect.
pip assumes that once it has finished its work, the process will terminate. It doesn’t need to handle the possibility that other code will continue to run after that point, so (for example) calling pip twice in the same process is likely to have issues.
This does not mean that the pip developers are opposed in principle to the idea that pip could be used as a library — it’s just that this isn’t how it was written, and it would be a lot of work to redesign the internals for use as a library, handling all of the above issues, and designing a usable, robust and stable API that we could guarantee would remain available across multiple releases of pip. And we simply don’t currently have the resources to even consider such a task.
What this means in practice is that everything inside of pip is considered an implementation detail. Even the fact that the import name is pip is subject to change without notice. While we do try not to break things as much as possible, all the internal APIs can change at any time, for any reason. It also means that we generally won’t fix issues that are a result of using pip in an unsupported way.
It should also be noted that installing packages into sys.path in a running Python process is something that should only be done with care. The import system caches certain data, and installing new packages while a program is running may not always behave as expected. In practice, there is rarely an issue, but it is something to be aware of.
Having said all of the above, it is worth covering the options available if you decide that you do want to run pip from within your program. The most reliable approach, and the one that is fully supported, is to run pip in a subprocess. This is easily done using the standard subprocess module:
If you want to process the output further, use one of the other APIs in the module. We are using freeze here which outputs installed packages in requirements format.:
If you don’t want to use pip’s command line functionality, but are rather trying to implement code that works with Python packages, their metadata, or PyPI, then you should consider other, supported, packages that offer this type of ability. Some examples that you could consider include:
packaging — Utilities to work with standard package metadata (versions, requirements, etc.)
setuptools (specifically pkg_resources ) — Functions for querying what packages the user has installed on their system.
distlib — Packaging and distribution utilities (including functions for interacting with PyPI).
Changes to the pip dependency resolver in 20.3 (2020)#
pip 20.3 has a new dependency resolver, on by default for Python 3 users. (pip 20.1 and 20.2 included pre-release versions of the new dependency resolver, hidden behind optional user flags.) Read below for a migration guide, how to invoke the legacy resolver, and the deprecation timeline. We also made a two-minute video explanation you can watch.
We will continue to improve the pip dependency resolver in response to testers’ feedback. Please give us feedback through the resolver testing survey.
Watch out for#
The big change in this release is to the pip dependency resolver within pip.
Computers need to know the right order to install pieces of software (“to install x , you need to install y first”). So, when Python programmers share software as packages, they have to precisely describe those installation prerequisites, and pip needs to navigate tricky situations where it’s getting conflicting instructions. This new dependency resolver will make pip better at handling that tricky logic, and make pip easier for you to use and troubleshoot.
The most significant changes to the resolver are:
It will reduce inconsistency: it will no longer install a combination of packages that is mutually inconsistent. In older versions of pip, it is possible for pip to install a package which does not satisfy the declared requirements of another installed package. For example, in pip 20.0, pip install "six<1.12" "virtualenv==20.0.2" does the wrong thing, “successfully” installing six==1.11 , even though virtualenv==20.0.2 requires six>=1.12.0,<2 (defined here). The new resolver, instead, outright rejects installing anything if it gets that input.
It will be stricter — if you ask pip to install two packages with incompatible requirements, it will refuse (rather than installing a broken combination, like it did in previous versions).
So, if you have been using workarounds to force pip to deal with incompatible or inconsistent requirements combinations, now’s a good time to fix the underlying problem in the packages, because pip will be stricter from here on out.
This also means that, when you run a pip install command, pip only considers the packages you are installing in that command, and may break already-installed packages. It will not guarantee that your environment will be consistent all the time. If you pip install x and then pip install y , it’s possible that the version of y you get will be different than it would be if you had run pip install x y in a single command. We are considering changing this behavior (per #7744) and would like your thoughts on what pip’s behavior should be; please answer our survey on upgrades that create conflicts.
We are also changing our support for Constraints Files , editable installs, and related functionality. We did a fairly comprehensive overhaul and stripped constraints files down to being purely a way to specify global (version) limits for packages, and so some combinations that used to be allowed will now cause errors. Specifically:
Constraints don’t override the existing requirements; they simply constrain what versions are visible as input to the resolver (see #9020)
Providing an editable requirement ( -e . ) does not cause pip to ignore version specifiers or constraints (see #8076), and if you have a conflict between a pinned requirement and a local directory then pip will indicate that it cannot find a version satisfying both (see #8307)
Hash-checking mode requires that all requirements are specified as a == match on a version and may not work well in combination with constraints (see #9020 and #8792)
If necessary to satisfy constraints, pip will happily reinstall packages, upgrading or downgrading, without needing any additional command-line options (see #8115 and Options that control the installation process )
Unnamed requirements are not allowed as constraints (see #6628 and #8210)
Links are not allowed as constraints (see #8253)
Constraints cannot have extras (see #6628)
Per our Python 2 Support policy, pip 20.3 users who are using Python 2 will use the legacy resolver by default. Python 2 users should upgrade to Python 3 as soon as possible, since in pip 21.0 in January 2021, pip dropped support for Python 2 altogether.
How to upgrade and migrate#
Install pip 20.3 with python -m pip install —upgrade pip .
Validate your current environment by running pip check . This will report if you have any inconsistencies in your set of installed packages. Having a clean installation will make it much less likely that you will hit issues with the new resolver (and may address hidden problems in your current environment!). If you run pip check and run into stuff you can’t figure out, please ask for help in our issue tracker or chat.
Test the new version of pip.
While we have tried to make sure that pip’s test suite covers as many cases as we can, we are very aware that there are people using pip with many different workflows and build processes, and we will not be able to cover all of those without your help.
If you use pip to install your software, try out the new resolver and let us know if it works for you with pip install . Try:
installing several packages simultaneously
re-creating an environment using a requirements.txt file
using pip install —force-reinstall to check whether it does what you think it should
using constraints files
the “Setups to test with special attention” and “Examples to try” below
If you have a build pipeline that depends on pip installing your dependencies for you, check that the new resolver does what you need.
Run your project’s CI (test suite, build process, etc.) using the new resolver, and let us know of any issues.
If you have encountered resolver issues with pip in the past, check whether the new resolver fixes them, and read Dealing with dependency conflicts . Also, let us know if the new resolver has issues with any workarounds you put in to address the current resolver’s limitations. We’ll need to ensure that people can transition off such workarounds smoothly.
If you develop or support a tool that wraps pip or uses it to deliver part of your functionality, please test your integration with pip 20.3.
Troubleshoot and try these workarounds if necessary.
If pip is taking longer to install packages, read Dependency resolution backtracking for ways to reduce the time pip spends backtracking due to dependency conflicts.
If you don’t want pip to actually resolve dependencies, use the —no-deps option. This is useful when you have a set of package versions that work together in reality, even though their metadata says that they conflict. For guidance on a long-term fix, read Dealing with dependency conflicts .
If you run into resolution errors and need a workaround while you’re fixing their root causes, you can choose the old resolver behavior using the flag —use-deprecated=legacy-resolver . This will work until we release pip 21.0 (see Deprecation timeline ).
Please report bugs through the resolver testing survey.
Setups to test with special attention#
Requirements files with 100+ packages
Installation workflows that involve multiple requirements files
Requirements files that include hashes ( Hash-checking Mode ) or pinned dependencies (perhaps as output from pip-compile within pip-tools )
Пакетный менеджер pip
pip — система управления пакетами, которая используется для установки и управления программными пакетами, написанными на Python. Множество этих пакетов размещается в онлайн репозитории Python Package Index (PyPI).
Что такое пакет?
В Python пакет представляет из себя архив со следующими файлами:
- конфигурационные файлы самого пакета
- зависимости которые необходимо установить с пакетом, т.е. другие пакеты
- модули пакета, которые подключаются с помощью import
При этом в пакете может содержаться любое количество модулей, для которых нужно использовать свой import , а сами подключаемые модули могут состоять из иерархии подмодулей.
Проверка установки pip
Проверить установленный pip можно из командной строки, например, с флагом для вывода его версии:
Если у вас не добавлены пути до модулей в переменные окружения системы, но сам Python добавлен в PATH , то можно проверить следующим образом:
В данном случае флаг -m отвечает за вызов модуля pip , после которого идут аргументы уже для вызываемого модуля.
Последним вариантом будет перейти в директорию со скриптами Python и вызвать оттуда pip , если он там есть (для Windows):
Установка
Данный шаг для тех, у кого pip оказался не установленным. Необходимую инструкцию по установке можно найти на этой странице: https://pypi.org/project/pip/
Использование
Командный интерфейс
Если вы впервые с таким сталкиваетесь, то не стоит бояться. Чтобы получить вспомогательную информацию, достаточно использовать флаг —help с командой pip (аналогично и с самим интерпретатором Python):
Дальнейшая информация будет аналогично приводится с помощью прямого вызова модуля pip с помощью интерпретатора Python, но можно использовать и кратко, если у вас все в порядке с установкой:
Как установить пакет?
Простая установка осуществляется с указанием подкоманды install и названием пакета (или нескольких):
После успешной установки пакета его можно уже можно подключать в проекте и работать с ним:
Подсказка
Список установленных пакетов всегда можно посмотреть с помощью подкоманды list :
Как удалить пакет?
Удалить пакет также просто, как и установить. Достаточно использовать подкоманду uninstall и указать название пакета:
Иногда pip может уточнить что вы удаляется и спросить продолжить ли. В том случае надо набрать на клавиатуре y и жмакнуть кнопку Enter .
Name already in use
pip / docs / html / user_guide.rst
- Go to file T
- Go to line L
- Copy path
- Copy permalink
- Open with Desktop
- View raw
- Copy raw contents Copy raw contents
Copy raw contents
Copy raw contents
pip is a command line program. When you install pip, a pip command is added to your system, which can be run from the command prompt as follows:
pip supports installing from PyPI, version control, local projects, and directly from distribution files.
The most common scenario is to install from PyPI using :ref:`Requirement Specifiers`
For more information and examples, see the :ref:`pip install` reference.
Basic Authentication Credentials
Using a Proxy Server
When installing packages from PyPI, pip requires internet access, which in many corporate environments requires an outbound HTTP proxy server.
pip can be configured to connect through a proxy server in various ways:
- using the —proxy command-line option to specify a proxy in the form scheme://[user:passwd@]proxy.server:port
- using proxy in a :ref:`config-file`
- by setting the standard environment-variables http_proxy , https_proxy and no_proxy .
- using the environment variable PIP_USER_AGENT_USER_DATA to include a JSON-encoded string in the user-agent variable used in pip’s requests.
«Requirements files» are files containing a list of items to be installed using :ref:`pip install` like so:
Details on the format of the files are here: :ref:`requirements-file-format` .
Logically, a Requirements file is just a list of :ref:`pip install` arguments placed in a file. Note that you should not rely on the items in the file being installed by pip in any particular order.
Requirements files can also be served via a URL, e.g. http://example.com/requirements.txt besides as local files, so that they can be stored and served in a centralized place.
In practice, there are 4 common uses of Requirements files:
Requirements files are used to hold the result from :ref:`pip freeze` for the purpose of achieving :doc:`topics/repeatable-installs` . In this case, your requirement file contains a pinned version of everything that was installed when pip freeze was run.
Requirements files are used to force pip to properly resolve dependencies. pip 20.2 and earlier doesn’t have true dependency resolution, but instead simply uses the first specification it finds for a project. E.g. if pkg1 requires pkg3>=1.0 and pkg2 requires pkg3>=1.0,<=2.0 , and if pkg1 is resolved first, pip will only use pkg3>=1.0 , and could easily end up installing a version of pkg3 that conflicts with the needs of pkg2 . To solve this problem, you can place pkg3>=1.0,<=2.0 (i.e. the correct specification) into your requirements file directly along with the other top level requirements. Like so:
Requirements files are used to force pip to install an alternate version of a sub-dependency. For example, suppose ProjectA in your requirements file requires ProjectB , but the latest version (v1.3) has a bug, you can force pip to accept earlier versions like so:
Requirements files are used to override a dependency with a local patch that lives in version control. For example, suppose a dependency SomeDependency from PyPI has a bug, and you can’t wait for an upstream fix. You could clone/copy the src, make the fix, and place it in VCS with the tag sometag . You’d reference it in your requirements file with a line like so:
If SomeDependency was previously a top-level requirement in your requirements file, then replace that line with the new line. If SomeDependency is a sub-dependency, then add the new line.
It’s important to be clear that pip determines package dependencies using install_requires metadata, not by discovering requirements.txt files embedded in projects.
Constraints files are requirements files that only control which version of a requirement is installed, not whether it is installed or not. Their syntax and contents is a subset of :ref:`Requirements Files` , with several kinds of syntax not allowed: constraints must have a name, they cannot be editable, and they cannot specify extras. In terms of semantics, there is one key difference: Including a package in a constraints file does not trigger installation of the package.
Use a constraints file like so:
Constraints files are used for exactly the same reason as requirements files when you don’t know exactly what things you want to install. For instance, say that the «helloworld» package doesn’t work in your environment, so you have a local patched version. Some things you install depend on «helloworld», and some don’t.
One way to ensure that the patched version is used consistently is to manually audit the dependencies of everything you install, and if «helloworld» is present, write a requirements file to use when installing that thing.
Constraints files offer a better way: write a single constraints file for your organisation and use that everywhere. If the thing being installed requires «helloworld» to be installed, your fixed version specified in your constraints file will be used.
Constraints file support was added in pip 7.1. In :ref:`Resolver changes 2020` we did a fairly comprehensive overhaul, removing several undocumented and unsupported quirks from the previous implementation, and stripped constraints files down to being purely a way to specify global (version) limits for packages.
Same as requirements files, constraints files can also be served via a URL, e.g. http://example.com/constraints.txt, so that your organization can store and serve them in a centralized place.
Installing from Wheels
«Wheel» is a built, archive format that can greatly speed installation compared to building and installing from source archives. For more information, see the Wheel docs , PEP 427, and PEP 425.
pip prefers Wheels where they are available. To disable this, use the :ref:`—no-binary <install_—no-binary>` flag for :ref:`pip install` .
If no satisfactory wheels are found, pip will default to finding source archives.
To install directly from a wheel archive:
To include optional dependencies provided in the provides_extras metadata in the wheel, you must add quotes around the install target name:
In the future, the path[extras] syntax may become deprecated. It is recommended to use PEP 508 syntax wherever possible.
For the cases where wheels are not available, pip offers :ref:`pip wheel` as a convenience, to build wheels for all your requirements and dependencies.
:ref:`pip wheel` requires the wheel package to be installed, which provides the «bdist_wheel» setuptools extension that it uses.
To build wheels for your requirements and all their dependencies to a local directory:
And then to install those requirements just using your local directory of wheels (and not from PyPI):
pip is able to uninstall most packages like so:
pip also performs an automatic uninstall of an old version of a package before upgrading to a newer version.
For more information and examples, see the :ref:`pip uninstall` reference.
To list installed packages:
To list outdated packages, and show the latest version available:
To show details about an installed package:
For more information and examples, see the :ref:`pip list` and :ref:`pip show` reference pages.
Searching for Packages
pip can search PyPI for packages using the pip search command:
The query will be used to search the names and summaries of all packages.
For more information and examples, see the :ref:`pip search` reference.
pip comes with support for command line completion in bash, zsh and fish.
To setup for bash:
To setup for zsh:
To setup for fish:
To setup for powershell:
Alternatively, you can use the result of the completion command directly with the eval function of your shell, e.g. by adding the following to your startup file:
Installing from local packages
In some cases, you may want to install from local packages only, with no traffic to PyPI.
First, download the archives that fulfill your requirements:
Note that pip download will look in your wheel cache first, before trying to download from PyPI. If you’ve never installed your requirements before, you won’t have a wheel cache for those items. In that case, if some of your requirements don’t come as wheels from PyPI, and you want wheels, then run this instead:
«Only if needed» Recursive Upgrade
pip install —upgrade now has a —upgrade-strategy option which controls how pip handles upgrading of dependencies. There are 2 upgrade strategies supported:
- eager : upgrades all dependencies regardless of whether they still satisfy the new parent requirements
- only-if-needed : upgrades a dependency only if it does not satisfy the new parent requirements
The default strategy is only-if-needed . This was changed in pip 10.0 due to the breaking nature of eager when upgrading conflicting dependencies.
It is important to note that —upgrade affects direct requirements (e.g. those specified on the command-line or via a requirements file) while —upgrade-strategy affects indirect requirements (dependencies of direct requirements).
As an example, say SomePackage has a dependency, SomeDependency , and both of them are already installed but are not the latest available versions:
- pip install SomePackage : will not upgrade the existing SomePackage or SomeDependency .
- pip install —upgrade SomePackage : will upgrade SomePackage , but not SomeDependency (unless a minimum requirement is not met).
- pip install —upgrade SomePackage —upgrade-strategy=eager : upgrades both SomePackage and SomeDependency .
As an historic note, an earlier «fix» for getting the only-if-needed behaviour was:
A proposal for an upgrade-all command is being considered as a safer alternative to the behaviour of eager upgrading.
With Python 2.6 came the «user scheme» for installation, which means that all Python distributions support an alternative install location that is specific to a user. The default location for each OS is explained in the python documentation for the site.USER_BASE variable. This mode of installation can be turned on by specifying the :ref:`—user <install_—user>` option to pip install .
Moreover, the «user scheme» can be customized by setting the PYTHONUSERBASE environment variable, which updates the value of site.USER_BASE .
To install «SomePackage» into an environment with site.USER_BASE customized to ‘/myappenv’, do the following:
pip install —user follows four rules:
- When globally installed packages are on the python path, and they conflict with the installation requirements, they are ignored, and not uninstalled.
- When globally installed packages are on the python path, and they satisfy the installation requirements, pip does nothing, and reports that requirement is satisfied (similar to how global packages can satisfy requirements when installing packages in a —system-site-packages virtualenv).
- pip will not perform a —user install in a —no-site-packages virtualenv (i.e. the default kind of virtualenv), due to the user site not being on the python path. The installation would be pointless.
- In a —system-site-packages virtualenv, pip will not install a package that conflicts with a package in the virtualenv site-packages. The —user installation would lack sys.path precedence and be pointless.
To make the rules clearer, here are some examples:
From within a —no-site-packages virtualenv (i.e. the default kind):
From within a —system-site-packages virtualenv where SomePackage==0.3 is already installed in the virtualenv:
From within a real python, where SomePackage is not installed globally:
From within a real python, where SomePackage is installed globally, but is not the latest version:
From within a real python, where SomePackage is installed globally, and is the latest version:
Fixing conflicting dependencies
Using pip from your program
As noted previously, pip is a command line program. While it is implemented in Python, and so is available from your Python code via import pip , you must not use pip’s internal APIs in this way. There are a number of reasons for this:
- The pip code assumes that it is in sole control of the global state of the program. pip manages things like the logging system configuration, or the values of the standard IO streams, without considering the possibility that user code might be affected.
- pip’s code is not thread safe. If you were to run pip in a thread, there is no guarantee that either your code or pip’s would work as you expect.
- pip assumes that once it has finished its work, the process will terminate. It doesn’t need to handle the possibility that other code will continue to run after that point, so (for example) calling pip twice in the same process is likely to have issues.
This does not mean that the pip developers are opposed in principle to the idea that pip could be used as a library — it’s just that this isn’t how it was written, and it would be a lot of work to redesign the internals for use as a library, handling all of the above issues, and designing a usable, robust and stable API that we could guarantee would remain available across multiple releases of pip. And we simply don’t currently have the resources to even consider such a task.
What this means in practice is that everything inside of pip is considered an implementation detail. Even the fact that the import name is pip is subject to change without notice. While we do try not to break things as much as possible, all the internal APIs can change at any time, for any reason. It also means that we generally won’t fix issues that are a result of using pip in an unsupported way.
It should also be noted that installing packages into sys.path in a running Python process is something that should only be done with care. The import system caches certain data, and installing new packages while a program is running may not always behave as expected. In practice, there is rarely an issue, but it is something to be aware of.
Having said all of the above, it is worth covering the options available if you decide that you do want to run pip from within your program. The most reliable approach, and the one that is fully supported, is to run pip in a subprocess. This is easily done using the standard subprocess module:
If you want to process the output further, use one of the other APIs in the module. We are using freeze here which outputs installed packages in requirements format.:
If you don’t want to use pip’s command line functionality, but are rather trying to implement code that works with Python packages, their metadata, or PyPI, then you should consider other, supported, packages that offer this type of ability. Some examples that you could consider include:
- packaging — Utilities to work with standard package metadata (versions, requirements, etc.)
- setuptools (specifically pkg_resources ) — Functions for querying what packages the user has installed on their system.
- distlib — Packaging and distribution utilities (including functions for interacting with PyPI).
Changes to the pip dependency resolver in 20.3 (2020)
pip 20.3 has a new dependency resolver, on by default for Python 3 users. (pip 20.1 and 20.2 included pre-release versions of the new dependency resolver, hidden behind optional user flags.) Read below for a migration guide, how to invoke the legacy resolver, and the deprecation timeline. We also made a two-minute video explanation you can watch.
We will continue to improve the pip dependency resolver in response to testers’ feedback. Please give us feedback through the resolver testing survey.
The big change in this release is to the pip dependency resolver within pip.
Computers need to know the right order to install pieces of software («to install x , you need to install y first»). So, when Python programmers share software as packages, they have to precisely describe those installation prerequisites, and pip needs to navigate tricky situations where it’s getting conflicting instructions. This new dependency resolver will make pip better at handling that tricky logic, and make pip easier for you to use and troubleshoot.
The most significant changes to the resolver are:
- It will reduce inconsistency: it will no longer install a combination of packages that is mutually inconsistent. In older versions of pip, it is possible for pip to install a package which does not satisfy the declared requirements of another installed package. For example, in pip 20.0, pip install «six<1.12» «virtualenv==20.0.2» does the wrong thing, “successfully” installing six==1.11 , even though virtualenv==20.0.2 requires six>=1.12.0,<2 (defined here). The new resolver, instead, outright rejects installing anything if it gets that input.
- It will be stricter — if you ask pip to install two packages with incompatible requirements, it will refuse (rather than installing a broken combination, like it did in previous versions).
So, if you have been using workarounds to force pip to deal with incompatible or inconsistent requirements combinations, now’s a good time to fix the underlying problem in the packages, because pip will be stricter from here on out.
This also means that, when you run a pip install command, pip only considers the packages you are installing in that command, and may break already-installed packages. It will not guarantee that your environment will be consistent all the time. If you pip install x and then pip install y , it’s possible that the version of y you get will be different than it would be if you had run pip install x y in a single command. We are considering changing this behavior (per :issue:`7744` ) and would like your thoughts on what pip’s behavior should be; please answer our survey on upgrades that create conflicts.
We are also changing our support for :ref:`Constraints Files` , editable installs, and related functionality. We did a fairly comprehensive overhaul and stripped constraints files down to being purely a way to specify global (version) limits for packages, and so some combinations that used to be allowed will now cause errors. Specifically:
- Constraints don’t override the existing requirements; they simply constrain what versions are visible as input to the resolver (see :issue:`9020` )
- Providing an editable requirement ( -e . ) does not cause pip to ignore version specifiers or constraints (see :issue:`8076` ), and if you have a conflict between a pinned requirement and a local directory then pip will indicate that it cannot find a version satisfying both (see :issue:`8307` )
- Hash-checking mode requires that all requirements are specified as a == match on a version and may not work well in combination with constraints (see :issue:`9020` and :issue:`8792` )
- If necessary to satisfy constraints, pip will happily reinstall packages, upgrading or downgrading, without needing any additional command-line options (see :issue:`8115` and :doc:`development/architecture/upgrade-options` )
- Unnamed requirements are not allowed as constraints (see :issue:`6628` and :issue:`8210` )
- Links are not allowed as constraints (see :issue:`8253` )
- Constraints cannot have extras (see :issue:`6628` )
Per our :ref:`Python 2 Support` policy, pip 20.3 users who are using Python 2 will use the legacy resolver by default. Python 2 users should upgrade to Python 3 as soon as possible, since in pip 21.0 in January 2021, pip dropped support for Python 2 altogether.
How to upgrade and migrate
Install pip 20.3 with python -m pip install —upgrade pip .
Validate your current environment by running pip check . This will report if you have any inconsistencies in your set of installed packages. Having a clean installation will make it much less likely that you will hit issues with the new resolver (and may address hidden problems in your current environment!). If you run pip check and run into stuff you can’t figure out, please ask for help in our issue tracker or chat.
Test the new version of pip.
While we have tried to make sure that pip’s test suite covers as many cases as we can, we are very aware that there are people using pip with many different workflows and build processes, and we will not be able to cover all of those without your help.
- If you use pip to install your software, try out the new resolver and let us know if it works for you with pip install . Try:
- installing several packages simultaneously
- re-creating an environment using a requirements.txt file
- using pip install —force-reinstall to check whether it does what you think it should
- using constraints files
- the «Setups to test with special attention» and «Examples to try» below
Troubleshoot and try these workarounds if necessary.
- If pip is taking longer to install packages, read :doc:`Dependency resolution backtracking <topics/dependency-resolution>` for ways to reduce the time pip spends backtracking due to dependency conflicts.
- If you don’t want pip to actually resolve dependencies, use the —no-deps option. This is useful when you have a set of package versions that work together in reality, even though their metadata says that they conflict. For guidance on a long-term fix, read :ref:`Fixing conflicting dependencies` .
- If you run into resolution errors and need a workaround while you’re fixing their root causes, you can choose the old resolver behavior using the flag —use-deprecated=legacy-resolver . This will work until we release pip 21.0 (see :ref:`Deprecation timeline for 2020 resolver changes` ).
Please report bugs through the resolver testing survey.
Setups to test with special attention
- Requirements files with 100+ packages
- Installation workflows that involve multiple requirements files
- Requirements files that include hashes ( :ref:`hash-checking mode` ) or pinned dependencies (perhaps as output from pip-compile within pip-tools )
- Using :ref:`Constraints Files`
- Continuous integration/continuous deployment setups
- Installing from any kind of version control systems (i.e., Git, Subversion, Mercurial, or CVS), per :doc:`topics/vcs-support`
- Installing from source code held in local directories
Examples to try
- pip install
- pip uninstall
- pip check
- pip cache
Specific things we’d love to get feedback on:
- Cases where the new resolver produces the wrong result, obviously. We hope there won’t be too many of these, but we’d like to trap such bugs before we remove the legacy resolver.
- Cases where the resolver produced an error when you believe it should have been able to work out what to do.
- Cases where the resolver gives an error because there’s a problem with your requirements, but you need better information to work out what’s wrong.
- If you have workarounds to address issues with the current resolver, does the new resolver let you remove those workarounds? Tell us!
Please let us know through the resolver testing survey.
We plan for the resolver changeover to proceed as follows, using :ref:`Feature Flags` and following our :ref:`Release Cadence` :
- pip 20.1: an alpha version of the new resolver was available, opt-in, using the optional flag —unstable-feature=resolver . pip defaulted to legacy behavior.
- pip 20.2: a beta of the new resolver was available, opt-in, using the flag —use-feature=2020-resolver . pip defaulted to legacy behavior. Users of pip 20.2 who want pip to default to using the new resolver can run pip config set global.use-feature 2020-resolver (for more on that and the alternate PIP_USE_FEATURE environment variable option, see issue 8661).
- pip 20.3: pip defaults to the new resolver in Python 3 environments, but a user can opt-out and choose the old resolver behavior, using the flag —use-deprecated=legacy-resolver . In Python 2 environments, pip defaults to the old resolver, and the new one is available using the flag —use-feature=2020-resolver .
- pip 21.0: pip uses new resolver by default, and the old resolver is no longer supported. It will be removed after a currently undecided amount of time, as the removal is dependent on pip’s volunteer maintainers’ availability. Python 2 support is removed per our :ref:`Python 2 Support` policy.
Since this work will not change user-visible behavior described in the pip documentation, this change is not covered by the :ref:`Deprecation Policy` .
Context and followup
As discussed in our announcement on the PSF blog, the pip team are in the process of developing a new «dependency resolver» (the part of pip that works out what to install based on your requirements).
We’re tracking our rollout in :issue:`6536` and you can watch for announcements on the low-traffic packaging announcements list and the official Python blog.