install python packages from github
This week, I wanted to turn a simple python script into a package and pull it into another project. Basically, I had some functionality that I wanted to isolate so why not make it a package. I’ve worked on other python packages before, but have never rolled my own so I thought this would be fun.
The python package
First up, the package file structure. After a bit of researching, a python package must contain a setup.py file and a subfolder. This stackoverflow answer was helpful for a basic package format.
After setting up my project structure, I looked into adding a very basic setup.py file. This was my starting setup.py file (based on this so question). Of course, check the docs for more details…
This is the test package I created: python_people_names. This basically takes a person’s name as a string and returns a dictionary with the name split into parts (first_name, middle_name, last_name, suffix_name). It’s also a wip so no guarantees if you use it… 🙂
Importing the package from github
So now I have a python package on github and want to pull it into my project without submitting it to the official Python Package Index. First step, install git on my virtual machine (else it won’t work and you’ll get an error).
Then pip install from github. You can add the —update flag if you make changes to your package and need the latest version of the package. In this case, I ran:
Then to use the package in the project, import & use:
This article is similar to one I wrote about installing node modules from github…
Import python modules straight from github
In Go we have ability to import modules from github, like:
It’s a bit controversial feature, but sometimes it’s useful. And I was interested, is it possible to implement something like that in python. TLDR it’s possible with import_from_github_com package:
So, how it works, according to PEP-0302 we have special sys.meta_path with importer objects and every importer should implement finder protocol with find_module(module_name: str, package_path: [str]) -> Loader|None . Now we need to implement finder that handles modules, which path starts with github_com , like:
And now we need GithubComLoader that implements loader protocol with load_module(fullname: str) -> None , I’ll skip private methods of the loader here, they’re straightforward and not interesting in context of the article:
MineRobber9000 / README.md
Allows you to import Python modules from the top level of a GitHub repository. Basically, golang’s import semantics but in Python fashion.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| from requests import get |
| from enum import auto , IntEnum |
| from importlib . machinery import ModuleSpec |
| from urllib . parse import urljoin |
| from os . path import join |
| class GithubImportState ( IntEnum ): |
| USER = auto () |
| REPO = auto () |
| FILE = auto () |
| class GithubImportFinder : |
| def find_spec ( self , modname , path = None , mod = None ): |
| package , dot , module = modname . rpartition ( "." ) |
| if not dot : |
| spec = ModuleSpec ( |
| modname , |
| GithubImportLoader (), |
| origin = "https://github.com/" + module , |
| loader_state = GithubImportState . USER , |
| is_package = True ) |
| spec . submodule_search_locations = [] |
| return spec |
| else : |
| user , dot2 , repo = package . rpartition ( "." ) |
| if not dot2 : |
| spec = ModuleSpec ( |
| modname , |
| GithubImportLoader (), |
| origin = "https://github.com/" + "/" . join ([ package , module ]), |
| loader_state = GithubImportState . REPO , |
| is_package = True ) |
| spec . submodule_search_locations = [] |
| return spec |
| path = urljoin ( "https://github.com" , |
| join ( user , repo , "raw" , "master" , module + ".py" )) |
| return ModuleSpec ( |
| modname , |
| GithubImportLoader (), |
| origin = path , |
| loader_state = GithubImportState . FILE ) |
| class GithubImportLoader : |
| def create_module ( self , spec ): |
| return None # default semantics |
| def exec_module ( self , module ): |
| path = module . __spec__ . origin |
| if module . __spec__ . loader_state != GithubImportState . USER : |
| setattr ( sys . modules [ module . __package__ ], |
| module . __name__ . rpartition ( "." )[ — 1 ], module ) |
| if module . __spec__ . loader_state == GithubImportState . FILE : |
| # load the module |
| code = get ( path ) |
| if code . status_code != 200 : |
| print ( path , code ) |
| raise ModuleNotFoundError ( f"No module named < module . __name__ >" ) |
| code = code . text |
| exec ( code , module . __dict__ ) |
| import sys |
| FINDER = GithubImportFinder () |
| sys . meta_path . append ( FINDER ) |
There’s actually a python module that allows you to do that: https://github.com/operatorequals/httpimport — supports GitHub/BitBucket/GitLab as well.
@operatorequals check the implementation of sys.meta_path finder!
BTW: It was suggested to include that in built-in python — didn’t go as planned: https://lwn.net/Articles/732194/
There’s actually a python module that allows you to do that: https://github.com/operatorequals/httpimport — supports GitHub/BitBucket/GitLab as well.
Thanks for mentioning the module señor.
@operatorequals check the implementation of sys.meta_path finder!
I’m using a similar idea in:
https://github.com/operatorequals/httpimport/blob/master/httpimport.py#L345
Yet, I use insert so I can put the Finder as the first element in sys.meta_path so it gets queried first about the module and getting general priority.
There’s actually a python module that allows you to do that: https://github.com/operatorequals/httpimport — supports GitHub/BitBucket/GitLab as well.
@dzervas Nice! I actually didn’t know about that. I was messing around with the Meta Path Finder stuff and accidentally made this. One of my friends mentioned that it was the same import model Golang used, so I figured I’d post it.
I actually wrote one that specifically allows you to register modules as coming from anywhere, given a package name (so for instance you could define a package as coming from anywhere, even not a GitHub/BitBucket/GitLab/Gitea/whatever), but I realized I could use the package import semantics to create a simple way to import from GitHub.
Install Python Package from Github [Linux and Windows]
This article is about how to install python package from GitHub, for this, you must have the latest version of python installed in your system.
How to check if python is installed or not?
Before getting into the procedure of how to install python package from GitHub the latest python version installed is a must. So to check if your system has python go to the command prompt and type
This will be the output if the version is installed
But if it is not installed, python not found will be displayed on the command prompt.
Install Python Package from Github on Windows
How to install a python package using pip install?
Usually, in python, a package is installed using the command
But sometimes, if a newer version of the library of the package is available, it might not get installed using this command, for that, we will learn how to download and install python package from Github.
Install Python Package from Github in VSCode
First of all, go to Github, and search for the package that you need to install. You need to make sure that the package has a “ setup.py ” file, for example in our case if we want to install the flask package from GitHub, I will search for the flask python package, and then check if it has the file “ setup.py ”.
After making sure that there is a setup file, go to your VS Code, in case you do not have VS Code, you can download it and set up a virtual environment.
The command to set up the virtual environment is :
After setting up the environment, go to your source using the following command
And then write the following command to install the package using GitHub
In my case, if I want to download the flask package I will write
Install a Python Package from Github Using Windows Command Prompt
This method is simpler, just open the windows command prompt to install python package from GitHub and write the following command
In my case, if I want to download the flask package I will write
The command prompt will show some output and will notify if the package is installed or not.
If successful, the following output will be shown :
Install Python Package from GitHub on Linux
Install git on Linux (pre-requisite)
To install python package from Github on Linux you need to have git installed on your system. for that please type the following command on your Linux terminal.
The following output will be shown :
Install python package on Linux using pip
Let’s suppose we want to install a package «moviespy» from GitHub, go to the Linux terminal and type the following command:
The following output will be shown
Install Python Package from Github by cloning the git repository
Another way to install Python Package from Github by cloning the git repository, for this process write the following commands
the output will be as follows
Then move to the project directory using the cd command
Run the setup.py file to install the package
Conclusion
In this article we have studied how to install a package from python using GitHub, we have studied installation in two environments, one is the virtual environment created using vs code and one is using the windows command prompt.
Further study
Didn’t find what you were looking for? Perform a quick search across GoLinuxCloud
If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.
For any other feedbacks or questions you can either use the comments section or contact me form.