How to run .py files (python scripts) on Google Colab
We all know how to run jupyter notebook(.ipynb) files on Google colab. Python script files can also be run on Google colab by a very simple trick. This is a very useful trick, as it enables you to make use of the GPU/TPU provided by Colab to run your .py file.
First, you have to create a new notebook or open an existing one.
Then, click on this icon(shown in red arrow) to upload files.
Upload your .py files.
Insert a code cell in the notebook and write a line in this format :
Execute the cell. You will see output as usual.
You can also view and edit an uploaded python file by double-clicking the filename in files section.
Python Installation — Colab Notebook
The Earth Engine Python API can be deployed in a Google Colaboratory notebook. Colab notebooks are Jupyter notebooks that run in the cloud and are highly integrated with Google Drive, making them easy to set up, access, and share. If you are unfamiliar with Google Colab or Jupyter notebooks, please spend some time exploring the Colab welcome site.
The following sections describe deploying Earth Engine in Google Colab and visualizing maps and charts using third‑party Python packages.
Open a Colab notebook
Notebooks can be opened from either Google Drive or the Colaboratory interface.
New notebook
Google Drive
Open Google Drive and create a new file.
- New > More > Colaboratory
- Right click in a folder and select More > Colaboratory from the context menu.
Colab interface
Visit the Colab site and create a new file.
- File > New > New Python 3 notebook
- If you have interacted with Colab previously, visiting the above linked site will provide you with a file explorer where you can start a new file using the dropdown menu at the bottom of the window.
Existing notebook
Existing notebook files (.ipynb) can be opened from Google Drive and the Colab interface.
Google Drive
Colab notebooks can exist in various folders in Google Drive depending on where notebooks files were created. Notebooks created in Google Drive will exist in the folder they were created or moved to. Notebooks created from the Colab interface will default to a folder called 'Colab Notebooks' which is automatically added to the 'My Drive' folder of your Google Drive when you start working with Colab.
Colab files can be identified by a yellow 'CO' symbol and '.ipynb' file extension. Open files by either doubling clicking on them and selecting Open with > Colaboratory from the button found at the top of the resulting page or by right clicking on a file and selecting Open with > Colaboratory from the file's context menu.
Colab interface
Opening notebooks from the Colab interface allows you to access existing files from Google Drive, GitHub, and local hardware. Visiting the Colab interface after initial use will result in a file explorer modal appearing. From the tabs at the top of the file explorer, select a source and navigate to the .ipynb file you wish to open. The file explorer can also be accessed from the Colab interface by selecting File > Open notebook or using the Ctrl+O keyboard combination.
Import API and get credentials
This section demonstrates how to import the Earth Engine Python API and authenticate access. This content is also available as a Colab notebook:
The Earth Engine API is included by default in Google Colaboratory so requires only importing and authenticating. These steps must be completed for each new Colab session or if you restart your Colab kernel or if your Colab virtual machine is recycled due to inactivity.
Import the API
Run the following cell to import the API into your session.
Authenticate and initialize
Run the ee.Authenticate function to authenticate your access to Earth Engine servers and ee.Initialize to initialize it. Add a code cell, enter the following lines, and run the cell.
You'll be asked to authorize access to your Earth Engine account. Follow the instructions printed to the cell to complete this step.
Test the API
Test the API by printing the elevation of Mount Everest. Note that before using the API you must initialize it. Run the following Python script in a new cell.
Map visualization
ee.Image objects can be displayed to notebook output cells. The following two examples demonstrate displaying a static image and an interactive map.
Static image
The IPython.display module contains the Image function, which can display the results of a URL representing an image generated from a call to the Earth Engine getThumbUrl function. The following script will display a thumbnail of a global elevation model.
Interactive map
The folium package can be used to display ee.Image objects on an interactive Leaflet map. Folium has no default method for handling tiles from Earth Engine, so one must be defined and added to the folium.Map module before use.
The following script provides an example of adding a method for handing Earth Engine tiles and using it to display an elevation model to a Leaflet map.
Chart visualization
Some Earth Engine functions produce tabular data that can be plotted by data visualization packages such as matplotlib . The following example demonstrates the display of tabular data from Earth Engine as a scatter plot. See Charting in Colaboratory for more information.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
import its own ipynb files on google colab
I am working with jupyter notebook using google colab(all the files are in the drive). I have 2 files: Exploratory_Data_Analysis.ipynb et PCA.ipynb. I want to import to use the data comming from first one in the secon one. using only the jupyter notebook locally(not with google colaboratory), the importing is working simply by doing this:
But with google colab I tried the following:
and getting this:
are there any other ways to import its own ipynb files on google colab?
3 Answers 3
Have you managed to import your notebook/ipynb file to a Google Colab project? The way I migrate my ongoing work in local jupyter-notebook to Google Colab is by using Github and Clouderizer. This method also allows me work on jupyter notebook environment that as if I do it locally but able to sync my work to Google Colab instantly. In addition, this method allows me to import modules .ipynb/.py to my ongoing notebook that I’m working on by simply do something like import <my own python/ipynb module> . I would recommend this setup instead of using hairy linux command line on Google Colab.
Here is the tutorial how to easily setup your notebook from github to Google Colab using Clouderizer: Medium tutorial.
Basically, these are the steps you need to setup your ipynb notebook as well as the dataset folder using Clouderizer:
How to use Google Colab
Google Colab is an interactive, all-cloud, easy-to-use, and colaboratory programming environment for those working on artificial intelligence and deep learning projects. It is free of cost and no software installation is required. If you are familiar with Jupyter notebook, you can think of Google Colab as a powered version of Jupyter notebook with multiple useful features hosted on Google’s cloud servers.
How to use Google Colab?
You can click here to start using Google Colab. When you click on the link, a window like the one below will open. We will call our workspace notebook.
Before moving on to the notebook, let’s get acquainted with the simple settings. The settings button is next to our profile picture. From here we can control the settings of our Colab. Let’s start with the theme of our page first.
In terms of theme, Google Colab gives you 3 options:
- light – work on a white sheet.
- dark – a black work.
- adaptive – adjusting the colour of the page according to the time of day.
When you click the settings button, you can customise many settings such as font size, indentation, showing line numbers, connecting to Github, etc., apart from the theme, from the selections in the small window that opens.
Now let’s move on to the Colab Notebook. To create a new notebook, click File , then New notebook.
- New notebook – to create a new Interactive Python notebook,
- Upload notebook – to open an existing .ipynb file in Colab,
- Open notebook – to view and open a notebook already existing on the drive.
We see a page like the one below and by default the notebook named Untitled0.ipynb opens. We can change the name of the notebook as we wish.
Let’s create our first code block on Google Colab :
Now let’s explore the Google Colab toolbar more closely:
- File – Changing the location or name of the notebook, saving, opening/creating/loading a new notebook, etc.,
- Edit – Selecting, copying, deleting, pasting, finding and replacing cells, etc.
- View – Check the metadata related to the notebook, view the code history, hide the code or output in the selected cell, etc.
- Insert – Adding a new code cell, text cell or heading, etc.
- Runtime – Run all or only selected cells, stop execution, change hardware accelerator (GPU, TPU), etc.
- Tools – Like going to shortcuts, settings and commands etc.
- Help – Provides options for reading documentation, giving feedback, etc.
Now let’s see how we can comment on someone else’s or our own notebook or share our code with others:
Library installations
We can use the Linux terminal commands by having “!” character before the command. For example, we can load the libraries that are not installed by default by typing the following command in the code cell:
| !pip install [library_name] |
Default useful shortcuts:
Shortcut settings with the help of Control (or Command) + M + H. Top 10 useful shortcuts:
| Run the selected cell | Ctrl / Cmd + Enter |
| Run the selected cell and switch to the new cell | Shift + Enter |
| Adding code cell above | Ctrl / Cmd + M + A |
| Add code cell below | Ctrl / Cmd + M + B |
| Delete the selected cell | Ctrl / Cmd + M + D |
| Stop code execution | Ctrl / Cmd + M + I |
| Convert selected cell to code cell | Ctrl / Cmd + M + Y |
| Convert selected cell to text cell | Ctrl / Cmd + M + M |
| Convert selected lines to comments | Ctrl / Cmd + / |
| Find and replace | Ctrl / Cmd + M |
Making the above shortcuts a habit will definitely speed up your work. At the same time, you can customise all the shortcuts according to your wishes from the shortcut settings!
Google Colab tricks that save lives
“Snippets”!
A “snippet” is a reusable piece of code that you can use whenever you want. If you’ve done exploratory data analysis in python at least once, you probably remember the following code:
| import numpy as np import pandas as pd import matplotlib.pyplot as plt |
We can save these or more libraries so that we don’t have to rewrite the import process or any heavily used code every time. To do this, Tools -> Settings -> Paste custom snippet notebook URL! It is sufficient to perform the operations. Afterwards, you can make your work easier by accessing this or a ready-made “snippet” for a while.
Let’s choose one of the random snippets and click on insert:
2. To stop the disconnection
You decide to take a break from working at Google Colab, but when you return, your computer is disconnected from Colab! In fact, Google Colab will automatically disconnect if the computer is left idle for more than 30 minutes. If we run the code below in the console, which is opened by clicking F12, a click is made on the colab screen every 60 seconds. Thus, Colab assumes that the computer is not idle and your connection is not interrupted.
| function KeepClicking() < console.log( “Clicking” ); document.querySelector( “colab-toolbar-button#connect” ).click() >setInterval(KeepClicking, 60000 ) |
Google Colab advantages and disadvantages
Advantages:
- Pre-installed libraries: Google Colab includes predefined libraries for machine learning and deep learning and requires no installation for them.
- Collaboration: Another great feature that Google Colab offers is the collaboration feature. If you’re working on a project with more than one friend, Google Colab is an ideal choice.
- Access from any device: You can access your codes from any device with a simple Google sign-in process. All your Google Colab notebooks are saved in your Google Drive account, just like your Google Docs and Google Sheets files.
Disadvantages:
- Repeating the same operations every time: Except for connecting to the local runtime, you have to reinstall libraries that are not included in the standard Colab package each time you use it.
- Working at the same time: Sharing the code you wrote with your friend allows you to collaborate. However, the inability of two people to write or edit code at the same time slows down the project.
- Storage issue: Since Google Colab does not provide permanent storage, the uploaded files are removed when the session is restarted. Therefore, data is lost when the device is turned off and it is necessary to reload it each time.
- Limited space: Google Colab files are stored in 15GB free space in Google Drive; however, working on larger datasets requires more space.
- Limited time: Google Colab allows users to run their notebooks for a maximum of 12 hours a day, but to work longer, users need access to the paid version, Colab Pro, which allows programmers to stay connected around the clock.
What is Google Colab Pro / Colab Pro +?
Apart from the plan offered to us for free, there are also Colab Pro and Colab Pro + plans. Let’s take a look at the differences between them:
| colab | Colab Pro | Colab Pro + | |
| GPU | K80 | K80, T4, and T100 | K80, T4, and T100 |
| RAM | 16 GB | 32GB | 52GB |
| Operating environment | 12 hours | 24 hours | 24 hours |
| Running in the background | No | No | Yup |
| Price | Free | $9.99/month | $49.99/month |
The free version is useful for people who are at the beginning of their careers in this field and working on machine learning and deep learning projects that do not require very large memory, Colab Pro is useful for people who have been working in this field for one or more years, and Colab Pro + is useful for people who do projects that require heavy and very large memory.
You’ve learned all the basics you need to start coding at Colab, and more. No setup required, so you can start building your machine learning models from your phone, tablet, or laptop as long as you have an internet connection!
If you want to learn Python, don’t hesitate to sign up and begin your educational journey with Global AI Hub.