Как открыть папку в python

Working With Files in Python

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Practical Recipes for Working With Files in Python

Python has several built-in modules and functions for handling files. These functions are spread out over several modules such as os , os.path , shutil , and pathlib , to name a few. This article gathers in one place many of the functions you need to know in order to perform the most common operations on files in Python.

In this tutorial, you’ll learn how to:

• Retrieve file properties
• Create directories
• Match patterns in filenames
• Traverse directory trees
• Make temporary files and directories
• Delete files and directories
• Copy, move, or rename files and directories
• Create and extract ZIP and TAR archives
• Open multiple files using the fileinput module

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you’ll need to take your Python skills to the next level.

Python’s “with open(…) as …” Pattern

Reading and writing data to files using Python is pretty straightforward. To do this, you must first open files in the appropriate mode. Here’s an example of how to use Python’s “with open(…) as …” pattern to open a text file and read its contents:

open() takes a filename and a mode as its arguments. r opens the file in read only mode. To write data to a file, pass in w as an argument instead:

In the examples above, open() opens files for reading or writing and returns a file handle ( f in this case) that provides methods that can be used to read or write data to the file. Check out Reading and Writing Files in Python and Working With File I/O in Python for more information on how to read and write to files.

Getting a Directory Listing

Suppose your current working directory has a subdirectory called my_directory that has the following contents:

The built-in os module has a number of useful functions that can be used to list directory contents and filter the results. To get a list of all the files and folders in a particular directory in the filesystem, use os.listdir() in legacy versions of Python or os.scandir() in Python 3.x. os.scandir() is the preferred method to use if you also want to get file and directory properties such as file size and modification date.

Directory Listing in Legacy Python Versions

In versions of Python prior to Python 3, os.listdir() is the method to use to get a directory listing:

os.listdir() returns a Python list containing the names of the files and subdirectories in the directory given by the path argument:

A directory listing like that isn’t easy to read. Printing out the output of a call to os.listdir() using a loop helps clean things up:

Directory Listing in Modern Python Versions

In modern versions of Python, an alternative to os.listdir() is to use os.scandir() and pathlib.Path() .

os.scandir() was introduced in Python 3.5 and is documented in PEP 471. os.scandir() returns an iterator as opposed to a list when called:

The ScandirIterator points to all the entries in the current directory. You can loop over the contents of the iterator and print out the filenames:

Here, os.scandir() is used in conjunction with the with statement because it supports the context manager protocol. Using a context manager closes the iterator and frees up acquired resources automatically after the iterator has been exhausted. The result is a print out of the filenames in my_directory/ just like you saw in the os.listdir() example:

Another way to get a directory listing is to use the pathlib module:

The objects returned by Path are either PosixPath or WindowsPath objects depending on the OS.

pathlib.Path() objects have an .iterdir() method for creating an iterator of all files and folders in a directory. Each entry yielded by .iterdir() contains information about the file or directory such as its name and file attributes. pathlib was first introduced in Python 3.4 and is a great addition to Python that provides an object oriented interface to the filesystem.

In the example above, you call pathlib.Path() and pass a path argument to it. Next is the call to .iterdir() to get a list of all files and directories in my_directory .

pathlib offers a set of classes featuring most of the common operations on paths in an easy, object-oriented way. Using pathlib is more if not equally efficient as using the functions in os . Another benefit of using pathlib over os is that it reduces the number of imports you need to make to manipulate filesystem paths. For more information, read Python 3’s pathlib Module: Taming the File System.

Note: To get started with pathlib , check out Python Basics: File System Operations and the associated exercises.

Running the code above produces the following:

Using pathlib.Path() or os.scandir() instead of os.listdir() is the preferred way of getting a directory listing, especially when you’re working with code that needs the file type and file attribute information. pathlib.Path() offers much of the file and path handling functionality found in os and shutil , and it’s methods are more efficient than some found in these modules. We will discuss how to get file properties shortly.

Here are the directory-listing functions again:

Function	Description
os.listdir()	Returns a list of all files and folders in a directory
os.scandir()	Returns an iterator of all the objects in a directory including file attribute information
pathlib.Path.iterdir()	Returns an iterator of all the objects in a directory including file attribute information

These functions return a list of everything in the directory, including subdirectories. This might not always be the behavior you want. The next section will show you how to filter the results from a directory listing.

Listing All Files in a Directory

This section will show you how to print out the names of files in a directory using os.listdir() , os.scandir() , and pathlib.Path() . To filter out directories and only list files from a directory listing produced by os.listdir() , use os.path :

Here, the call to os.listdir() returns a list of everything in the specified path, and then that list is filtered by os.path.isfile() to only print out files and not directories. This produces the following output:

An easier way to list files in a directory is to use os.scandir() or pathlib.Path() :

Using os.scandir() has the advantage of looking cleaner and being easier to understand than using os.listdir() , even though it is one line of code longer. Calling entry.is_file() on each item in the ScandirIterator returns True if the object is a file. Printing out the names of all files in the directory gives you the following output:

Here’s how to list files in a directory using pathlib.Path() :

Here, you call .is_file() on each entry yielded by .iterdir() . The output produced is the same:

The code above can be made more concise if you combine the for loop and the if statement into a single generator expression. Dan Bader has an excellent article on generator expressions and list comprehensions.

The modified version looks like this:

This produces exactly the same output as the example before it. This section showed that filtering files or directories using os.scandir() and pathlib.Path() feels more intuitive and looks cleaner than using os.listdir() in conjunction with os.path .

Listing Subdirectories

To list subdirectories instead of files, use one of the methods below. Here’s how to use os.listdir() and os.path() :

Manipulating filesystem paths this way can quickly become cumbersome when you have multiple calls to os.path.join() . Running this on my computer produces the following output:

Here’s how to use os.scandir() :

As in the file listing example, here you call .is_dir() on each entry returned by os.scandir() . If the entry is a directory, .is_dir() returns True , and the directory’s name is printed out. The output is the same as above:

Here’s how to use pathlib.Path() :

Calling .is_dir() on each entry of the basepath iterator checks if an entry is a file or a directory. If the entry is a directory, its name is printed out to the screen, and the output produced is the same as the one from the previous example:

Getting File Attributes

Python makes retrieving file attributes such as file size and modified times easy. This is done through os.stat() , os.scandir() , or pathlib.Path() .

os.scandir() and pathlib.Path() retrieve a directory listing with file attributes combined. This can be potentially more efficient than using os.listdir() to list files and then getting file attribute information for each file.

The examples below show how to get the time the files in my_directory/ were last modified. The output is in seconds:

os.scandir() returns a ScandirIterator object. Each entry in a ScandirIterator object has a .stat() method that retrieves information about the file or directory it points to. .stat() provides information such as file size and the time of last modification. In the example above, the code prints out the st_mtime attribute, which is the time the content of the file was last modified.

The pathlib module has corresponding methods for retrieving file information that give the same results:

In the example above, the code loops through the object returned by .iterdir() and retrieves file attributes through a .stat() call for each file in the directory list. The st_mtime attribute returns a float value that represents seconds since the epoch. To convert the values returned by st_mtime for display purposes, you could write a helper function to convert the seconds into a datetime object:

This will first get a list of files in my_directory and their attributes and then call convert_date() to convert each file’s last modified time into a human readable form. convert_date() makes use of .strftime() to convert the time in seconds into a string.

The arguments passed to .strftime() are the following:

• %d : the day of the month
• %b : the month, in abbreviated form
• %Y : the year

Together, these directives produce output that looks like this:

The syntax for converting dates and times into strings can be quite confusing. To read more about it, check out the official documentation on it. Another handy reference that is easy to remember is http://strftime.org/ .

Making Directories

Sooner or later, the programs you write will have to create directories in order to store data in them. os and pathlib include functions for creating directories. We’ll consider these:

Function	Description
os.mkdir()	Creates a single subdirectory
pathlib.Path.mkdir()	Creates single or multiple directories
os.makedirs()	Creates multiple directories, including intermediate directories

Creating a Single Directory

To create a single directory, pass a path to the directory as a parameter to os.mkdir() :

If a directory already exists, os.mkdir() raises FileExistsError . Alternatively, you can create a directory using pathlib :

If the path already exists, mkdir() raises a FileExistsError :

To avoid errors like this, catch the error when it happens and let your user know:

Alternatively, you can ignore the FileExistsError by passing the exist_ok=True argument to .mkdir() :

This will not raise an error if the directory already exists.

Creating Multiple Directories

os.makedirs() is similar to os.mkdir() . The difference between the two is that not only can os.makedirs() create individual directories, it can also be used to create directory trees. In other words, it can create any necessary intermediate folders in order to ensure a full path exists.

os.makedirs() is similar to running mkdir -p in Bash. For example, to create a group of directories like 2018/10/05 , all you have to do is the following:

This will create a nested directory structure that contains the folders 2018, 10, and 05:

.makedirs() creates directories with default permissions. If you need to create directories with different permissions call .makedirs() and pass in the mode you would like the directories to be created in:

This creates the 2018/10/05 directory structure and gives the owner and group users read, write, and execute permissions. The default mode is 0o777 , and the file permission bits of existing parent directories are not changed. For more details on file permissions, and how the mode is applied, see the docs.

Run tree to confirm that the right permissions were applied:

This prints out a directory tree of the current directory. tree is normally used to list contents of directories in a tree-like format. Passing the -p and -i arguments to it prints out the directory names and their file permission information in a vertical list. -p prints out the file permissions, and -i makes tree produce a vertical list without indentation lines.

As you can see, all of the directories have 770 permissions. An alternative way to create directories is to use .mkdir() from pathlib.Path :

Passing parents=True to Path.mkdir() makes it create the directory 05 and any parent directories necessary to make the path valid.

By default, os.makedirs() and Path.mkdir() raise an OSError if the target directory already exists. This behavior can be overridden (as of Python 3.2) by passing exist_ok=True as a keyword argument when calling each function.

Running the code above produces a directory structure like the one below in one go:

I prefer using pathlib when creating directories because I can use the same function to create single or nested directories.

Filename Pattern Matching

After getting a list of files in a directory using one of the methods above, you will most probably want to search for files that match a particular pattern.

These are the methods and functions available to you:

• endswith() and startswith() string methods
• fnmatch.fnmatch()
• glob.glob()
• pathlib.Path.glob()

Each of these is discussed below. The examples in this section will be performed on a directory called some_directory that has the following structure:

If you’re following along using a Bash shell, you can create the above directory structure using the following commands:

This will create the some_directory/ directory, change into it, and then create sub_dir . The next line creates file1.py and file2.py in sub_dir , and the last line creates all the other files using expansion. To learn more about shell expansion, visit this site.

Using String Methods

Python has several built-in methods for modifying and manipulating strings. Two of these methods, .startswith() and .endswith() , are useful when you’re searching for patterns in filenames. To do this, first get a directory listing and then iterate over it:

The code above finds all the files in some_directory/ , iterates over them and uses .endswith() to print out the filenames that have the .txt file extension. Running this on my computer produces the following output:

Simple Filename Pattern Matching Using fnmatch

String methods are limited in their matching abilities. fnmatch has more advanced functions and methods for pattern matching. We will consider fnmatch.fnmatch() , a function that supports the use of wildcards such as * and ? to match filenames. For example, in order to find all .txt files in a directory using fnmatch , you would do the following:

This iterates over the list of files in some_directory and uses .fnmatch() to perform a wildcard search for files that have the .txt extension.

More Advanced Pattern Matching

Let’s suppose you want to find .txt files that meet certain criteria. For example, you could be only interested in finding .txt files that contain the word data , a number between a set of underscores, and the word backup in their filename. Something similar to data_01_backup , data_02_backup , or data_03_backup .

Using fnmatch.fnmatch() , you could do it this way:

Here, you print only the names of files that match the data_*_backup.txt pattern. The asterisk in the pattern will match any character, so running this will find all text files whose filenames start with the word data and end in backup.txt , as you can see from the output below:

Filename Pattern Matching Using glob

Another useful module for pattern matching is glob .

.glob() in the glob module works just like fnmatch.fnmatch() , but unlike fnmatch.fnmatch() , it treats files beginning with a period ( . ) as special.

UNIX and related systems translate name patterns with wildcards like ? and * into a list of files. This is called globbing.

For example, typing mv *.py python_files/ in a UNIX shell moves ( mv ) all files with the .py extension from the current directory to the directory python_files . The * character is a wildcard that means “any number of characters,” and *.py is the glob pattern. This shell capability is not available in the Windows Operating System. The glob module adds this capability in Python, which enables Windows programs to use this feature.

Here’s an example of how to use glob to search for all Python ( .py ) source files in the current directory:

glob.glob(‘*.py’) searches for all files that have the .py extension in the current directory and returns them as a list. glob also supports shell-style wildcards to match patterns:

This finds all text ( .txt ) files that contain digits in the filename:

glob makes it easy to search for files recursively in subdirectories too:

This example makes use of glob.iglob() to search for .py files in the current directory and subdirectories. Passing recursive=True as an argument to .iglob() makes it search for .py files in the current directory and any subdirectories. The difference between glob.iglob() and glob.glob() is that .iglob() returns an iterator instead of a list.

Running the program above produces the following:

pathlib contains similar methods for making flexible file listings. The example below shows how you can use .Path.glob() to list file types that start with the letter p :

Calling p.glob(‘*.p*’) returns a generator object that points to all files in the current directory that start with the letter p in their file extension.

Path.glob() is similar to os.glob() discussed above. As you can see, pathlib combines many of the best features of the os , os.path , and glob modules into one single module, which makes it a joy to use.

To recap, here is a table of the functions we have covered in this section:

Function	Description
startswith()	Tests if a string starts with a specified pattern and returns True or False
endswith()	Tests if a string ends with a specified pattern and returns True or False
fnmatch.fnmatch(filename, pattern)	Tests whether the filename matches the pattern and returns True or False
glob.glob()	Returns a list of filenames that match a pattern
pathlib.Path.glob()	Finds patterns in path names and returns a generator object

Traversing Directories and Processing Files

A common programming task is walking a directory tree and processing files in the tree. Let’s explore how the built-in Python function os.walk() can be used to do this. os.walk() is used to generate filename in a directory tree by walking the tree either top-down or bottom-up. For the purposes of this section, we’ll be manipulating the following directory tree:

The following is an example that shows you how to list all files and directories in a directory tree using os.walk() .

os.walk() defaults to traversing directories in a top-down manner:

os.walk() returns three values on each iteration of the loop:

The name of the current folder

A list of folders in the current folder

A list of files in the current folder

On each iteration, it prints out the names of the subdirectories and files it finds:

To traverse the directory tree in a bottom-up manner, pass in a topdown=False keyword argument to os.walk() :

Passing the topdown=False argument will make os.walk() print out the files it finds in the subdirectories first:

As you can see, the program started by listing the contents of the subdirectories before listing the contents of the root directory. This is very useful in situations where you want to recursively delete files and directories. You will learn how to do this in the sections below. By default, os.walk does not walk down into symbolic links that resolve to directories. This behavior can be overridden by calling it with a followlinks=True argument.

Making Temporary Files and Directories

Python provides a handy module for creating temporary files and directories called tempfile .

tempfile can be used to open and store data temporarily in a file or directory while your program is running. tempfile handles the deletion of the temporary files when your program is done with them.

Here’s how to create a temporary file:

The first step is to import TemporaryFile from the tempfile module. Next, create a file like object using the TemporaryFile() method by calling it and passing the mode you want to open the file in. This will create and open a file that can be used as a temporary storage area.

In the example above, the mode is ‘w+t’ , which makes tempfile create a temporary text file in write mode. There is no need to give the temporary file a filename since it will be destroyed after the script is done running.

After writing to the file, you can read from it and close it when you’re done processing it. Once the file is closed, it will be deleted from the filesystem. If you need to name the temporary files produced using tempfile , use tempfile.NamedTemporaryFile() .

The temporary files and directories created using tempfile are stored in a special system directory for storing temporary files. Python searches a standard list of directories to find one that the user can create files in.

On Windows, the directories are C:\TEMP , C:\TMP , \TEMP , and \TMP , in that order. On all other platforms, the directories are /tmp , /var/tmp , and /usr/tmp , in that order. As a last resort, tempfile will save temporary files and directories in the current directory.

.TemporaryFile() is also a context manager so it can be used in conjunction with the with statement. Using a context manager takes care of closing and deleting the file automatically after it has been read:

This creates a temporary file and reads data from it. As soon as the file’s contents are read, the temporary file is closed and deleted from the file system.

tempfile can also be used to create temporary directories. Let’s look at how you can do this using tempfile.TemporaryDirectory() :

Calling tempfile.TemporaryDirectory() creates a temporary directory in the file system and returns an object representing this directory. In the example above, the directory is created using a context manager, and the name of the directory is stored in tmpdir . The third line prints out the name of the temporary directory, and os.path.exists(tmpdir) confirms if the directory was actually created in the file system.

After the context manager goes out of context, the temporary directory is deleted and a call to os.path.exists(tmpdir) returns False , which means that the directory was succesfully deleted.

Deleting Files and Directories

You can delete single files, directories, and entire directory trees using the methods found in the os , shutil , and pathlib modules. The following sections describe how to delete files and directories that you no longer need.

Deleting Files in Python

To delete a single file, use pathlib.Path.unlink() , os.remove() . or os.unlink() .

os.remove() and os.unlink() are semantically identical. To delete a file using os.remove() , do the following:

Deleting a file using os.unlink() is similar to how you do it using os.remove() :

Calling .unlink() or .remove() on a file deletes the file from the filesystem. These two functions will throw an OSError if the path passed to them points to a directory instead of a file. To avoid this, you can either check that what you’re trying to delete is actually a file and only delete it if it is, or you can use exception handling to handle the OSError :

os.path.isfile() checks whether data_file is actually a file. If it is, it is deleted by the call to os.remove() . If data_file points to a folder, an error message is printed to the console.

The following example shows how to use exception handling to handle errors when deleting files:

The code above attempts to delete the file first before checking its type. If data_file isn’t actually a file, the OSError that is thrown is handled in the except clause, and an error message is printed to the console. The error message that gets printed out is formatted using Python f-strings.

Finally, you can also use pathlib.Path.unlink() to delete files:

This creates a Path object called data_file that points to a file. Calling .remove() on data_file will delete home/data.txt . If data_file points to a directory, an IsADirectoryError is raised. It is worth noting that the Python program above has the same permissions as the user running it. If the user does not have permission to delete the file, a PermissionError is raised.

Deleting Directories

The standard library offers the following functions for deleting directories:

• os.rmdir()
• pathlib.Path.rmdir()
• shutil.rmtree()

To delete a single directory or folder, use os.rmdir() or pathlib.rmdir() . These two functions only work if the directory you’re trying to delete is empty. If the directory isn’t empty, an OSError is raised. Here is how to delete a folder:

Here, the trash_dir directory is deleted by passing its path to os.rmdir() . If the directory isn’t empty, an error message is printed to the screen:

Alternatively, you can use pathlib to delete directories:

Here, you create a Path object that points to the directory to be deleted. Calling .rmdir() on the Path object will delete it if it is empty.

Deleting Entire Directory Trees

To delete non-empty directories and entire directory trees, Python offers shutil.rmtree() :

Everything in trash_dir is deleted when shutil.rmtree() is called on it. There may be cases where you want to delete empty folders recursively. You can do this using one of the methods discussed above in conjunction with os.walk() :

This walks down the directory tree and tries to delete each directory it finds. If the directory isn’t empty, an OSError is raised and that directory is skipped. The table below lists the functions covered in this section:

Function	Description
os.remove()	Deletes a file and does not delete directories
os.unlink()	Is identical to os.remove() and deletes a single file
pathlib.Path.unlink()	Deletes a file and cannot delete directories
os.rmdir()	Deletes an empty directory
pathlib.Path.rmdir()	Deletes an empty directory
shutil.rmtree()	Deletes entire directory tree and can be used to delete non-empty directories

Copying, Moving, and Renaming Files and Directories

Python ships with the shutil module. shutil is short for shell utilities. It provides a number of high-level operations on files to support copying, archiving, and removal of files and directories. In this section, you’ll learn how to move and copy files and directories.

Copying Files in Python

shutil offers a couple of functions for copying files. The most commonly used functions are shutil.copy() and shutil.copy2() . To copy a file from one location to another using shutil.copy() , do the following:

shutil.copy() is comparable to the cp command in UNIX based systems. shutil.copy(src, dst) will copy the file src to the location specified in dst . If dst is a file, the contents of that file are replaced with the contents of src . If dst is a directory, then src will be copied into that directory. shutil.copy() only copies the file’s contents and the file’s permissions. Other metadata like the file’s creation and modification times are not preserved.

To preserve all file metadata when copying, use shutil.copy2() :

Using .copy2() preserves details about the file such as last access time, permission bits, last modification time, and flags.

Copying Directories

While shutil.copy() only copies a single file, shutil.copytree() will copy an entire directory and everything contained in it. shutil.copytree(src, dest) takes two arguments: a source directory and the destination directory where files and folders will be copied to.

Here’s an example of how to copy the contents of one folder to a different location:

In this example, .copytree() copies the contents of data_1 to a new location data1_backup and returns the destination directory. The destination directory must not already exist. It will be created as well as missing parent directories. shutil.copytree() is a good way to back up your files.

Moving Files and Directories

To move a file or directory to another location, use shutil.move(src, dst) .

src is the file or directory to be moved and dst is the destination:

shutil.move(‘dir_1/’, ‘backup/’) moves dir_1/ into backup/ if backup/ exists. If backup/ does not exist, dir_1/ will be renamed to backup .

Renaming Files and Directories

Python includes os.rename(src, dst) for renaming files and directories:

The line above will rename first.zip to first_01.zip . If the destination path points to a directory, it will raise an OSError .

Another way to rename files or directories is to use rename() from the pathlib module:

To rename files using pathlib , you first create a pathlib.Path() object that contains a path to the file you want to replace. The next step is to call rename() on the path object and pass a new filename for the file or directory you’re renaming.

Archiving

Archives are a convenient way to package several files into one. The two most common archive types are ZIP and TAR. The Python programs you write can create, read, and extract data from archives. You will learn how to read and write to both archive formats in this section.

Reading ZIP Files

The zipfile module is a low level module that is part of the Python Standard Library. zipfile has functions that make it easy to open and extract ZIP files. To read the contents of a ZIP file, the first thing to do is to create a ZipFile object. ZipFile objects are similar to file objects created using open() . ZipFile is also a context manager and therefore supports the with statement:

Here, you create a ZipFile object, passing in the name of the ZIP file to open in read mode. After opening a ZIP file, information about the archive can be accessed through functions provided by the zipfile module. The data.zip archive in the example above was created from a directory named data that contains a total of 5 files and 1 subdirectory:

To get a list of files in the archive, call namelist() on the ZipFile object:

This produces a list:

.namelist() returns a list of names of the files and directories in the archive. To retrieve information about the files in the archive, use .getinfo() :

Here’s the output:

.getinfo() returns a ZipInfo object that stores information about a single member of the archive. To get information about a file in the archive, you pass its path as an argument to .getinfo() . Using getinfo() , you’re able to retrieve information about archive members such as the date the files were last modified, their compressed sizes, and their full filenames. Accessing .file_size retrieves the file’s original size in bytes.

The following example shows how to retrieve more details about archived files in a Python REPL. Assume that the zipfile module has been imported and bar_info is the same object you created in previous examples:

bar_info contains details about bar.py such as its size when compressed and its full path.

The first line shows how to retrieve a file’s last modified date. The next line shows how to get the size of the file after compression. The last line shows the full path of bar.py in the archive.

ZipFile supports the context manager protocol, which is why you’re able to use it with the with statement. Doing this automatically closes the ZipFile object after you’re done with it. Trying to open or extract files from a closed ZipFile object will result in an error.

Extracting ZIP Archives

The zipfile module allows you to extract one or more files from ZIP archives through .extract() and .extractall() .

These methods extract files to the current directory by default. They both take an optional path parameter that allows you to specify a different directory to extract files to. If the directory does not exist, it is automatically created. To extract files from the archive, do the following:

The third line of code is a call to os.listdir() , which shows that the current directory has only one file, data.zip .

Next, you open data.zip in read mode and call .extract() to extract file1.py from it. .extract() returns the full file path of the extracted file. Since there’s no path specified, .extract() extracts file1.py to the current directory.

The next line prints a directory listing showing that the current directory now includes the extracted file in addition to the original archive. The line after that shows how to extract the entire archive into the zip_extract directory. .extractall() creates the extract_dir and extracts the contents of data.zip into it. The last line closes the ZIP archive.

Extracting Data From Password Protected Archives

zipfile supports extracting password protected ZIPs. To extract password protected ZIP files, pass in the password to the .extract() or .extractall() method as an argument:

This opens the secret.zip archive in read mode. A password is supplied to .extractall() , and the archive contents are extracted to extract_dir . The archive is closed automatically after the extraction is complete thanks to the with statement.

Creating New ZIP Archives

To create a new ZIP archive, you open a ZipFile object in write mode ( w ) and add the files you want to archive:

In the example, new_zip is opened in write mode and each file in file_list is added to the archive. When the with statement suite is finished, new_zip is closed. Opening a ZIP file in write mode erases the contents of the archive and creates a new archive.

To add files to an existing archive, open a ZipFile object in append mode and then add the files:

Here, you open the new.zip archive you created in the previous example in append mode. Opening the ZipFile object in append mode allows you to add new files to the ZIP file without deleting its current contents. After adding files to the ZIP file, the with statement goes out of context and closes the ZIP file.

Opening TAR Archives

TAR files are uncompressed file archives like ZIP. They can be compressed using gzip, bzip2, and lzma compression methods. The TarFile class allows reading and writing of TAR archives.

Do this to read from an archive:

tarfile objects open like most file-like objects. They have an open() function that takes a mode that determines how the file is to be opened.

Use the ‘r’ , ‘w’ or ‘a’ modes to open an uncompressed TAR file for reading, writing, and appending, respectively. To open compressed TAR files, pass in a mode argument to tarfile.open() that is in the form filemode[:compression] . The table below lists the possible modes TAR files can be opened in:

Mode	Action
r	Opens archive for reading with transparent compression
r:gz	Opens archive for reading with gzip compression
r:bz2	Opens archive for reading with bzip2 compression
r:xz	Opens archive for reading with lzma compression
w	Opens archive for uncompressed writing
w:gz	Opens archive for gzip compressed writing
w:xz	Opens archive for lzma compressed writing
a	Opens archive for appending with no compression

.open() defaults to ‘r’ mode. To read an uncompressed TAR file and retrieve the names of the files in it, use .getnames() :

This returns a list with the names of the archive contents.

Note: For the purposes of showing you how to use different tarfile object methods, the TAR file in the examples is opened and closed manually in an interactive REPL session.

Interacting with the TAR file this way allows you to see the output of running each command. Normally, you would want to use a context manager to open file-like objects.

The metadata of each entry in the archive can be accessed using special attributes:

In this example, you loop through the list of files returned by .getmembers() and print out each file’s attributes. The objects returned by .getmembers() have attributes that can be accessed programmatically such as the name, size, and last modified time of each of the files in the archive. After reading or writing to the archive, it must be closed to free up system resources.

Extracting Files From a TAR Archive

In this section, you’ll learn how to extract files from TAR archives using the following methods:

• .extract()
• .extractfile()
• .extractall()

To extract a single file from a TAR archive, use extract() , passing in the filename:

The README.md file is extracted from the archive to the file system. Calling os.listdir() confirms that README.md file was successfully extracted into the current directory. To unpack or extract everything from the archive, use .extractall() :

.extractall() has an optional path argument to specify where extracted files should go. Here, the archive is unpacked into the extracted directory. The following commands show that the archive was successfully extracted:

To extract a file object for reading or writing, use .extractfile() , which takes a filename or TarInfo object to extract as an argument. .extractfile() returns a file-like object that can be read and used:

Opened archives should always be closed after they have been read or written to. To close an archive, call .close() on the archive file handle or use the with statement when creating tarfile objects to automatically close the archive when you’re done. This frees up system resources and writes any changes you made to the archive to the filesystem.

Creating New TAR Archives

Here’s how you do it:

First, you make a list of files to be added to the archive so that you don’t have to add each file manually.

The next line uses the with context manager to open a new archive called packages.tar in write mode. Opening an archive in write mode( ‘w’ ) enables you to write new files to the archive. Any existing files in the archive are deleted and a new archive is created.

After the archive is created and populated, the with context manager automatically closes it and saves it to the filesystem. The last three lines open the archive you just created and print out the names of the files contained in it.

To add new files to an existing archive, open the archive in append mode ( ‘a’ ):

Opening an archive in append mode allows you to add new files to it without deleting the ones already in it.

Working With Compressed Archives

tarfile can also read and write TAR archives compressed using gzip, bzip2, and lzma compression. To read or write to a compressed archive, use tarfile.open() , passing in the appropriate mode for the compression type.

For example, to read or write data to a TAR archive compressed using gzip, use the ‘r:gz’ or ‘w:gz’ modes respectively:

The ‘w:gz’ mode opens the archive for gzip compressed writing and ‘r:gz’ opens the archive for gzip compressed reading. Opening compressed archives in append mode is not possible. To add files to a compressed archive, you have to create a new archive.

An Easier Way of Creating Archives

The Python Standard Library also supports creating TAR and ZIP archives using the high-level methods in the shutil module. The archiving utilities in shutil allow you to create, read, and extract ZIP and TAR archives. These utilities rely on the lower level tarfile and zipfile modules.

Working With Archives Using shutil.make_archive()

shutil.make_archive() takes at least two arguments: the name of the archive and an archive format.

By default, it compresses all the files in the current directory into the archive format specified in the format argument. You can pass in an optional root_dir argument to compress files in a different directory. .make_archive() supports the zip , tar , bztar , and gztar archive formats.

This is how to create a TAR archive using shutil :

This copies everything in data/ and creates an archive called backup.tar in the filesystem and returns its name. To extract the archive, call .unpack_archive() :

Calling .unpack_archive() and passing in an archive name and destination directory extracts the contents of backup.tar into extract_dir/ . ZIP archives can be created and extracted in the same way.

Reading Multiple Files

Python supports reading data from multiple input streams or from a list of files through the fileinput module. This module allows you to loop over the contents of one or more text files quickly and easily. Here’s the typical way fileinput is used:

fileinput gets its input from command line arguments passed to sys.argv by default.

Using fileinput to Loop Over Multiple Files

Let’s use fileinput to build a crude version of the common UNIX utility cat . The cat utility reads files sequentially, writing them to standard output. When given more than one file in its command line arguments, cat will concatenate the text files and display the result in the terminal:

Running this on two text files in my current directory produces the following output:

fileinput allows you to retrieve more information about each line such as whether or not it is the first line ( .isfirstline() ), the line number ( .lineno() ), and the filename ( .filename() ). You can read more about it here.

Conclusion

You now know how to use Python to perform the most common operations on files and groups of files. You’ve learned about the different built-in modules used to read, find, and manipulate them.

You’re now equipped to use Python to:

• Get directory contents and file properties
• Create directories and directory trees
• Find patterns in filenames
• Create temporary files and directories
• Move, rename, copy, and delete files or directories
• Read and extract data from different types of archives
• Read multiple files simultaneously using fileinput

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Practical Recipes for Working With Files in Python

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

About Vuyisile Ndlovu

Django developer and open source enthusiast.

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Master Real-World Python Skills
With Unlimited Access to Real Python

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

What Do You Think?

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal. Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!

os — Miscellaneous operating system interfaces¶

This module provides a portable way of using operating system dependent functionality. If you just want to read or write a file see open() , if you want to manipulate paths, see the os.path module, and if you want to read all the lines in all the files on the command line see the fileinput module. For creating temporary files and directories see the tempfile module, and for high-level file and directory handling see the shutil module.

Notes on the availability of these functions:

The design of all built-in operating system dependent modules of Python is such that as long as the same functionality is available, it uses the same interface; for example, the function os.stat(path) returns stat information about path in the same format (which happens to have originated with the POSIX interface).

Extensions peculiar to a particular operating system are also available through the os module, but using them is of course a threat to portability.

All functions accepting path or file names accept both bytes and string objects, and result in an object of the same type, if a path or file name is returned.

On VxWorks, os.popen, os.fork, os.execv and os.spawn*p* are not supported.

On WebAssembly platforms wasm32-emscripten and wasm32-wasi , large parts of the os module are not available or behave differently. API related to processes (e.g. fork() , execve() ), signals (e.g. kill() , wait() ), and resources (e.g. nice() ) are not available. Others like getuid() and getpid() are emulated or stubs.

All functions in this module raise OSError (or subclasses thereof) in the case of invalid or inaccessible file names and paths, or other arguments that have the correct type, but are not accepted by the operating system.

An alias for the built-in OSError exception.

The name of the operating system dependent module imported. The following names have currently been registered: ‘posix’ , ‘nt’ , ‘java’ .

sys.platform has a finer granularity. os.uname() gives system-dependent version information.

The platform module provides detailed checks for the system’s identity.

File Names, Command Line Arguments, and Environment Variables¶

In Python, file names, command line arguments, and environment variables are represented using the string type. On some systems, decoding these strings to and from bytes is necessary before passing them to the operating system. Python uses the filesystem encoding and error handler to perform this conversion (see sys.getfilesystemencoding() ).

The filesystem encoding and error handler are configured at Python startup by the PyConfig_Read() function: see filesystem_encoding and filesystem_errors members of PyConfig .

Changed in version 3.1: On some systems, conversion using the file system encoding may fail. In this case, Python uses the surrogateescape encoding error handler , which means that undecodable bytes are replaced by a Unicode character U+DCxx on decoding, and these are again translated to the original byte on encoding.

The file system encoding must guarantee to successfully decode all bytes below 128. If the file system encoding fails to provide this guarantee, API functions can raise UnicodeError .

Python UTF-8 Mode¶

New in version 3.7: See PEP 540 for more details.

The Python UTF-8 Mode ignores the locale encoding and forces the usage of the UTF-8 encoding:

locale.getpreferredencoding() returns ‘utf-8’ (the do_setlocale argument has no effect).

sys.stdin , sys.stdout , and sys.stderr all use UTF-8 as their text encoding, with the surrogateescape error handler being enabled for sys.stdin and sys.stdout ( sys.stderr continues to use backslashreplace as it does in the default locale-aware mode)

On Unix, os.device_encoding() returns ‘utf-8’ rather than the device encoding.

Note that the standard stream settings in UTF-8 mode can be overridden by PYTHONIOENCODING (just as they can be in the default locale-aware mode).

As a consequence of the changes in those lower level APIs, other higher level APIs also exhibit different default behaviours:

Command line arguments, environment variables and filenames are decoded to text using the UTF-8 encoding.

open() , io.open() , and codecs.open() use the UTF-8 encoding by default. However, they still use the strict error handler by default so that attempting to open a binary file in text mode is likely to raise an exception rather than producing nonsense data.

The Python UTF-8 Mode is enabled if the LC_CTYPE locale is C or POSIX at Python startup (see the PyConfig_Read() function).

It can be enabled or disabled using the -X utf8 command line option and the PYTHONUTF8 environment variable.

If the PYTHONUTF8 environment variable is not set at all, then the interpreter defaults to using the current locale settings, unless the current locale is identified as a legacy ASCII-based locale (as described for PYTHONCOERCECLOCALE ), and locale coercion is either disabled or fails. In such legacy locales, the interpreter will default to enabling UTF-8 mode unless explicitly instructed not to do so.

The Python UTF-8 Mode can only be enabled at the Python startup. Its value can be read from sys.flags.utf8_mode .

Python 3.15 will make Python UTF-8 Mode default.

Process Parameters¶

These functions and data items provide information and operate on the current process and user.

Return the filename corresponding to the controlling terminal of the process.

Availability : Unix, not Emscripten, not WASI.

A mapping object where keys and values are strings that represent the process environment. For example, environ[‘HOME’] is the pathname of your home directory (on some platforms), and is equivalent to getenv("HOME") in C.

This mapping is captured the first time the os module is imported, typically during Python startup as part of processing site.py . Changes to the environment made after this time are not reflected in os.environ , except for changes made by modifying os.environ directly.

This mapping may be used to modify the environment as well as query the environment. putenv() will be called automatically when the mapping is modified.

On Unix, keys and values use sys.getfilesystemencoding() and ‘surrogateescape’ error handler. Use environb if you would like to use a different encoding.

On Windows, the keys are converted to uppercase. This also applies when getting, setting, or deleting an item. For example, environ[‘monty’] = ‘python’ maps the key ‘MONTY’ to the value ‘python’ .

Calling putenv() directly does not change os.environ , so it’s better to modify os.environ .

On some platforms, including FreeBSD and macOS, setting environ may cause memory leaks. Refer to the system documentation for putenv() .

You can delete items in this mapping to unset environment variables. unsetenv() will be called automatically when an item is deleted from os.environ , and when one of the pop() or clear() methods is called.

Changed in version 3.9: Updated to support PEP 584’s merge ( | ) and update ( |= ) operators.

Bytes version of environ : a mapping object where both keys and values are bytes objects representing the process environment. environ and environb are synchronized (modifying environb updates environ , and vice versa).

New in version 3.2.

Changed in version 3.9: Updated to support PEP 584’s merge ( | ) and update ( |= ) operators.

These functions are described in Files and Directories .

os. fsencode ( filename ) ¶

fsdecode() is the reverse function.

New in version 3.2.

Changed in version 3.6: Support added to accept objects implementing the os.PathLike interface.

fsencode() is the reverse function.

New in version 3.2.

Changed in version 3.6: Support added to accept objects implementing the os.PathLike interface.

Return the file system representation of the path.

If str or bytes is passed in, it is returned unchanged. Otherwise __fspath__() is called and its value is returned as long as it is a str or bytes object. In all other cases, TypeError is raised.

New in version 3.6.

An abstract base class for objects representing a file system path, e.g. pathlib.PurePath .

New in version 3.6.

Return the file system path representation of the object.

The method should only return a str or bytes object, with the preference being for str .

os. getenv ( key , default = None ) ¶

Return the value of the environment variable key as a string if it exists, or default if it doesn’t. key is a string. Note that since getenv() uses os.environ , the mapping of getenv() is similarly also captured on import, and the function may not reflect future environment changes.

On Unix, keys and values are decoded with sys.getfilesystemencoding() and ‘surrogateescape’ error handler. Use os.getenvb() if you would like to use a different encoding.

Return the value of the environment variable key as bytes if it exists, or default if it doesn’t. key must be bytes. Note that since getenvb() uses os.environb , the mapping of getenvb() is similarly also captured on import, and the function may not reflect future environment changes.

New in version 3.2.

Returns the list of directories that will be searched for a named executable, similar to a shell, when launching a process. env, when specified, should be an environment variable dictionary to lookup the PATH in. By default, when env is None , environ is used.

New in version 3.2.

Return the effective group id of the current process. This corresponds to the “set id” bit on the file being executed in the current process.

Availability : Unix, not Emscripten, not WASI.

Return the current process’s effective user id.

Availability : Unix, not Emscripten, not WASI.

Return the real group id of the current process.

The function is a stub on Emscripten and WASI, see WebAssembly platforms for more information.

Return list of group ids that user belongs to. If group is not in the list, it is included; typically, group is specified as the group ID field from the password record for user, because that group ID will otherwise be potentially omitted.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Return list of supplemental group ids associated with the current process.

Availability : Unix, not Emscripten, not WASI.

On macOS, getgroups() behavior differs somewhat from other Unix platforms. If the Python interpreter was built with a deployment target of 10.5 or earlier, getgroups() returns the list of effective group ids associated with the current user process; this list is limited to a system-defined number of entries, typically 16, and may be modified by calls to setgroups() if suitably privileged. If built with a deployment target greater than 10.5 , getgroups() returns the current group access list for the user associated with the effective user id of the process; the group access list may change over the lifetime of the process, it is not affected by calls to setgroups() , and its length is not limited to 16. The deployment target value, MACOSX_DEPLOYMENT_TARGET , can be obtained with sysconfig.get_config_var() .

Return the name of the user logged in on the controlling terminal of the process. For most purposes, it is more useful to use getpass.getuser() since the latter checks the environment variables LOGNAME or USERNAME to find out who the user is, and falls back to pwd.getpwuid(os.getuid())[0] to get the login name of the current real user id.

Availability : Unix, Windows, not Emscripten, not WASI.

Return the process group id of the process with process id pid. If pid is 0, the process group id of the current process is returned.

Availability : Unix, not Emscripten, not WASI.

Return the id of the current process group.

Availability : Unix, not Emscripten, not WASI.

Return the current process id.

The function is a stub on Emscripten and WASI, see WebAssembly platforms for more information.

Return the parent’s process id. When the parent process has exited, on Unix the id returned is the one of the init process (1), on Windows it is still the same id, which may be already reused by another process.

Availability : Unix, Windows, not Emscripten, not WASI.

Changed in version 3.2: Added support for Windows.

Get program scheduling priority. The value which is one of PRIO_PROCESS , PRIO_PGRP , or PRIO_USER , and who is interpreted relative to which (a process identifier for PRIO_PROCESS , process group identifier for PRIO_PGRP , and a user ID for PRIO_USER ). A zero value for who denotes (respectively) the calling process, the process group of the calling process, or the real user ID of the calling process.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Parameters for the getpriority() and setpriority() functions.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Return a tuple (ruid, euid, suid) denoting the current process’s real, effective, and saved user ids.

Availability : Unix, not Emscripten, not WASI.

New in version 3.2.

Return a tuple (rgid, egid, sgid) denoting the current process’s real, effective, and saved group ids.

Availability : Unix, not Emscripten, not WASI.

New in version 3.2.

Return the current process’s real user id.

The function is a stub on Emscripten and WASI, see WebAssembly platforms for more information.

Call the system initgroups() to initialize the group access list with all of the groups of which the specified username is a member, plus the specified group id.

Availability : Unix, not Emscripten, not WASI.

New in version 3.2.

Set the environment variable named key to the string value. Such changes to the environment affect subprocesses started with os.system() , popen() or fork() and execv() .

Assignments to items in os.environ are automatically translated into corresponding calls to putenv() ; however, calls to putenv() don’t update os.environ , so it is actually preferable to assign to items of os.environ . This also applies to getenv() and getenvb() , which respectively use os.environ and os.environb in their implementations.

On some platforms, including FreeBSD and macOS, setting environ may cause memory leaks. Refer to the system documentation for putenv() .

Raises an auditing event os.putenv with arguments key , value .

Changed in version 3.9: The function is now always available.

Set the current process’s effective group id.

Availability : Unix, not Emscripten, not WASI.

Set the current process’s effective user id.

Availability : Unix, not Emscripten, not WASI.

Set the current process’ group id.

Availability : Unix, not Emscripten, not WASI.

Set the list of supplemental group ids associated with the current process to groups. groups must be a sequence, and each element must be an integer identifying a group. This operation is typically available only to the superuser.

Availability : Unix, not Emscripten, not WASI.

On macOS, the length of groups may not exceed the system-defined maximum number of effective group ids, typically 16. See the documentation for getgroups() for cases where it may not return the same group list set by calling setgroups().

Call the system call setpgrp() or setpgrp(0, 0) depending on which version is implemented (if any). See the Unix manual for the semantics.

Availability : Unix, not Emscripten, not WASI.

Call the system call setpgid() to set the process group id of the process with id pid to the process group with id pgrp. See the Unix manual for the semantics.

Availability : Unix, not Emscripten, not WASI.

Set program scheduling priority. The value which is one of PRIO_PROCESS , PRIO_PGRP , or PRIO_USER , and who is interpreted relative to which (a process identifier for PRIO_PROCESS , process group identifier for PRIO_PGRP , and a user ID for PRIO_USER ). A zero value for who denotes (respectively) the calling process, the process group of the calling process, or the real user ID of the calling process. priority is a value in the range -20 to 19. The default priority is 0; lower priorities cause more favorable scheduling.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Set the current process’s real and effective group ids.

Availability : Unix, not Emscripten, not WASI.

Set the current process’s real, effective, and saved group ids.

Availability : Unix, not Emscripten, not WASI.

New in version 3.2.

Set the current process’s real, effective, and saved user ids.

Availability : Unix, not Emscripten, not WASI.

New in version 3.2.

Set the current process’s real and effective user ids.

Availability : Unix, not Emscripten, not WASI.

Call the system call getsid() . See the Unix manual for the semantics.

Availability : Unix, not Emscripten, not WASI.

Call the system call setsid() . See the Unix manual for the semantics.

Availability : Unix, not Emscripten, not WASI.

Set the current process’s user id.

Availability : Unix, not Emscripten, not WASI.

Return the error message corresponding to the error code in code. On platforms where strerror() returns NULL when given an unknown error number, ValueError is raised.

True if the native OS type of the environment is bytes (eg. False on Windows).

New in version 3.2.

Set the current numeric umask and return the previous umask.

The function is a stub on Emscripten and WASI, see WebAssembly platforms for more information.

Returns information identifying the current operating system. The return value is an object with five attributes:

sysname — operating system name

nodename — name of machine on network (implementation-defined)

release — operating system release

version — operating system version

machine — hardware identifier

For backwards compatibility, this object is also iterable, behaving like a five-tuple containing sysname , nodename , release , version , and machine in that order.

Some systems truncate nodename to 8 characters or to the leading component; a better way to get the hostname is socket.gethostname() or even socket.gethostbyaddr(socket.gethostname()) .

Changed in version 3.3: Return type changed from a tuple to a tuple-like object with named attributes.

Unset (delete) the environment variable named key. Such changes to the environment affect subprocesses started with os.system() , popen() or fork() and execv() .

Deletion of items in os.environ is automatically translated into a corresponding call to unsetenv() ; however, calls to unsetenv() don’t update os.environ , so it is actually preferable to delete items of os.environ .

Raises an auditing event os.unsetenv with argument key .

Changed in version 3.9: The function is now always available and is also available on Windows.

File Object Creation¶

These functions create new file objects . (See also open() for opening file descriptors.)

Return an open file object connected to the file descriptor fd. This is an alias of the open() built-in function and accepts the same arguments. The only difference is that the first argument of fdopen() must always be an integer.

File Descriptor Operations¶

These functions operate on I/O streams referenced using file descriptors.

File descriptors are small integers corresponding to a file that has been opened by the current process. For example, standard input is usually file descriptor 0, standard output is 1, and standard error is 2. Further files opened by a process will then be assigned 3, 4, 5, and so forth. The name “file descriptor” is slightly deceptive; on Unix platforms, sockets and pipes are also referenced by file descriptors.

The fileno() method can be used to obtain the file descriptor associated with a file object when required. Note that using the file descriptor directly will bypass the file object methods, ignoring aspects such as internal buffering of data.

Close file descriptor fd.

This function is intended for low-level I/O and must be applied to a file descriptor as returned by os.open() or pipe() . To close a “file object” returned by the built-in function open() or by popen() or fdopen() , use its close() method.

Close all file descriptors from fd_low (inclusive) to fd_high (exclusive), ignoring errors. Equivalent to (but much faster than):

Copy count bytes from file descriptor src, starting from offset offset_src, to file descriptor dst, starting from offset offset_dst. If offset_src is None, then src is read from the current position; respectively for offset_dst. The files pointed by src and dst must reside in the same filesystem, otherwise an OSError is raised with errno set to errno.EXDEV .

This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, some filesystems could implement extra optimizations. The copy is done as if both files are opened as binary.

The return value is the amount of bytes copied. This could be less than the amount requested.

New in version 3.8.

Return a string describing the encoding of the device associated with fd if it is connected to a terminal; else return None .

On Unix, if the Python UTF-8 Mode is enabled, return ‘UTF-8’ rather than the device encoding.

Changed in version 3.10: On Unix, the function now implements the Python UTF-8 Mode.

Return a duplicate of file descriptor fd. The new file descriptor is non-inheritable .

On Windows, when duplicating a standard stream (0: stdin, 1: stdout, 2: stderr), the new file descriptor is inheritable .

Changed in version 3.4: The new file descriptor is now non-inheritable.

Duplicate file descriptor fd to fd2, closing the latter first if necessary. Return fd2. The new file descriptor is inheritable by default or non-inheritable if inheritable is False .

Changed in version 3.4: Add the optional inheritable parameter.

Changed in version 3.7: Return fd2 on success. Previously, None was always returned.

Change the mode of the file given by fd to the numeric mode. See the docs for chmod() for possible values of mode. As of Python 3.3, this is equivalent to os.chmod(fd, mode) .

Raises an auditing event os.chmod with arguments path , mode , dir_fd .

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

Change the owner and group id of the file given by fd to the numeric uid and gid. To leave one of the ids unchanged, set it to -1. See chown() . As of Python 3.3, this is equivalent to os.chown(fd, uid, gid) .

Raises an auditing event os.chown with arguments path , uid , gid , dir_fd .

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

Force write of file with filedescriptor fd to disk. Does not force update of metadata.

This function is not available on MacOS.

Return system configuration information relevant to an open file. name specifies the configuration value to retrieve; it may be a string which is the name of a defined system value; these names are specified in a number of standards (POSIX.1, Unix 95, Unix 98, and others). Some platforms define additional names as well. The names known to the host operating system are given in the pathconf_names dictionary. For configuration variables not included in that mapping, passing an integer for name is also accepted.

If name is a string and is not known, ValueError is raised. If a specific value for name is not supported by the host system, even if it is included in pathconf_names , an OSError is raised with errno.EINVAL for the error number.

As of Python 3.3, this is equivalent to os.pathconf(fd, name) .

Get the status of the file descriptor fd. Return a stat_result object.

As of Python 3.3, this is equivalent to os.stat(fd) .

Return information about the filesystem containing the file associated with file descriptor fd, like statvfs() . As of Python 3.3, this is equivalent to os.statvfs(fd) .

Force write of file with filedescriptor fd to disk. On Unix, this calls the native fsync() function; on Windows, the MS _commit() function.

If you’re starting with a buffered Python file object f, first do f.flush() , and then do os.fsync(f.fileno()) , to ensure that all internal buffers associated with f are written to disk.

Truncate the file corresponding to file descriptor fd, so that it is at most length bytes in size. As of Python 3.3, this is equivalent to os.truncate(fd, length) .

Raises an auditing event os.truncate with arguments fd , length .

Changed in version 3.5: Added support for Windows

Get the blocking mode of the file descriptor: False if the O_NONBLOCK flag is set, True if the flag is cleared.

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

New in version 3.5.

Return True if the file descriptor fd is open and connected to a tty(-like) device, else False .

Apply, test or remove a POSIX lock on an open file descriptor. fd is an open file descriptor. cmd specifies the command to use — one of F_LOCK , F_TLOCK , F_ULOCK or F_TEST . len specifies the section of the file to lock.

Raises an auditing event os.lockf with arguments fd , cmd , len .

New in version 3.3.

Flags that specify what action lockf() will take.

New in version 3.3.

Prepare the tty of which fd is a file descriptor for a new login session. Make the calling process a session leader; make the tty the controlling tty, the stdin, the stdout, and the stderr of the calling process; close fd.

Availability : Unix, not Emscripten, not WASI.

New in version 3.11.

Set the current position of file descriptor fd to position pos, modified by how: SEEK_SET or 0 to set the position relative to the beginning of the file; SEEK_CUR or 1 to set it relative to the current position; SEEK_END or 2 to set it relative to the end of the file. Return the new cursor position in bytes, starting from the beginning.

os. SEEK_SET ¶ os. SEEK_CUR ¶ os. SEEK_END ¶

Parameters to the lseek() function. Their values are 0, 1, and 2, respectively.

New in version 3.3: Some operating systems could support additional values, like os.SEEK_HOLE or os.SEEK_DATA .

Open the file path and set various flags according to flags and possibly its mode according to mode. When computing mode, the current umask value is first masked out. Return the file descriptor for the newly opened file. The new file descriptor is non-inheritable .

For a description of the flag and mode values, see the C run-time documentation; flag constants (like O_RDONLY and O_WRONLY ) are defined in the os module. In particular, on Windows adding O_BINARY is needed to open files in binary mode.

This function can support paths relative to directory descriptors with the dir_fd parameter.

Raises an auditing event open with arguments path , mode , flags .

Changed in version 3.4: The new file descriptor is now non-inheritable.

This function is intended for low-level I/O. For normal usage, use the built-in function open() , which returns a file object with read() and write() methods (and many more). To wrap a file descriptor in a file object, use fdopen() .

New in version 3.3: The dir_fd argument.

Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an InterruptedError exception (see PEP 475 for the rationale).

Changed in version 3.6: Accepts a path-like object .

The following constants are options for the flags parameter to the open() function. They can be combined using the bitwise OR operator | . Some of them are not available on all platforms. For descriptions of their availability and use, consult the open(2) manual page on Unix or the MSDN on Windows.

os. O_RDONLY ¶ os. O_WRONLY ¶ os. O_RDWR ¶ os. O_APPEND ¶ os. O_CREAT ¶ os. O_EXCL ¶ os. O_TRUNC ¶

The above constants are available on Unix and Windows.

os. O_DSYNC ¶ os. O_RSYNC ¶ os. O_SYNC ¶ os. O_NDELAY ¶ os. O_NONBLOCK ¶ os. O_NOCTTY ¶ os. O_CLOEXEC ¶

The above constants are only available on Unix.

Changed in version 3.3: Add O_CLOEXEC constant.

The above constants are only available on Windows.

os. O_EVTONLY ¶ os. O_FSYNC ¶ os. O_SYMLINK ¶ os. O_NOFOLLOW_ANY ¶

The above constants are only available on macOS.

Changed in version 3.10: Add O_EVTONLY , O_FSYNC , O_SYMLINK and O_NOFOLLOW_ANY constants.

The above constants are extensions and not present if they are not defined by the C library.

Changed in version 3.4: Add O_PATH on systems that support it. Add O_TMPFILE , only available on Linux Kernel 3.11 or newer.

Open a new pseudo-terminal pair. Return a pair of file descriptors (master, slave) for the pty and the tty, respectively. The new file descriptors are non-inheritable . For a (slightly) more portable approach, use the pty module.

Availability : Unix, not Emscripten, not WASI.

Changed in version 3.4: The new file descriptors are now non-inheritable.

Create a pipe. Return a pair of file descriptors (r, w) usable for reading and writing, respectively. The new file descriptor is non-inheritable .

Changed in version 3.4: The new file descriptors are now non-inheritable.

Create a pipe with flags set atomically. flags can be constructed by ORing together one or more of these values: O_NONBLOCK , O_CLOEXEC . Return a pair of file descriptors (r, w) usable for reading and writing, respectively.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Ensures that enough disk space is allocated for the file specified by fd starting from offset and continuing for len bytes.

New in version 3.3.

Announces an intention to access data in a specific pattern thus allowing the kernel to make optimizations. The advice applies to the region of the file specified by fd starting at offset and continuing for len bytes. advice is one of POSIX_FADV_NORMAL , POSIX_FADV_SEQUENTIAL , POSIX_FADV_RANDOM , POSIX_FADV_NOREUSE , POSIX_FADV_WILLNEED or POSIX_FADV_DONTNEED .

New in version 3.3.

Flags that can be used in advice in posix_fadvise() that specify the access pattern that is likely to be used.

New in version 3.3.

Read at most n bytes from file descriptor fd at a position of offset, leaving the file offset unchanged.

Return a bytestring containing the bytes read. If the end of the file referred to by fd has been reached, an empty bytes object is returned.

New in version 3.3.

Read from a file descriptor fd at a position of offset into mutable bytes-like objects buffers, leaving the file offset unchanged. Transfer data into each buffer until it is full and then move on to the next buffer in the sequence to hold the rest of the data.

The flags argument contains a bitwise OR of zero or more of the following flags:

Return the total number of bytes actually read which can be less than the total capacity of all the objects.

The operating system may set a limit ( sysconf() value ‘SC_IOV_MAX’ ) on the number of buffers that can be used.

Combine the functionality of os.readv() and os.pread() .

Availability : Linux >= 2.6.30, FreeBSD >= 6.0, OpenBSD >= 2.7, AIX >= 7.1.

Using flags requires Linux >= 4.6.

New in version 3.7.

Do not wait for data which is not immediately available. If this flag is specified, the system call will return instantly if it would have to read data from the backing storage or wait for a lock.

If some data was successfully read, it will return the number of bytes read. If no bytes were read, it will return -1 and set errno to errno.EAGAIN .

New in version 3.7.

High priority read/write. Allows block-based filesystems to use polling of the device, which provides lower latency, but may use additional resources.

Currently, on Linux, this feature is usable only on a file descriptor opened using the O_DIRECT flag.

New in version 3.7.

Write the bytestring in str to file descriptor fd at position of offset, leaving the file offset unchanged.

Return the number of bytes actually written.

New in version 3.3.

Write the buffers contents to file descriptor fd at a offset offset, leaving the file offset unchanged. buffers must be a sequence of bytes-like objects . Buffers are processed in array order. Entire contents of the first buffer is written before proceeding to the second, and so on.

The flags argument contains a bitwise OR of zero or more of the following flags:

Return the total number of bytes actually written.

The operating system may set a limit ( sysconf() value ‘SC_IOV_MAX’ ) on the number of buffers that can be used.

Combine the functionality of os.writev() and os.pwrite() .

Availability : Linux >= 2.6.30, FreeBSD >= 6.0, OpenBSD >= 2.7, AIX >= 7.1.

Using flags requires Linux >= 4.6.

New in version 3.7.

Provide a per-write equivalent of the O_DSYNC os.open() flag. This flag effect applies only to the data range written by the system call.

New in version 3.7.

Provide a per-write equivalent of the O_SYNC os.open() flag. This flag effect applies only to the data range written by the system call.

New in version 3.7.

Provide a per-write equivalent of the O_APPEND os.open() flag. This flag is meaningful only for os.pwritev() , and its effect applies only to the data range written by the system call. The offset argument does not affect the write operation; the data is always appended to the end of the file. However, if the offset argument is -1 , the current file offset is updated.

New in version 3.10.

Read at most n bytes from file descriptor fd.

Return a bytestring containing the bytes read. If the end of the file referred to by fd has been reached, an empty bytes object is returned.

This function is intended for low-level I/O and must be applied to a file descriptor as returned by os.open() or pipe() . To read a “file object” returned by the built-in function open() or by popen() or fdopen() , or sys.stdin , use its read() or readline() methods.

Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an InterruptedError exception (see PEP 475 for the rationale).

Copy count bytes from file descriptor in_fd to file descriptor out_fd starting at offset. Return the number of bytes sent. When EOF is reached return 0 .

The first function notation is supported by all platforms that define sendfile() .

On Linux, if offset is given as None , the bytes are read from the current position of in_fd and the position of in_fd is updated.

The second case may be used on macOS and FreeBSD where headers and trailers are arbitrary sequences of buffers that are written before and after the data from in_fd is written. It returns the same as the first case.

On macOS and FreeBSD, a value of 0 for count specifies to send until the end of in_fd is reached.

All platforms support sockets as out_fd file descriptor, and some platforms allow other types (e.g. regular file, pipe) as well.

Cross-platform applications should not use headers, trailers and flags arguments.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Changed in version 3.9: Parameters out and in was renamed to out_fd and in_fd.

Set the blocking mode of the specified file descriptor. Set the O_NONBLOCK flag if blocking is False , clear the flag otherwise.

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

New in version 3.5.

Parameters to the sendfile() function, if the implementation supports them.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3.

Parameter to the sendfile() function, if the implementation supports it. The data won’t be cached in the virtual memory and will be freed afterwards.

Availability : Unix, not Emscripten, not WASI.

New in version 3.11.

Transfer count bytes from file descriptor src, starting from offset offset_src, to file descriptor dst, starting from offset offset_dst. At least one of the file descriptors must refer to a pipe. If offset_src is None, then src is read from the current position; respectively for offset_dst. The offset associated to the file descriptor that refers to a pipe must be None . The files pointed by src and dst must reside in the same filesystem, otherwise an OSError is raised with errno set to errno.EXDEV .

This copy is done without the additional cost of transferring data from the kernel to user space and then back into the kernel. Additionally, some filesystems could implement extra optimizations. The copy is done as if both files are opened as binary.

Upon successful completion, returns the number of bytes spliced to or from the pipe. A return value of 0 means end of input. If src refers to a pipe, then this means that there was no data to transfer, and it would not make sense to block because there are no writers connected to the write end of the pipe.

New in version 3.10.

New in version 3.10.

Read from a file descriptor fd into a number of mutable bytes-like objects buffers. Transfer data into each buffer until it is full and then move on to the next buffer in the sequence to hold the rest of the data.

Return the total number of bytes actually read which can be less than the total capacity of all the objects.

The operating system may set a limit ( sysconf() value ‘SC_IOV_MAX’ ) on the number of buffers that can be used.

New in version 3.3.

Return the process group associated with the terminal given by fd (an open file descriptor as returned by os.open() ).

Set the process group associated with the terminal given by fd (an open file descriptor as returned by os.open() ) to pg.

Return a string which specifies the terminal device associated with file descriptor fd. If fd is not associated with a terminal device, an exception is raised.

Write the bytestring in str to file descriptor fd.

Return the number of bytes actually written.

This function is intended for low-level I/O and must be applied to a file descriptor as returned by os.open() or pipe() . To write a “file object” returned by the built-in function open() or by popen() or fdopen() , or sys.stdout or sys.stderr , use its write() method.

Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an InterruptedError exception (see PEP 475 for the rationale).

Write the contents of buffers to file descriptor fd. buffers must be a sequence of bytes-like objects . Buffers are processed in array order. Entire contents of the first buffer is written before proceeding to the second, and so on.

Returns the total number of bytes actually written.

The operating system may set a limit ( sysconf() value ‘SC_IOV_MAX’ ) on the number of buffers that can be used.

New in version 3.3.

Querying the size of a terminal¶

New in version 3.3.

Return the size of the terminal window as (columns, lines) , tuple of type terminal_size .

The optional argument fd (default STDOUT_FILENO , or standard output) specifies which file descriptor should be queried.

If the file descriptor is not connected to a terminal, an OSError is raised.

shutil.get_terminal_size() is the high-level function which should normally be used, os.get_terminal_size is the low-level implementation.

A subclass of tuple, holding (columns, lines) of the terminal window size.

Width of the terminal window in characters.

Height of the terminal window in characters.

Inheritance of File Descriptors¶

New in version 3.4.

A file descriptor has an “inheritable” flag which indicates if the file descriptor can be inherited by child processes. Since Python 3.4, file descriptors created by Python are non-inheritable by default.

On UNIX, non-inheritable file descriptors are closed in child processes at the execution of a new program, other file descriptors are inherited.

On Windows, non-inheritable handles and file descriptors are closed in child processes, except for standard streams (file descriptors 0, 1 and 2: stdin, stdout and stderr), which are always inherited. Using spawn* functions, all inheritable handles and all inheritable file descriptors are inherited. Using the subprocess module, all file descriptors except standard streams are closed, and inheritable handles are only inherited if the close_fds parameter is False .

On WebAssembly platforms wasm32-emscripten and wasm32-wasi , the file descriptor cannot be modified.

Get the “inheritable” flag of the specified file descriptor (a boolean).

Set the “inheritable” flag of the specified file descriptor.

os. get_handle_inheritable ( handle , / ) ¶

Get the “inheritable” flag of the specified handle (a boolean).

Set the “inheritable” flag of the specified handle.

Files and Directories¶

On some Unix platforms, many of these functions support one or more of these features:

specifying a file descriptor: Normally the path argument provided to functions in the os module must be a string specifying a file path. However, some functions now alternatively accept an open file descriptor for their path argument. The function will then operate on the file referred to by the descriptor. (For POSIX systems, Python will call the variant of the function prefixed with f (e.g. call fchdir instead of chdir ).)

You can check whether or not path can be specified as a file descriptor for a particular function on your platform using os.supports_fd . If this functionality is unavailable, using it will raise a NotImplementedError .

If the function also supports dir_fd or follow_symlinks arguments, it’s an error to specify one of those when supplying path as a file descriptor.

paths relative to directory descriptors: If dir_fd is not None , it should be a file descriptor referring to a directory, and the path to operate on should be relative; path will then be relative to that directory. If the path is absolute, dir_fd is ignored. (For POSIX systems, Python will call the variant of the function with an at suffix and possibly prefixed with f (e.g. call faccessat instead of access ).

You can check whether or not dir_fd is supported for a particular function on your platform using os.supports_dir_fd . If it’s unavailable, using it will raise a NotImplementedError .

not following symlinks: If follow_symlinks is False , and the last element of the path to operate on is a symbolic link, the function will operate on the symbolic link itself rather than the file pointed to by the link. (For POSIX systems, Python will call the l. variant of the function.)

You can check whether or not follow_symlinks is supported for a particular function on your platform using os.supports_follow_symlinks . If it’s unavailable, using it will raise a NotImplementedError .

Use the real uid/gid to test for access to path. Note that most operations will use the effective uid/gid, therefore this routine can be used in a suid/sgid environment to test if the invoking user has the specified access to path. mode should be F_OK to test the existence of path, or it can be the inclusive OR of one or more of R_OK , W_OK , and X_OK to test permissions. Return True if access is allowed, False if not. See the Unix man page access(2) for more information.

If effective_ids is True , access() will perform its access checks using the effective uid/gid instead of the real uid/gid. effective_ids may not be supported on your platform; you can check whether or not it is available using os.supports_effective_ids . If it is unavailable, using it will raise a NotImplementedError .

Using access() to check if a user is authorized to e.g. open a file before actually doing so using open() creates a security hole, because the user might exploit the short time interval between checking and opening the file to manipulate it. It’s preferable to use EAFP techniques. For example:

is better written as:

I/O operations may fail even when access() indicates that they would succeed, particularly for operations on network filesystems which may have permissions semantics beyond the usual POSIX permission-bit model.

Changed in version 3.3: Added the dir_fd, effective_ids, and follow_symlinks parameters.

Changed in version 3.6: Accepts a path-like object .

Values to pass as the mode parameter of access() to test the existence, readability, writability and executability of path, respectively.

Change the current working directory to path.

This function can support specifying a file descriptor . The descriptor must refer to an opened directory, not an open file.

This function can raise OSError and subclasses such as FileNotFoundError , PermissionError , and NotADirectoryError .

Raises an auditing event os.chdir with argument path .

New in version 3.3: Added support for specifying path as a file descriptor on some platforms.

Changed in version 3.6: Accepts a path-like object .

Set the flags of path to the numeric flags. flags may take a combination (bitwise OR) of the following values (as defined in the stat module):

This function can support not following symlinks .

Raises an auditing event os.chflags with arguments path , flags .

Availability : Unix, not Emscripten, not WASI.

New in version 3.3: The follow_symlinks argument.

Changed in version 3.6: Accepts a path-like object .

Change the mode of path to the numeric mode. mode may take one of the following values (as defined in the stat module) or bitwise ORed combinations of them:

Although Windows supports chmod() , you can only set the file’s read-only flag with it (via the stat.S_IWRITE and stat.S_IREAD constants or a corresponding integer value). All other bits are ignored.

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

Raises an auditing event os.chmod with arguments path , mode , dir_fd .

New in version 3.3: Added support for specifying path as an open file descriptor, and the dir_fd and follow_symlinks arguments.

Changed in version 3.6: Accepts a path-like object .

Change the owner and group id of path to the numeric uid and gid. To leave one of the ids unchanged, set it to -1.

See shutil.chown() for a higher-level function that accepts names in addition to numeric ids.

Raises an auditing event os.chown with arguments path , uid , gid , dir_fd .

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

New in version 3.3: Added support for specifying path as an open file descriptor, and the dir_fd and follow_symlinks arguments.

Changed in version 3.6: Supports a path-like object .

Change the root directory of the current process to path.

Availability : Unix, not Emscripten, not WASI.

Changed in version 3.6: Accepts a path-like object .

Change the current working directory to the directory represented by the file descriptor fd. The descriptor must refer to an opened directory, not an open file. As of Python 3.3, this is equivalent to os.chdir(fd) .

Raises an auditing event os.chdir with argument path .

Return a string representing the current working directory.

Return a bytestring representing the current working directory.

Changed in version 3.8: The function now uses the UTF-8 encoding on Windows, rather than the ANSI code page: see PEP 529 for the rationale. The function is no longer deprecated on Windows.

Set the flags of path to the numeric flags, like chflags() , but do not follow symbolic links. As of Python 3.3, this is equivalent to os.chflags(path, flags, follow_symlinks=False) .

Raises an auditing event os.chflags with arguments path , flags .

Availability : Unix, not Emscripten, not WASI.

Changed in version 3.6: Accepts a path-like object .

Change the mode of path to the numeric mode. If path is a symlink, this affects the symlink rather than the target. See the docs for chmod() for possible values of mode. As of Python 3.3, this is equivalent to os.chmod(path, mode, follow_symlinks=False) .

Raises an auditing event os.chmod with arguments path , mode , dir_fd .

Changed in version 3.6: Accepts a path-like object .

Change the owner and group id of path to the numeric uid and gid. This function will not follow symbolic links. As of Python 3.3, this is equivalent to os.chown(path, uid, gid, follow_symlinks=False) .

Raises an auditing event os.chown with arguments path , uid , gid , dir_fd .

Changed in version 3.6: Accepts a path-like object .

Create a hard link pointing to src named dst.

This function can support specifying src_dir_fd and/or dst_dir_fd to supply paths relative to directory descriptors , and not following symlinks .

Raises an auditing event os.link with arguments src , dst , src_dir_fd , dst_dir_fd .

Changed in version 3.2: Added Windows support.

New in version 3.3: Added the src_dir_fd, dst_dir_fd, and follow_symlinks arguments.

Changed in version 3.6: Accepts a path-like object for src and dst.

Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order, and does not include the special entries ‘.’ and ‘..’ even if they are present in the directory. If a file is removed from or added to the directory during the call of this function, whether a name for that file be included is unspecified.

path may be a path-like object . If path is of type bytes (directly or indirectly through the PathLike interface), the filenames returned will also be of type bytes ; in all other circumstances, they will be of type str .

This function can also support specifying a file descriptor ; the file descriptor must refer to a directory.

Raises an auditing event os.listdir with argument path .

To encode str filenames to bytes , use fsencode() .

The scandir() function returns directory entries along with file attribute information, giving better performance for many common use cases.

Changed in version 3.2: The path parameter became optional.

New in version 3.3: Added support for specifying path as an open file descriptor.

Changed in version 3.6: Accepts a path-like object .

Perform the equivalent of an lstat() system call on the given path. Similar to stat() , but does not follow symbolic links. Return a stat_result object.

On platforms that do not support symbolic links, this is an alias for stat() .

As of Python 3.3, this is equivalent to os.stat(path, dir_fd=dir_fd, follow_symlinks=False) .

Changed in version 3.2: Added support for Windows 6.0 (Vista) symbolic links.

Changed in version 3.3: Added the dir_fd parameter.

Changed in version 3.6: Accepts a path-like object .

Changed in version 3.8: On Windows, now opens reparse points that represent another path (name surrogates), including symbolic links and directory junctions. Other kinds of reparse points are resolved by the operating system as for stat() .

Create a directory named path with numeric mode mode.

If the directory already exists, FileExistsError is raised. If a parent directory in the path does not exist, FileNotFoundError is raised.

On some systems, mode is ignored. Where it is used, the current umask value is first masked out. If bits other than the last 9 (i.e. the last 3 digits of the octal representation of the mode) are set, their meaning is platform-dependent. On some platforms, they are ignored and you should call chmod() explicitly to set them.

It is also possible to create temporary directories; see the tempfile module’s tempfile.mkdtemp() function.

Raises an auditing event os.mkdir with arguments path , mode , dir_fd .

New in version 3.3: The dir_fd argument.

Changed in version 3.6: Accepts a path-like object .

Recursive directory creation function. Like mkdir() , but makes all intermediate-level directories needed to contain the leaf directory.

The mode parameter is passed to mkdir() for creating the leaf directory; see the mkdir() description for how it is interpreted. To set the file permission bits of any newly created parent directories you can set the umask before invoking makedirs() . The file permission bits of existing parent directories are not changed.

If exist_ok is False (the default), a FileExistsError is raised if the target directory already exists.

makedirs() will become confused if the path elements to create include pardir (eg. “..” on UNIX systems).

This function handles UNC paths correctly.

Raises an auditing event os.mkdir with arguments path , mode , dir_fd .

New in version 3.2: The exist_ok parameter.

Changed in version 3.4.1: Before Python 3.4.1, if exist_ok was True and the directory existed, makedirs() would still raise an error if mode did not match the mode of the existing directory. Since this behavior was impossible to implement safely, it was removed in Python 3.4.1. See bpo-21082.

Changed in version 3.6: Accepts a path-like object .

Changed in version 3.7: The mode argument no longer affects the file permission bits of newly created intermediate-level directories.

Create a FIFO (a named pipe) named path with numeric mode mode. The current umask value is first masked out from the mode.

FIFOs are pipes that can be accessed like regular files. FIFOs exist until they are deleted (for example with os.unlink() ). Generally, FIFOs are used as rendezvous between “client” and “server” type processes: the server opens the FIFO for reading, and the client opens it for writing. Note that mkfifo() doesn’t open the FIFO — it just creates the rendezvous point.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3: The dir_fd argument.

Changed in version 3.6: Accepts a path-like object .

Create a filesystem node (file, device special file or named pipe) named path. mode specifies both the permissions to use and the type of node to be created, being combined (bitwise OR) with one of stat.S_IFREG , stat.S_IFCHR , stat.S_IFBLK , and stat.S_IFIFO (those constants are available in stat ). For stat.S_IFCHR and stat.S_IFBLK , device defines the newly created device special file (probably using os.makedev() ), otherwise it is ignored.

Availability : Unix, not Emscripten, not WASI.

New in version 3.3: The dir_fd argument.

Changed in version 3.6: Accepts a path-like object .

Extract the device major number from a raw device number (usually the st_dev or st_rdev field from stat ).

Extract the device minor number from a raw device number (usually the st_dev or st_rdev field from stat ).

Compose a raw device number from the major and minor device numbers.

Return system configuration information relevant to a named file. name specifies the configuration value to retrieve; it may be a string which is the name of a defined system value; these names are specified in a number of standards (POSIX.1, Unix 95, Unix 98, and others). Some platforms define additional names as well. The names known to the host operating system are given in the pathconf_names dictionary. For configuration variables not included in that mapping, passing an integer for name is also accepted.

If name is a string and is not known, ValueError is raised. If a specific value for name is not supported by the host system, even if it is included in pathconf_names , an OSError is raised with errno.EINVAL for the error number.

Changed in version 3.6: Accepts a path-like object .

Dictionary mapping names accepted by pathconf() and fpathconf() to the integer values defined for those names by the host operating system. This can be used to determine the set of names known to the system.

Return a string representing the path to which the symbolic link points. The result may be either an absolute or relative pathname; if it is relative, it may be converted to an absolute pathname using os.path.join(os.path.dirname(path), result) .

If the path is a string object (directly or indirectly through a PathLike interface), the result will also be a string object, and the call may raise a UnicodeDecodeError. If the path is a bytes object (direct or indirectly), the result will be a bytes object.

When trying to resolve a path that may contain links, use realpath() to properly handle recursion and platform differences.

Changed in version 3.2: Added support for Windows 6.0 (Vista) symbolic links.

New in version 3.3: The dir_fd argument.

Changed in version 3.6: Accepts a path-like object on Unix.

Changed in version 3.8: Accepts a path-like object and a bytes object on Windows.

Changed in version 3.8: Added support for directory junctions, and changed to return the substitution path (which typically includes \\?\ prefix) rather than the optional “print name” field that was previously returned.

Remove (delete) the file path. If path is a directory, an OSError is raised. Use rmdir() to remove directories. If the file does not exist, a FileNotFoundError is raised.

On Windows, attempting to remove a file that is in use causes an exception to be raised; on Unix, the directory entry is removed but the storage allocated to the file is not made available until the original file is no longer in use.

This function is semantically identical to unlink() .

Raises an auditing event os.remove with arguments path , dir_fd .

New in version 3.3: The dir_fd argument.

Changed in version 3.6: Accepts a path-like object .

Remove directories recursively. Works like rmdir() except that, if the leaf directory is successfully removed, removedirs() tries to successively remove every parent directory mentioned in path until an error is raised (which is ignored, because it generally means that a parent directory is not empty). For example, os.removedirs(‘foo/bar/baz’) will first remove the directory ‘foo/bar/baz’ , and then remove ‘foo/bar’ and ‘foo’ if they are empty. Raises OSError if the leaf directory could not be successfully removed.

Raises an auditing event os.remove with arguments path , dir_fd .

Changed in version 3.6: Accepts a path-like object .

Rename the file or directory src to dst. If dst exists, the operation will fail with an OSError subclass in a number of cases:

On Windows, if dst exists a FileExistsError is always raised. The operation may fail if src and dst are on different filesystems. Use shutil.move() to support moves to a different filesystem.

On Unix, if src is a file and dst is a directory or vice-versa, an IsADirectoryError or a NotADirectoryError will be raised respectively. If both are directories and dst is empty, dst will be silently replaced. If dst is a non-empty directory, an OSError is raised. If both are files, dst will be replaced silently if the user has permission. The operation may fail on some Unix flavors if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement).

This function can support specifying src_dir_fd and/or dst_dir_fd to supply paths relative to directory descriptors .

If you want cross-platform overwriting of the destination, use replace() .

Raises an auditing event os.rename with arguments src , dst , src_dir_fd , dst_dir_fd .

New in version 3.3: The src_dir_fd and dst_dir_fd arguments.

Changed in version 3.6: Accepts a path-like object for src and dst.

Recursive directory or file renaming function. Works like rename() , except creation of any intermediate directories needed to make the new pathname good is attempted first. After the rename, directories corresponding to rightmost path segments of the old name will be pruned away using removedirs() .

This function can fail with the new directory structure made if you lack permissions needed to remove the leaf directory or file.

Raises an auditing event os.rename with arguments src , dst , src_dir_fd , dst_dir_fd .

Changed in version 3.6: Accepts a path-like object for old and new.

Rename the file or directory src to dst. If dst is a non-empty directory, OSError will be raised. If dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement).

This function can support specifying src_dir_fd and/or dst_dir_fd to supply paths relative to directory descriptors .

Raises an auditing event os.rename with arguments src , dst , src_dir_fd , dst_dir_fd .

New in version 3.3.

Changed in version 3.6: Accepts a path-like object for src and dst.

Remove (delete) the directory path. If the directory does not exist or is not empty, a FileNotFoundError or an OSError is raised respectively. In order to remove whole directory trees, shutil.rmtree() can be used.

Raises an auditing event os.rmdir with arguments path , dir_fd .

New in version 3.3: The dir_fd parameter.

Changed in version 3.6: Accepts a path-like object .

Return an iterator of os.DirEntry objects corresponding to the entries in the directory given by path. The entries are yielded in arbitrary order, and the special entries ‘.’ and ‘..’ are not included. If a file is removed from or added to the directory after creating the iterator, whether an entry for that file be included is unspecified.

Using scandir() instead of listdir() can significantly increase the performance of code that also needs file type or file attribute information, because os.DirEntry objects expose this information if the operating system provides it when scanning a directory. All os.DirEntry methods may perform a system call, but is_dir() and is_file() usually only require a system call for symbolic links; os.DirEntry.stat() always requires a system call on Unix but only requires one for symbolic links on Windows.

path may be a path-like object . If path is of type bytes (directly or indirectly through the PathLike interface), the type of the name and path attributes of each os.DirEntry will be bytes ; in all other circumstances, they will be of type str .

This function can also support specifying a file descriptor ; the file descriptor must refer to a directory.

Raises an auditing event os.scandir with argument path .

The scandir() iterator supports the context manager protocol and has the following method:

Close the iterator and free acquired resources.

This is called automatically when the iterator is exhausted or garbage collected, or when an error happens during iterating. However it is advisable to call it explicitly or use the with statement.

New in version 3.6.

The following example shows a simple use of scandir() to display all the files (excluding directories) in the given path that don’t start with ‘.’ . The entry.is_file() call will generally not make an additional system call:

On Unix-based systems, scandir() uses the system’s opendir() and readdir() functions. On Windows, it uses the Win32 FindFirstFileW and FindNextFileW functions.

New in version 3.5.

New in version 3.6: Added support for the context manager protocol and the close() method. If a scandir() iterator is neither exhausted nor explicitly closed a ResourceWarning will be emitted in its destructor.

The function accepts a path-like object .

Changed in version 3.7: Added support for file descriptors on Unix.

Object yielded by scandir() to expose the file path and other file attributes of a directory entry.

scandir() will provide as much of this information as possible without making additional system calls. When a stat() or lstat() system call is made, the os.DirEntry object will cache the result.

os.DirEntry instances are not intended to be stored in long-lived data structures; if you know the file metadata has changed or if a long time has elapsed since calling scandir() , call os.stat(entry.path) to fetch up-to-date information.

Because the os.DirEntry methods can make operating system calls, they may also raise OSError . If you need very fine-grained control over errors, you can catch OSError when calling one of the os.DirEntry methods and handle as appropriate.

To be directly usable as a path-like object , os.DirEntry implements the PathLike interface.

Attributes and methods on a os.DirEntry instance are as follows:

The entry’s base filename, relative to the scandir() path argument.

The name attribute will be bytes if the scandir() path argument is of type bytes and str otherwise. Use fsdecode() to decode byte filenames.

The entry’s full path name: equivalent to os.path.join(scandir_path, entry.name) where scandir_path is the scandir() path argument. The path is only absolute if the scandir() path argument was absolute. If the scandir() path argument was a file descriptor , the path attribute is the same as the name attribute.

The path attribute will be bytes if the scandir() path argument is of type bytes and str otherwise. Use fsdecode() to decode byte filenames.

Return the inode number of the entry.

The result is cached on the os.DirEntry object. Use os.stat(entry.path, follow_symlinks=False).st_ino to fetch up-to-date information.

On the first, uncached call, a system call is required on Windows but not on Unix.

is_dir ( * , follow_symlinks = True ) ¶

Return True if this entry is a directory or a symbolic link pointing to a directory; return False if the entry is or points to any other kind of file, or if it doesn’t exist anymore.

If follow_symlinks is False , return True only if this entry is a directory (without following symlinks); return False if the entry is any other kind of file or if it doesn’t exist anymore.

The result is cached on the os.DirEntry object, with a separate cache for follow_symlinks True and False . Call os.stat() along with stat.S_ISDIR() to fetch up-to-date information.

On the first, uncached call, no system call is required in most cases. Specifically, for non-symlinks, neither Windows or Unix require a system call, except on certain Unix file systems, such as network file systems, that return dirent.d_type == DT_UNKNOWN . If the entry is a symlink, a system call will be required to follow the symlink unless follow_symlinks is False .

This method can raise OSError , such as PermissionError , but FileNotFoundError is caught and not raised.

is_file ( * , follow_symlinks = True ) ¶

Return True if this entry is a file or a symbolic link pointing to a file; return False if the entry is or points to a directory or other non-file entry, or if it doesn’t exist anymore.

If follow_symlinks is False , return True only if this entry is a file (without following symlinks); return False if the entry is a directory or other non-file entry, or if it doesn’t exist anymore.

The result is cached on the os.DirEntry object. Caching, system calls made, and exceptions raised are as per is_dir() .

Return True if this entry is a symbolic link (even if broken); return False if the entry points to a directory or any kind of file, or if it doesn’t exist anymore.

The result is cached on the os.DirEntry object. Call os.path.islink() to fetch up-to-date information.

On the first, uncached call, no system call is required in most cases. Specifically, neither Windows or Unix require a system call, except on certain Unix file systems, such as network file systems, that return dirent.d_type == DT_UNKNOWN .

This method can raise OSError , such as PermissionError , but FileNotFoundError is caught and not raised.

stat ( * , follow_symlinks = True ) ¶

Return a stat_result object for this entry. This method follows symbolic links by default; to stat a symbolic link add the follow_symlinks=False argument.

On Unix, this method always requires a system call. On Windows, it only requires a system call if follow_symlinks is True and the entry is a reparse point (for example, a symbolic link or directory junction).

On Windows, the st_ino , st_dev and st_nlink attributes of the stat_result are always set to zero. Call os.stat() to get these attributes.

The result is cached on the os.DirEntry object, with a separate cache for follow_symlinks True and False . Call os.stat() to fetch up-to-date information.

Note that there is a nice correspondence between several attributes and methods of os.DirEntry and of pathlib.Path . In particular, the name attribute has the same meaning, as do the is_dir() , is_file() , is_symlink() and stat() methods.

New in version 3.5.

Changed in version 3.6: Added support for the PathLike interface. Added support for bytes paths on Windows.

Get the status of a file or a file descriptor. Perform the equivalent of a stat() system call on the given path. path may be specified as either a string or bytes – directly or indirectly through the PathLike interface – or as an open file descriptor. Return a stat_result object.

This function normally follows symlinks; to stat a symlink add the argument follow_symlinks=False , or use lstat() .

On Windows, passing follow_symlinks=False will disable following all name-surrogate reparse points, which includes symlinks and directory junctions. Other types of reparse points that do not resemble links or that the operating system is unable to follow will be opened directly. When following a chain of multiple links, this may result in the original link being returned instead of the non-link that prevented full traversal. To obtain stat results for the final path in this case, use the os.path.realpath() function to resolve the path name as far as possible and call lstat() on the result. This does not apply to dangling symlinks or junction points, which will raise the usual exceptions.

New in version 3.3: Added the dir_fd and follow_symlinks arguments, specifying a file descriptor instead of a path.

Changed in version 3.6: Accepts a path-like object .

Changed in version 3.8: On Windows, all reparse points that can be resolved by the operating system are now followed, and passing follow_symlinks=False disables following all name surrogate reparse points. If the operating system reaches a reparse point that it is not able to follow, stat now returns the information for the original path as if follow_symlinks=False had been specified instead of raising an error.

Object whose attributes correspond roughly to the members of the stat structure. It is used for the result of os.stat() , os.fstat() and os.lstat() .

File mode: file type and file mode bits (permissions).

Platform dependent, but if non-zero, uniquely identifies the file for a given value of st_dev . Typically:

the inode number on Unix,

the file index on Windows

Identifier of the device on which this file resides.

Number of hard links.

User identifier of the file owner.

Group identifier of the file owner.

Size of the file in bytes, if it is a regular file or a symbolic link. The size of a symbolic link is the length of the pathname it contains, without a terminating null byte.

Time of most recent access expressed in seconds.

Time of most recent content modification expressed in seconds.

the time of most recent metadata change on Unix,

the time of creation on Windows, expressed in seconds.

Time of most recent access expressed in nanoseconds as an integer.

Time of most recent content modification expressed in nanoseconds as an integer.

the time of most recent metadata change on Unix,

the time of creation on Windows, expressed in nanoseconds as an integer.

The exact meaning and resolution of the st_atime , st_mtime , and st_ctime attributes depend on the operating system and the file system. For example, on Windows systems using the FAT or FAT32 file systems, st_mtime has 2-second resolution, and st_atime has only 1-day resolution. See your operating system documentation for details.

Similarly, although st_atime_ns , st_mtime_ns , and st_ctime_ns are always expressed in nanoseconds, many systems do not provide nanosecond precision. On systems that do provide nanosecond precision, the floating-point object used to store st_atime , st_mtime , and st_ctime cannot preserve all of it, and as such will be slightly inexact. If you need the exact timestamps you should always use st_atime_ns , st_mtime_ns , and st_ctime_ns .

On some Unix systems (such as Linux), the following attributes may also be available:

Number of 512-byte blocks allocated for file. This may be smaller than st_size /512 when the file has holes.

“Preferred” blocksize for efficient file system I/O. Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite.

Type of device if an inode device.

User defined flags for file.

On other Unix systems (such as FreeBSD), the following attributes may be available (but may be only filled out if root tries to use them):

File generation number.

Time of file creation.

On Solaris and derivatives, the following attributes may also be available:

String that uniquely identifies the type of the filesystem that contains the file.

On macOS systems, the following attributes may also be available:

Real size of the file.

Creator of the file.

On Windows systems, the following attributes are also available:

Windows file attributes: dwFileAttributes member of the BY_HANDLE_FILE_INFORMATION structure returned by GetFileInformationByHandle() . See the FILE_ATTRIBUTE_* constants in the stat module.

When st_file_attributes has the FILE_ATTRIBUTE_REPARSE_POINT set, this field contains the tag identifying the type of reparse point. See the IO_REPARSE_TAG_* constants in the stat module.

The standard module stat defines functions and constants that are useful for extracting information from a stat structure. (On Windows, some items are filled with dummy values.)

For backward compatibility, a stat_result instance is also accessible as a tuple of at least 10 integers giving the most important (and portable) members of the stat structure, in the order st_mode , st_ino , st_dev , st_nlink , st_uid , st_gid , st_size , st_atime , st_mtime , st_ctime . More items may be added at the end by some implementations. For compatibility with older Python versions, accessing stat_result as a tuple always returns integers.

New in version 3.3: Added the st_atime_ns , st_mtime_ns , and st_ctime_ns members.

New in version 3.5: Added the st_file_attributes member on Windows.

Changed in version 3.5: Windows now returns the file index as st_ino when available.

New in version 3.7: Added the st_fstype member to Solaris/derivatives.

New in version 3.8: Added the st_reparse_tag member on Windows.

Changed in version 3.8: On Windows, the st_mode member now identifies special files as S_IFCHR , S_IFIFO or S_IFBLK as appropriate.

Perform a statvfs() system call on the given path. The return value is an object whose attributes describe the filesystem on the given path, and correspond to the members of the statvfs structure, namely: f_bsize , f_frsize , f_blocks , f_bfree , f_bavail , f_files , f_ffree , f_favail , f_flag , f_namemax , f_fsid .

Two module-level constants are defined for the f_flag attribute’s bit-flags: if ST_RDONLY is set, the filesystem is mounted read-only, and if ST_NOSUID is set, the semantics of setuid/setgid bits are disabled or not supported.

Additional module-level constants are defined for GNU/glibc based systems. These are ST_NODEV (disallow access to device special files), ST_NOEXEC (disallow program execution), ST_SYNCHRONOUS (writes are synced at once), ST_MANDLOCK (allow mandatory locks on an FS), ST_WRITE (write on file/directory/symlink), ST_APPEND (append-only file), ST_IMMUTABLE (immutable file), ST_NOATIME (do not update access times), ST_NODIRATIME (do not update directory access times), ST_RELATIME (update atime relative to mtime/ctime).

Changed in version 3.2: The ST_RDONLY and ST_NOSUID constants were added.

New in version 3.3: Added support for specifying path as an open file descriptor.

Changed in version 3.4: The ST_NODEV , ST_NOEXEC , ST_SYNCHRONOUS , ST_MANDLOCK , ST_WRITE , ST_APPEND , ST_IMMUTABLE , ST_NOATIME , ST_NODIRATIME , and ST_RELATIME constants were added.

Changed in version 3.6: Accepts a path-like object .

New in version 3.7: Added f_fsid .

A set object indicating which functions in the os module accept an open file descriptor for their dir_fd parameter. Different platforms provide different features, and the underlying functionality Python uses to implement the dir_fd parameter is not available on all platforms Python supports. For consistency’s sake, functions that may support dir_fd always allow specifying the parameter, but will throw an exception if the functionality is used when it’s not locally available. (Specifying None for dir_fd is always supported on all platforms.)

To check whether a particular function accepts an open file descriptor for its dir_fd parameter, use the in operator on supports_dir_fd . As an example, this expression evaluates to True if os.stat() accepts open file descriptors for dir_fd on the local platform:

Currently dir_fd parameters only work on Unix platforms; none of them work on Windows.

New in version 3.3.

A set object indicating whether os.access() permits specifying True for its effective_ids parameter on the local platform. (Specifying False for effective_ids is always supported on all platforms.) If the local platform supports it, the collection will contain os.access() ; otherwise it will be empty.

This expression evaluates to True if os.access() supports effective_ids=True on the local platform:

Currently effective_ids is only supported on Unix platforms; it does not work on Windows.

New in version 3.3.

A set object indicating which functions in the os module permit specifying their path parameter as an open file descriptor on the local platform. Different platforms provide different features, and the underlying functionality Python uses to accept open file descriptors as path arguments is not available on all platforms Python supports.

To determine whether a particular function permits specifying an open file descriptor for its path parameter, use the in operator on supports_fd . As an example, this expression evaluates to True if os.chdir() accepts open file descriptors for path on your local platform:

New in version 3.3.

A set object indicating which functions in the os module accept False for their follow_symlinks parameter on the local platform. Different platforms provide different features, and the underlying functionality Python uses to implement follow_symlinks is not available on all platforms Python supports. For consistency’s sake, functions that may support follow_symlinks always allow specifying the parameter, but will throw an exception if the functionality is used when it’s not locally available. (Specifying True for follow_symlinks is always supported on all platforms.)

To check whether a particular function accepts False for its follow_symlinks parameter, use the in operator on supports_follow_symlinks . As an example, this expression evaluates to True if you may specify follow_symlinks=False when calling os.stat() on the local platform:

New in version 3.3.

Create a symbolic link pointing to src named dst.

On Windows, a symlink represents either a file or a directory, and does not morph to the target dynamically. If the target is present, the type of the symlink will be created to match. Otherwise, the symlink will be created as a directory if target_is_directory is True or a file symlink (the default) otherwise. On non-Windows platforms, target_is_directory is ignored.

On newer versions of Windows 10, unprivileged accounts can create symlinks if Developer Mode is enabled. When Developer Mode is not available/enabled, the SeCreateSymbolicLinkPrivilege privilege is required, or the process must be run as an administrator.

OSError is raised when the function is called by an unprivileged user.

Raises an auditing event os.symlink with arguments src , dst , dir_fd .

The function is limited on Emscripten and WASI, see WebAssembly platforms for more information.

Changed in version 3.2: Added support for Windows 6.0 (Vista) symbolic links.

New in version 3.3: Added the dir_fd argument, and now allow target_is_directory on non-Windows platforms.

Changed in version 3.6: Accepts a path-like object for src and dst.

Changed in version 3.8: Added support for unelevated symlinks on Windows with Developer Mode.

Force write of everything to disk.

New in version 3.3.

Truncate the file corresponding to path, so that it is at most length bytes in size.

Raises an auditing event os.truncate with arguments path , length .

New in version 3.3.

Changed in version 3.5: Added support for Windows

Changed in version 3.6: Accepts a path-like object .

Remove (delete) the file path. This function is semantically identical to remove() ; the unlink name is its traditional Unix name. Please see the documentation for remove() for further information.

Raises an auditing event os.remove with arguments path , dir_fd .

New in version 3.3: The dir_fd parameter.

Changed in version 3.6: Accepts a path-like object .

Set the access and modified times of the file specified by path.

utime() takes two optional parameters, times and ns. These specify the times set on path and are used as follows:

If ns is specified, it must be a 2-tuple of the form (atime_ns, mtime_ns) where each member is an int expressing nanoseconds.

If times is not None , it must be a 2-tuple of the form (atime, mtime) where each member is an int or float expressing seconds.

If times is None and ns is unspecified, this is equivalent to specifying ns=(atime_ns, mtime_ns) where both times are the current time.

It is an error to specify tuples for both times and ns.

Note that the exact times you set here may not be returned by a subsequent stat() call, depending on the resolution with which your operating system records access and modification times; see stat() . The best way to preserve exact times is to use the st_atime_ns and st_mtime_ns fields from the os.stat() result object with the ns parameter to utime() .

Raises an auditing event os.utime with arguments path , times , ns , dir_fd .

New in version 3.3: Added support for specifying path as an open file descriptor, and the dir_fd, follow_symlinks, and ns parameters.

Changed in version 3.6: Accepts a path-like object .

Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at directory top (including top itself), it yields a 3-tuple (dirpath, dirnames, filenames) .

dirpath is a string, the path to the directory. dirnames is a list of the names of the subdirectories in dirpath (including symlinks to directories, and excluding ‘.’ and ‘..’ ). filenames is a list of the names of the non-directory files in dirpath. Note that the names in the lists contain no path components. To get a full path (which begins with top) to a file or directory in dirpath, do os.path.join(dirpath, name) . Whether or not the lists are sorted depends on the file system. If a file is removed from or added to the dirpath directory during generating the lists, whether a name for that file be included is unspecified.

If optional argument topdown is True or not specified, the triple for a directory is generated before the triples for any of its subdirectories (directories are generated top-down). If topdown is False , the triple for a directory is generated after the triples for all of its subdirectories (directories are generated bottom-up). No matter the value of topdown, the list of subdirectories is retrieved before the tuples for the directory and its subdirectories are generated.

When topdown is True , the caller can modify the dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into the subdirectories whose names remain in dirnames; this can be used to prune the search, impose a specific order of visiting, or even to inform walk() about directories the caller creates or renames before it resumes walk() again. Modifying dirnames when topdown is False has no effect on the behavior of the walk, because in bottom-up mode the directories in dirnames are generated before dirpath itself is generated.

By default, errors from the scandir() call are ignored. If optional argument onerror is specified, it should be a function; it will be called with one argument, an OSError instance. It can report the error to continue with the walk, or raise the exception to abort the walk. Note that the filename is available as the filename attribute of the exception object.

By default, walk() will not walk down into symbolic links that resolve to directories. Set followlinks to True to visit directories pointed to by symlinks, on systems that support them.

Be aware that setting followlinks to True can lead to infinite recursion if a link points to a parent directory of itself. walk() does not keep track of the directories it visited already.

If you pass a relative pathname, don’t change the current working directory between resumptions of walk() . walk() never changes the current directory, and assumes that its caller doesn’t either.

This example displays the number of bytes taken by non-directory files in each directory under the starting directory, except that it doesn’t look under any CVS subdirectory:

In the next example (simple implementation of shutil.rmtree() ), walking the tree bottom-up is essential, rmdir() doesn’t allow deleting a directory before the directory is empty:

Raises an auditing event os.walk with arguments top , topdown , onerror , followlinks .

Changed in version 3.5: This function now calls os.scandir() instead of os.listdir() , making it faster by reducing the number of calls to os.stat() .

Changed in version 3.6: Accepts a path-like object .

This behaves exactly like walk() , except that it yields a 4-tuple (dirpath, dirnames, filenames, dirfd) , and it supports dir_fd .

dirpath, dirnames and filenames are identical to walk() output, and dirfd is a file descriptor referring to the directory dirpath.

This function always supports paths relative to directory descriptors and not following symlinks . Note however that, unlike other functions, the fwalk() default value for follow_symlinks is False .

Since fwalk() yields file descriptors, those are only valid until the next iteration step, so you should duplicate them (e.g. with dup() ) if you want to keep them longer.

This example displays the number of bytes taken by non-directory files in each directory under the starting directory, except that it doesn’t look under any CVS subdirectory:

In the next example, walking the tree bottom-up is essential: rmdir() doesn’t allow deleting a directory before the directory is empty:

Name already in use

py_training_ru / lecture_08.md

• Go to file T
• Go to line L
• Copy path
• Copy permalink

• Open with Desktop
• View raw
• Copy raw contents Copy raw contents

Copy raw contents

Copy raw contents

Лекция 8. Ввод-вывод в файлы. Конструкция with

Значки, используемые в тексте

• �� — отмечает, что высказывание не совсем верно, но будет уточнено позднее.
• ⭐ — выделяет определение нового понятия.
• ❗ — привлекает внимание к высказыванию.
• ☝️ — указывает на небольшое отступление от темы или уточнение.
• �� — индикатор традиций и договорённостей.
• ⚠️ — требуется осторожность: возможно, сложное высказывание.

Повторение терминов из седьмой лекции

• Конструкция try-except-else — синтаскическая конструкция (сложное утверждение), предназначенная для обработки исключений из блока try в блоках except и выполнения блока else в случае отсутствия ошибок.
• Утверждение raise — утверждение, дающее возможность непосредственно вызвать исключение в коде.

Что такое файл? С точки зрения Python, это объект файловой системы, поддерживающий потоковое чтение и запись.

Для того, чтобы начать работу с файлом (чтение или запись), необходимо открыть файл, с указанием вида работы.

Файл обычно идентифицируется именем, и Python также использует имя файла для его открытия, которое осуществляется встроенной функцией open() :

Функция open() принимает как минимум 1 аргумент — имя файла — и возвращает объект типа file (также встроенный в Python тип), с которым далее и ведётся работа. Вторым аргументом функции указывается способ работы, то есть чтение или запись; по умолчанию, файл открывается на чтение.

У объекта файла есть набор атрибутов и методов, которые в дальнейшем используются для работы с файлом.

Имя файла, связанного с объектом, доступно через атрибут file.name :

При отсутствии такого файла или невозможности открыть его (например, нет разрешений, либо файл является каталогом) возникнет исключение IOError .

Чтобы прочитать содержимое файла, используется метод file.read() . Он возвращает Python-строку ( str ) состоящую из всего содержимого файла, либо если передать ему 1 аргумент — целое число N, то строка будет состоять из первых N байт файла (либо файла целиком, если его размер меньше N байт).

Можно написать программу из 2 строк, которая выведет собственный код, для этого достаточно сохранить её в файл например program1.py и написать:

Указатель позиции в файле

Если вызвать метод file.read() без аргументов более одного раза, то он вернёт пустую строку:

Объяснение этому таково. Файл можно представить как упорядоченную последовательность байт, каждый из которых имеет индекс, начиная с 0 (первый байт) и до «размер файла — 1». При чтении данных из файла, каждый байт условно по очереди передаётся в строку-результат, и индекс (указатель на текущий байт) увеличивается. По окончании чтения, индекс остаётся в том же месте, и следующее чтение начнётся с этого индекса. Если мы уже прочитали весь файл, то логично, что данных после текущего указателя уже нет.

Метод file.read() не только читает данные из файла, но и увеличивает указатель на кол-во прочитанных байт. Если в цикле читать из файла по одному байту, то каждый раз в результате чтения будет следующий байт, до тех пор, пока мы не достигнем конца файла.

Текстовые и бинарные файлы

Во многих системах файлы бывают двух «разновидностей»: текстовые и двоичные (бинарные). В общем любой файл можно считать бинарным, особенно если нам не требуется работать с файлом по частям. Если мы считываем весь файл в строку с помощью метода file.read() , то далее мы можем работать со строкой, полностью игнорируя вид файла.

Главное отличие этих видов в семантике доступа: для текстовых файлов имеет смысл построчный доступ, для бинарных — посимвольный или буферизованный.

С точки зрения пользователя, вид файла не всегда имеет значение, скажем, файл с данными xml или json по содержанию является текстовым, но едва ли имеет смысл обрабатывать его построчно, так как данные представляют собой единую структуру. В то же время файл типа doc для пользователя представляет двоичную кодировку текста.

Работа с текстовыми файлами

Типичным текстовым файлом является файл программы на Python. Для построчного чтения текстовых файлов существует метод file.readline() , который читает строку из файла и увеличивает указатель на длину прочитанной строки. При этом конец строки определяется настройками текущей ОС: для Linux концом строки служит символ перевода строки «\n» (или «\x0a» ), для Windows — последовательность «\r\n» (или «\x0d\x0a» ).

Этот метод удобно использовать в цикле while , опять же проверяя, что считанная строка не пустая. Также в этом методе можно задавать максимальную длину считываемой строки.

Для большего удобства построчного чтения файлов целиком, в цикле for мы можем использовать метод file.readlines() :

Или даже ещё проще:

Объект типа файл является итерируемым, как списки и строки, и итерируется по строкам, содержащимся в файле.

Работа с бинарными файлами

Для итеративной (поблочной) работы с бинарными файлами используется метод file.read(N) , где N — размер считываемого блока. Проверять, что достигнут конец файла, можно, сравнивая длину результата с N, либо также считывать до получения пустой строки.

У бинарных файлов есть ещё одно отличие: мы можем узнать, где сейчас указатель, и передвинуть указатель вручную на нужное место. (То есть, мы можем это делать и в текстовых файлах, но там это бессмысленно.) Для этого предназначены два соответствующих метода: file.tell() и file.seek() .

Первый возвращает индекс байта, на котором сейчас указатель; второй — принимает число-индекс в качестве аргумента и устанавливает указатель на этот индекс.

Таким образом, мы в теории сможем при достижении конца файла вернуть указатель на начало и прочитать содержимое заново. Впрочем, на практике это крайне редко используется.

Для разрешения записи в файл, как мы уже упомянули выше, необходимо сообщить функции open() , что мы открываем файл на запись. Режим работы с файлом передаётся вторым аргументом функции open() и представляет собой строку.

Запись может быть двух видов: перезапись (создание файла, если его ещё нет, либо пересоздание файла, если он уже есть) и дозапись (добавление в конец файла, если он есть, либо ошибка IOError в случае отсутствия).

Режим записи обозначается строкой «w» , режим добавления — строкой «a» .

Далее, собственно писать в файл можно исользуя метод file.write() , который принимает параметр — Python-строку с содержимым для записи. При записи, как и при чтении, указатель позиции в файле увеличивается на количество записанных байт.

Однако если просто записать в открытый файл, записанные данные не сохранятся. Для сохранения данных необходимо закрыть файл, это делается методом file.close() .

Примечание: файлы сами закрываются при выходе из программы, но крайне рекомендуется закрывать их вручную во избежание ошибок.

Смешанные режимы открытия файла

Возможно также открыть файл в режиме, совмещающем чтение и запись. Существует три таких режима:

• чтение с возможностью записи, режим «r+»
• перезапись с возможностью чтения, режим «w+»
• дозапись с возможностью чтения, режим «a+»

Они в целом соответствуют своим «основным» режимам.

Некоторые синтаксические конструкции, облегчающие работу с файлами

Представим себе ситуацию, что мы обрабатываем файл построчно и ищем какую-то строку, после которой дальнейшая обработка уже не имеет смысла. Используя цикл for , нам бы пришлось создавать переменную-флаг и «вхолостую» пробегать остаток файла, проверяя этот флаг в каждой итерации.

К счастью, в Python можно обойтись куда более простой конструкцией.

Утверждение break даёт команду немедленно выйти из цикла к следующему за ним утверждению, полностью пропустив оставшиеся итерации и также все утверждения, идущие в блоке кода цикла после break .

В данном случае будет выводиться на экран каждая строка до строки, содержащей «end», не включая её, потому что цикл закончится до вывода её на экран.

Утверждение break можно использовать не только в обработке файлов, и не только в цикле for : циклы while также поддерживают его. Но в любом другом блоке кода, который не относится к циклам, использование break является синтаксической ошибкой.

Утверждение continue является дополнением break : оно пропускает все следующие за ним утверждения в блоке кода цикла и переходит к следующей его итерации. В случае с циклом for это означает — к обработке следующего элемента (если он есть), в случае while — к следующей проверке условия.

Его удобно использовать, если нам нужно, наоборот, начать обработку файла с какой-то строки файла или пропустить обработку строки.

Этот код будет пропускать все строки файла до строки, содержащей «start», начиная с неё, он начнёт выводить все те строки, которые не содержат «skip».

Объект файла, будучи итерируемым объектом (типа контейнера) является ещё и так называемым контекстным менеджером.

Контекстный менеджер умеет автоматически выполнять некторые действия по окончанию выполнения определённого контекста, который задаётся конструкцией with :

БЛОК_КОДА в данном случае и является контекстом. Независимо от того, произошли ли ошибки в БЛОКе_КОДА, контекстный менеджер всегда выполнит завершающие действия, а именно для файлов — их закрытие.

Возьмём пример, использованный выше:

Его можно переписать так:

И файл автоматически будет закрыт по окончанию блока кода в with .

Записать в новый текстовый файл числа от 1 до 10 по одному в строке.

• в файл можно записать только строковые данные.
• В конец записываемой строки необходимо добавлять символы перевода строки, для Windows это последовательность «\r\n» .

Прочитать из записанного файла числа от 3 до 7 включительно, используя break и continue . В результате должен получиться список из 5 чисел.