BPLCK105B/205B : Introduction to Python Programming
Module - 4
Organizing Files: The shutil Module, Walking a Directory Tree, Compressing Files with the zipfile Module, Project: Renaming Files
with American-Style Dates to European-Style Dates,Project: Backing Up a Folder into a ZIP File,
Debugging: Raising Exceptions, Getting the Traceback as a String, Assertions, Logging, IDLE‟s Debugger.
MODULE 4
ORGANIZING FILES
DEBUGGING
Organizing Files
A folder full of dozens, hundreds, or
even thousands of files and copying,
renaming, moving, or compressing
them all by manual or by programs
What is OS module?
This module helps interact with the operating system and
provides a way of using OS dependent functionalities.
It has many functions such startfile(), getcwd() , listdr() ,
the path.isfile() functions and many more.
4
The shutil Module (shell utility)
This module offers a number of high-level operations on
files and collection of files.
The shutil (or shell utilities) module has functions to let you
copy, move, rename, and delete files in your Python
programs.
To use the shutil functions, you will first need to use import
shutil
What is pathlib module?
The pathlib module is part of Python’s standard library, and it helps you deal with
challenges such as
• list all files of a given type in a directory,
• find the parent directory of a given file,
• or create a unique filename that doesn’t already exist.
It gathers the necessary functionality in one place and makes it available through methods
and properties on a convenient Path object.
6
Copying Files and Folders
The shutil module provides functions for copying files, as well as entire
folders
Calling shutil.copy(source, destination) will copy the file at the path source
to the folder at the path destination. (Both source and destination can be
strings or Path objects.)
If destination is a filename, it will be used as the new name of the copied file.
This function returns a string or Path object of the copied file.
Copy file:
import shutil
import os
shutil.copy('C:\\Desktop\\IP\\text.txt‘ , 'C:\\Desktop\\IPP' )
Copy folder:
import shutil
import os
shutil.copy('C:\\Desktop\\IP ‘ , 'C:\\Desktop\\IPP' )
8
Prof. Roopashree S, Dept. of CSE
Move file:
import shutil
import os
shutil.move('C:\\Desktop\\IP\\text.txt‘ , 'C:\\Desktop\\IPP' )
rename file:
import shutil
import os
shutil.move('C:\\Desktop\\text.txt‘ , 'C:\\Desktop\\heee.txt' )
9
Prof. Roopashree S, Dept. of CSE
Deleting file:-should be empty, should not have sub folder
os.unlink(path) delete one file
os.rmdir(path) delete folder
Shutil.rmtree(path) can delete any folder-(folder with files)
Deleting file:-permanent
import os
For fname in os listdir():
if fname.endswith(‘.txt’)
os.unlink(fname)
Deleting file:-send to recycle bin
import os
send2trash
10
Prof. Roopashree S, Dept. of CSE
Create new zip file and add:-
import zipfile
import os
azip=zipfile.ZipFile('new.zip‘, ‘w’)
azip.write(spam.txt,compress_type=zipfile.ZIP_DEFL
ATED)
azip.close()
11
Prof. Roopashree S, Dept. of CSE
Extract zip file:-
import zipfile
import os
azip=zipfile.ZipFile('new.zip')
azip.extractall()
azip.extract(‘spam.txt’)
azip.extract(‘spam.txt’, ‘C:\\my’) #extract to other
location
azip.close()
12
Prof. Roopashree S, Dept. of CSE
Read zip file:-
import zipfile
import os
azip=zipfile.ZipFile('new.zip')
azip.namelist() -#to get file and folder
.filesize
.compres_size
azip.close()
13
Prof. Roopashree S, Dept. of CSE
Copying Files and Folders
The first shutil.copy() call copies the file at
C:\Users\Al\spam.txt to the folder
C:\Users\Al\some_folder.
The return value is the path of the newly
copied file.
Note that since a folder was specified as the
destination ➊, the original spam.txt
filename is used for the new, copied file’s
filename.
15
File copied into another file in another folder
16
Copying Files and Folders
The second shutil.copy() call ➋ also copies
the file at C:\Users\Al\eggs.txt to the folder
C:\Users\Al\some_folder but gives the
copied file the name eggs2.txt
18
Copying Files and Folders
While shutil.copy() will copy a single file, shutil.copytree() will copy an
entire folder and every folder and file contained in it.
Calling shutil.copytree(source, destination) will copy the folder at the
path source, along with all of its files and subfolders, to the folder at the
path destination
The source and destination parameters are both strings
The function returns a string of the path of the copied folder
Copying Files and Folders
The shutil.copytree() call creates a new folder named spam_backup with the same
content as the original spam folder.
It is now safely backed up the precious, precious spam.
Moving and Renaming Files and Folders
Calling shutil.move(source, destination) will move the file or folder at the path
source to the path destination and will return a string of the absolute path of the new
location
If destination points to a folder, the source file gets moved into destination and keeps
its current filename
Permanently Deleting Files and Folders
You can delete a single file or a single empty folder with functions in the os
module, whereas to delete a folder and all of its contents, you use the shutil
module
• Calling os.unlink(path) will delete the file at path
• Calling os.rmdir(path) will delete the folder at path. This folder must be
empty of any files or folders
• Calling shutil.rmtree(path) will remove the folder at path, and all files and
folders it contains will also be deleted
Compressing Files with zipfile Module
•You may be familiar with ZIP files.
•Compressing a file reduces its size, which is useful
when transferring it over the Internet.
•This single file, called an archive file, can then
be, say, attached to an email.
•Your Python programs can both create and open (or
extract) ZIP files using functions in the zipfile module.
Compressing Files with the zipfile Module
Python programs can create and open (or extract) ZIP files using functions in the
zipfile module.
Say you have a ZIP file named example.zip that has the contents shown
Reading ZIP Files
• To read the contents of a ZIP file, first you must create a ZipFile object
(note the capital letters Z and F).
• ZipFile objects are conceptually similar to the File objects you saw
returned by the open() function in the previous chapter: they are values
through which the program interacts with the file.
• To create a ZipFile object, call the zipfile.ZipFile() function, passing it a
string of the .zip file’s filename.
• Note that zipfile is the name of the Python module, and ZipFile() is the
name of the function.
Reading ZIP Files
Reading ZIP Files
A ZipFile object has a namelist() method that returns a list of strings for all the files and
folders contained in the ZIP file.
These strings can be passed to the getinfo() ZipFile method to return a ZipInfo object about
that particular file.
ZipInfo objects have their own attributes, such as file_size and compress_size in bytes,
which hold integers of the original file size and compressed file size, respectively.
While a ZipFile object represents an entire archive file, a ZipInfo object holds useful
information about a single file in the archive.
The command at ➊ calculates how efficiently example.zip is compressed by dividing the
original file size by the compressed file size and prints this information.
Extracting from ZIP Files
The extractall() method for ZipFile objects extracts all the files and folders from a ZIP file into the
current working directory
• The contents of example.zip will be extracted to C:\. Optionally, you can pass a folder name to
extractall() to have it extract the files into a folder other than the current working directory.
• If the folder passed to the extractall() method does not exist, it will be created.
• For instance, if you replaced the call at ➊ with exampleZip.extractall('C:\\delicious'), the code would
extract the files from example.zip into a newly created C:\delicious folder
Extracting from ZIP Files
The extract() method for ZipFile objects will extract a single file from the ZIP file
• The string you pass to extract() must match one of the strings in the list returned by
namelist().
• Optionally, you can pass a second argument to extract() to extract the file into a
folder other than the current working directory.
• If this second argument is a folder that doesn’t yet exist, Python will create the
folder.
• The value that extract() returns is the absolute path to which the file was extracted.
Creating and Adding to ZIP Files
To create your own compressed ZIP files, you must open the ZipFile object in write
mode by passing 'w' as the second argument. (This is similar to opening a text file in
write mode by passing 'w' to the open() function.)
When you pass a path to the write() method of a ZipFile object, Python will compress
the file at that path and add it into the ZIP file.
The write() method’s first argument is a string of the filename to add.
The second argument is the compression type parameter, which tells the computer
what algorithm it should use to compress the files; you can always just set this value to
zipfile.ZIP_DEFLATED. (This specifies the deflate compression algorithm, which
works well on all types of data.)
Creating and Adding to ZIP Files
This code will create a new ZIP file named new.zip that has the compressed
contents of spam.txt
Walking a Directory Tree
Say you want to rename every file in some folder and also every
file in every subfolder of that folder.
That is, you want to walk through the directory tree, touching
each file as you go.
Writing a program to do this could get tricky; fortunately,
Python provides a function to handle this process for you
Walking a Directory Tree
Let’s look at the C:\delicious folder with its
contents
Walking a Directory Tree
Here is an example program that uses the os.walk() function on the directory tree
Walking a Directory Tree
The os.walk() function is passed a single string value: the path of a folder
os.walk() function will return three values on each iteration through the
loop:
• A string of the current folder’s name
• A list of strings of the folders in the current folder
• A list of strings of the files in the current folder
Debugging
•Tools and techniques for finding the root cause of bugs in your
program to help you fix bugs faster and with less effort.
•Few tools and techniques to identify what exactly your code is
doing and where it’s going wrong.
•First, you will look at logging and assertions, two features that can
help you detect bugs early.
•Second, you will look at how to use the debugger(IDLE-inspect the
values in variables).
logging functions (debug, info, warning, error, critical)
37
38
39
Prof. Roopashree S, Dept. of CSE
40
Prof. Roopashree S, Dept. of CSE
41
Prof. Roopashree S, Dept. of CSE
42
43
44
Raising Exceptions
• Python raises an exception whenever it tries to execute invalid code.
• Python’s exceptions with try and except statements.
• Exceptions are raised with a raise statement.
• A raise statement consists of the following:
• The raise keyword:
A call to the Exception() function
A string with a helpful error message passed to the Exception() function
Program example
Getting the Traceback as a String
•When Python encounters an error, it produces error
information called the traceback.
•The traceback includes the error message, the line
number of the line that caused the error, and the sequence
of the function calls that led to the error.
•This sequence of calls is called the call stack
Getting the Traceback as a String
• Program example:
• Open a new file editor window in IDLE, enter the following program, and save it as errorExample.py
def spam():
bacon()
def bacon():
raise Exception('This is the error message.’)
spam()
When you run errorExample.py, the output will look like this:
Traceback (most recent call last):
File "errorExample.py", line 7, in <module> spam()
File "errorExample.py", line 2, in spam bacon()
File "errorExample.py", line 5, in bacon
raise Exception('This is the error message.')
Assertions
• An assertion is a sanity check to make sure your code isn’t doing
something obviously wrong.
• These sanity checks are performed by assert statements. If the sanity
check fails, then an AssertionError exception is raised.
• An assert statement consists of the following:
The assert keyword
A condition (that is, an expression that evaluates to True or False)
A comma
A string to display when the condition is False
49
Assertions
Assertions
51
Assertions
52
Disabling Assertions
• Assertions can be disabled by passing the -O option when running Python.
• Assertions are for development, not the final product.
Logging
• Print()
• Logging is a great way to understand what’s happening in your
program and in what order its happening.
• Python’s logging module makes it easy to create a record of
custom messages that you write.
• example
Logging Levels
• Logging levels provide a way to categorize your log messages by
importance. There are five logging levels.
Logging Levels
Logging to a File
• Instead of displaying the log messages to the screen,
you can write them to a text file.
IDLE’s Debugger
• The debugger is a feature of IDLE that allows you to execute your
program one line at a time.
Useful links
• https://nptel.ac.in/courses/106/106/106106212/
• https://www.tutorialspoint.com/python/python_useful
_resources.htm
• http://www.python.org/doc/
• http://diveintopython.org/
Saving Variables with the shelve Module
Save variables in your Python programs to binary shelf files using the shelve module
This way, your program can restore data to variables from the hard drive
The shelve module will let you add Save and Open features to your program
For example, if you ran a program and entered some configuration settings, you could save those
settings to a shelf file and then have the program load them the next time it is run.
60
Saving Variables with the shelve Module
61
Saving Variables with the shelve Module
To read and write data using the shelve module, you first import shelve
Call shelve.open() and pass it a filename, and then store the returned shelf value in a variable
You can make changes to the shelf value as if it were a dictionary
When you’re done, call close() on the shelf value
Here, the shelf value is stored in shelfFile
We create a list cats and write shelfFile['cats'] = cats to store the list in shelfFile as a value
associated with the key 'cats' (like in a dictionary)
Then we call close() on shelfFile
62
Saving Variables with the shelve Module
After running the previous code on Windows, you will see three new files in the current working
directory: mydata.bak, mydata.dat, and mydata.dir
These binary files contain the data you stored in your shelf
The format of these binary files is not important; you only need to know what the shelve
module does, not how it does it
The module frees you from worrying about how to store your program’s data to a file
63
Saving Variables with the shelve Module
Here, we open the shelf files to check that our data was stored correctly.
Entering shelfFile['cats'] returns the same list that we stored earlier, so we know that the list is
correctly stored, and we call close()
64
Saving Variables with the shelve Module
Just like dictionaries, shelf values have keys() and values() methods that will return list-like
values of the keys and values in the shelf
Plaintext is useful for creating files that you’ll read in a text editor such as
Notepad or TextEdit, but if you want to save data from your Python
programs, use the shelve module
65
Write a function to calculate factorial of a number. Develop a program to
compute binomial coefficient (Given N and R).
66
Read N numbers from the console and create a list. Develop a program to
print mean, variance and standard deviation with suitable messages.
67
68
Read a multi-digit number (as chars) from the console. Develop a
program to print the frequency of each digit with suitable message.
69
70
Develop a program to print 10 most frequently appearing words in a text
file.
71
72
73
Prof. Roopashree S, Dept. of CSE
6. Develop a program to sort the contents of a text file and write the sorted
contents into a separate text file.
74
7. Develop a program to backing Up a given Folder (Folder in a current working directory) into a
ZIP File by using relevant modules and suitable methods.
75
76
77
