Python Profiling Code Tutorial – Complete Guide

If you’ve been working with Python for a little while, there’s a good chance you’ve encountered a situation where your code runs slower than you’d like. Or perhaps, your script eats more memory than you anticipated. This is where Python profiling comes into play and becomes an invaluable tool in your toolkit.

What is Python Profiling?

Python profiling is a process that helps us understand the behavior of our Python code. With it, we can find bottlenecks in the code, spots where the code slows down or uses excessive amounts of memory.

Profiling allows us to optimize our code and make it run more efficiently. It provides us with insights on where to focus our optimization efforts. This leads to an overall performance boost in our code and ensures that we deliver a better user experience.

Every Python programmer, regardless of their experience level, should learn about profiling. Knowing how to profile your code is an essential skill, especially when you work with larger, more complex projects.

Profiling can be used not only in boosting the code’s performance but also in managing resources better, debugging, and even in testing. It’s important in the development of efficient, scalable software.

Now that we’ve outlined the importance and uses of Python profiling, let’s dive into some practical examples in the next parts of this article.

CTA Small Image
FREE COURSES AT ZENVA
LEARN GAME DEVELOPMENT, PYTHON AND MORE
ACCESS FOR FREE
AVAILABLE FOR A LIMITED TIME ONLY

Getting Started With Python Profiling: A Basic Example

Python standard library has a built-in module for profiling, known as the cProfile module. Let’s first start with a simple example:

import cProfile

def sum_of_numbers():
    return sum([i for i in range(1000000)])

cProfile.run('sum_of_numbers()')

When this code is run, the output will display a detailed performance of every invoked function, including the number of calls and the time spent on each call.

Python Profiling Using Time Module

The time module is another simple way to measure time taken for a process. Here’s how you can use it:

import time

def sum_of_numbers():
    return sum([i for i in range(1000000)])

start_time = time.time()

sum_of_numbers()

end_time = time.time()

print("Time taken: ", end_time - start_time)

This code will print the time taken to calculate the sum of numbers from 1 to 1000000.

Python Profiling Using memory-profiler

For memory profiling, we can use memory-profiler, a third-party library. Let’s see how to use it:

from memory_profiler import profile

@profile
def sum_of_numbers():
    return sum([i for i in range(1000000)])

sum_of_numbers()

This script will print the memory usage in each line of our function. To execute this correctly, make sure to install the library by running pip install memory-profiler

Python Profiling Using line_profiler

To get a line-by-line report of time consumed, we use another third-party library called line_profiler. Here’s a basic example:

from line_profiler import LineProfiler

def do_profile(func):
    profiler = LineProfiler()
    profiler.add_function(func)
    profiler.runcall(func)
    profiler.print_stats()

do_profile(sum_of_numbers)

Here, do_profile function will print details for each line of function sum_of_numbers. Install the library by running pip install line_profiler.

Python Profiling Using Py-Spy

Py-Spy is a sampling profiler for python programs. It lets us visualize what the python program is doing without changing the code or slowing down the program. Let’s see an example:

# Run pip install py-spy
# Then on the command-line, run
# py-spy top -- python your_file_name.py

The above command will run and generate a live view of functions running at the top of CPU usage.

Python Profiling Using Py-Instrument

Py-Instrument is another tool providing statistical python profiling. Here is how to use it:

from pyinstrument import Profiler

profiler = Profiler()
profiler.start()

# code you want to profile
sum_of_numbers()

profiler.stop()

print(profiler.output_text(unicode=True, color=False))

This profiler code will profile the execution of the sum_of_numbers function and give us a detailed output with timing metrics.

Python Profiling Using profiling package

The profiling module is another powerful third-party package that provides visualization in a web browser. Here’s an example of how to use:

# Install the package
# pip install profiling

# Run the command-line profiling
# profiling your-script.py

This command will generate a web server running at localhost:8000 displaying a live profiling of your script.

Python Profiling With SnakeViz

SnakeViz is a browser-based graphical viewer for the output of Python’s built-in cProfile module and other profiling format. Here is an example:

# Install using pip
# pip install snakeviz

# Run your script with cProfile, and save the result in a file
# python -m cProfile -o my_script.prof my_script.py

# Now run snakeviz with the saved file
# snakeviz my_script.prof

SnakeViz will create a visualization for the profile in your default web browser.

Python Profiling With Yappi

Yappi is yet another Python profiler, but this one is multi-thread aware. It’s a low-overhead, robust, and easy-to-use Python profiler. Here’s an example:

import yappi

yappi.start()

# your code here
sum_of_numbers()

yappi.get_func_stats().print_all()
yappi.get_thread_stats().print_all()

This code snippet will start the Yappi profiler, run your function, and print a summary of both the function statistics and the thread statistics.

Where To Go Next?

As we have just scratched the surface of Python profiling, it is essential to continue learning and practicing to master the art. Profiling Python code is a skill that comes with experience and regular practice.

Python is a powerful language and to build a strong foundation in it, we at Zenva have put together a comprehensive collection of courses – the Python Mini-Degree. The Python Mini-Degree is a comprehensive and step-by-step program, perfect for both beginners and seasoned programmers. It covers a vast variety of topics such as coding basics, algorithms, object-oriented programming, game development, and app development. What’s more, you’ll be creating your own games, algorithms, and apps.

The courses include quick challenges and in-course quizzes to cement your learning. Moreover, the course material is regularly updated to keep up with industry practices, ensuring you always stay ahead of the curve. Completing these courses will help you build a rich portfolio of Python projects and prepare for many career opportunities.

If you wish to explore our broader collection of Python courses, you can find them here. Our range of Python courses covers every aspect of Python programming, from beginner to advanced levels.

Conclusion

Python profiling is an essential aspect of writing optimized and efficient code. Given its importance and wide-ranging utilities, it’s a skill that every Python programmer must possess. With the tools and techniques mentioned in this tutorial, you’re well on your way to master Python profiling.

At Zenva, we aim to provide you with the best resources to enhance your coding skills further. We encourage you to take a look at our comprehensive Python Mini-Degree. Let it serve as your guide in this exciting journey into mastering Python, leading you to become an adept and efficient Python programmer.

Did you come across any errors in this tutorial? Please let us know by completing this form and we’ll look into it!

FREE COURSES
Python Blog Image

FINAL DAYS: Unlock coding courses in Unity, Godot, Unreal, Python and more.