debugging-and-profiling.md

Debugging and Profiling

Using pdb and ipdb for Interactive Debugging

pdb (Python Debugger):

• pdb is the built-in debugger in Python. It allows you to set breakpoints, step through code, inspect variables, and evaluate expressions.

Example usage of pdb:

def buggy_function(a, b):
    import pdb; pdb.set_trace()  # Set a breakpoint
    result = a / b
    return result

buggy_function(5, 0)

In the code above, when pdb.set_trace() is executed, the debugger will pause execution and provide an interactive prompt where you can inspect variables and step through the code.

Common pdb commands:

• n (next): Continue to the next line within the same function.
• s (step): Step into the function called at the current line.
• c (continue): Continue execution until the next breakpoint.
• l (list): List source code around the current line.
• p (print): Print the value of a variable, e.g., p result.
• q (quit): Exit the debugger.

ipdb (IPython Debugger):

• ipdb provides the same functionality as pdb but with IPython enhancements, such as syntax highlighting and tab completion.

Example usage of ipdb:

def buggy_function(a, b):
    import ipdb; ipdb.set_trace()  # Set a breakpoint
    result = a / b
    return result

buggy_function(5, 0)

Install ipdb using pip:

pip install ipdb

Profiling Python Code with cProfile and line_profiler

cProfile:

• cProfile is a built-in Python module for profiling the performance of your code, providing statistics on function call times and frequencies.

Example usage of cProfile:

import cProfile

def example_function():
    total = 0
    for i in range(10000):
        total += i
    return total

cProfile.run('example_function()')

To save profiling results to a file:

import cProfile

def example_function():
    total = 0
    for i in range(10000):
        total += i
    return total

cProfile.run('example_function()', 'profile_results')

To view saved profiling results:

import pstats

p = pstats.Stats('profile_results')
p.sort_stats('cumulative').print_stats(10)

line_profiler:

• line_profiler is a third-party module that profiles the execution time of individual lines within functions, providing a more granular view of performance.

Install line_profiler using pip:

pip install line_profiler

Example usage of line_profiler:

from line_profiler import LineProfiler

def example_function():
    total = 0
    for i in range(10000):
        total += i
    return total

profiler = LineProfiler()
profiler.add_function(example_function)
profiler.run('example_function()')
profiler.print_stats()

To use the @profile decorator with line_profiler:

1. Create a script, example.py:

   @profile
   def example_function():
       total = 0
       for i in range(10000):
           total += i
       return total
   
   if __name__ == '__main__':
       example_function()

2. Run the script with line_profiler:

   kernprof -l -v example.py

Analyzing Memory Usage and Performance Bottlenecks

Memory Profiling:

• Tools like memory_profiler help track memory usage in Python applications, identifying memory leaks and optimization opportunities.

Install memory_profiler using pip:

pip install memory_profiler

Example usage of memory_profiler:

from memory_profiler import profile

@profile
def example_function():
    a = [i for i in range(10000)]
    b = [i * 2 for i in range(10000)]
    return a, b

example_function()

Run the script:

python -m memory_profiler example.py

Performance Bottlenecks:

• Identify performance bottlenecks using a combination of cProfile and line_profiler to understand which functions and lines of code are consuming the most time.
• Optimize critical sections by improving algorithms, using more efficient data structures, or parallelizing tasks.

Benefits of Debugging and Profiling

• Debugging: Provides insights into code behavior, helping identify and fix bugs efficiently.
• Profiling: Helps understand code performance, guiding optimization efforts for faster execution and reduced resource usage.
• Memory Analysis: Identifies memory usage patterns and leaks, ensuring efficient memory management.

Go Back