Data Debugging Performance
See in schedule: Fri, Jul 30, 19:00-19:30 CEST (30 min) Download/View SlidesIf your program is using too much memory, it can crash, thrash, or just run more slowly. To reduce memory usage, you need to measure it somehow.
There are a number of tools to measure Python memory usage, however. Which one should you pick? This depends heavily on your use case.
In this talk you will learn about the different ways memory problems manifest in Python, the memory usage patterns of different application types, and which tools to use in each situation.
For web applications, the typical problem is memory leaks: memory that is never freed adds up if your program runs forever. That means you need tools to help you identify the source of memory leaks, whether in Python or in C extensions.
For data processing applications like data science or scientific computing, the problem is typically caused by processing large amounts of data. Here you need to find the peak memory usage, and what code what responsible for it.
Type: Talk (30 mins); Python level: Intermediate; Domain level: Intermediate
Itamar has been using Python since 1999, and was inspired to move to a new country by his trip to EuroPython in 2002. He writes extensively on Docker packaging for Python (https://pythonspeed.com/docker/), Python performance (https://pythonspeed.com/performance/), and memory usage(https://pythonspeed.com/memory/). In the past he spent many years as a maintainer of the Twisted framework, and nowadays is the maintainer of the Fil memory profiler (https://github.com/pythonspeed/filprofiler) and the Eliot logging library (https://eliot.readthedocs.io).
