Is Python too slow?¶
At some point, you end up with one little piece of your system, as a whole, where you end up spending all your time. If you write that just as a sort of simple-minded Python loop, at some point you will see that that is the bottleneck in your system. It is usually much more effective to take that one piece and replace that one function or module with a little bit of code you wrote in C or C++ rather than rewriting your entire system in a faster language, because for most of what you're doing, the speed of the language is irrelevant.
- Guido van Rossum1
It depends...¶
Ways to Speed Up Python Programs¶
(in ascending order of effort)¶
- Make your code more efficient with better data structures, algorithms, or design
- Divide up the work between n copies of the program
- Use a shared object (.so, .dll) with python bindings
- Replace your python interpreter (PyPy)
Creating shared objects¶
- Use case: a single hot spot module or function that would benefit from being close to metal
- good candidates: tight loops, mathematical operations
- write in C/C++, use something like swig or Boost Python to create python interface
Cython - Overview¶
- history in scientific computing (Sage)
- forked from Pyrex
- Interface to the Python C API
- Python + annotations -> Cython compiler -> C code -> yourmodule.so ->
import yourmodule
- big gains with just a little effort
Cython - Normal Python Tools¶
pip install cython
- use setup.py to compile resource to an importable python module inside the package
- includes profiling (cProfile) to encourage iterative approach
- Caveat emptor: code generation is black magic