[pycrypto] PyCrypto status update & release plans
dlitz at dlitz.net
Mon Sep 9 00:03:06 PDT 2013
Hi folks! I've begun work on clearing my backlog, and it's been a while
since I've sent a status update, so here goes:
== Where we are right now ==
I've received pull requests for a bunch of new features, including AEAD
modes (CCM, EAX, GCM, and SIV), HKDF (HMAC KDF), SHA-3 (Keccak),
Diffie-Hellman, a better DSA API, scrypt, Salsa20, encrypted PKCS#8
private keys, AES-NI support, ElGamal blinding, and ARC4-drop[n]. There
have also been many bugfixes and documentation improvements.
However, I don't think PyCrypto's core is in good enough shape for me to
feel comfortable adding a lot of new features. I think I made a few
mistakes in some of our newer APIs, which I want to fix before they get
- .verify methods that return False instead of raising an exception
- PBKDF1 and PBKDF2 default to using SHA1.
We also have a number of overall design issues that affect performance
- Our PBKDF2 implementation is about 8.5x slower than OpenSSL. That
means users are going to use 8.5x lower iteration counts than they
should, thus giving attackers who use a faster PBKDF2 implementation a
sizeable advantage. I've tested a C-based implementation of PBKDF2,
but that currently doesn't help much, because our coding style has
been to write Python wrappers around all of our C code.
- The `b()` function in Python 3 can also be expensive, since it
allocates memory for a new bytestring on every invocation.
My solutions to both of these problems require Python 2.2 or later.
== Roadmap ==
My rough near-term plan looks something like this:
* "2.6.x" branch - Bugfixes & general quality improvements
- Last version to support Python 2.1
- DeprecationWarnings on some things that suck (e.g. HMAC without
- (Maybe) Do something about .verify not raising an exception.
- (Maybe) AES-NI and ElGamal blinding
* 2.7 - Remove Python 2.1 compatibility stuff.
- Python 2.2 or later required.
- New-style classes in C, with the Python code inheriting from them.
(This can be much faster than proxying.)
- Replace b() function calls with b"" literals in Python.
- Use better idioms for reference counting in C.
- Do something about .verify not raising an exception by default.
- Fix the remaining bugs that couldn't be fixed in the 2.6.x
- (Maybe) Fix the docs. We need a user FAQ and some contributor
- (Maybe) Run cpychecker and/or Coverity Scan.
* 2.8 and beyond - Major new functionality, such as:
- AEAD modes
- Crypto.Random.random performance improvements
- Diffie-Hellman support
- Encrypted PKCS#8 keys
- Performance improvements for Crypto.Random.random
- scrypt, Salsa20, Camellia, new DSA API
- SHA-3 (assuming the standard is released by this time)
- Some PyPy-related improvements
Hopefully I can get to v2.8 fairly quickly. Depending on how things go,
I might release some things sooner (e.g. AEAD support), because I've
been promising my colleagues that I'd finish it for a while now, and the
major API design work is almost done.
Anyway, that's the current thinking. This should help me get the
project to a point where I'm not so nervous about merging people's
== Dropping support for Python 2.1 ==
In case you're only reading the headings: I'm planning to drop support
for Python 2.1 in PyCrypto 2.7 and later; The 2.6.x series will be the
last versions to support Python 2.1.
== Bug tracker migration: Launchpad -> GitHub ===
At some point, I'm thinking of moving the bug tracker from Launchpad to
GitHub. It's weird that some people might have to create accounts on
two separate services in order to report a bug and submit a fix for the
bug. On the other hand, GitHub's bug tracker is a lot more immature
than Launchpad's, but I think it will be good enough.
== I've moved to the USA ==
I moved to San Francisco last year (July 7, 2012) to work at Dropbox:
- I don't think this affects anyone as far as export control regulations
are concerned (I had already been accepting US-origin patches for a
while before I moved), but just in case---now you know.
- I'll occasionally be getting contributions from my co-workers or other
people that I meet in the Bay Area, so there will inevitably be more
conversations that happen off-list. I will be encouraging people to
move these conversations online (or at least to summarize the
important points online) so that online contributors are not left in
== Conclusion ==
That's it for now.
Dwayne C. Litzenberger <dlitz at dlitz.net>
OpenPGP: 19E1 1FE8 B3CF F273 ED17 4A24 928C EC13 39C2 5CF7
 I compared three PBKDF2-HMAC-SHA1 implementations on my laptop. For
a 48-byte output with 1,000,000 iterations, I got:
OpenSSL: 3.9 seconds
PyCrypto v2.6: 44.0 seconds (11x OpenSSL)
PyCrypto master: 33.2 seconds (8.5x OpenSSL)
PyCrypto master PBKDF2.c: 32.1 seconds (8.5x OpenSSL)
 See some of the discussion at
More information about the pycrypto