[pycrypto] ERROR: testRsaUnversionedSignAndVerify failed

[Dwayne C. Litzenberger]

On Thu, Aug 20, 2009 at 01:42:12PM -0700, Steve Weis wrote:
>Are there advantages to using Pycrypto's Randpool over
>Random.SystemRandom()?
>In another Keyczar thread, someone reported that Randpool was a performance
>bottleneck and got a big improvement by switching to SystemRandom.
>
>I don't know enough about the underlying implementations to make any
>security judgement. If anyone can comment authoritatively, please do.

Quoting myself in https://bugs.launchpad.net/pycrypto/+bug/249765:

     RandomPool really really needs to die, at least in its current form.
     It's not thread-safe, it's not fork-safe, and if it can't get entropy
     from the OS, it silently produces predictable output. What's worse is
     that everyone assumes that it's a portable substitute for reading from
     /dev/urandom on Linux, but it's actually way too fragile to be safely
     used that way.

It's also much slower than just reading from /dev/urandom on Linux.  I 
don't remember how it compares on Windows.

You could use random.SystemRandom(), which is basically just a wrapper 
around reading from /dev/urandom of Windows's CryptGenRandom.  That's 
better than using RandomPool, but there are still caveats:

- Some systems are adversely affected when you read a lot of data from the 
  operating system's PRNG.  For example, old versions of Linux (I don't 
  remember how old, but it wasn't too long ago) starved /dev/random if you 
  had a process continually reading from /dev/urandom.

- Windows XP's (and probably earlier's) PRNG is vulnerable to a 
  state-extension attack, because it inappropriately uses RC4:
    http://en.wikipedia.org/w/index.php?title=CryptGenRandom&oldid=292725857#Security

- I'm told that some old *nix's /dev/urandom devices are far less 
  trustworthy than Linux's.

- Even the security of Linux's /dev/urandom is debated (search lkml.org for 
  Jean-Luc Cooke & Fortuna).

Why trust them when you can trust me? (Hah!)

Regarding this patch:

>> --- util.py.orig        2009-08-20 10:40:19.248702303 +0200
>> +++ util.py     2009-08-20 10:57:27.765198430 +0200
>> @@ -30,7 +30,12 @@ except ImportError:
>>   from sha import sha as sha1
>>   from Crypto.Hash.SHA256 import new as sha256
>>
>> -from Crypto.Util import randpool
>> +try:
>> +  # Import RandomPoolCompat, if available
>> +  from Crypto.Random import RandomPoolCompat as RandomPool
>> +except ImportError:
>> +  from Crypto.Util.randpool import RandomPool
>> +
>>  from pyasn1.codec.der import decoder
>>  from pyasn1.codec.der import encoder
>>  from pyasn1.type import univ
>> @@ -291,7 +296,7 @@ def TrimBytes(bytes):
>>
>>  def RandBytes(n):
>>   """Return n random bytes."""
>> -  return randpool.RandomPool(512).get_bytes(n)
>> +  return RandomPool(512).get_bytes(n)
>>
>>  def Hash(digest, *inputs):
>>   """Return a SHA-1 hash over a variable number of inputs."""

That will work, but it will perform better if you invoke Random.new() once, 
then .read() from the resulting object every time you want random data.  

I don't recommend using RandomPoolCompat, and after seeing it used here I 
think I will drop it before the next release.  RandomPool is *broken*, and 
on Windows it is a catastrophic security hole.  You should not use it even 
to maintain backwards compatibility.

Instead, use os.urandom() if Crypto.Random is not available (but you should 
use Crypto.Random if it's available: Crypto.Random performs better than 
os.urandom on Linux, and on Windows it provides a workaround for the 
weaknesses in Microsoft's RC4-based CryptGenRandom implementation).

You can use this code.  I hereby release it into the public domain:

try:
    from Crypto import Random
    _rng_instance = Random.new()
except ImportError:
    class RandomStub:
        def read(n):
            return os.urandom(n)
    _rng_instance = RandomStub()

def RandBytes(n):
    """Return n random bytes."""
    return _rng_instance.read(n)

Does that help clarify things?

- Dwayne

-- 
Dwayne C. Litzenberger <dlitz at dlitz.net>
  Key-signing key   - 19E1 1FE8 B3CF F273 ED17  4A24 928C EC13 39C2 5CF7