[pycrypto] Test code - Random

[Dwayne C. Litzenberger]

On Fri, Nov 07, 2008 at 04:06:26PM -0500, Dwayne C. Litzenberger wrote:
>  class SimpleTest(unittest.TestCase):
>      def runTest(self):
>          """Crypto.Random.new()"""
>          # Import the OSRNG module and try to use it
>          from Crypto import Random
>          randobj = Random.new()
>          x = randobj.read(16)
>          y = randobj.read(16)
>          self.assertNotEqual(x, y)
> +class TestNotAlwaysEqual(unittest.TestCase):
> +    def runTest(self):
> +        from Crypto import Random
> +        randobj = Random.new()
> +        k = 10
> +        DataArray = []
> +        x = randobj.read(16)
> +        count = 0
> +        for i in range(0,k):
> +            y = randobj.read(16)
> +            if x == y:
> +                count += 1
> +        self.assertNotEqual(count,k)

I'm not sure what your proposed TestNotAlwaysEqual test offers that 
SimpleTest does.  Presumably the purpose of this new test is to avoid the 
case where a correctly-functioning RNG returns two identical 128-bit 
numbers in a row.  While that may be possible, I'm really not concerned 
about it, since according to my calculations*, the probability that 
SimpleTest fails is 2**-128.  TestNotAlwaysEqual adds another test that can  
fail in the same way, but with a probability of 2**-1280.  I don't see why 
that would be necessary.

(* Someone please correct me if I'm wrong, since I'm still not very good at 
reasoning with probability.)

> +class TestAverage(unittest.TestCase):
> +    def runTest(self):
> +        from Crypto import Random
> +        randobj = Random.new()
> +        x = randobj.read(2**15)
> +        values = []
> +        for i in x:
> +            values.append(ord(i))
> +        total = 0
> +        for i in values:
> +            total += i
> +        average = total/len(values)
> +        print average
> +        self.assertEqual(average in range((256/2)-28,(256/2)+28),True)
> +

This looks like the beginning of a statistical test suite, but it seems out 
of place here on its own, and I fear that it might cause more confusion 
than anything else.  Why did you choose the constants 2**15 and +/- 28, for 
example?  What's the likelihood that this fails under normal circumstances.  
I don't want to end up with bug reports and confused users (or worse: users 
who learn to ignore test failures) as a result of too many false positives, 
for example.  Also, I want to avoid adding unnecessary tests to SelfTest, 
since SelfTest should be fairly quick to run.

It might be better to implement a full set of statistical tests like the 
ones specified in FIPS 140-2.  I would consider such a thing for a future 
release.  If someone were to tackle this, I would suggest adding it as a 
module under Crypto.Random or Crypto.Util, and then calling it from 
SelfTest.  i.e. It should provide a usable, well-documented API, not just 
be buried in the SelfTest module.

We definitely could use a more comprehensive set of self-tests, especially 
for the random number generators.

Cheers,
  - Dwayne

-- 
Dwayne C. Litzenberger <dlitz at dlitz.net>
  Key-signing key   - 19E1 1FE8 B3CF F273 ED17  4A24 928C EC13 39C2 5CF7
  Annual key (2008) - 4B2A FD82 FC7D 9E38 38D9  179F 1C11 B877 E780 4B45
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://lists.dlitz.net/pipermail/pycrypto/attachments/20081107/cc393bc9/attachment.pgp