How random is enough to be random? [closed]

Question

I download the atmospheric noise data from random.org in form:

10111011 01101111 01100001 00001001 00100110 10111011 01110110 11010110 00000111 01111110 10001110 01101010 11100000 11110111 10101011 01001011 11010111 00100110 01001011 11101111 01100111 01011000 11000000 00011010 00111101 10000100 11011111 10001100 11100010 10010011 11001101 00000011

I remove the \n and spaces and test the resulting 100k+ bits with Python NIST test-suite module (https://pypi.org/project/nistrng/):

Test results:

• PASSED - score: 0.543 - Monobit - elapsed time: 1 ms
• PASSED - score: 0.546 - Frequency Within Block - elapsed time: 1 ms
• PASSED - score: 0.986 - Runs - elapsed time: 26 ms
• PASSED - score: 0.939 - Longest Run Ones In A Block - elapsed time: 1 ms
• PASSED - score: 0.522 - Binary Matrix Rank - elapsed time: 247 ms
• FAILED - score: 0.0 - Discrete Fourier Transform - elapsed time: 6 ms
• PASSED - score: 0.341 - Non Overlapping Template Matching - elapsed time: 265 ms
• FAILED - score: nan - Serial - elapsed time: 3696 ms
• FAILED - score: 0.0 - Approximate Entropy - elapsed time: 2443 ms
• FAILED - score: 0.0 - Cumulative Sums - elapsed time: 81 ms
• FAILED - score: 0.63 - Random Excursion - elapsed time: 290 ms
• FAILED - score: 0.0 - Random Excursion Variant - elapsed time: 1 ms

I generated an array of random bits via python.numpy (exactly same length) and the results came out as:

Test results:

• PASSED - score: 0.426 - Monobit - elapsed time: 0 ms
• PASSED - score: 0.156 - Frequency Within Block - elapsed time: 1 ms
• PASSED - score: 0.268 - Runs - elapsed time: 27 ms
• PASSED - score: 0.405 - Longest Run Ones In A Block - elapsed time: 2 ms
• PASSED - score: 0.017 - Binary Matrix Rank - elapsed time: 238 ms
• FAILED - score: 0.0 - Discrete Fourier Transform - elapsed time: 6 ms
• PASSED - score: 0.279 - Non Overlapping Template Matching - elapsed time: 261 ms
• FAILED - score: nan - Serial - elapsed time: 3712 ms
• FAILED - score: 0.0 - Approximate Entropy - elapsed time: 2498 ms
• FAILED - score: 0.0 - Cumulative Sums - elapsed time: 83 ms
• FAILED - score: 0.683 - Random Excursion - elapsed time: 289 ms
• PASSED - score: 0.17 - Random Excursion Variant - elapsed time: 2 ms

I create my own method using 2d pictures of random processes and I get similar results.

Purely according to the text output of the tests I could assume something like:

• atmospheric noise from random.org is not fully random
• numpy pseudorandom generator is not fully random
• numpy pseudorandom generator is more random than atmospheric noise (in a matter of passed tests)

However, these outcomes does not sound correct to me - should not atmospheric noise be the measure of full randomness? How pseudorandom generator can be better than random numbers?

My question:

How to understand the results? What tests are the important?

Answer 1

The random.org site themselves discusses this at length here: https://www.random.org/analysis/ - and even cites two different research papers that have done the same thing as you and analyze the results.

I think the root of your confusion is that you apparently think "randomness" is like a number you can measure and compare, and that there is such a thing as "full randomness". And that's just not true. It's very much more complicated.

But specifically, a pseudorandom number generator has the property that it only pretends to be random while actually being completely deterministic. If you know its internal state, you can predict all future output with 100% accuracy. But if you don't, the pretense can be really good and fulfill all kinds of statistical tests.

Atmospheric noise, on the other hand, is not predictable at all, at least given the current state of knowledge about how phsyics work. But that doesn't mean that it's automatically also very good at passing statistical tests.

As a very simplified example: consider a biased coin flip where God told you it's absolutely impossible to predict whether it will be heads or tails, but there is a 60% chance of it being heads. How "good" is this as randomness? You have the word of God that it cannot be predicted - in a way, that's a lot better than any pseudorandom number generator which can be predicted. But that 10% bias is huge and would be ridiculously easy to exploit if you did straightforward bets on the result.

The atmospheric noise is a bit like that biased coin, just with much more subtle statistical flaws.

Answer 2

In fact, a "statistically useful" PRNG has the characteristic that it will produce values that are fairly evenly spread across its range. The "next" value is taken to be "unpredictable" (even though it isn't), but if you generated a thousand numbers and plotted them on a graph, the graph would be fairly evenly covered with dots. In this way, a "random" sample can be expected to examine "a good cross-section of" the entire population, which is what statisticians want.

But, cryptographers have an entirely different need for pseudo-randomness, and they therefore use different algorithms to get what they need.

Neither one is true randomness, which actually might not be nearly as useful.