Randomness Testing with Bayesian Stats
If Bayesian stats is so great, it should be able to do everything that ordinary stats can do. Indeed, as a fan of Bayesian stats, I have not made much of an attempt to learn normal stats with its p-values and t-tests outside what was required for my highschool statistics class many years ago. But was this really a wise decision on my part? [1] Randomness Testing Consider the following problem: We have a random bit generator. It should generate independent random bits, each bit at 1:1 odds of being 0 vs 1. We would like to test if it is truly a good RNG, or is broken in some way. This is actually a really deep problem, and people invent barrages of statistical tests to divine subtle correlations between bits. But we will start out by setting a very humble goal for ourselves: Determine if the generator is making too many 0's or too many 1's. This is really easy for normal stats, so it had better be really easy for Bayesian methods too. First of all, what is the thing that is going to replace the p-value here? In Bayesian statistics we will use a likelihood ratio for this. Just like we set a cutoff for the p-value in classical stats, we'll likewise set a cutoff for likelihood ratio. If it becomes too large, it tells us that the random number generator has failed the test. In testing a random number generator, we'll observe some large number of bits it generates, all strung together into a bitstring mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container