Adventures in testing go randomness
In a recent feature I have worked on the need for using randomness has risen. This feature had to most basic requirement of rolling a number which helps decides a simple yes/no route.
After completing the required changes the need to unit test the new and affected code has risen, On first look, this seemed like a quick hack, as rand.Rand accepts a source for generating randomness. But alas, after running writing and running the unit tests, something seemed wrong.
Since I can’t share the original code, and as it will probably bloat this post as well, I’ll share some code examples.
Here is an example of a function which might resemble what I wanted to test-
func DoRandomCalculation(rand *rand.Rand) int64 {
return int64(rand.Intn(5) * 5)
}
And here is an example of a unit test attempting to test such a function -
func TestDoRandomCalculation(t *testing.T) {
type args struct {
rand *rand.Rand
}
tests := []struct {
name string
args args
want int64
}{
{"When zero is rolled, zero is returned", args{rand.New(&PredictableRandomSource{mockedRandomValue: 0})}, 0},
{"When one is rolled, five is returned", args{rand.New(&PredictableRandomSource{mockedRandomValue: 1})}, 5},
{"When two is rolled, ten is returned", args{rand.New(&PredictableRandomSource{mockedRandomValue: 2})}, 10},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := DoRandomCalculation(tt.args.rand); got != tt.want {
t.Errorf("DoRandomCalculation() = %v, want %v", got, tt.want)
}
})
}
}
Where PredictableRandomSource has the following implementation -
type PredictableRandomSource struct {
mockedRandomValue int64
}
func (p *PredictableRandomSource) Int63() int64 {
return p.mockedRandomValue
}
func (p *PredictableRandomSource) Seed(seed int64) {
//do nothing
}
Running the unit test as-is will produce the following output -
=== RUN TestDoRandomCalculation/When_zero_is_rolled,_zero_is_returned
=== RUN TestDoRandomCalculation/When_one_is_rolled,_five_is_returned
random_test.go:28: DoRandomCalculation() = 0, want 5
=== RUN TestDoRandomCalculation/When_two_is_rolled,_ten_is_returned
random_test.go:28: DoRandomCalculation() = 0, want 10
--- FAIL: TestDoRandomCalculation (0.00s)
--- PASS: TestDoRandomCalculation/When_zero_is_rolled,_zero_is_returned (0.00s)
--- FAIL: TestDoRandomCalculation/When_one_is_rolled,_five_is_returned (0.00s)
--- FAIL: TestDoRandomCalculation/When_two_is_rolled,_ten_is_returned (0.00s)
So what is going on here ?
Poking inside the actual random implementation inside go, I observed the following culprit -
func (r *Rand) Int31() int32 { return int32(r.Int63() >> 32) }
it seems that Intn here is doing a “binary shift” to make our “random” int64 fit an int, so if we were to mimic our second unit test, this will translate to “1 » 32”, here is short and fast way to test this theory -
func main() {
fmt.Printf("the output of the binary shift is %v", 1>>32)
}
this, not surprisingly, outputs the output of the binary shift is 0.
So now that we know why our unit tests are failing, we can probably get away by using an 64 bit variant of Intn - Int63n.
Replacing the above DoRandomCalculation implementation with it proves it by making all our tests pass as expected.
But this is not the end of the adventure, as the post started - our basic need is a yes/no, random answer.
Poking around inside the implementation of Int(63)n has made me think whether it’s a good idea to actually use Intn, as it might loop looking for our target random number.
Maybe it will be faster to check for a range of numbers, i.e. is the random number larger than half of math.MaxInt ?
bench time.
Let’s start with the closed group (int(63)n) version -
func BenchmarkInt64ClosedGroup(b *testing.B) {
r := rand.New(rand.NewSource(85166485))
for i := 0; i < b.N; i++ {
_ = r.Intn(2)
}
}
Has produced on my machine -
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkInt64ClosedGroup
BenchmarkInt64ClosedGroup-16 225519601 5.637 ns/op
PASS
And for the proposed, simpler and faster version -
func BenchmarkInt64WithSimpleRange(b *testing.B) {
r := rand.New(rand.NewSource(85166485))
for i := 0; i < b.N; i++ {
_ = r.Int63() > (math.MaxInt64 / 2)
}
}
got us -
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkInt64WithSimpleRange
BenchmarkInt64WithSimpleRange-16 399359359 2.864 ns/op
PASS
To summarize, just as suspected, for our use case, avoiding the closed group (Int(63)n) variant for a simple yes/no question, it will probably be faster to simply ask for a larger random number.