Unraveling the Bayesian knot: the magic of probability

There is a wonderful maths problem called "the two children problem" invented by Martin Gardner and it runs like this:

you meet a friend in the street, you know he has two children but not their sexes. As you are talking a boy runs up who is introduced as your friend's son: what is the probability that the other child is also boy?

Intuition would say 50%, in this case intuition is wrong; I will now try to explain why before giving the answer.

First I want to explain some fundamentals of statistics. First and foremost is the concept of probability, this is intuitively easy to describe: "the likelihood of an event occurring" the actual calculation is also deceptively simple:
\[ \frac{\text{desired outcome}}{\text{all outcomes}} \]
where  desired outcome is all the ways of achieving the outcome you're interested in (for example a boy in the above problem) and all outcomes are all the possible outcomes (eg boy or girl) so for the sex of a child the result, as you would expect, is:
\[\frac{\text{boy}}{\text{boy}+\text{girl}} = \frac{1}{2} = 50\%\]

The above of calculation is very simple but problems arise when you start looking at adding probabilities. For example the probability of having 2 sons is $\frac{1}{4}$ there are 4 possible combinations of 2 children (2 sons, 2 girls, 1 (eldest) son 1 daughter, 1 (eldest) daughter 1 son) while  the probability of having a son or a daughter remains $\frac{1}{2}$ the probability of a particular number of sons or daughters changes. While this may not make immediate sense it is easier to understand by going to an extreme: instead of 2 sons what are the odds of 200 sons? (answer below, it's very unlikely; even without considering the physical problems). A series of trials like this is where the 'Gambler's fallacy' comes from, each trial is independent, it has no effect on the outcome of any of the other trials, but any particular combination has a different chance: having 199 sons then a daughter is as likely as 199 sons then another son. The chance of having just 199 sons and daughter is more likely as the order of birth doesn't matter. There is only one way to have 199 sons then a daughter but there are 200 ways to have 199 sons and a daughter.

This is the one of the big pit-falls of probability (hence the fallacy) at each trial (in this case birth) the probability is the same, any one of those 200 children has a 50-50 chance of being a son or a daughter but any particular combination of children will have its own probability. At this point the essence of the 2 children problem should start to become apparent.

So what the answer to the 'two children problem'? It's not actually $\frac{1}{2}$ or  $\frac{1}{4}$ , it's $\frac{1}{3}$. To understand this think back to the definition of probability, we know that for 2 trials with each trial having one of two outcomes we have 4 possible outcomes:

2 sons, 2 daughters, 1 son 1 daughter and 1 daughter 1 son

Now we know that one of those options, 2 daughters, is impossible because we already have already met one son so that leaves us 3 options: 2 sons, a daughter or a different daughter (say, an older or younger daughter). The two options for a daughter may seem needless but it is vital: we know that there were 4 states and only one of those states is removed (2 daughters). We know at least one child is a boy but nothing more, if we knew the eldest child was a son the probability of the other child being a son would be $\frac{1}{2}$  but unless we know that this is a similar case to having 199 sons and a daughter: more likely because there are more possible 'desired' outcomes.

This sort of calculations relies on a branch of probability known as Bayesian statistics which concerns itself with the combination of different probabilities. This example is among the simplest and already it clashes against classical intuition, the most impressive part is that the probabilities are exactly what we see when we go and test these systems (you can do it yourself with two coins instead of children).

Of course one other reason the two children problem causes such confusion is the utterly artificial way in which it is presented but real world examples do exist, for example: given the probability of a disease test giving the correct result and the probability of having that disease how likely are you to have the disease if you test positive? Just so you know the answer is not $\frac{1}{4}, it is different as each outcome is not equally likely (for example you might have a 0.2% chance of having the disease and a 1% chance of a the wrong result from the test). For those that want an even more mind-bending problem I direct you to the Monte-hall problem, or the inspiration for this post: the Tuesday two children problem (if you know the son you meet was born on a Tuesday how likely is it your friend has another son).



200 heads from 200 flips? its:
\[ \left( \frac{1}{2}\right)^{200} = \frac{1}{1606938044258990275541962092341162602522202993782792835301376}\]