4 Likes
Ok, thinking on it more, NOW it’s clicked. It’s weird how these things work cognitively - like the Monty Hall problem, the Raven problem initially resists intuitive analysis. But then, once the switch flips, you wonder how you could ever have thought about it any other way, because the resolution is obvious.
1 Like
So, what you have to do at the beginning of an application of Bayesian statistics, is come up with some concrete models and assign probabilities. These form the prior. Then you make observations which update the probabilities via Bayes’ rule. The design of the models is important here for both logical and computability reasons. I’d suggest, if you want to test the proposition that there are no red ravens, you might start by modeling ravens as having a probability, θ, of being red. If there are no red ravens, then θ=0. Any other value of θ implies that there are some red ravens.
Now, if you e.g. have never heard of ravens before and have absolutely no idea about their color, you would also have no idea of the value of θ. So we give θ a prior distribution that is uniform between 0 and 1. (This is an “uninformative” prior and you and I obviously know better than this, but we at least want that the probability density is positive on [0,1].)
Now you go out and find a raven and it is black. So we update via Bayes’ rule. The one you want has an integral in it b/c we are updating functions on a continuous space. You’ll start from P(θ)=constant and go to P(θ|found 1 black raven)=a beta distribution. The beta distribution is a probability density on [0,1] and is what you get when the model is telling you that, for each θ, your observations are coming from a binomial distribution.
What does that update do? It shift the probability mass of P(θ| observations) over towards 0. Keep doing this and θ=0 will keep accumulating more and more probability mass. The distribution will be proportionate to (1-θ)^num_observations.
Fig1: Observed 10 non-red ravens.
Unless you ever observe a red raven, then P(θ| observations) will acquire a zero at θ=0. (It will still have a lot of probability mass near θ=0 because you are estimating the proportion of red ravens and that’s going to be pretty small.) That’s because θ=0 is now impossible. The beta distribution will now be proportionate to θ^1*(1-θ)^(num_observations-1).
Fig2: 11th raven is red
1 Like
Big green apples maybe proof that ravens are black, but everyone knows that little green apples are
This topic was automatically closed after 5 days. New replies are no longer allowed.
