For twenty years, as a scientific computer programmer, I've written programs based on ideas related to Bayesian Inference. But I've never had an intuitive understand of it until now. I never had an intuitive exemplar which illustrated the components and how they work together.
Now I have one. It related to how a friend is so vocal every year right around the beginning of Daylight Saving Time (DST). He posts editorial after editorial, cartoon after cartoon, and shoots down any alternative other than totally abolishing DST.
With only a moment's reflection, I realized abolishing DST would have substantial negative impact on a wide variety of human social, recreational, educational, entertainment, and religious activities...all the things that people may choose to do among other-than-immediate-family, in short, social activities.
Restaurants, bars, gyms, concerts, lectures, museums, zoos, resorts, professional athletics, golf courses, after-school-activies, church activities would suffer, and the downstream affected industries such as wedding services, childbirth and care services, and marriage counselors, among many others. Both the people who enjoy these things and the people who provide the services.
This is probably why, though it seems counterintuitive (and/or evil) to many people, the general direction of proposed changes at the Federal and State levels seems to be toward Permanent DST. The vast majority of proposed bills have been to that effect. At the Federal level, all the bills I know of would give states the ability to choose either permanent Standard OR Daylight Saving time, and make exceptions. Here's a updated source:
My friend always shoots that down, complaining that the cold winter mornings would be deadly, especially among northern states. Well that's exactly why per-state control makes sense, they could decide based on such factors. Southern states might...depending on how extreme climate evolves...prefer the longer evenings year round anyway. Florida has been most consistently pushing for year round DST.
I think my idea of Daylight Saving Seconds is the best idea of all...and it eliminates the cold winter morning issue without causing any more than a few seconds shift every day.
But anyway, I find it interesting (and not surprising actually) that my friend's campaign to abolish DST seems to have never grasped the negative effect such a change would have on social activities. If I were to launch the kind of ad hominem attack I all too often have towards this worthy fellow, I might wrongly shout out.
"Well, this just shows how anti-social you are."
But clearly, that would not be a valid inference from just this one strongly felt concern. Many people oppose DST--reportedly a plurality, including some who would consider themselves very concerned about the extra human lives lost because of Spring Forward day. That's a social issue, right? This issue isn't completely resolved by the reduction in human lives lost on Fall Back day compared to baseline, because of having the added hour. The effects on these two days almost but not completely cancel each other out. (I would point out that the process also affects many other days, for good or ill, and that life is about more than just avoiding death.) And possibly any number of concerns, which might not necessarily be described as anti-social.
But suppose for a moment we could assume that nearly all people who ARE anti-social, oppose DST because they don't like to see other people enjoying social activities, and they don't care about its benefits. Then, what would know about my friend's being anti-social or not based on his also being opposed to DST?
Ahah, this is just the arrangement to deploy Bayesian Inference! Otherwise, you can't get that answer easily from logic or math. It's not a proper syllogism, as I would alway know in my conscience if shouting out a flame like the above.
We start from the Prior Probability, P(H), which is the probability of someone being anti-social. There are a lot of estimates of anti-social disorder, from 0.3% to 3.3%. I've frequently referred to this group of people, from hardened violent felons to bankers and CEO's and US Presidents, as "2%." But sadly I believe it may be larger than that, so I'll go with 3%. This is called the prior probability because it's the best we can guess without knowing the ultimate fact here (that my friend opposes DST), and knowing nothing else about him either in this example. I won't talk about whether I have other priors or not. I can't think of any offhand at this moment, and I'm not trying either. The wonderful thing about Bayesian Inference is that you can follow the same process over and over to add all your priors together, which once again is not possible in elementary logic or math because of faulty syllogism or disjoint sets.
We multiply that by the Posterior Probability, P(E | H), which in this problem is the probability of someone being opposed to DST given the fact (or theory) that they are anti-social? I've just postulated that this probability is effectively 1, at least for the sake of this argument, all anti-social people oppose DST, they have no reason to support and every reason to oppose, to want other people suffer lack of their precious social activities. That might not be true, actually, but at least it's a coherent theory. If we had actual evidence, from surveys or whatever, we could plug it in here. You could even plug in less than 0.5, Bayesian Inference still does it's best whatever the number (or function) is.
Finally, we divide by Model Evidence, P(H). This is the probability of being opposed to DST. That's been measured in polls as 45% in favor of ending the clock switching, and 37% keeping the current system. The popularity of other alternatives like Permanent DST is hard to tell because most people haven't thought about them much, which might actually be a problem with this claimed model evidence as well. If you told people they might lose church and school activities, they might change their minds, etc. Finally, the remaining number are not reported and we can assume they don't fit well into this analysis. So we'll have to go with 45/(45+37) as the current model evidence, though I personally think if people understood ALL the ramifications, they wouldn't be so easily persuaded to end the current system, and there are better options IMO.
So now the fill out the Bayesian Inference equation:
P(H | E) = P(E | H) * P(E) / P(H)
In this case:
Probability of a person being anti-social, given that we only know they oppose DST is equal to
Probability of being opposed to DST given that one is anti-social, times the probability of being anti-social, and divided by the probability of opposing DST.
It still blows my mind that works.
Here the proposed numbers are: 1*.03/(45/(45+37))
Answer: 0.055%
So this one bit of information, that person opposes DST, tells us that they're about twice as likely to be anti-social as the general population, but it's still not a large number.
Now, if surveys are wrong about the popularity of DST, in the direction more people preferring DST if they actually understand all the implications, then that above estimation goes up! As it should, because the remaining pool of people is more and more relatively populated by the anti-social. Or, if more people would be opposed to DST, then the number goes down. (Thinking about the pools of people involved is how Bayesian Analysis actually works.)
Likewise, and obviously, if the number of anti-social in the general population goes up or down, then the result does so in exactly the same proportion.
Finally, if the estimation of the preference of the anti-social towards eliminating DST goes down, then so does the result in exactly the same proportion.
These all make perfect sense, still the derivation of the equation seems unintuitive to me. I'm hoping it will come to me soon. The "proof" of Bayes Theorem that's easiest to follow is algebraic, but seeing that hasn't helped my intuition.
Aha, here is the intuitive Venn Diagram explanation I've been looking for!
It is easy to see that P(A|B) is P(AB)/P(B)...the intersection of A and B viewed from the universe of B.
Also note that P(AB) = P(A|B)P(B), deriving the universal view from the B view.
Also that P(B|A) is P(AB)/P(A), the intersection of A and B viewed from the universe of A.
And now we also have P(AB) = P(B|A)P(A).
Now the two ways of looking at the intersection yield the same actual result.
P(AB) = P(A|B)P(B) = P(B|A)P(A)
Bayes just rearranged the two term equalities to
P(A|B) = P(B|A)P(A)/P(B)
No comments:
Post a Comment