The theory that would not die : How bayes's rule cracked the enigma code, ... 베이지안 통계학

 아직 한국에 발간되지 않은 책이다. 베이지안 통계학에 관심이 있는 사람이라면 읽기 좋은 책이다. 토마스 베이즈의 이야기부터 최근 2010년까지의 베이지안 통계학의 발전사에 대해 설명되어 있다. 특히 책의 부제인 How Bayes' rule cracked the enigma code, hunted down Russian submarines & emerged triumphant from two centuries of controversy.에 관한 이야기가 정말 자세하게 설명되어 있다. 미국 Amazon에서 Bayes 관련 책으로는 #1인데 왜 한국에서는 번역 발간되지 않았는지 아쉽다. 대신 한국에서는 고지마 히로유키의


책이 번역되어 있다. 고지마 히로유키의 책과 비교해보았을 때 이 책은 좀 더 통시적으로 베이지안 통계학의 역사에 대해 다루고 있다.

 베이지안이란 Frequentist와 구분되는 개념으로서, 경제학으로 치면 Keynesian and classical economics 의 구분과 비슷하다. 케인지안과 전통경제학의 차이가 정부의 역할에서 온다면 베이지안과 Frequentist의 차이는 'Subjective probability'의 인정 여부에서 온다.
 이 책에서는 "Bayes is a measure of belief"라고 한다.

 베이지안 통계학은 인간의 사고 체계를 닮아있기 때문에 꽤 Make sense한다. 이와 관련하여 책의 저자는 빌게이츠를 베이지안으로 분류한다.

이 책의 저자는 빌게이츠는 베이지안이라고 한다. (관련 동영상)

이번 학기부터 베이지안 수업을 듣는다... 열심히 해서 꼭 도널드 루빈 교수님의 눈에 띄고 싶다.

----------------------------------------------------------------------------------------------------------------
Part 1. Enlightenment and the Anti-Bayesian Reaction

 - By updating our initial belief about something with objective new information, we get a new and improved belief.
 - Learning from experience. : Hume `We can rely only on what we learn from experience.`


 1. Causes in the Air

  - Bayes was quantifying ignorance
  - By modern standards, we should refer to the Bayes-Price rule. Price discovered Bayes' work.
  - Bayes did not mention Hume, religion, or God.

 2. The Man Who Did Everything

 - Laplace `We owe to the frailty of the human mind one of the most delicate and ingenious of mathematical theories, namely the science of chance or probabilities.` : 확률에 대한 의미
  -> He mathematized the Bayes rule. He insisted that theories had to be based on actual fact.
  -> Where a reasonable person could develop a hypothesis and then evaluate it relentlessly in light of new knowledge, he became the first modern Bayesian.
 - 판사가 무죄였을 때 사형을 내렸을 확률 P(사형 선고|무죄) 에 대한 Bayes` rule을 계산함으로서 사형 반대의 이유로도 Bayes rule은 사용되었다.
 * It was Laplace who named the meter, centimeter, and millimeter.
 - Laplace `Sire(Napoleon), I have no need of that hypothesis.`
 - Laplace announced the central limit theorem.
 - Laplace had owned Bayes' rule in all but name since 1781.

 3. Many Doubts, Few Defenders

 - Main doubts: Prior를 반반으로 두는 것을 과연 justify할 수 있는가?
 - Karl Pearson was a zealous atheist, socialist, feminist, Darwinist, Germanophile, and eugenicist.
 - Laplace's frequency-based theories, the methods Laplace himself preferred toward the end of his life.
 - Fisher about Bayes: `a mistake, perhaps the only mistake to which the mathematical world has so deeply committed itself.`
  -> Fisher, regarded samples as their only source of information: and viewed each new set of data as a separate problem.

 * A popular in-house riddle described the situation: "What`s the collective noun for a group of statisticians?" "A quarrel."

 - Jeffreys concluded that p-values fundamentally distorted science. Frequentists, he complained, "appear to regard observations as a basis for possibly rejecting hypotheses, but in no case for supporting them."

 - Fisher의 방법이 더 appealing했던 이유는, 통계를 잘 모르는 사람이라도 p-value가 0.05면 된다는 간단한 사실을 이용할 수 있게 했기 때문이다. 더 알기 쉽게 책을 쓰고, 사회과학 연구자들이 더 쉽게 적용할 수 있게 textbook을 작성했다.



Part 2. Second World War Era

 4. Bayes Goes to War : Turing에 관한 이야기

 -  Kolmogorov said, "Strictly speaking," he told the generals, starting with Bayes' 50-50 prior odds was "not only arbitrary but surely wrong because it contradicts the main requirements of the probability theory"
 -> But Kolmogorov published two problems of Bayesian artillery

 - Turing test: a computer is thinking if, after five minutes of questioning, a person cannot distinguish its responses from those of a human in the next room
 - Turing to choose between prison and chemical castration (Since Turing was found guilty as a Nazi contribute)

 5. Dead and Buried Again
 - A bureau statistician half-jokingly called a colleague "un-American because he was Bayesian, and ... undermining the United States Government." Professors at Harvard Business School called their Bayesian colleagues "socialists and so-called scientists."



Part 3. The Glorious Revival

 6. Arthur Bailey
 - Some people live in the past, some people live in the future, but the wisest ones live in the present.

 7. From Tool to Theology
 - Neyman: When a student tried to solve a blackboard problem unconventionally, Neyman grabbed his hand and forced it to write the answer Neyman's way. For 40 years most of his hires were frequentists, and outsiders called the group "Jesus and his disciples." Neyman continued to run his institute into his 80s.
- Zero event: Counting those missing species as zero has the deleterious effect of asserting that missing species can never be found. Turing decided to assign those missing species a tiny possibility, a probability that is not zero.
- During the 1960s Bayes' rule in the United States was concentrated at the universities of Chigago and Wisconsin and at Harvard and Carnegie Mellon.
- Lindley said, "Bayesian statistics is not a branch of statistics. It is a way of looking at the whole of statistics."

 8. Jerome Cornfield, Lung Cancer, and Heart Attacks
- Fisher was as paid consultant to the tobacco industry. So, he refused to say that tobacco is the cause of the lung cancer.
 -> Fisher developed two remarkable hypothesis. The first, believe it or not, was that lung cancer might cause smoking. The second was that a latent genetic factor might give some people hereditary predilections for both smoking and lung cancer. In neither case would smoking cause lung cancer.

 - Cornfield helped design the Framingham Heart Study
 -> His report also showed researchers how to use Bayes' rule to analyze several risk factors at a time; his multiple logistic risk function has been called one of epidemiology's greatest methodologies.

 9. There`s Always a First Time
 10. 46,656 Varieties
 - "Inverse probability" was disappearing, and a modern term, "Bayesian inference," was taking its place.
 -  When Box organized a statistics department at the University of Wisconsin, he said, "I prepared my notes very carefully. But the more I did, the more I became convinced that the standard stuff I'd studied under Egon Pearson was wrong. So gradually my course became more and more Bayesian... People used to make fun of it and say it was all nonsense."

**************************************************
- Written by George Box, tune of "There`s no business like show business" -
There`s no Theorem like Bayes theorem
Like no theorem we know
Everything about it is appealing
Everything about it is a wow
Let out all that a priori feeling
You've been concealing right up to now.
....There's no theorem like Bayes theorem
Like no theorem we know.
***************************************************

 - Fisher declared, "Thinking that you don't know and thinking that the probabilities of all the possibilities are equal is not the same thing".
 - A compromise between Bayesian and anti-Bayesian method is called "Empirical Bayes"
 - After several suggestions that Bayes might be useful in assessing legal evidence, Professor Laurence H. Tribe of Harvard Law School published a blistering and influential article about the method in 1971. Drawing on his bachelor's degree in mathematics, Tribe condemned Bayes and other "mathematical or pseudo-mathematical devices" because they are able to "distort-and, in some instances, to destroy- important values" by "enshrouding the legal process in mathematical obscurity". After that, many courtroom doors slammed shut on Bayes.

Part 4. To Prove Its Worth

 11. Business Decisions
 - Robert Osher Schlaifer and Howard Raiffa in HBS: Decision making under uncertainty(DUU)
 - Savage thought Schlaifer was "hot as a pistol, sharp as a knife, clear as a bell, quick as a whip, and as exhausting as a marathon run."
 - By 2000 Bayesian methods were often centered in university business schools rather than statistics department.
 - (Story about Schlaifer when he was in HBS)
 Then Schlaifer leaned intently forward: "Tell me. There is one thing that really interests me. This book is printed on very good, very glossy paper. It must have burned very poorly. How do you burn them?" "Well, sir," the student answered respectfully, "we burn them page by page."

 12. Who Wrote The Federalist?
 - Mosteller pioneered meta-analysis and strongly advocated randomized clinical trials, fair tests of medical treatments, and data-based clinical trials.

 13. The Cold Warrior
 - Tukey: "A New Englander through and through". His conversation was quiet and measured and excluded personal comments and idle chatter.
  -> Tukey had foreseen the intimate connection between computers and statistics years earlier.
  -> "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise."
  -> "A natural, but dangerous desire for a unified approach, the greatest danger I see from Bayesian analysis stems from the belief that everything that is important can be stuffed into a single quantitative framework.", "The belief in a unified structure for inference is a dangerous form of hubris."

 14. Three Mile Island
 15. The Navy Searches
 - LEP: Local Effectiveness Probability
 - Physicists on the Manhattan Project had pioneered Monte Carlo techniques for tracking the probable paths of neutrons in a chain reaction explosion. Richardson substituted "little hypothetical submarines" for the neutrons. Academic Bayesian would not adopt Monte Carlo methods for another 20 years.
 - Stone argued, "To leave out subjective information is to throw away valuable information because there is no unique or 'scientific' way to quantify it."

 * Richardson said, "That`s kind of the ultimate happiness, when you can make things move around in the world based on your ideas."

 - The limit of approaches just isn't obvious until you actually have to make some decisions.

Part 5. Victory

 16. Eureka!
 - "Curse of dimensionality"
 - Lindley and Adrian F.M. Smith developed Hierarchical Model in Bayesian statistics.
 - Meta analysis: Combining the results of similar trials, were too complex for frequentists to address.
  -> Bayesian meta-analysis

 - By selecting a single model for large samples, frequentists ignored uncertainties about the model. Raftery told his colleagues, "The point is that we should be comparing the models, not just looking for possibly minor discrepancies between one of them and the data."
 - Markov chain can be composed of scores of links, and for each one a range of possible variables must be sampled and calculated one after another. So you replace one high-dimensional draw with lots of low-dimensional draws that are easy. => "Markov Chain Monte Carlo (MCMC)"

 17. Rosetta Stones
 - Harsanyi also showed that Nash's equilibrium for games with incomplete or imperfect information was a form of Bayes' rule
 - Bayesian purists like Michael I. Jordan of Berkeley and Philip Dawid of Cambridge object to the term "Bayesian networks"; they regard Judea Pearl's nomenclature as a misnomer because Bayesian networks do not always have priors and Bayes without priors is not Bayes.
 - Hidden Markov Model: useful for recognizing patterns that involve likely time sequences
 - Progress was coming, not from better algorithms, but from more training data
  -> For example, French and English translations of the Canadian parliament's daily debates, about 100 million words in computer-readable form.



Epilogue
 - Clearly the frequentists are metaphorical Catholics; dividing results into "significant" and "nonsignificant" instead of dividing sins into "mortal" and venial
 - Similarly, the bootstrappers cannot see why you need to bring God into it at all






------------------------------------------------------------------------------------------------------
<Vocabulary>
- amok : 날뛰다
- verboten: 금지된
- Presbyterian: 장로교
- ordained: 정하다
- feisty: 적극적인, 성마른
- conjuror: 마법사
- usury: 고리대금업
- shews: show의 고어
- celibacy: 독신
- debauchery: 방탕함
- bucolic: 목가적인
- preposterous: 터무니없는
- tenacious: 끈질긴
- judical tribunal: 재판소
- rapprochement: 화해
- papacy: 교황
- perused: 통독하다, 읽다
- about-face: 180도 전환
- allegiance: 충성
- chugged: 칙칙폭폭 소리를 내며 나아가다
- unsubstantiated: 증거가 없는는
- servility: 노예 근성
- tramp: 떠돌이
- bane: 죽음
- feud: 다투다
- raking: 모으다
- muck-heap: 엉망진창
- pedigree: 족보
- tarnished: 실추되다
- lashing out: 채찍질을 하다
- wrath: 분노
- self-effacing: 전면에 나서지 않는
- asunder: 뿔뿔이
- maelstrom: 소용돌이
- stymied: 방해받다
- disparaged: 비난받다

- doldrums: 우울
- escapade: 엉뚱한 것
- incredulity: 의심
- brigadier: 준장
- barrage: 일제 사격
- canteen: 매점
- convoy: 호위함
- whimsical: 장난스러운
- torpedo: 어뢰
- ethos: 기풍
- derisory: 보잘 것 없는

- melding: 섞다
- fell on deaf ears: 무시받다
- whimsical: 기발한
- proselytize: 개종하다
- panacea: 만병통치약

- hubris: 오만
- placidly: 잔잔하게
- condescension: 겸손
- gloom: 어둠
- acquit: 무죄를 선고하다
- harrumphed: 헛기침을 하다
- quarantined: 격리되다
- convene: 소집하다
- antiquated: 한물 간

- oxymoron: 모순어법
- cachet: 격식
- sectarian: 종파의
- skits: 희극
- mammography: 유방암 검진
- chastised: 꾸짖다

- nomenclature: 명명법
- surrogate: 대리의, 대용의

댓글