51:11
2. Conditioning and Bayes' Rule
MIT OpenCourseWare
Overview
This video introduces conditional probability as a way to update beliefs when new information is received. It reviews basic probability concepts like sample spaces and probability assignments, then delves into conditional probability, explaining its definition and properties. The lecture illustrates its use with examples involving dice rolls and a radar system detecting airplanes, highlighting how conditional probabilities help in inference and decision-making, and introduces Bayes' Rule as a fundamental tool for updating beliefs based on evidence.
How was this?
Save this permanently with flashcards, quizzes, and AI chat
Chapters
- A sample space is the set of all possible, mutually exclusive, and collectively exhaustive outcomes of a random experiment.
- Probabilities are assigned to subsets (events) of the sample space, satisfying non-negativity, the probability of the sample space being 1, and additivity for disjoint events.
- Countable additivity allows summing probabilities of an infinite sequence of disjoint events, but this has subtleties with continuous sample spaces where individual points have zero probability.
Understanding these foundational concepts is crucial for correctly setting up probability models and interpreting the results of probability calculations, especially when dealing with continuous outcomes.
The unit square example demonstrates that while the union of all points (the square) has probability 1, the probability of any single point is 0, illustrating the limitations of applying countable additivity to uncountable sets.
- Conditional probability, denoted P(A|B), represents the updated probability of event A occurring given that event B has already occurred.
- When event B occurs, it becomes the new sample space, and probabilities are revised proportionally to their original values within B.
- The formal definition is P(A|B) = P(A intersection B) / P(B), provided P(B) is not zero.
- Conditional probabilities behave like regular probabilities and satisfy the same axioms, forming a new probability law on the reduced sample space.
Conditional probability is the core mechanism for updating our beliefs and making predictions in light of new evidence, moving from initial probabilities to revised ones.
If event B (e.g., minimum of two dice rolls is 0) has occurred, the probability of event A (e.g., maximum roll is 2) is recalculated based only on the outcomes within B, maintaining the relative likelihoods of those outcomes.
- Conditional probabilities are used in 'divide and conquer' strategies to break down complex problems.
- The probability of a sequence of events (A, then B, then C) can be found by multiplying conditional probabilities along the branches of a probability tree: P(A) * P(B|A) * P(C|A and B).
- The Law of Total Probability allows calculating the probability of an event B by summing the probabilities of B occurring under each scenario in a partition of the sample space: P(B) = sum(P(B|Ai) * P(Ai)).
These techniques provide systematic methods for calculating probabilities in complex scenarios, enabling us to quantify the likelihood of combined events and overall outcomes.
In a radar example, the probability of a plane being present and the radar detecting it is calculated as P(Plane) * P(Detect | Plane), and the total probability of the radar detecting something is the sum of probabilities of detection with and without a plane, weighted by their respective likelihoods.
- Bayes' Rule provides a formal way to reverse conditional probabilities, allowing inference about causes given observed effects.
- It updates prior beliefs (probabilities of scenarios) into posterior beliefs (updated probabilities of scenarios) based on new evidence (an observed event).
- The formula P(Ai|B) = [P(B|Ai) * P(Ai)] / P(B) uses the probability of the evidence given the scenario and the prior probability of the scenario to calculate the updated probability of the scenario.
- This process is fundamental for learning from data and is counter-intuitive at times, as demonstrated by the radar example where good radar specs don't guarantee high confidence in detection due to false alarms.
Bayes' Rule is a cornerstone of statistical inference, enabling us to systematically update our understanding of the world as we gather more information, crucial in fields from medicine to artificial intelligence.
Given a radar detects a blip (event B), Bayes' Rule calculates the updated probability that there was actually an airplane (scenario Ai) by combining the radar's accuracy (P(B|Ai)) with the initial probability of a plane being there (P(Ai)).
Key takeaways
- Information is always partial, and conditional probability is the mathematical tool to update beliefs when new information arrives.
- Zero probability does not mean impossible; it signifies an extremely unlikely event, especially in continuous probability models.
- Probability of one signifies essential certainty but still allows for the possibility of an outcome outside the event.
- Conditional probabilities form a valid probability law on a reduced sample space, maintaining internal consistency.
- The probability tree multiplication rule simplifies calculating the probability of sequential events.
- The Law of Total Probability allows calculating the probability of an event by considering all possible scenarios under which it could occur.
- Bayes' Rule is essential for inference, allowing us to update our beliefs about causes (scenarios) based on observed effects (evidence).
Key terms
Sample SpaceEventProbabilityConditional ProbabilityIntersection of EventsLaw of Total ProbabilityBayes' RulePrior ProbabilityPosterior ProbabilityInference
Test your understanding
- How does the concept of a sample space change when we are given new information, and what mathematical tool addresses this change?
- Explain why individual points in a continuous probability model have zero probability, and what this implies about the interpretation of zero probability events.
- How can the probability of a complex event, like the intersection of three events (A, B, and C), be calculated using conditional probabilities?
- What is the core idea behind Bayes' Rule, and how does it allow us to update our beliefs based on new data?
- Describe a scenario where conditional probabilities might lead to counter-intuitive results, and explain why this happens.