Statistics - Probability Theory

Probability Theory


Probability theory forms the basis for statistics and is reviewed in outline below...

Experience shows that if a random experiment is performed very often, the occurence of events obeys certain laws.  Tossing a coin 100 times results in a head approximately 50 times.  Throwing a true dice a 1200 time results in a number 6 approximately 200 times.  .  These experiments show the statistical regularity, of the relative frequencies.

Experiments such as these provide evidence that there is a number P(E) called the probability which indicates the there is a (50/100= 0,5) chance of a head appearing in the toss of a coin and a (200/1200 = 1/6) chance of a 6 resulting from the throw of a dice.  If the number of tosses or throws are increased the relative frequencies h(E) approach the theoretical probabilities P(E).

The statement E has the probability P(E) identifies that the if an event is repeated more and more times it is almost certain that the relative frequency h(E) will be equal to P(E)..

S = Sample Space
P(x) = Probability of event x
A, B ... = Event outcome.
A' .. Complement of event. (Event A does not occur)
A I B ..Outcome A assuming B has taken place>
φ ..The empty set. An impossible event

Probability theory..

A probability provides a quantitative description of the likely occurrence of a particular event.   Probability is conventionally expressed on a scale from 0 to 1; a rare event has a probability close to 0, a very common event has a probability close to 1.

The sample space is an exhaustive list of all the possible outcomes of an event.   Each possible result is represented by one and only one point in the sample space, which is usually denoted by S.

If S is the set of outcomes of an event all of which are assumed to be equally likely and A is satisfied by a subset A of all of the outcomes.

Then P(A) = N(A) /N(S)

The probability of dealing an Ace from a pack of cards = 4 (number of aces) /52 (total number of cards) = 1/13.

Probability Conditions..

Venn Diagram

The venn diagrams below illustrate various probability conditions.
The whole rectangle represents the sum off all possible events. It represents a sample space S
The shape A represents event A
The shape B represents event B
The hatched area illustrates different cases.

P(A) 0  Event A has a probability P(A)
h(A) = Relative frequency = Number of events including A / total number of possible events

If E is any event in a sample space S then

1 P(E) 0

For the entire sample space there corresponds the relationship

P(S) = 1

Alternatively .. i P(A i) = 1   The sum of the probabilities of all possible events A i taking place must be 1.

Example: The entire sample space of tossing a dice S= 1,2,3,4,5,6. The probability P(S)= P(1,2,3,4,5,6) is 1

Addition Rule

If events are mutually exclusive P( A ∩ B) = 0 Therefore

For independent events, that is events which have no influence on each other:

Conditional probability

It is often required to find the probability of event B if it is known that event A has taken . This probability is is called the conditional probability if B given A. P(B I A).  In this case A serves as the reduced sample space and the probability is the fraction of P(A) which corresponds to A B.

example.. A teacher gave his class two tests.  20% of the class passed both tests and 40% of the class passed the first test. What percent of those who passed the first test also passed the second test.

P (Second I First) = P(First and Second) /First = 0,2 /0,4 = 0,5

Multiplication Rule

The multiplication rule is a result used to determine the probability that two events, A and B, both occur.

For independent events, that is events which have no influence on each other. This simplifies to:

This results from

Useful Related Links
  1. Learning Math- Data Analysis, Statistics and Probability ...Clear tuturials on statistics and probability
  2. Probability Venn Applet.. Useful applet illustrating various probability conditions
  3. Venn Diagrams .... Notes of Venn Diagrams
  4. Wolfram- Venn Diagrams .... High Quality Information source
  5. Statistics Glossary .... Very accessible notes with some detail.
  6. NIST Engineering Statistics Handbook 1.3.6 Probability Distributions ...Comprehensive quality notes.