Applied Econometrics

2 - REVIEW OF PROBABILITY

2.1 Random Variables and Probability Distributions

The set of all possible outcomes is called the sample space. An event is a subset of the sample

space, that is, an event is a set of one or more outcomes. The event “my computer will crash no

more than once” is the set consisting of two outcomes: “no crashes” and “one crash.”

A random variable is a numerical summary of a random outcome.

A formula giving the probabilities for different values of the r.v. X is called probability

distribution in the case of discrete r.vs (can take a finite or countably infinite set of values), and

probability density function for continuous r.vs (can assume an infinite number of different

outcomes, f.e. any value in [0,1]).

Bernoulli distribution. It’s the distribution of a discrete random binary variable, whose outcomes

are 0 or 1. One outcome has probability p, the other 1-p.E(G)=p. Var(G)=p(1-p)

Expected Value: E(Y) or μY, is the long-run average value of a random variable over many

repeated trials.

Variance: var(Y), is a measure of the dispersion or spread of a probability distribution.

2.3 Two Random Variables

Joint probability distribution of two discrete random variables, say X and Y, is the probability

that the random variables simultaneously take on certain values, say x and y.

Conditional distribution of Y given X, is the distribution of a random variable Y conditional on

another random variable X taking a specific value. Pr(Y=y|X=x).

-law of iterated expectations: the expectation of Y is the expectation of the conditional expectation

of Y given X, E(Y)=E[E(Y|X)]

where the inner expectation on the right-hand side is computed using the conditional distribution of

Y given X and the outer expectation is computed using the marginal distribution of X. For example,

the mean height of adults is the weighted average of the mean height of men and the mean height of

women, weighted by the propor- tions of men and women. The law of iterated expectations implies

that if the conditional mean of Y given X is zero, then the mean of Y is zero.

Two random variables are independent if knowing the value of one of them provides no

information about the other. Pr(Y=y|X=x)=Pr(Y=y)

A first measure of the extent to which two random variables move together is the covariance.

Another measure of the dependence between X and Y is the correlation, which, differently from

the covariance, is unitless.

2.4 The Normal, Chi-Squared, Student t and F Distributions

The Normal Distribution

A continuous random variable with a normal distribution has the familiar bell-shaped probability

62 62).”

density. The normal distribution with mean μ and variance is expressed concisely as “N(μ,

62

The standard normal distribution is the normal distribution with mean m = 0 and variance = 1

and is denoted N(0, 1). Random variables that have a N(0, 1) distribution are often denoted Z, and

the standard normal cumulative distribution function is denoted by the Greek letter Φ; Pr(Z ... c) =

Φ(c).

The Chi-squared Distribution

The chi-squared distribution is used when testing certain types of hypotheses in statistics and

econometrics.

The chi-squared distribution is the distribution of the sum of m squared independent standard

normal random variables. This distribution depends on m, which is called the degrees of freedom of

the chi-squared distribution. For example, let Z Z 2, and Z be independent standard normal

1, 3

21 2 23

random variables. Then Z + Z + Z has a chi-squared distribution with 3 degrees of freedom.

The name for this distribution derives from the Greek letter used to denote it: A chi- squared

2

distribution with m degrees of freedom is denoted χ .

m

The Student Distribution

The Student t distribution with m degrees of freedom is defined to be the distribution of the ratio

of a standard normal random variable, divided by the square root of an independently distributed

chi-squared random variable with m degrees of freedom divided by m. That is, let Z be a standard

normal random variable, let W be a random variable with a chi-squared distribution with m degrees

of freedom, and let Z and W be independently distributed. Then the random variable Z/√(W/m) has

Review of Probability

CHAPTER 2 a Student t distribution (also called t distribution) with m degrees of freedom; denoted with tm.

The Student t distribution has a bell shape similar to that of the normal distribution, but when m is

small (20 or less), it has more mass in the tails—that is, it is a “fatter” bell shape than the normal.

Simple Random Sampling and i.i.d. Random Variables

CONCEPT When m is 30 or more, the Student t distribution is well approximated by the standard normal

2.5 distribution and the t distribution equals the standard normal distribution.

∞

In a simple random sample, objects are drawn at random from a population and

n

The F Distribution

each object is equally likely to be drawn. The value of the random variable for

Y

The F distribution with m and n degrees of freedom, denoted Fm,n , is defined to be the distribution

th

the randomly drawn object is denoted . Because each object is equally likely

i Y i

of the ratio of a chi-squared random variable with degrees of freedom m, divided by m, to an

to be drawn and the distribution of is the same for all the random variables

Y i,

i

c,

independently distributed chi-squared random variable with degrees of freedom n, divided by n. To

, are independently and identically distributed (i.i.d.); that is, the distri-

Y Y

1 n c,

state this mathematically, let W be a chi-squared random variable with m degrees of freedom and let

bution of is the same for all 1 , and is distributed independently

Y

Y i n

= 1

i

c,

V be a chi-squared random variable with n degrees of freedom, where W and V are independently

of , and so forth.

Y Y

2 n

distributed. Then (W/m)/(V/n) has an Fm,n distribution, that is an F distribution with numerator

degrees of freedom m and denominator degrees of freedom n.

In statistics and econometrics, an important special case of the F distribution arises when the

different members of the population are chosen, their values of will differ. Thus

Y

denominator degrees of freedom is large enough that the Fm,n distribution can be approximated by

c,

the act of random sampling means that , can be treated as random vari-

Y Y

1 n

the Fm,∞ distribution. In this limiting case, the denominator random variable V/n is the mean of

c,

ables. Before they are sampled, , can take on many possible values;

Y Y

1 n

infinitely many squared standard normal random variables, and that mean is 1 because the mean of

after they are sampled, a specific value is recorded for each observation.

a squared normal random variables is 1 Thus, the Fm,∞ distribution is the distribution of a chi-

squared random variable with m degrees of freedom, divided by m: W/m is distributed Fm,∞.

c,

i.i.d. draws. Because , are randomly drawn from the same population,

Y Y

1 n

2.5 Random Sampling and the Distribution of the sample Average

c,

the marginal distribution of is the same for each 1 , this marginal

Y i n;

=

i

The simple random sampling is a situation in which n objects are selected at random from a

distribution is the distribution of in the population being sampled. When has

Y Y i

population and each member of the population is equally likely to be included in the sample. The n

c, c,

the same marginal distribution for 1 , then , are said to be

i n, Y Y

= 1 n

observations in the sample are denoted Y , …, Y

identically distributed. 1 n.

When Yi has the same marginal distribution for i = 1, …, n, then Y 1, …, Yn are said to be

Under simple random sampling, knowing the value of provides no infor-

Y 1

identically distributed. Under simple random sampling, knowing the value of Y provides no

mation about , so the conditional distribution of given is the same as the

Y Y Y 1

2 2 1

information about Y , so the conditional distribution of Y given Y is the same as the marginal

marginal distribution of . In other words, under simple random sampling, is

Y Y

2 2 1

2 1

c,

distributed independently of , .

Y Y

distribution of Y . In other words, under simple random sampling, Y is distributed independently of

2 n

2 1

c,

When , are drawn from the same distribution and are indepen-

Y Y

Y , …, Y . 1 n

2 n independently and identically distributed

dently distributed, they are said to be

When Y 1, c, Yn are drawn from the same distribution and are independently distributed, they are

i.i.d.).

(or

said to be independently and identically distributed (or i.i.d.).

Simple random sampling and i.i.d. draws are summarized in Key Concept 2.5.

(However, please note: time series are not random samples. They are often autocorrelated (large

values follow large values, etc.) The textbook theory does not always apply!)

The Sampling Distribution of the Sample Average

The Sampling Distribution of the Sample Average c,

The sample average or sample mean, Y, of the n observations Y 1, …, Y is

sample average sample mean,

The or of the observations , is

Y, n Y Y n

1 n

a

n

1 1

g

+ + +Y ) . (2.43)

(Y

Y Y Y

= =

1 2 n i

n n 1

i =

An essential concept is that the act of drawing a random sample has the effect of

c, role in statistics and econometrics. We start our discu

samples , .

Y Y

For example, suppose our student commuter selected five days at random to

1 n

sampling distribution

is called the of because it is the probability distribution

Y bution of by computing its mean and variance und

Y

The sampling distribution of averages and weighted averages plays a central

record her commute times, then computed the average of those five times. Had

associated with possible values of that could be computed for different possible

Y population distribution of Y.

role in statistics and econometrics. We start our discussion of the sampling distri-

she chosen five different days, she would have recorded five different times—and

c,

samples , .

Y Y

1 n _

bution of by computing its mean and variance under general conditions on the

Y

thus would have computed a different value of the sample average.

The sampling distribution of averages and weighted averages plays a central

Mean and variance of Y

. Suppose that the observati

population distribution of Y.

Because is random, it has a probability distribution. The distribution of

Y Y

role in statistics and econometrics. We start our discussion of the sampling distri-

2

let and denote the mean and variance of (beca

Y

m s Y

Y i

_

sampling distribution

is called the of because it is the probability distribution

Y

bution of by computing its mean and variance under general conditions on the

Y c

the mean and variance is the same for all 1,

i

c, =

Mean and variance of Y

. Suppose that the observations , are i.i.d., and

Y Y

associated with possible values of that could be computed for different possible

Y 1 n

population distribution of Y. +

of the sum is given by applying Equation

Y Y

2

let and denote the mean and variance of (because the observations are i.i.d.

Y

c, 1 2

m s

samples , .

Y Y Y

Y i

_

1 n 2m . Thus the mean of the sample average is E

c,

=

An essential concept is that the act of drawing a random sample has the effect of making the sample

m

the mean and variance is the same for all 1, When 2, the mean

i n). n

= =

Y Y

c,

Mean and variance of Y

.

The sampling distribution of averages and weighted averages plays a central

Suppose that the observations , are i.i.d., and

Y Y

1 n

. In general,

m

average Y a random variable. Because Y is random, it has a probability distribution. The distribution

+ + +

of the sum is given by applying Equation (2.28): )

Y Y E(Y Y =

Y m

2

role in statistics and econometrics. We start our discussion of the sampling distri-

1 2 1 2 Y

let and denote the mean and variance of (because the observations are i.i.d.

Y

m s Y

Y i 1 1

+ *

2m . Thus the mean of the sample average is (Y )4 2m

E3 Y

= = =

of Y is called the sampling distribution of Y because it is the probability distribution associated a

c,

m

bution of by computing its mean and variance under general conditions on the

Y

the mean and variance is the same for all 1, When 2, the mean

1 2

i n). n

Y Y Y

= = 2 2 n

1

. In general, )

E(Y) E(Y

= =

with possible values of Y that could be computed for different possible samples Y , …, Yn .

m m

population distribution of Y. 1

+ + +

of the sum is given by applying Equation (2.28): )

Y Y E(Y Y

Y = i Y

n

m

1 2 1 2 Y 1

i =

12 12

+ *

2m . Thus the mean of the sample average is (Y )4 2m

E3 Y

= = =

_ a

Mean and variance of Y

m 1 2

Y Y Y

n

1

c, The variance of is found by applying Equatio

Y

Mean and variance of Y

. Suppose that the observations , are i.i.d., and

Y Y ) . (2.44)

E(Y) E(Y

= =

. In general, m

1 n

m

Suppose that the observations Y1,…,Yn are i.i.d., and let μY and 6^2Y denote the mean and

i Y

n

Y 2

+

2, var(Y ) 2s , so [by applying Equatio

Y

n

2 = =

let and denote the mean and variance of (because the observations are i.i.d.

Y 1

i =

m s 1 2 Y

Y

Y i

variance of Yi (because the observations are i.i.d. the mean and variance is the same for all i=1,

a 1

c, 2

4,

n cov(Y , ) 0 var(Y) . For general bec

Y n,

= =

1

the mean and variance is the same for all 1, When 2, the mean

i n). n

= = s

The variance of is found by applying Equation (2.37). For example, for

Y 1 2 Y

2

) . (2.44)

E(Y) E(Y

= =

…,n). m

i Y

n ≠

and are independently distributed for so co

Y i j,

+ + +

of the sum is given by applying Equation (2.28): )

Y Y E(Y Y = 1

2 m

+

2, var(Y ) 2s , so [by applying Equation (2.31) with and

Y

n a b

= = = =

j

1

i =

1 2 1 2 Y

1 2 Y 2

When n=2, the mean of the sum Y1+Y2 is given by applying equation 2.28: E(Y1Y2)= μY+ μY=2

1 1

+ *

2m . Thus the mean of the sample average is (Y )4 2m

E3 Y c,

= = =

1 2

m 4, a

cov(Y , ) 0 var(Y) . For general because , are i.i.d.,

Y n, Y Y Y

= = 1 2

Y Y Y

s

2 2

The variance of is found by applying Equation (2.37). For example, for

Y n

1 2 1

Y n i

2 1

μY. Thus the mean of the sample average is E[1/2(Y1+Y2)]=1/2*2 μY= μY. In general:

. In general, a b

var(Y ) var Y

=

m ≠

and are independently distributed for so cov(Y , ) 0 . Thus,

Y i j, Y =

Y 1

2 i

+

2, var(Y ) 2s , so [by applying Equation (2.31) with and n

Y

n a b

= = = =

j i j

1 2 Y 2 1

i =

c,

12 a

2

4,

cov(Y , ) 0 var(Y) . For general because , are i.i.d.,

Y n, Y Y Y

= = a a a a

s

n

1

1 2 1

Y n i

n n n n

1 1 1

) . (2.44)

E(Y) E(Y

= = a b

var(Y ) var Y

m

≠ +

= var(Y )

and are independently distributed for so cov(Y , ) 0 . Thus,

Y i j, Y =

=

i Y

n i

n i

j i j 2 2

n n

1

i = 1

i ≠

= 1 1 1,

i i j j i

= = =

a a a a

n

1 n n n

The variance of is found by applying Equation (2.37). For example, for

Y 1 1 2

The variance is: s

a b

var(Y ) var Y

= Y

+

var(Y ) cov(Y ,Y )

= .

=

i

n 1

i i j

2

+

2, var(Y ) 2s , so [by applying Equation (2.31) with and

2 2

Y

n a b

= = = =

n n n

1

i =

1 2 Y ≠

1 1 1,

i i j j i

2

= = =

c,

a a a

12 2

4,

cov(Y , ) 0 var(Y) . For general because , are i.i.d.,

Y n, Y Y Y

= = s n n n

1 1

1 2 1

Y n i

2 The standard deviation of is the square root of the

Y

s

+

var(Y ) cov(Y ,Y )

= Y

≠

and are independently distributed for so cov(Y , ) 0 . Thus,

Y i j, Y =

. (2.45)

=

i i j

2 2

n n

j i j

n

These result hold whatever the distribution of Y is; that is, the distribution of Y does not need to

≠

1 1 1,

i i j j i

= = =

i i

a

n 2

take on a specific form, such as the normal distribution, for these results to hold.

1 2n.

The standard deviation of is the square root of the variance,

Y

s s

Y

a b Y

var(Y ) var Y

= . (2.45)

= i

n n

1

i =

a a a

2.6 Large Sample Approximations to Sampling Distributions

n n n 2n.

The standard deviation of is the square root of the variance,

Y

1 1 s

Y

+

var(Y ) cov(Y ,Y )

= i i j

2 2

It’s important to know what the sampling distribution of Y is. There are two approaches to

n n ≠

1 1 1,

i i j j i

= = =

characterizing sampling distributions: an “exact” approach and an “approximate” approach.

2 M02_STOC1312_Ch02_pp060-110.indd 91

s Y . (2.45)

=

The exact approach entails deriving a formula for the sampling distribution that holds exactly for

n

any value of n. 2n.

The standard deviation of is the square root of the variance,

Y s

M02_STOC1312_Ch02_pp060-110.indd 91 29/07/14 2:06 PM

Y

The “approximate” approach uses approximations to the sampling distribution that rely on the

sample size being large. The large-sample approximation to the samplin