Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
Scarica il documento per vederlo tutto.
vuoi
o PayPal
tutte le volte che vuoi
Schemes - Statistics
Introduction
In Statistic, we always start from data and we try to get some informations in order to learn something about the phenomenon ⇒ we want to make inference
- (z1, z2, ..., zn), n ≥ 1 ⇒ Sample (realizations)
- (X1, X2, ..., Xn), n ≥ 1 ⇒ Random sample (random variables)
We assume that the random sample comes from the model Xn ~ FX(., θ), θ∈Θ
It means that we assume that the random variables X1, X2, ..., Xn are independent and identically distributed, following a model parameterized by θ.
Notice that if I fix θ I know everything about the model.
What do we want to do? → Starting from data we want to estimate the parameter θ, in order to be able to make inference on the population.
Example:
Xn ~ N(μ, σ2), (μ, σ2) ∈ ℝ x ℝ+
In this example the parameters are two: θ = (μ, σ2)So, θ is bi-dimensional (and in general it can be n-dimensional)
Now, let's consider a random sample (x1, ..., xn) from Xn ~ FX(., θ), θ∈Θand suppose that the sample mean X̄n = (1/n) Σ xi is an estimator for θ.
But, is it a good estimator? Could we have a better estimator?We'll try to answer to this kind of questions.
First of all: What is an estimator?
- An Estimator Ŷ = f(X1, ..., Xn) is a function of the random variables X1, ..., Xn
- We want to study his distributional properties
- Examples of estimators: 1) Sample mean 2) Sample variance
The best we can do is to find the distribution of the estimator but it's almost impossible.Exception: Since the Gaussian model is a particular casewe are able to find the distribution of the sample mean and the sample variance.
So, let's assume that we know the model Xn ~ FX(., θ), θ∈Θ and the estimator Ŷ = f(X1, ..., Xn).
We need to know if the estimator is good, that means if it has some specific desirable properties.In particular, we'll introduce tools and methods that allow us to get estimators with good properties in order to be a good estimator.
NOTATION:
- fX(z;θ) : density function (continuous)
- pX(z;θ) : probability mass function (discrete)
- FX(z;θ) : cumulative distribution function
- z = (z1,...,zn) : observed sample
- X = (X1,...,Xn) : random sample
- Xn∼f(.,θ), θ∈Θ : the random sample X comes from a model parameterized by θ
- Xi are independent and identically distributed as X
From the definition of random sample, we can find the distribution of X :
fz(x;θ) = ∏i=1fXi(zi;θ) => JOINT DENSITY FUNCTION
We have used INDEPENDENCE : joint = product of marginals
In general, we are not interested in X = (X1,...,Xn) and in its distribution
Instead, we are interested in some functions T such that :
T : Rn → Rm
(X1,...,Xn) → T(X1,...,Xn)
This kind of function T is called STATISTIC.
In the point estimation theory T is an ESTIMATOR, instead in the testing theory T is a TEST.
COMMON EXAMPLES:
- Sample mean : Ⅹ̄n = 1/n ∑i=1n Xi
- Sample variance : S2n = 1/n ∑i=1n(Xi - Ⅹ̄n)2
- The correct version has 1/n-1 instead of 1/n
- Sample moment of order x > 1 : Ⅹ̄x,n = 1/n ∑i=1nXxi
- Centered sample moment of order x > 1 : x,n = 1/n ∑i=1n(Xi - Ⅹ̄n)x
- Sample minimum : X(1) = min { X1,...,Xn }
- Sample maximum : X(n) = max { X1,...,Xn }
These are all examples of estimators T : Rn → Rm with m = 1
In general m ≤ n since we want to summarize data and the information that they contain.
If m = 1 (or almost m = 2) the statistic T is more informational.
Theorem (Fisher-Cochran)
If Q, Q1, Q2 are random variables such that Q = Q1 + Q2 and if Q1 ∼ χ2(g1), Q ∼ χ2(g), then:
Q2 ∼ χ2(g2), where g2 = g - g1 and Q2 is independent of Q1.
Theorem
If X1, ..., Xn is a random sample from N(μ, σ2), then:
- Xn and Sn2 are independent random variables.
- Moreover, if Xn and Sn2 are independent random variables,
- (X1, ..., Xn) is a random sample from a Gaussian model.
So, now we can find the distribution of Sn2 (**)
Sn2 = (1/n) ∑ (Xi - Xn)2
Intuition: We expect a sort of chi-squared distribution, since (Xi - Xn) is a Gaussian and so (Xi - Xn)2 is the square of a Gaussian → chi-squared.
First of all let's consider Sn* = (1/n) ∑ (Xi - μ)2 and multiply it by n/σ2:
n/σ2 Sn* = (1/σ2) ∑ (Xi - μ)2 = (1/σ2) ∑ (Xi - Xn + Xn - μ)2 =
= ∑ ((Xi - Xn)/σ + (Xn - μ)/σ)2 = ∑ (Xi - Xn)/σ)2 + 2 ∑ ((Xi - Xn)/σ) ((Xn - μ)/σ) + n((Xn - μ)/σ)2
Notice that:
- (Xi - μ)/σ is a Gaussian since Xi is a Gaussian
- (Xi - Xn)/σ is a Gaussian since Xi is a Gaussian
- (Xn - μ)/(σ/√n) is a Gaussian since Xn is a Gaussian
We can use the theorem (Fisher - Cochran), so we have :
∑ (Xi - Xn)/σ)2 ∼ χ2n-1 (**)
Condition of Identifiability
We say that a statistical model is identifiable if for θ1, θ2 ∈ Θ
there exists at least one event E such that:
P[X ∈ E, θ1] ≠ P[X ∈ E, θ2]Let's consider X=(X1,...,Xn) from a regular model Xn∼fX(⋅;θ) , θ∈Θ
Let's call Vn(θ) = log L(θ;X) = log∏i=1nfXi(xi;θ) = ∑i=1nlog fXi(xi;θ)
We define the score function as the first derivative of the log-likelihood random variable Vn(θ) :
V'n(θ) = d/dθ log L(θ; X) = L'(θ; X)/L(θ; X)Under the condition of regularity it can be proved that:
- E [ V'n(θ)] = 0
- In(θ) = Var [ V'n(θ)] = E [ (V'n(θ))2 ] − E [ (V'n(θ)) ]2 = E [ (V'n(θ))2 ] = − E [ V''n(θ)]
Remark:
All the notions presented above are valid for a parametric space of dimension k:
- Vn(θ) is a k-dimensional vector
- In(θ) is a k×k matrix
Now, let's prove that E [ V'n(θ)] = 0 :
E [ V'n(θ)] = ∫Rn V'n(θ)⋅fX(z;θ) dz = ∫Rn d/dθ fX(z;θ) / fX(z;θ) ⋅ fX(z;θ) dz = ∫Rn d/dθ fX(z;θ) dzBy Leibnitz Theorem we can swap integral and derivative
= d/dθ ∫ fz(z;θ) dz = d/dθ (1) = 0⇒ E [ V'n(θ)] = 0
Recall:
V'n(θ) is a function of θ and also of the data z=(z1,...,zn)
V'n(θ) = d/dθ log L(θ; X) = d/dθ L(θ; X) / L(θ; X) =
Accedi a tutti gli appunti
Tutor AI: studia meglio e in meno tempo
