This post is a continuation of the preceding post on bivariate normal distribution. The preceding post gives one characterization of the bivariate normal distribution. This post gives another characterization and other properties, providing further insight into bivariate normal distribution.

Practice problems to reinforce these concepts are available here.

**Recap**

First we summarize the preceding post. Consider a pair of random variables and with the following probability density function (pdf):

.

**(1)**……..

……..

where

……..

.

for all and . Any random variables and that are jointly distributed according to the pdf in (1) are said to have a bivariate normal distribution with parameters , , , and . This definition of bivariate normal does not give much insight. The following characterization gives further insight.

*Theorem 1*

The following two conditions (Condition 1 and Condition 2) are equivalent.

**Condition 1**. The joint pdf of the random variables and is the same as (1) above.

**Condition 2**. The random variables and satisfy these four conditions: (1) The conditional distribution of , given any , is a normal distribution. (2) The mean of the conditional distribution of given , , is a linear function of . (3) The variance of the conditional distribution of given , , is a constant, that is, it is not a function of . (4) The marginal distribution of is a normal distribution.

Condition 1 is the definition of bivariate normal stated at the beginning. Condition 1 implying Condition 2 is shown in Theorem 1 in the preceding post. Condition 2 shows that the conditional distributions are normal with linear mean and constant variance and that the marginal distributions are normal. Condition 2 implying Condition 1 is shown in Theorem 3 in the preceding post. Thus bivariate normality can be defined by either condition. The following theorem gives the specific form of the linear conditional mean and constant variance.

*Theorem 2*

Whenever the random variables and have a bivariate normal distribution with parameters , , , and (by satisfying Condition 1 or Condition 2), the conditional mean and the conditional variance are of the form:

.

**(2)**……..

.

**(3)**……..

.

The equation in (2) is also called the least squares regression line. Whenever the conditional mean is a linear function, it must be of the exact same form as in (2) (this fact is Theorem 2 in this previous post). Given that the linear form of is part of the definition of bivariate normal, equation (2) is not surprising.

**How to Generate a Bivariate Normal Distribution**

There is a way to generate a bivariate normal distribution. This process is interesting in its own right. It also points toward another characterization of bivariate normal. Let and be a pair of independent standard normal random variables (standard normal is a normal distribution with mean 0 and standard deviation 1). We generate a bivariate normal distribution as follows:

.

**How to generate a bivariate normal distribution**. Let and be a pair of independent standard normal random variables. For any set of 5 parameters , , , and , let and . Then and have a bivariate normal distribution with parameters , , , and . These parameters have the usual meaning as indicated in Theorem 1 and Theorem 2 above.

.

The mu-parameters ( and ) can be any real numbers. The sigma-parameters ( and ) can be any positive real numbers. The parameter must satisfy .

Because and are independent standard normal, it can be readily verified that , , , . Because both and are linear combinations of and , both and have normal distributions. So the marginal distribution of is normal with mean and standard deviation . Likewise the marginal distribution of is normal with mean and standard deviation .

Next, confirm the role of the parameter . First, evaluate the expectation .

.

.

In the above derivation, some algebraic manipulation is needed to get to a workable form. Also note that since and are independent and standard normal. Furthermore, and since is standard normal.

The covariance is , which becomes . This means that . This confirms that is the correlation coefficient of and .

With the parameters squared away, we next obtain the joint pdf of and . To this end, we use a Jacobian transformation argument. Note that the two equations and is a one-to-one transformation from to . We know the pdf of and because and are independent standard normal. Through the transformation, we want to express the pdf of and via . The following is the inverse of the transformation (it expresses and in terms of and ).

.

**(4)**……..

**(5)**……..

.

Consider the following two functions and that derive from (4) and (5).

.

……..

……..

.

Now calculate the Jacobian.

.

……..

.

The pdf of and is then where is the joint pdf of and . The pdf is known since and are independent standard normal.

.

……..

.

where is the following quantity.

.

……..

.

Note that the pdf is identical to the one in (1). Thus the approach of starting with a pair of independent standard normal random variables does generate a bivariate normal distribution.

**Another Characterization of Bivariate Normal**

The reverse of the preceding section is also true. Starting with any bivariate normal and , the and described in (4) and (5) are a pair of independent standard normal random variables. We have the following theorem.

*Theorem 3*

Let and be continuous random variables. Then and have a bivariate normal distribution with parameters , , , and if and only if the following condition holds.

**Condition 3**. There exists a pair of independent standard normal random variables and such that and ……………. .

One direction of Theorem 3 is already established in the preceding section – Condition 3 implying that and are bivariate normal. We now sketch out a proof of the other direction – the fact that and are bivariate normal implies Condition 3.

Suppose that and are bivariate normal with parameters , , , and . Consider the and as defined in (4) and (5) above. It is straightforward to show that and and and . More importantly, it can be shown that the joint pdf of and is a product of two standard normal density functions, one in terms of and the other in terms of . If this is done, then we know and are independent and standard normal random variables.

The equations (4) and (5) above is a one-to-one transformation from to . The inverse of that transformation is given by the following two equations.

.

……..

……..

.

Now calculate the Jacobian.

.

……..

.

Recall that is the bivariate normal pdf as described in (1). The following derivation gives the pdf of and .

.

……..

.

where is the expression found in (1) above. The derivation continues.

.

……..

.

The last result shows that the pdf is a product of two standard normal pdfs (one in terms of and the other one in terms of . This means that and are independent and standard normal random variables. This concludes the proof of Theorem 3.

**An Application**

We have discussed three characterizations of bivariate normal distribution. One is the basic definition using the pdf described in (1). The second one is Condition 2 in terms of the conditional distribution of and the conditional mean . The third one is Condition 3, which essentially says that any bivariate normal and can be generated by a pair of independent standard normal random variables.

Whenever we say and have a bivariate normal distribution, we have any of the three conditions at our disposal. We now use Condition 3 to prove the following theorem.

*Theorem 4*

Suppose that the random variables and have a bivariate normal distribution. Then any linear combination of and has a normal distribution. More specifically, is a normal random variable for any real constants and .

One comment. When , , which clearly is not a normal distribution. One way to get around this is to declare a single point as a normal distribution with zero variance. This is the approach some authors take. Another approach is to exclude the case . If the second approach is used, the theorem should say is a normal random variable for any real constants and not both zero. In any case, the scenario is not a very interesting one.

Based on Condition 3, and can be expressed in terms of a pair of independent and , both have standard normal distributions.

.

……..

……..

.

Express in terms of and .

.

……..

.

The last expression is a linear combination of and plus a constant. Linear combinations of independent normal distributions are normal. Any normal distribution adding a constant is normal. Thus the last expression is a normal distribution. As a result, is a normal distribution. Note that the mean of is what it should be: . The variance of is what it should be: . This completes the proof of Theorem 4.

**The Moment Generating Function**

Theorem 4 can also be accomplished using the moment generating function of the bivariate normal distribution. The following theorem is stated without proof.

*Theorem 5*

Suppose that the random variables and have a bivariate normal distribution with parameters , , , and . Then its moment generating function (mgf) is given by:

.

……..

.

where and . Many properties can be derived from the mgf. For example, it can be used to establish Theorem 4. The above mgf can be used to show that the mgf of is a normal mgf. The following is another application of the bivariate normal mgf.

*Theorem 6*

Suppose that the random variables and have a bivariate normal distribution. Then any pair of linear combinations of and also have a bivariate normal distribution. More specifically, and also have a bivariate normal distribution for any constants .

One way to prove Theorem 6 is to show that the mgf of and is of the same form in Theorem 5.

**Remarks**

One prominent characteristic of bivariate normal is that normal distribution is found in many directions. If and are bivariate normal, then the marginal distributions are normal (both and ). Furthermore, the conditional distributions of and are normal. On top of that, any linear combination of and is normal. What makes the last result possible is that and can be expressed by a pair of independent standard normal random variables. Theorem 6 indicates that and transformed linearly also have a bivariate normal distribution. So bivariate normality is a versatile property and is definitely an interesting and rich mathematical property.

Based on the discussion in the preceding paragraph, whenever and are bivariate normal, their sum and differences and are also normal. In general, the sum of two normal distributions is not necessarily normal. On the other hand, though bivariate normality of and implies the normality of the marginal distributions, in general the normality of the marginals does not mean the joint distribution is bivariate normal. The following example shows why.

*Example 1*

Let be a standard normal random variable. Let be Bernoulli such that . Furthermore, and are assumed to be independent. Let , which can be also be explicitly defined as follows:

.

……..

.

The random variable is also standard normal. The following shows that its CDF is identical to the standard normal CDF.

.

……..

.

However, is the following random variable, which is not normal.

.

……..

.

The sum of independent normal distributions is normal. But in general the sum two normal distributions do not have to be normal as this example demonstrates. The example also shows that when the marginal distributions are normal, the joint distribution does not have to be bivariate normal. If and were bivariate normal, then would have to be normal. Thus the and in this example cannot be bivariate normal.

Another way to show that and in this example are not bivariate normal is through the covariance. Note that since ………….. . Thus . For bivariate normal and , zero correlation means independence. However, and are obviously dependent.

We conclude by presenting two calculation examples.

*Example 2*

Suppose that the height (husband) and the height (wife) of a married couple are modeled by a bivariate normal distribution with parameters inches, inches, inches, inches, and . For a randomly selected married couple, determine the probability that the wife is taller than the husband.

Since this is a bivariate normal model, the difference has a normal distribution with mean inches and standard deviation as the variance is:

.

……..

.

The following calculates the probability.

.

……..

.

In this bivariate normal model, there is about a 7% chance that the female member of a couple is taller than the male member of the couple.

*Example 3*

For the same bivariate normal and discussed in Example 2, any pair of linear combinations of and is also bivariate normal according to Theorem 6. In particular, and are bivariate normal. This fact can be used to answer this question: for a randomly selected married couple, if the wife is three inches taller than the husband, what is the probability that she is taller than 5 feet 10 inches (70 inches)?

The probability to be calculated is:

.

……..

.

The first is to determine the 5 parameters of the bivariate normal and .

.

……..

……..

……..

……..

.

……..

.

……..

.

Next, find the mean and variance of .

.

……..

.

……..

.

Finally, the desired probability:

.

……..

.

For those couples whose wives are three inches taller, there is roughly a 62% chance that the wife is over 70 inches tall.

Practice problems are available here.

.

.

Dan Ma mathematical statistics

Daniel Ma mathematical statistics

Dan Ma math

Daniel Ma mathematics

Dan Ma stats

Daniel Ma statistics

Dan Ma statistical

Daniel Ma statistical

2019 – Dan Ma

Practice Problem Set 5 – bivariate normal distribution | Probability and Statistics Problem Solve(00:05:51) :[…] posts – one is a detailed introduction to bivariate normal distribution and the other is a further discussion that brings out more mathematical properties of the bivariate normal distribution. The properties […]

LikeLike

Introducing bivariate normal distribution | Mathematical Statistics(00:19:10) :[…] The next post is a further discussion on bivariate normal distribution. Practice problems on bivariate normal distribution are available here. […]

LikeLike