Conditional Distribution and Conditional Expectation

The conditional probability mass function of Y given X is:

$$p(y\vert x) = P(Y=y\vert X=x) = \frac{P(Y=y,X=x)}{P(X=x)} = \frac{p(x,y)}{p(x)}.$$ (34)

For continuous random variables, we can define the conditional probability density function:

$$f(y\vert x)= \frac{f(x,y)}{f(x)}.$$ (35)

Rewriting the above equation yields:

$$f(x,y) = f(x) \cdot f(y\vert x).$$ (36)

The marginal density of Y can then be obtained from:

$$f(y) = \int_{-\infty}^{\infty} f(x) \cdot f(y\vert x) dx.$$ (37)

The conditional expectation of a random variable Y is the expected value of Y given [X=x], and is denoted: E[Y|X=x] or E[Y|x]. If the conditional probability density function is known, then the conditional expectation can be found using:

$$E[Y\vert X=x] = \left\{ \begin{array}{ll} \int_{-\infty}^{\in... ...vert x) & \mbox{if $Y$\space is discrete} \end{array} \right.$$ (38)

To obtain the unconditional expectation of Y, we can take the expectation of E[Y|X]. The result is the theorem of total expectation:

$$E[Y] = \left\{ \begin{array}{ll} \int_{-\infty}^{\infty} E[Y\... ...=x]p(x) & \mbox{if $X$\space is discrete}. \end{array} \right.$$ (39)

A similar result is the theorem of total moments:

$$E[Y^{k}] = \left\{ \begin{array}{ll} \int_{-\infty}^{\infty} ... ...{0.3in} & \mbox{if $X$\space is discrete}. \end{array} \right.$$ (40)

Example: Consider a computer system with different classes of jobs.

Y = service time of a job.

X = class of job. X takes on the values $i=1,2,\ldots,r$ with probabilities $p(i)=\alpha_{i}$.

For job class i, service time is exponentially distributed with parameter $\lambda_{i}$.

The conditional expectation of Y given X=i is:

$$E[Y\vert X=i] = \frac{1}{\lambda_{i}}$$ (41)

The unconditional expectation is then:

$$E[Y] = \sum_{i=1}^{r} E[Y\vert X=i] P(X=i) = \sum_{i=1}^{r}\frac{\alpha_{i}}{\lambda_{i}}$$ (42)

and

$$E[Y^{2}] = \sum_{i=1}^{r} E[Y^{2}\vert X=i] P(X=i) = \sum_{i... ...a_{i}y}dy = \sum_{i=1}^{r}\frac{2\alpha_{i}}{\lambda_{i}^{2}}$$ (43)

then

$$Var(Y)=\sum_{i=1}^{r} \frac{2 \alpha_{i}}{\lambda_{i}^{2}} - (\sum_{i=1}^{r} \frac{\alpha_{i}}{\lambda_{i}})^{2}$$ (44)

Another approach to calculating the unconditional variance of a random variable is by using the following equation:

Var(Y) = E[Var(Y|X)] + Var[E(Y|X)] (45)