I love the subject of probability. Almost every book I’ve read on Probability, the explanation of expectation has been unintuitive and de-motivating. Except a few – one that is written by my professor, Prof. Norm Matloff, which is available here.

Consider that I am buying 10 oranges, which sums up to a weight of 1lb. Now the weight of one orange on an average is {\frac{1}{10}}lb. This concept is so simple that there is nothing to explain here. We all know what an average value means.

Expectation is nothing but an average value. Assume the case of throwing a die {n} times. Let us assume we have a variable that marks the value we get for every throw. Let {X_i} denote the value we get in the {i^{\text{th}}} throw. So after {n} throws I get a sum of values:

\displaystyle X_1 + X_2 + ... + X_i + ... + X_n

.

Let us denote the sum by a variable {S}; then: {S = X_1 + X_2 + ... + X_i + ... + X_n}.

Now what is the average value that we get for each throw. That would be {\frac{S}{n}}.

As a side note, these variables, the {\left(X_i'\right)s} are called random variables. Why are they called random variables? Just because they are initialized to some value randomly. I can not say {X_1} will always be 1. All I can say is {X_1} will be an integer between 1 and 6.

Let us try rewriting our equation for the sum {S} in a fancy way:

\displaystyle \begin{array}{rcl} S = & & (0. \text{number of times we throw a 0}\\ & + & 1.\text{number of times we throw a 1}\\ & + & ... \\ & + & 6.\text{number of times we throw a 6}) \end{array}

If we take this sum over our {n} throws, we get the exact sum {S}.

With this new equation if we rewrite our average:

\displaystyle \begin{array}{rcl} \frac{S}{n} = & & (0. \text{number of times we throw a 0}+ \\ & & 1.\text{number of times we throw a 1}+\\ & & ...+\\ & & 6.\text{number of times we throw a 6}) / n \end{array}

That is

\displaystyle \begin{array}{rcl} \frac{S}{n} & = &(\frac{0. \text{number of times we throw a 0}}{n})+\\ & &(\frac{1. \text{number of times we throw a 1}}{n})+\\ & & ...+\\ & &(\frac{6. \text{number of times we throw a 6}}{n}) \end{array}

But we already know, intuitively:

\displaystyle \frac{\text{number of times we throw a 0}}{n} =\\ \text{probability of getting a 0 on throwing a die}

\displaystyle \frac{\text{number of times we throw a 1}}{n} =\\ \text{probability of getting a 1 on throwing a die}

And so on.

Then our average becomes:

\displaystyle \begin{array}{rcl} \frac{S}{n} & = & 0.\text{probability of getting a 0 in throwing a die}\\ & + & 1. \text{probability of getting a 1 in throwing a die} \\ & + & .... \\ & + & 6.\text{probability of getting a 6 in throwing a die} \end{array}

We can write this concisely as:

\displaystyle \begin{array}{rcl} \frac{S}{n} & = & \sum_{i=1}^{6}i.[\text{probability of getting an }i\text{ in throwing a die}]\\ & = & \sum_{i=1}^{6}i.P[X=i] \end{array}

where we wrote a generic single random variable {X} instead of differentiating each of them with {X_i}‘s.

We can call this average {\frac{S}{n}} by a fancy name too, expectation. The mathematical notation of which would be {E[X]}.

This gives {E[X] = \sum_{i=1}^{6}i.P[X=i] = 3.5\text{(by doing the cumbersome arithmetic)}}, in case of rolling a die. Here the sum is taken over all possible values that {X} can take.

Let us denote that as a set, named {A}, which in our case will be {A = \{1, 2, ...., 6\}}.

Using this we can write our generic equation for expectation (as found in many textbooks):

\displaystyle E[X] = \sum_{i \in A}i.P[X=i]

And finally, the word expectation, as my professor explains, says only one thing about that value; “you should never expect the expected value”.

Advertisement