Question 1
This question is about applying ideas from logarithms, calculus and probability to an unfamiliar mathematical theory called information theory.
Claude Shannon developed a mathematical theory called information theory to measure the information gained when random events occur. He defined the information, I, that is gained when an event with probability p occurs as
where . For example, no information is gained ( I=0 ) when an event is certain to .
When a coin is flipped, the outcome is either heads or tails. The coin may be biased. Let p be the probability of the outcome being heads.
Find, in terms of p, the information gained when the outcome is tails.