The Mathematical Theory of Communication, Part 11

approximation arbitrarily small average power band binary digits bits per second calculated channel capacity chosen coding communication system communication theory conditional entropy consider constraints continuous corresponding defined definition digram dimensional discrete channel distribution duration dx dy encoding English ensemble of functions entropy power equal equivocation ergodic example fidelity frequency of errors function p(x given graph H₁ H₁(x Hence high probability independent information source joint entropy letter frequencies letters limited log p(x logarithm Markoff process mathematical maximizing maximum entropy meaning measure n-gram N₁ noise power noiseless number of possible obtain output p₁ perturbed prob probability measure problem produced R₁ random rate of transmission received signal redundancy relative entropy samples semantic Shannon situation space statistical structure stochastic process subset Suppose telegraphy Theorem thermal noise tion trans transducer transition probabilities transmitted signal uncertainty white noise words y₁ zero