Discrete Memory less Channel

DISCRETE MEMORYLESS CHANNEL:

·        Transmission rate over a noisy channel

Repetition code

Transmission rate

·        Capacity of DMC

Capacity of a noisy channel

Examples

^Ø   All these transition probabilities from xi to yj are gathered in a transition matrix.

^Ø   The (i ; j) entry of the matrix is P(Y = yj /jX = xi ), which is called forward transition probability.

^Ø   In DMC the output of the channel depends only on the input of the channel at the same instant and not on the input before or after.

^Ø   The input of a DMC is a RV (random variable) X who selects its value from a discrete limited set X.

^Ø   The cardinality of X is the number of the point in the used constellation.

^Ø   In an ideal channel, the output is equal to the input.

^Ø   In a non-ideal channel, the output can be different from the input with a given probability.

·        Transmission rate:

^Ø   H(X) is the amount of information per symbol at the input of the channel.

^Ø   H(Y ) is the amount of information per symbol at the output of the channel.

^Ø   H(XjY ) is the amount of uncertainty remaining on X knowing Y .

^Ø   The information transmission is given by:I (X; Y ) = H(X) − H(XjY ) bits/channel use

^Ø   For an ideal channel X = Y , there is no uncertainty over X when we observe Y . So all the information is transmitted for each channel use: I (X;Y ) = H(X)

^Ø   If the channel is too noisy, X and Y are independent. So the uncertainty over X remains the same knowing or not Y , i.e. no information passes through the channel: I (X; Y ) = 0.

·        Hard and soft decision:

^Ø   Normally the size of constellation at the input and at the output are the same, i.e., jXj = jYj

^Ø   In this case the receiver employs hard-decision decoding.

^Ø   It means that the decoder makes a decision about the transmitted symbol.

^Ø   It is possible also that jXj 6= jY j.

^Ø   In this case the receiver employs a soft-decision.

^ü   Channel models and channel capacity:

1.     The encoding process is a process that takes a k information bits at a time and maps each k-bit sequence into a unique n-bit sequence. Such an n-bit sequence is called a code word.

2. The code rate is defined as k/n.

3.     If the transmitted symbols are M-ary (for example, M levels), and at the receiver the output of the detector, which follows the demodulator, has an estimate of the transmitted data symbol with

(a). M levels, the same as that of the transmitted symbols, then we say the detector has made a hard decision;

(b). Q levels, Q being greater than M, then we say the detector has made a soft decision.

^ü   Channels models:

1. Binary symmetric channel (BSC):

If (a) the channel is an additive noise channel, and (b) the modulator and demodulator/detector are included as parts of the channel. Furthermore, if the modulator employs binary waveforms, and the detector makes hard decision, then the channel has a discrete-time binary input sequence and a discrete-time binary output sequence.

Note that if the channel noise and other interferences cause statistically independent errors in the transmitted binary sequence with average probability p, the channel is called a BSC. Besides, since each output bit from the channel depends only upon the corresponding input bit, the channel is also memoryless.

2. Discrete memoryless channels (DMC):

A channel is the same as above, but with q-ary symbols at the output of the channel encoder, and Q-ary symbols at the output of the detector, where Q ³ q . If the channel and the modulator are memoryless, then it can be described by a set of qQ conditional probabilities

P (Y = y _i  | X = x _j ) º P ( y _i  | x _j ), i = 0,1,...,Q - 1; j = 0,1,..., q -1

Such a channel is called discrete memory channel (DSC).

If the input to a DMC is a sequence of n symbols u₁ , u₂ ,..., u_n selected from the alphabet X and the corresponding output is the sequence v₁ , v ₂ ,..., v_n of symbols from the alphabet Y, the joint conditional probability is

the probability transition matrix for the channel.

3. Discrete-input, continuous-output channels:

Suppose the output of the channel encoder has q-ary symbols as above, but the output of the detector is unquantized (Q = ¥) . The conditional probability density functions

p ( y | X = x _k ),      k = 0,1, 2,..., q -1

AWGN is the most important channel of this type.

Y = X + G

For any given sequence X _i , i = 1, 2,..., n , the corresponding output is Y_i , i = 1, 2,..., n

Y_i  = X _i + G_i , i = 1, 2,..., n

If, further, the channel is memoryless, then the joint conditional pdf of the detector‘s output is

4. Waveform channels:

If such a channel has bandwidth W with ideal frequency response C ( f ) = 1 , and if the bandwidth-limited input signal to the channel is x ( t) , and the output signal, y ( t) of the channel is corrupted by AWGN, then

y ( t ) = x ( t ) + n ( t)

The channel can be described by a complete set of orthonormal functions:

Since { n_i } are uncorrelated and are Gaussian, therefore, statistically independent. So

^ü   Channel Capacity:

Channel model: DMC

Input alphabet: X = {x₀ , x₁ , x ₂ ,..., x_q-1}

Output alphabet: Y = {y ₀ , y₁ , y ₂ ,..., y_q-1}

Suppose x _j is transmitted,  y_i is received, then

The mutual information (MI) provided about the event {X = x _j } by the occurrence of the event

Hence, the average mutual information (AMI) provided by the output Y about the input X is

To maximize the AMI, we examine the above equation: 

(1). P ( y i) represents the jth output of the detector; 

(2). P ( y _i  | x _j ) represents the channel characteristic, on which we cannot do anything;

(3). P ( x _j ) represents the probabilities of the input symbols, and we may do something or control them. Therefore, the channel capacity is defined by