Joint source channel coding using arithmetic codes

Dongsheng Bi, Michael W. Hoffman, Khalid Sayood

Research output: Contribution to journalArticle

Abstract

Based on the encoding process, arithmetic codes can be viewed as tree codes and current proposals for decoding arithmetic codes with forbidden symbols belong to sequential decoding algorithms and their variants. In this monograph, we propose a new way of looking at arithmetic codes with forbidden symbols. If a limit is imposed on the maximum value of a key parameter in the encoder, this modified arithmetic encoder can also be modeled as a finite state machine and the code generated can be treated as a variable-length trellis code. The number of states used can be reduced and techniques used for decoding convolutional codes, such as the list Viterbi decoding algorithm, can be applied directly on the trellis. The finite state machine interpretation can be easily migrated to Markov source case. We can encode Markov sources without considering the conditional probabilities, while using the list Viterbi decoding algorithm which utilizes the conditional probabilities. We can also use context-based arithmetic coding to exploit the conditional probabilities of the Markov source and apply a finite state machine interpretation to this problem. The finite state machine interpretation also allows us to more systematically understand arithmetic codes with forbidden symbols. It allows us to find the partial distance spectrum of arithmetic codes with forbidden symbols. We also propose arithmetic codes with memories which use high memory but low implementation precision arithmetic codes. The low implementation precision results in a state machine with less complexity. The introduced input memories allow us to switch the probability functions used for arithmetic coding. Combining these two methods give us a huge parameter space of the arithmetic codes with forbidden symbols. Hence we can choose codes with better distance properties while maintaining the encoding efficiency and decoding complexity. A construction and search method is proposed and simulation results show that we can achieve a similar performance as turbo codes when we apply this approach to rate 2/3 arithmetic codes.

Original languageEnglish (US)
Pages (from-to)1-77
Number of pages77
JournalSynthesis Lectures on Communications
Volume4
DOIs
StatePublished - Nov 12 2009

Fingerprint

Channel coding
Decoding
Finite automata
Data storage equipment
Trellis codes
Turbo codes
Convolutional codes
Switches

Keywords

  • arithmetic coding
  • channel coding
  • digital communications
  • joing source channel coding
  • source coding

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Joint source channel coding using arithmetic codes. / Bi, Dongsheng; Hoffman, Michael W.; Sayood, Khalid.

In: Synthesis Lectures on Communications, Vol. 4, 12.11.2009, p. 1-77.

Research output: Contribution to journalArticle

Bi, Dongsheng ; Hoffman, Michael W. ; Sayood, Khalid. / Joint source channel coding using arithmetic codes. In: Synthesis Lectures on Communications. 2009 ; Vol. 4. pp. 1-77.
@article{abe6a1b378764022a7b9202e1df7197e,
title = "Joint source channel coding using arithmetic codes",
abstract = "Based on the encoding process, arithmetic codes can be viewed as tree codes and current proposals for decoding arithmetic codes with forbidden symbols belong to sequential decoding algorithms and their variants. In this monograph, we propose a new way of looking at arithmetic codes with forbidden symbols. If a limit is imposed on the maximum value of a key parameter in the encoder, this modified arithmetic encoder can also be modeled as a finite state machine and the code generated can be treated as a variable-length trellis code. The number of states used can be reduced and techniques used for decoding convolutional codes, such as the list Viterbi decoding algorithm, can be applied directly on the trellis. The finite state machine interpretation can be easily migrated to Markov source case. We can encode Markov sources without considering the conditional probabilities, while using the list Viterbi decoding algorithm which utilizes the conditional probabilities. We can also use context-based arithmetic coding to exploit the conditional probabilities of the Markov source and apply a finite state machine interpretation to this problem. The finite state machine interpretation also allows us to more systematically understand arithmetic codes with forbidden symbols. It allows us to find the partial distance spectrum of arithmetic codes with forbidden symbols. We also propose arithmetic codes with memories which use high memory but low implementation precision arithmetic codes. The low implementation precision results in a state machine with less complexity. The introduced input memories allow us to switch the probability functions used for arithmetic coding. Combining these two methods give us a huge parameter space of the arithmetic codes with forbidden symbols. Hence we can choose codes with better distance properties while maintaining the encoding efficiency and decoding complexity. A construction and search method is proposed and simulation results show that we can achieve a similar performance as turbo codes when we apply this approach to rate 2/3 arithmetic codes.",
keywords = "arithmetic coding, channel coding, digital communications, joing source channel coding, source coding",
author = "Dongsheng Bi and Hoffman, {Michael W.} and Khalid Sayood",
year = "2009",
month = "11",
day = "12",
doi = "10.2200/S00222ED1V01Y200911COM004",
language = "English (US)",
volume = "4",
pages = "1--77",
journal = "Synthesis Lectures on Communications",
issn = "1932-1244",
publisher = "Morgan and Claypool Publishers",

}

TY - JOUR

T1 - Joint source channel coding using arithmetic codes

AU - Bi, Dongsheng

AU - Hoffman, Michael W.

AU - Sayood, Khalid

PY - 2009/11/12

Y1 - 2009/11/12

N2 - Based on the encoding process, arithmetic codes can be viewed as tree codes and current proposals for decoding arithmetic codes with forbidden symbols belong to sequential decoding algorithms and their variants. In this monograph, we propose a new way of looking at arithmetic codes with forbidden symbols. If a limit is imposed on the maximum value of a key parameter in the encoder, this modified arithmetic encoder can also be modeled as a finite state machine and the code generated can be treated as a variable-length trellis code. The number of states used can be reduced and techniques used for decoding convolutional codes, such as the list Viterbi decoding algorithm, can be applied directly on the trellis. The finite state machine interpretation can be easily migrated to Markov source case. We can encode Markov sources without considering the conditional probabilities, while using the list Viterbi decoding algorithm which utilizes the conditional probabilities. We can also use context-based arithmetic coding to exploit the conditional probabilities of the Markov source and apply a finite state machine interpretation to this problem. The finite state machine interpretation also allows us to more systematically understand arithmetic codes with forbidden symbols. It allows us to find the partial distance spectrum of arithmetic codes with forbidden symbols. We also propose arithmetic codes with memories which use high memory but low implementation precision arithmetic codes. The low implementation precision results in a state machine with less complexity. The introduced input memories allow us to switch the probability functions used for arithmetic coding. Combining these two methods give us a huge parameter space of the arithmetic codes with forbidden symbols. Hence we can choose codes with better distance properties while maintaining the encoding efficiency and decoding complexity. A construction and search method is proposed and simulation results show that we can achieve a similar performance as turbo codes when we apply this approach to rate 2/3 arithmetic codes.

AB - Based on the encoding process, arithmetic codes can be viewed as tree codes and current proposals for decoding arithmetic codes with forbidden symbols belong to sequential decoding algorithms and their variants. In this monograph, we propose a new way of looking at arithmetic codes with forbidden symbols. If a limit is imposed on the maximum value of a key parameter in the encoder, this modified arithmetic encoder can also be modeled as a finite state machine and the code generated can be treated as a variable-length trellis code. The number of states used can be reduced and techniques used for decoding convolutional codes, such as the list Viterbi decoding algorithm, can be applied directly on the trellis. The finite state machine interpretation can be easily migrated to Markov source case. We can encode Markov sources without considering the conditional probabilities, while using the list Viterbi decoding algorithm which utilizes the conditional probabilities. We can also use context-based arithmetic coding to exploit the conditional probabilities of the Markov source and apply a finite state machine interpretation to this problem. The finite state machine interpretation also allows us to more systematically understand arithmetic codes with forbidden symbols. It allows us to find the partial distance spectrum of arithmetic codes with forbidden symbols. We also propose arithmetic codes with memories which use high memory but low implementation precision arithmetic codes. The low implementation precision results in a state machine with less complexity. The introduced input memories allow us to switch the probability functions used for arithmetic coding. Combining these two methods give us a huge parameter space of the arithmetic codes with forbidden symbols. Hence we can choose codes with better distance properties while maintaining the encoding efficiency and decoding complexity. A construction and search method is proposed and simulation results show that we can achieve a similar performance as turbo codes when we apply this approach to rate 2/3 arithmetic codes.

KW - arithmetic coding

KW - channel coding

KW - digital communications

KW - joing source channel coding

KW - source coding

UR - http://www.scopus.com/inward/record.url?scp=77956158825&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956158825&partnerID=8YFLogxK

U2 - 10.2200/S00222ED1V01Y200911COM004

DO - 10.2200/S00222ED1V01Y200911COM004

M3 - Article

AN - SCOPUS:77956158825

VL - 4

SP - 1

EP - 77

JO - Synthesis Lectures on Communications

JF - Synthesis Lectures on Communications

SN - 1932-1244

ER -