多媒体通信系统：技术、标准及网络

Preface

Acknowledgments

List of Acronyms

Chapter 1

Multimedia Communications

1.1 Introduction

1.2 Multimedia Communication Model

1.3 Elements of Multimedia Systems

1.4 User Requirements

1.5 Network Requirements

1.6 Packet Transfer Concept

1.7 Multimedia Requirements and ATM Networks

1.8 Multimedia Terminals

1.9 Concluding Remarks

Chapter 2

Audio-Visual Integration

2.1 Introduction

2.2 Media Interaction

2.3 Bimodality of Human Speech

2.4 Lip Reading

2.5 Speech-Driven Talking Heads

2.6 Lip Synchronization

2.7 Lip Tracking

2.8 Audio-to-Visual Mapping

2.8.1 Classification-Based Conversion

2.8.2 HMM for Audio-to-Visual Conversion

2.8.3 Audio and Visual Integration for Lip-Reading Applications

2.8.4 Audio-Visual Information Preprocessing

2.8.5 Pattern-Recognition Strategies

2.8.6 Integration Strategy

2.9 Bimodal Person Verification

2.10 Joint Audio-Video Coding

2.11 Concluding Remarks

Chapter 3

Multimedia Processing in Communications

3.1 Introduction

3.2 Digital Media

3.3 Signal-Processing Elements

3.4 Challenges of Multimedia Information Processing

3.4.1 Pre and Postprocessing

3.4.2 Speech, Audio and Acoustic Processing for Multimedia

3.4.3 video Signal Processing

3.4.4 Content-Based Image Retrieval

Texture-Based Methods

Shape-Based Methods

Color-Based Methods

3.5 Perceptual Coding of Digital Audio Signals

3.5.1 General Perceptual Audio-Coding Architecture

3.5.2 Review of Psychoacoustic Fundamentals

Absolute Threshold of Hearing

Critical Band Frequency Analysis

Simultaneous Masking and the Spread of Masking

Temporal Masking

PE

3.6 Transform Audio Coders

3.6.1 Optimum Coding in the Frequency Domain

3.6.2 Perceptual Transform Coder

3.6.3 Hybrid Coder

3.6.4 Transform Coding Using DFT Interblock Redundancy

3.6.5 ASPEC

3.6.6 Differential Perceptual Audio Coder

3.6.7 DFT Noise Substitution

3.6.8 DCT with Vector Quantization

3.6.9 MDCT

3.6.10 MDCT with VQ

3.7 Audio Subband Coders

3.7.1 Wavelet Decompositions

3.7.2 DWT-based Subband Coders

3.8 Speech Coder Attributes

3.9 CD Audio Coding for Multimedia Applications

3.10 Image Coding

3.11 Video Coding

3.11.1 TC and Subband Coding (SBC)

3.11.2 Predictive Coding

3.11.3 Motion-Compensated Video Coding

3.12 Watermarking

3.12.1 Watermarking Techniques

3.12.2 Main Features of Watermarking

3.12.3 Application Domains

3.13 Organization, Storage and Retrieval Issues

3.13.1 Streaming Issues for Speech and Audio

3.13.2 Streaming Issues for Video

3.14 Signal Processing for Networked Multimedia

3.15 NNs for Multimedia Processing

3.15.1 NNs for Optimal Visualization

3.15.2 Neural Techniques for Motion Estimation

3.15.3 NN Application to Face Detection and Recognition

3.15.4 Personal Authentication by Fusing Image and Speech

3.15.5 Subject-Based Retrieval for Image and Video Databases

3.15.6 Face-Based Video Indexing and Browsing

3.16 Multimedia Processors

3.16.1 Image-Processing Hardware and Software

3.16.2 Multimedia Processors' Classification

3.16.3 General Purpose Microprocessors

3.16.4 Microprocessors for Embedded Applications

3.17 Concluding Remarks

Chapter 4

Distributed Multimedia Systems

4.1 Introduction

4.2 Main Features of a DMS

4.3 Resource Management of DMS

4.4 Networking

4.4.1 IP Networking

IP Multicast

Resource Reservation Protocol (RSVP)

RTP

4.4.2 Integrated Management Architecture for IP-Based Networks

Performance Management

Fault Management

Configuration Management

Security Management

Accounting and Billing Management

4.4.3 ATM

4.4.4 Integration of IP and ATM

4.4.5 Real-Time Multimedia over ATM (RMOA)

4.5 Multimedia Operating Systems

CPU Management

Memory Management

I/0 Management

File System Management

4.6 Distributed Multimedia Servers

4.6.1 Multimedia Packing

4.7 Distributed Multimedia Applications

4.7.1 1TV

VoD

4.7.2 Telecooperation

Telecooperation Infrastructure

Telecooperative Applications

Telemedicine

4.7.3 Hypermedia Applications

Basic Features of a Hypermedia System

The Web

4.8 Concluding Remarks

Chapter 5

Multimedia Communication Standards

5.1 Introduction

5.2 MPEG Approach to Multimedia Standardization

5.3 MPEG-1 (Coding of Moving

Pictures and Associated Audio)

5.3.1 The Basic MPEG-1 Intefframe Coding Scheme

5.3.2 Conditional Replenishment

5.3.3 Specific Storage Media Functionalities

5.3.4 Rate Control

5.4 MPEG-2 (Generic Coding of

Moving Pictures and Associated Audio)

5.4.1 MPEG-2 Video

MPEG-2 Video--The Basics

MPEG-2 Video Syntax

MPEG-2 Video Scalability

MPEG-2 Video: Profiles and Levels

5.4.2 MPEG-2 Audio

5.4.3 MPEG-2 Systems

5.4.4 MPEG-2 DSM-CC

MPEG-4---Coding of Audiovisual Objects

5.5.1 Overview of MPEG4:

Motivations, Achievement, Process and Requirements

Media Objects

MPEG-4 Version 1

MPEG-4 Version 2

Extensions to MPEG-4 Beyond Version 2

Profiles in MPEG-4

Verification Testing: Checking MPEG's Performance

MPEG-4 Standardization Process

Requirements for MPEG-4

5.5.2 MPEG-4 Systems

MPEG-4 Systems Architecture

Elementary Stream Management (ESM)

Auxiliary Descriptors and Streams

Structuring Content by Grouping of Streams

Managing Content Complexity

Distributed Content-Handling Considerations

System Decoder Model (SDM) for ES Synchronization

MPEG-4 Systems BIFS

5.5.3 DMIF

DMIF Computational Model

5.5.4 MPEG-4 Video

Shape-Coding Tools for MPEG-4 Natural Video

Motion Estimation and Compensation

Texture-Coding Tools

Multifunctional Coding

Sprite Coding

Scalability

Error Resilience

Relationship Between Natural and Synthetic Video Coding

Synthetic Images

Integration of Face Animation with Natural Video

FAPs

Face Model

Coding of FAPs

FIT

Integration of Face Animation and Text-to-Speech (TI'S) Synthesis

BIFS for Facial Animation

2l) Mesh Coding

VO Tracking

2D-Mesh Object Encoder/Decoder

5.5.5 MPEG-4 Audio

MPEG4 Natural Audio Coding

General Audio Coding (Advanced Audio Coding Based)

Twin VQ

Speech Coding in MPEG-4 Audio

Scalability in MPEG-4 Natural Audio

Synthetic Audio in MPEG-4

Audio BIFS

5.5.6 Profiles and Levels in MPEG-4

Visual Object Types

Visual Profiles

Audio Object Types

Audio Profiles

Graphics

Systems Profiles

5.6 MPEG-4 Visual Texture Coding (VTC)

and JPEG 2000 Image Compression Standards

5.6.1 JPEG 2000 Development Process

5.6.2 Overview of Still-Image Coding Standards

MPEG-4 VTC

JPEG

PNG

5.6.3 Significant Features of JPEG 2000

Region of Interest (ROI) Coding

Scalability

Error Resilience

IPRs

5.6.4 Architecture of JPEG 2000

5.6.5 JPEG 2000 Bit Stream

5.6.6 Compression Efficiency Comparisons

Error Resilience

5.7 MPEG-7 Standardization Process

of Multimedia Content Description

5.7.1 Objective of the MPEG-7 Standard

5.7.2 Status of the MPEG-7 Standard

5.7.3 Major Functionalities in MPEG-7

MPEG-7 Systems

MPEG- 7 DDL

MPEG- 7 Audio

MPEG- 7 Visual

MPEG- 7 MMDSs

MPEG-7 Reference Software (XM)

MPEG- 7 Conformance

5.7.4 Applications Enabled by MPEG-7

5.8 MPEG-21 Multimedia Framework

5.8.1 Audiovisual Content Representation Issues

5.8.2 Description of a Multimedia Framework Architecture

MPEG-21 Digital Item Declaration

Content Representation

Digital Item Identification and Description

Content Management and Usage

Intellectual Property Management and Protection

Terminals and Networks

Event Reporting

5.8.3 Requirements for Digital Item Declaration

5.9 ITU-T Standardization of

Audiovisual Communication Systems

5.9.1 ITU-T Standardization Process

5.9.2 Audiovisual Systems

(H.310, H.320, H.321, H.322, H.323, and H.324)

H.320 Standard

Standards for Audiovisual Services Across ATM H.310 and H. 321

Standard H. 322--Guaranteed QoS LAN Systems

ITU-T H. 323 Standard

H.324 Standard

5.9.3 Video-Coding Standards (H.261, H.263 and H.26L)

H.261 Standard

H. 263 Standard

H. 263+ (H. 263 Version 2) Standard

H. 263+ + Standard Development

H. 26L Standard

5.9.4 ITU-T Speech-Coding Standards

Bit Rate

Delay

Complexity

Quality

5.9.5 Multimedia Multiplex and Synchronization Standards

ITU- T Recommendation H. 221

ITU-T Recommendation H. 223

ITU-T Recommendation H. 225

Common Control Protocol H. 245

5.10 IETF and Intemet Standards

5.10.1 IETF Standardization Process

5.10.2 Internet Network Architecture

5.10.3 Internet Protocols

Classical IP Stack

IP Version 6

Priority Field

Flow Label

IPv6 Addresses

Hop-by-Hop Options Header

Fragment Header

Routing Header

lPv6 Security

5.10.4 Real-Time Multimedia Transmission Across the Internet

Signaling

Session Control

Transport

Network Infrastructure

Multimedia Data for Network Use

5.10.5 MPEG-4 Video Transport Across the Internet

Use of RTP

System Architecture

MPEG-4 Server

MPEG-4 Client

5.11 Concluding Remarks

Chapter 6

Multimedia Communications Across Networks

6.1 Packet Audio/Video in the Network Environment

6.1.1 Packet Voice

6.1.2 Integrated Packet Networks

6.1.3 Packet Video

6.2 Video Transport Across Generic Networks

6.2.1 Layered Video Ceding

Layered Compression

Layered Transmission

6.2.2 Error-Resilient Video Coding Techniques

Error-Resilient Encoding

Decoder Error Concealment

Error-Resilient Entropy Code

6.2.3 Scalable Rate Control

Rate Control Techniques

Theoretical Foundation of the SRC

6.2.4 Streaming Video Across the Internet

Video Compression

Requirements Imposed by Streaming Applications

Application Layer QoS Control

Continuous Media Distribution Services

Streaming Servers

Media Synchronization

Protocols for Streaming Video

6.3 Multimedia Transport Across ATM Networks

6.3.1 Multiplexing in ATM Networks

6.3.2 Video Delay in ATM Networks

6.3.3 Errors and Losses in ATM

6.3.4 MPEG Video Error Concealment

6.3.5 Loss Concealment

6.3.6 Video Across WATM Networks

6.3.7 Heterogeneous Networking

6.4 Multimedia Across IP Networks

6.4.1 Video Transmission Across IP Networks

6.4.2 Traffic Specification for

MPEG Video Transmission on the Internet

6.4.3 Bandwidth Allocation Mechanism

6.4.4 Fine-Grained Scalable

Video Coding for Multimedia Across IP

6.5 Multimedia Across DSLs

6.5.1 VoDSL Architecture

6.5.2 Delivering Voice Services Across DSL

6.5.3 Multimedia Across ADSL

Serial Transmission: TDM

Parallel Transmission Frequency Division Mulnplexing

6.6 Internet Access Networks

6.6.1 DSL Networks

6.6.2 Cable Access Networks

6.6.3 Fixed Wireless Routed for Intemet Access

6.7 Multimedia Across Wireless

6.7.1 Wireless Broadband

Communication System (WBCS) for Multimedia

6.7.2 Audiovisual Solutions for Wireless Communications

6.7.3 Mobile Networks

Speech Transmission in GSM

Video Across GSM

Mobile ATM

Mobile IP

Wireless Multimedia Delivery

SIP in Mobile Environment

Multicast Routing in Cellular Networks

Broadband Wireless Mobile

6.7.4 Broadcasting Networks

Digital Video Broadcasting ( DVB )

Data Transmission Using MPEG-2 and DVB

MPEG Program Stream

MPEG Transport Stream

Broadband Multimedia Satellite Systems

Multimedia Home Platform

Multimedia Car Platform

6.8 Digital Television Infrastructure

for Interactive Multimedia Services

Interactive Broadcast Data (IDB) Services

Data Carousel Concept

6.9 Concluding Remarks

References

Index

About the Authors