This monograph on Bayesian Signal Processing is published within the Book Series on Adaptive and Cognitive Dynamic Systems (editor: Simon Haykin). This second edition of the above-mentioned book contains significantly updated content from the first edition published in 2009 with an additional chapter on Sequential Bayesian Detection. The author filled in gaps left in the previous edition by more cohesive discussion with application examples, effectively serving the intention to help readers better understand the classical and modern methods of Kalman Filters, discrete hidden Markov models (HMMs), and related topics within Bayesian framework. HMMs are stochastic representation of processes used for dynamic (physical) systems. These models have found widespread use in various fields such as acoustics, communications, signal processing, and speech processing to name a few. The book contains a rich collection of examples embedded within the description of the context. Many chapters offer case studies on real-life applications of the methods discussed. Each chapter encloses matlab-notes as well, serving matlab-users effectively for their study and own implementation. Exercise problems are also included at the end of each chapter. They help interested readers, including practicing engineers, research scientists, graduate students, advanced undergraduates, and postdoctrates for their self-study, at the same time, efficiently suit under-/graduate teaching at higher educational institutions. Besides an introduction and appendix, the book contains the following major chapters:
Bayesian Estimation
Simulation-Based Bayesian Methods
State-Space Models for Bayesian Processing
Classical Bayesian State-Space Processors
Modern Bayesian State-Space Processors
Particle-Based Bayesian State-Space Processors
Joint Bayesian State/Parametric Processors
Discrete Hidden Markov Model Bayesian Processors
Sequential Bayesian Detection
Bayesian Processors for Physics-Based Applications
The book offers detailed, lucidly presented materials on a number of model-based Bayesian methods. The author prepares readers with fundamental knowledge and necessary notation and terminologies (Chapter 1), and introduces (in Chapter 2) a useful group of exponential distributions, a number of probabilistic operations, and the generic Bayesian processor—the so-called “maximum a posteriori solution.” This chapter makes a clear reference to the classical maximum likelihood method, which is actually a special case of the Bayesian estimator. Chapter 2 ends with luminously describing the basic idea of sequential or recursive Bayesian estimation.
Chapter 3 describes general Bayesian solutions by introducing simulation-based sampling methods, the author employs a tender approach to uniform, rejection, and importance sampling, some useful Markov Chain Monte Carlo methods including Metropolis-Hastings, Gibbs sampling, to advanced slice sampling and the sequential importance sampling. In Chapter 4, the author dedicates the entire chapter to developing models central to the model-based Bayesian signal processing. This chapter elaborates on the fact that most physical phenomena can be modeled by mathematical relations in the form of state-space representation—a complete generic form for almost any physical system, including: continuous-time, sampled-data, and discrete-time state-space models. They are also generalizable to multichannel, nonstationary, and nonlinear processes. These three types of models along with a variety of time series models developed in this chapter lead to the Gauss−Markov representations of both linear and nonlinear systems.
After introducing the model-based concept, the following three chapters (5−7) are dedicated to the Bayesian state-space processors, from classical (Chapter 5) and modern (Chapter 6) to the particle-based (Chapter 7) processors. A progressive handling of Kalman filters as model-based Bayesian estimators using state-space models starts with linear, linearized, extended, and iterative extended Bayesian processors, and goes on to the unscented Kalman filter. This also includes a significantly extended section (in Chapter 6) on the ensemble Kalman filter which can be considered as a hybrid of a regression-based processor and a particle-like sampling, heretofore not well known in engineering areas. Chapter 7 details the Bayesian state-space processors using particle filtering (a sequential Monte Carlo sampling method) including sequential “importance sampling” technique and resampling strategies (such as multinomial, systematic, residual resampling), plus Bootstrap, auxiliary, and linearized particle filters, to name a few. Chapter 7 ends with practical aspects of particle filter design which are helpful in guiding readers with their own implementation, to test and validate their models in a wide variety of ways. The author concludes Chapter 7 with a case study on a very challenging problem for any particle filter design. Besides a lengthy list of relevant references, the author thoughtfully incorporated a decent number of graphs, schematics, flow diagrams, and algorithms in table form to help readers comprehend the underlying theory and practical implementation of the particle filtering techniques.
Chapter 8 expands the Bayesian approach to a joint state and parameter estimation and system identification problem, based on joint posterior distributions, that joins simultaneous estimations of both dynamic state and parameter variables. At the end of the chapter, target tracking using a synthetic aperture towed array is elucidated as a case study. Chapter 9 elaborates on the viewpoint that classical hidden Markov processors are essentially model-based Bayesian processors. First, this chapter develops the concepts of Markov and discrete hidden Markov chains, then illustrates how they are related. All the Bayesian processors discussed in this book embody, in essence, hidden Markov processors, since the internal states are observed indirectly and are therefore “hidden,” and the processors carry out the state exploration randomly in a Markovian manner, independent of “past” states. Through analyzing properties of hidden Markov models (HMM), the author elucidates how Bayesian concepts easily transfer over to embody the hidden Markov models. To no surprise, as the author states, once placed in the state-space representation, all Bayesian properties apply to hidden Markov processors. Upon establishing the Bayesian concepts, state and parameter estimations for HMMs are examined. Chapter 9 ends again with a case study on time-reversal decoding.
An entirely new addition to this edition, Chapter 10 offers materials on the Bayesian decision and detection theory, illustrating how model-based Bayesian processors applied to sequential detection provide a powerful solution to a wide variety of decision problems. The final chapter (Chapter 11) examines a number of physics-based applications in solving real-world problems including case studies in ocean acoustics, and threat detection in biological and radioactive abnormities. The final chapter in this new edition is also extended to include sequential threat detection using an X-ray approach and model-based adaptive application in shallow ocean. All these applications provide a completely new perspective on classical problems, yet they have in common incorporating physics-based models for problem solving within the sequential Bayesian framework.
Overall, Bayesian signal processing is centered on a unique viewpoint from the Bayesian perspective. The author has dedicated the entire book to manifesting this perspective that “the more prior information we know about the data and its evolution, the more information we can incorporate into the processor in the form of mathematical models to improve its overall performance.” More profoundly, many classical approaches, such as maximum likelihood method, Kalman filters, and hidden Markov models, have been classified within the model-based Bayesian framework in this book, clearly reflecting the author's assertion that Bayesian signal processing is a natural way to solve these basic processing problems. This reviewer believes that the book in this second edition should receive its readership beyond that which comes from the control, dynamic systems, and signal processing communities. The Bayesian probability community will find it useful as a means of enhancing their understanding as to how Bayesian probability theory can be applied in these research and engineering fields.
