The Wavelet Lifting Scheme
At the beginning of my journey, I was naive. I didn't yet know that answers vanish as one continues to travel, that there is only further complexity, that there are still more interrelationships, and more questions.
Robert D. Kaplan, The Ends of the Earth: a journey to the frontiers of anarchy, Vintage Press, 1996
Wavelets and Their Applications
Wavelets have been applied in a wide range of areas. My interest in wavelets came from digital signal processing of non-stationary time series (see my web page Applying the Haar Wavelet Transform to Time Series Information). Data sets without obviously periodic components cannot be processed well using Fourier techniques. Wavelets allow complex filters to be constructed for this kind of data which can remove or enhance selected parts of the signal. There is a growing body of literature on wavelet techniques for noise reduction.
Wavelets have been used for data compression. For example, the United States FBI compresses their fingerprint data base using wavelets. Lifting scheme wavelets also form the basis of the emerging JPEG 2000 image compression standard.
Wavelet techniques have also been used in a variety of statistical applications, including signal variance estimation, frequency analysis and kernel regression.
The coverage of most of these topics requires a book length discussion and is way beyond the scope of these humble web pages.
What These Web Pages Contribute
There is a large and growing literature on wavelets ranging from papers in mathematical journals to applied areas like artificial intelligence (neural nets) and geology. Frequently mathematics papers deal with wavelets applied to an infinite series of data. This leaves a large gap between theory and practice. In the case of papers that apply wavelets, I often get the impression that many of the authors are using existing wavelet functions in Mathematica or MathLab. These articles often include the standard integral expression for wavelets on an infinite data series, but the complexities of wavelet implementation are not discussed.
The fact that the authors of a journal article applying wavelet analysis to the study of the geology of karsts, aquifers and water flow do not implement the wavelet algorithms themselves is understandable. I have spent months studying wavelets and I still feel that I have far to go. I am sure that there are are mathematical depths to wavelets that I will never understand. Compared with theoretical papers on wavelets, there is much less material on applying wavelets to real problems. As someone who has implemented wavelet algorithms in Java and C++, I have found the gap between theory and application frustrating. In the few cases where I have found wavelet software source code, it has provided little illumination. The software was not clearly written and I did not understand it until I already understood the wavelet algorithm that it implemented.
Wavelet techniques are remarkably powerful and general. Applying wavelet techniques is also surprisingly complicated (there are times when I find myself longing to the simplicity of the Fourier transform). I hope that this web page will fill some of the gap between wavelet theory and application. I am a software engineer with rather modest skills in mathematics. This web page provides an applied view of wavelet algorithms through the lens of the Lifting Scheme. I have tried to provide clearly written software that implements the algorithms I discuss.
The mathematically sophisticated may look at this web page as a "kindergarten" view of wavelets and mathematics. In most cases I list even simple algebraic steps in deriving an equation. This reflects the humble skills of the author of these web pages. For those who find the material on these web pages trivial, I suggest going directly to the wavelet literature where the reader may find far more sophistication.
Sadly this web page is drastically incomplete. In such a short discussion I cannot completely cover the wavelet mathematics which underlies these algorithms. To do so would require a book, not a web page or two. These web pages are best read along with the references listed at the end.
Perfect Reconstruction in an Finite World
One of the features of wavelets that is critical in areas like signal processing and compression is what is referred to in the wavelet literature as perfect reconstruction. A wavelet algorithm has perfect reconstruction when the inverse wavelet transform of the result of the wavelet transform yields exactly the original data set:
IWT( WT( D ) ) = D(Here IWT is the inverse wavelet transform and WT is the wavelet transform.)
If the wavelet literature is any guide, when mathematicians think about and write about wavelets equations, they think about wavelets applied to an infinite sequence of data. Many wavelet equations that have the property of perfect reconstruction for infinite data sequences do not have this property for finite data sequences. Since sound files, financial time series, images and other data sets to which wavelets are applied are finite this can be a problem. There are several methods proposed in the wavelet literature for dealing with the edges of finite data sets. See for example, my discussion of the Daubechies D4 wavelet transform.
The simplest wavelet algorithm that shows perfect reconstruction is the Haar wavelet algorithm. However, Haar wavelets can miss detail and do not always represent change at all resolution scales. See Limitations of the Haar Wavelet Transform.
Relative to the Haar wavelet algorithm, the Daubechies D4 algorithm shows better multi-scale resolution, at the cost of a more complex algorithm. Wim Sweldens' tutorialBuilding Your Own Wavlets at Home (see references, below) showed how Haar wavelets could be expressed in Lifting Scheme form and extended to provide accurate multiscale resolution. The Lifting Scheme provides a simpler view of many wavelet algorithms. The Lifting Scheme provides the basis for the book Ripples in Mathematics(referenced below). This book is the primary reference for these web pages.
Lifting Scheme Sub-pages
-
Predict Wavlets: wavelets viewed as compression
The wavelet lifting scheme divides the wavelet transform into a set of steps. One of the elegant qualities of wavelet algorithms expressed via the listing scheme is the fact that the inverse transform is a mirror of the forward transform. The simplest way to start thinking about the lifting scheme is via a one step wavelet that I refer to as a "predict wavelet".
-
"Predict" wavelets are a good model for how wavelet algorithms can approximate a set of data with a function like polynomial interpolation. However, "predict" wavelets are missing one of the central features of wavelet algorithms: the data is not represented at multiple resolutions. By not performing an averaging step (called an update step in the lifting scheme), the differences stored in the predict step are larger than they need to be. Wavelet based algorithms like noise reduction (a.k.a. signal estimation) do not work well without an averaging step. The Basic Lifting Scheme Wavelets web page shows how each of the algorithms described by the "predict" wavelet page can be implemented as full wavelet algorithms with an averaging (update) step.
-
Lifting Scheme version of the Daubechies D4 Transform
The first part of this web page discusses the "standard" version of the Daubechies D4 transform. The second part of the web page covers the lifting scheme version of the Daubechies D4 transform. The Lifting Scheme version is taken from Ripples in Mathematics.
References
-
Ripples in Mathematics: the Discrete Wavelet Transform by Arne Jense and Anders la Cour-Harbo, Springer, 2001
This is the central reference for these web pages. Of the seven or so wavelet books I've worked with this is the best for my purposes (implementing wavelet algorithms). The book is clearly written and is light on proofs. The authors take an applied approach to wavelets, rather than approaching wavelet from a theoretical mathematics point of view.
-
Gabriel Fernandez has published an excellent bibliography on the lifting scheme wavelets which can be found here. This bibliography has a pointer to Wim Sweldens' and Peter Schroder's lifting scheme tutorial Building Your Own Wavelets at Home.
Building Your Own Wavelets at Home is part of a course on Wavelets in Computer Graphics given at SigGraph 1994, 1995 and 1996. The sigGraph course covered an amazing amount of material. Building Your Own Wavelets at Home was apparently covered in a morning. There are a lot of mathematically gifted people in computer graphics. But even for these people, this looks like tough going for a morning. I've spent hours reading and rereading this tutorial before I understood it enough to implement the polynomial interpolation wavelets that it discusses. Although I found this tutorial a lot of work, it provided the central reference for the polynomial interpolation wavelets I discuss on my wavelet web pages.
Web Links
-
Ripples in Mathematics web page.
This is the web page by the authors of Ripples in Mathematics. It includes links to a number of sub-pages, including pages that publish C source code for the algorithms discussed in the book.
-
The JPEG 2000 standard is based on Lifting Scheme wavelets. The JJ2000 web site contains links to various publications and reference Java code. I've had a hard time understanding which Lifting Scheme algorithm was used.
For an industry standard it seems remarkably difficult to get any clear information on the standard. I could find very little information that explained the standard on thejpeg.org web site. The publications listed on the JJ2000 site were better.
There is a book out, JPEG2000: Image Compression Fundamentals, Standards and Practice by Taubman and Marcellin, Kluwer Academic Publishers, November 2001. At the time of this writing this book was listed on Amazon for $125. Only those actually implementing the JPEG 2000 algorithm are likely to pay for a book this expensive.
Local Links
-
Wavelets and Signal Processing
Main page for wavelets, the Fourier transform and signal processing.
-
This Web page collects all the links that allow the Java software source that implements the wavelet algorithms discussed on these web pages to be downloaded.
Ian Kaplan
The Wavelet Lifting Scheme
At the beginning of my journey, I was naive. I didn't yet know that answers vanish as one continues to travel, that there is only further complexity, that there are still more interrelationships, and more questions.
Robert D. Kaplan, The Ends of the Earth: a journey to the frontiers of anarchy, Vintage Press, 1996
Wavelets and Their Applications
Wavelets have been applied in a wide range of areas. My interest in wavelets came from digital signal processing of non-stationary time series (see my web page Applying the Haar Wavelet Transform to Time Series Information). Data sets without obviously periodic components cannot be processed well using Fourier techniques. Wavelets allow complex filters to be constructed for this kind of data which can remove or enhance selected parts of the signal. There is a growing body of literature on wavelet techniques for noise reduction.
Wavelets have been used for data compression. For example, the United States FBI compresses their fingerprint data base using wavelets. Lifting scheme wavelets also form the basis of the emerging JPEG 2000 image compression standard.
Wavelet techniques have also been used in a variety of statistical applications, including signal variance estimation, frequency analysis and kernel regression.
The coverage of most of these topics requires a book length discussion and is way beyond the scope of these humble web pages.
What These Web Pages Contribute
There is a large and growing literature on wavelets ranging from papers in mathematical journals to applied areas like artificial intelligence (neural nets) and geology. Frequently mathematics papers deal with wavelets applied to an infinite series of data. This leaves a large gap between theory and practice. In the case of papers that apply wavelets, I often get the impression that many of the authors are using existing wavelet functions in Mathematica or MathLab. These articles often include the standard integral expression for wavelets on an infinite data series, but the complexities of wavelet implementation are not discussed.
The fact that the authors of a journal article applying wavelet analysis to the study of the geology of karsts, aquifers and water flow do not implement the wavelet algorithms themselves is understandable. I have spent months studying wavelets and I still feel that I have far to go. I am sure that there are are mathematical depths to wavelets that I will never understand. Compared with theoretical papers on wavelets, there is much less material on applying wavelets to real problems. As someone who has implemented wavelet algorithms in Java and C++, I have found the gap between theory and application frustrating. In the few cases where I have found wavelet software source code, it has provided little illumination. The software was not clearly written and I did not understand it until I already understood the wavelet algorithm that it implemented.
Wavelet techniques are remarkably powerful and general. Applying wavelet techniques is also surprisingly complicated (there are times when I find myself longing to the simplicity of the Fourier transform). I hope that this web page will fill some of the gap between wavelet theory and application. I am a software engineer with rather modest skills in mathematics. This web page provides an applied view of wavelet algorithms through the lens of the Lifting Scheme. I have tried to provide clearly written software that implements the algorithms I discuss.
The mathematically sophisticated may look at this web page as a "kindergarten" view of wavelets and mathematics. In most cases I list even simple algebraic steps in deriving an equation. This reflects the humble skills of the author of these web pages. For those who find the material on these web pages trivial, I suggest going directly to the wavelet literature where the reader may find far more sophistication.
Sadly this web page is drastically incomplete. In such a short discussion I cannot completely cover the wavelet mathematics which underlies these algorithms. To do so would require a book, not a web page or two. These web pages are best read along with the references listed at the end.
Perfect Reconstruction in an Finite World
One of the features of wavelets that is critical in areas like signal processing and compression is what is referred to in the wavelet literature as perfect reconstruction. A wavelet algorithm has perfect reconstruction when the inverse wavelet transform of the result of the wavelet transform yields exactly the original data set:
IWT( WT( D ) ) = D(Here IWT is the inverse wavelet transform and WT is the wavelet transform.)
If the wavelet literature is any guide, when mathematicians think about and write about wavelets equations, they think about wavelets applied to an infinite sequence of data. Many wavelet equations that have the property of perfect reconstruction for infinite data sequences do not have this property for finite data sequences. Since sound files, financial time series, images and other data sets to which wavelets are applied are finite this can be a problem. There are several methods proposed in the wavelet literature for dealing with the edges of finite data sets. See for example, my discussion of the Daubechies D4 wavelet transform.
The simplest wavelet algorithm that shows perfect reconstruction is the Haar wavelet algorithm. However, Haar wavelets can miss detail and do not always represent change at all resolution scales. See Limitations of the Haar Wavelet Transform.
Relative to the Haar wavelet algorithm, the Daubechies D4 algorithm shows better multi-scale resolution, at the cost of a more complex algorithm. Wim Sweldens' tutorialBuilding Your Own Wavlets at Home (see references, below) showed how Haar wavelets could be expressed in Lifting Scheme form and extended to provide accurate multiscale resolution. The Lifting Scheme provides a simpler view of many wavelet algorithms. The Lifting Scheme provides the basis for the book Ripples in Mathematics(referenced below). This book is the primary reference for these web pages.
Lifting Scheme Sub-pages
-
Predict Wavlets: wavelets viewed as compression
The wavelet lifting scheme divides the wavelet transform into a set of steps. One of the elegant qualities of wavelet algorithms expressed via the listing scheme is the fact that the inverse transform is a mirror of the forward transform. The simplest way to start thinking about the lifting scheme is via a one step wavelet that I refer to as a "predict wavelet".
-
"Predict" wavelets are a good model for how wavelet algorithms can approximate a set of data with a function like polynomial interpolation. However, "predict" wavelets are missing one of the central features of wavelet algorithms: the data is not represented at multiple resolutions. By not performing an averaging step (called an update step in the lifting scheme), the differences stored in the predict step are larger than they need to be. Wavelet based algorithms like noise reduction (a.k.a. signal estimation) do not work well without an averaging step. The Basic Lifting Scheme Wavelets web page shows how each of the algorithms described by the "predict" wavelet page can be implemented as full wavelet algorithms with an averaging (update) step.
-
Lifting Scheme version of the Daubechies D4 Transform
The first part of this web page discusses the "standard" version of the Daubechies D4 transform. The second part of the web page covers the lifting scheme version of the Daubechies D4 transform. The Lifting Scheme version is taken from Ripples in Mathematics.
References
-
Ripples in Mathematics: the Discrete Wavelet Transform by Arne Jense and Anders la Cour-Harbo, Springer, 2001
This is the central reference for these web pages. Of the seven or so wavelet books I've worked with this is the best for my purposes (implementing wavelet algorithms). The book is clearly written and is light on proofs. The authors take an applied approach to wavelets, rather than approaching wavelet from a theoretical mathematics point of view.
-
Gabriel Fernandez has published an excellent bibliography on the lifting scheme wavelets which can be found here. This bibliography has a pointer to Wim Sweldens' and Peter Schroder's lifting scheme tutorial Building Your Own Wavelets at Home.
Building Your Own Wavelets at Home is part of a course on Wavelets in Computer Graphics given at SigGraph 1994, 1995 and 1996. The sigGraph course covered an amazing amount of material. Building Your Own Wavelets at Home was apparently covered in a morning. There are a lot of mathematically gifted people in computer graphics. But even for these people, this looks like tough going for a morning. I've spent hours reading and rereading this tutorial before I understood it enough to implement the polynomial interpolation wavelets that it discusses. Although I found this tutorial a lot of work, it provided the central reference for the polynomial interpolation wavelets I discuss on my wavelet web pages.
Web Links
-
Ripples in Mathematics web page.
This is the web page by the authors of Ripples in Mathematics. It includes links to a number of sub-pages, including pages that publish C source code for the algorithms discussed in the book.
-
The JPEG 2000 standard is based on Lifting Scheme wavelets. The JJ2000 web site contains links to various publications and reference Java code. I've had a hard time understanding which Lifting Scheme algorithm was used.
For an industry standard it seems remarkably difficult to get any clear information on the standard. I could find very little information that explained the standard on thejpeg.org web site. The publications listed on the JJ2000 site were better.
There is a book out, JPEG2000: Image Compression Fundamentals, Standards and Practice by Taubman and Marcellin, Kluwer Academic Publishers, November 2001. At the time of this writing this book was listed on Amazon for $125. Only those actually implementing the JPEG 2000 algorithm are likely to pay for a book this expensive.
Local Links
-
Wavelets and Signal Processing
Main page for wavelets, the Fourier transform and signal processing.
-
This Web page collects all the links that allow the Java software source that implements the wavelet algorithms discussed on these web pages to be downloaded.
Ian Kaplan
290
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
