bilateral_filter
bilateral_filter
1. 1. 1. algrithm_introduction
1.1 1.1 1.1 gaussian_linear_filter
1.1.1 1.1.1 1.1.1 guassian_function
The Gaussian Linear Filter functin is the basic filter function at spacial denoising, though not function well at edge preserving. To apply, we need to do a set of steps.
First of all, I want to introduce the gaussian function, as follow.
G
σ
(
x
)
=
e
x
p
−
x
2
2
σ
2
(1.1.1.1)
G_{\sigma}(x) = exp^{- \frac{x^2}{2\sigma^2}} \tag{1.1.1.1}
Gσ(x)=exp−2σ2x2(1.1.1.1)
maybe the unvisible expression is too difficult to understand, so we can plot gaussian function on MATLAB, code is given.
In the gragh, we can learn that guassian function empower a much larger value to the center part of the gragh, while empower a much smaller value to the surrouding areas.
f
i
g
.
1.1.1.1
fig. 1.1.1.1
fig.1.1.1.1 The right figure is guassian function plot on 2-dimension, the left fighre is guassian function plot on 3-dimension. The parameter
σ
\sigma
σ of gaussian function in left pic is
σ
=
0.2
,
0.4
,
0.6
,
0.8
\sigma = 0.2, 0.4, 0.6, 0.8
σ=0.2,0.4,0.6,0.8. The parameter
σ
\sigma
σ of guassian function in right pic is
σ
=
2
\sigma = 2
σ=2.
%% plot 2-dimension gaussian-function
x = [-5: 0.1: 5];
gaussian = @(x, sigma) exp((-x.^2) ./ (2*sigma^2));
y = zeros(size(x));
figure(1);
for i = 1: 4
y = gaussian(x, 0.2 * i);
subplot(2, 2, i);
plot(x, y);
end
%% plot 3-dimension gaussian-function
x = [-5: 0.1: 5];
map = zeros(size(x, 2), size(x, 2));
figure(2);
for i = 1: size(x, 2)
for j = 1: size(x, 2)
map(i, j) = x(i)^2 + x(j)^2;
end
end
mesh(x, x, gaussian(map, 2));
1.1.2 1.1.2 1.1.2 apply guassian function to filter
It is avaliable to get the empower map through guassian function, then we can apply it to a filter to carry some computational photogragh task like denoising. Here we gave the filter epxpression as follow.
G
σ
(
f
)
(
x
)
=
1
Z
∑
y
G
σ
(
x
−
y
)
f
(
y
)
(1.1.2.1)
G_{\sigma}(f)(x) = \frac{1}{Z}\sum_{y}{G_{\sigma}(x - y)f(y)} \tag{1.1.2.1}
Gσ(f)(x)=Z1y∑Gσ(x−y)f(y)(1.1.2.1)
In this function
Z
=
∑
y
G
σ
(
y
)
Z = \sum_{y}G_{\sigma}(y)
Z=∑yGσ(y), always we standlize all value in
f
(
y
)
f(y)
f(y) to make
Z
=
1
Z = 1
Z=1.
To understand expression
1.1.2.1
1.1.2.1
1.1.2.1 , I want to have the aid of some concept in coputational photogragh. If we see
x
x
x as target pixel, which have some noise in it. To denoise it, we can reference the surrounding pixels to repower the value to pixel
x
x
x. Moreover, we often apply surrounding pixels in a squre.
Suspend
v
a
l
m
,
n
val_{m, n}
valm,n is value matrix of pixel
x
x
x's surrounding pixels,
G
m
,
n
G_{m, n}
Gm,n is gaussian function matrix. Sum of
G
m
,
n
G_{m, n}
Gm,n equals
1
1
1.Apply
G
m
,
n
G_{m, n}
Gm,n to
v
a
l
m
,
n
val_{m, n}
valm,n, output the denoising temp martrix. After a series of steps, we can get the value of pixel
x
x
x. Details as follow.
G
u
a
s
s
i
a
n
(
x
)
m
,
n
=
v
a
l
m
,
n
.
∗
G
m
,
n
Guassian(x)_{m, n} = val_{m,n} .*G_{m, n}
Guassian(x)m,n=valm,n.∗Gm,n
x
=
s
u
m
(
G
u
a
s
s
i
a
n
(
x
)
m
,
n
)
x = sum(Guassian(x)_{m, n})
x=sum(Guassian(x)m,n)
Then we can get the
x
x
x output function, as follow:
x
=
s
u
m
(
v
a
l
m
,
n
.
∗
G
m
,
n
)
(1.1.2.2)
x = sum(val_{m, n } .* G_{m, n}) \tag{1.1.2.2}
x=sum(valm,n.∗Gm,n)(1.1.2.2)
Applying this function to all pixels, then we can get gaussian function denoising gragh.
1.1.3 1.1.3 1.1.3 accelerate gaussian function
Using FFT method, through two-dimensional fast fourier transform, we could reduce the calculating work.
Here is the method page, paper.
1.1.4 1.1.4 1.1.4 code is here
code is based on MATLAB.
% x - input data
% sigma - guassian funciton parameter
% n*n - size of gragh
gaussian = @(x, sigma) exp((-x.^2) ./ (2*sigma^2));
x = [0: n/2-1, -n/2: -1];
[Y, X] = meshgrid(x, x);
GaussianFilt = @(F, sigma) exp((-X.^2 -Y.^2) ./ (2*sigma^2));
Filter = @(F,s)real( ifft2( fft2(F).*repmat( fft2(GaussianFilt(s)), [1 1 size(F,3)])));
suggest we have a gragh whose size is n*n, we can process several steps to denoise by guassian filter(here we adopt FFT method).
% name - path of a RGB image
% n*n - size of gragh
% sigma - parameter of guassina function
f0 = imread(name);
f0 = rescale(crop(f0, 3), n);
imshow(Filter(f0, sigma));
1.2 1.2 1.2 bilateral_filter
1.2.1 1.2.1 1.2.1 basic funciton introduction
Basic function introduction is posted in this blog.
1.2.2 1.2.2 1.2.2 bilateral_filter_stacking
Based on basic bilateral filter function, it is of vital importance to accelerate the processing.
B
σ
x
,
σ
v
(
f
)
(
x
)
=
F
v
(
x
)
=
[
G
σ
x
(
f
⋅
W
v
)
]
(
x
)
[
G
σ
x
]
(
W
v
)
]
(
x
)
(1.2.2.1)
B_{\sigma_{x}, \sigma_{v}}(f)(x) = F_{v}(x) = \frac{[G_{\sigma_{x}} (f·W_{v})](x)}{[G_{\sigma_{x}]}(W_{v})](x)} \tag{1.2.2.1}
Bσx,σv(f)(x)=Fv(x)=[Gσx](Wv)](x)[Gσx(f⋅Wv)](x)(1.2.2.1)
W
v
(
x
)
=
G
σ
v
(
v
−
f
(
x
)
)
(1.2.2.2)
W_{v}(x) = G_{\sigma_{v}}(v - f(x)) \tag{1.2.2.2}
Wv(x)=Gσv(v−f(x))(1.2.2.2)
2. 2. 2. using
2.1 2.1 2.1 denoising
Ideas from Bilateral Filtering for Gray and Color Images.
2.1.1 2.1.1 2.1.1 introduction
If we treat bilateral fiter as “black box”, noise gragh can be denoise through the “black box”.
f i g 2.1.1.1 fig2.1.1.1 fig2.1.1.1 denoise processing, f f f is image with noise, f 0 f_{0} f0 is denoised image, blak box is bilateral filter.
2.1.2 2.1.2 2.1.2 code is here
Initially, we generate a noise gragh. Suspend f0 is a clear image without noise, we put some random noise on it.
% mu - put noise in a controled limit
% f - a noise gragh with noise, based on f0
mu = 0.05;
f = f0 + rand(n, n)*mu;
Then, apply bilateral filter to denoise the gragh.
% sx, sv - parameter of bilateral spacial, valued-dimensional
imshow(clamp(bilateral_lin(f, sx, sv)));
Now, processing is over.
2.2 2.2 2.2 detail_enhancement
2.2.1 2.2.1 2.2.1 introduction
Ideas from
p
a
r
t
2.1
part 2.1
part2.1, we konw that we can seperate a gragh to a 2-layer data structer. The first layer we define as base layer, which cotains the bisic contents of the image;the second layer we dined as detail layer, which contains the details of a image, sometimes noise. To enhance the detail, we need to work on the detail layer.
To seperate detail layer and base layer To understand the base layer and detail layer, there is a gragh that could help.
f
i
g
2.2.1.1
fig2.2.1.1
fig2.2.1.1 Right gragh is denoised gragh, which is processed by bilateral filter; left contains noise.
Contract to right gragh, left gragh contains lots of details. When it comes to a face picture, maybe there are some freckles in it, which is the detail we want to boost.
To get denoised gragh, jump to
p
a
r
t
2.1
part 2.1
part2.1.
To enhance detail To enhance gragh, initially is to find what to enhance. Idea from
p
a
r
t
2.1
part 2.1
part2.1. Define
d
e
t
a
i
l
detail
detail as detail layer,
f
f
f is denoised layer,
f
0
f_0
f0 is original layer with noise. Give the funciton.
d
e
t
a
i
l
=
f
−
f
0
detail = f - f_0
detail=f−f0Define
γ
\gamma
γ as enhance rate. Output enhanced gragh as
f
e
n
h
s
f_{enhs}
fenhs, give the funciton.
f
e
n
h
s
=
f
0
+
γ
∗
d
e
t
a
i
l
α
f_{enhs} = f_0 + \gamma *detail^{\alpha}
fenhs=f0+γ∗detailαAlwaysly, set
α
=
1
\alpha = 1
α=1.
Now, gragh enhancement is completed.
2.2.2 2.2.2 2.2.2 code is here
% f - input image matrix
% f0 - denoise image
% mu - enhance rate
f0 = clamp(bilateral_lin(f, sx, sv));
detail = f - f0;
im_output = f0 + mu*detail;
imshow(im_output);
2.3 2.3 2.3 HDR_range_tone
Idea from Fast Bilateral Filtering for the Display of High-Dynamic-Range Images.
2.3.1 2.3.1 2.3.1 introduction
B e g i n Begin Begin
1 s t 1st 1st, i n p u t I n t e n s i t y = 20 R + 40 G + B 61 inputIntensity = \frac{20R + 40G + B}{61} inputIntensity=6120R+40G+B
2 n d 2nd 2nd, r = R i n p u t I n t e n s i t y r = \frac{R}{inputIntensity} r=inputIntensityR; g = G i n p u t I n t e n s i t y g = \frac{G}{inputIntensity} g=inputIntensityG: b = B i n p u t I n t e n s i t y b = \frac{B}{inputIntensity} b=inputIntensityB
3 r d 3rd 3rd, b a s e l o g = B i l a t e r a l ( l o g ( I n p u t I n t e n s i t y ) ) base_{log} = Bilateral(log(InputIntensity)) baselog=Bilateral(log(InputIntensity))
4 t h 4th 4th, d e t a i l l o g = l o g ( i n p u t I n t e n s i t y ) − b a s e l o g detail_{log} = log(inputIntensity) - base_{log} detaillog=log(inputIntensity)−baselog
5 t h 5th 5th, o u t p u t I n t e n s i t y l o g = b a s e l o g ∗ c o m p r e s s i o n F a c t o r + d e t a i l l o g − a b s o l u t e S c a l e l o g outputIntensity_{log} = base_{log} * compressionFactor + detail_{log} - absoluteScale_{log} outputIntensitylog=baselog∗compressionFactor+detaillog−absoluteScalelog
5 t h 5th 5th, o u t p u t r = r ∗ 1 0 o u t p u t I n t e n s i t y l o g output_r = r * 10 ^{outputIntensity_{log}} outputr=r∗10outputIntensitylog
E n d End End
where :
c
o
m
p
r
e
s
s
i
o
o
n
F
a
c
t
o
r
=
t
r
a
g
e
t
C
o
n
t
r
a
s
t
m
a
x
(
b
a
s
e
l
o
g
)
−
m
i
n
(
b
a
s
e
l
o
g
)
compressioonFactor = \frac{tragetContrast}{max(base_{log})} - min(base_{log})
compressioonFactor=max(baselog)tragetContrast−min(baselog) Alwaysly,
t
a
r
g
e
t
C
o
n
t
r
a
c
t
=
l
o
g
(
5
)
targetContract = log(5)
targetContract=log(5)
a
b
s
o
l
u
t
e
S
c
a
l
e
l
o
g
=
m
a
x
(
b
a
s
e
l
o
g
∗
c
o
m
p
r
e
s
s
i
o
n
F
a
c
t
o
r
)
absoluteScale_{log} = max(base_{log} * compressionFactor)
absoluteScalelog=max(baselog∗compressionFactor)
To learn how to process this, use gragh as follow.
From numerical-tours, It can be learned that
A simple way to process a color image is to perform a change of color space and to process only the intensity channel.
For instance one can use a HSV color space and process the intensity channel
f
i
g
2.3.1.1
fig2.3.1.1
fig2.3.1.1 Graghs on top is HSV color space intensity for 3 channelsplot graghs. Graghs at bottom is standard RGB channels intensity plot gragh.
f
i
g
2.3.1.2
fig2.3.1.2
fig2.3.1.2 Graghs on top is HSV color space intensity for 3 channels value.
Therefore, in the following steps, our operations to details are based on the 3rd channel of HSV image, define this layer as f v f_v fv.
A popular contract modification is the γ \gamma γ correction, give the function.
ψ ( t ) = ( t + ϵ ) α (2.3.1.1) \psi(t) = (t + \epsilon)^{\alpha} \tag{2.3.1.1} ψ(t)=(t+ϵ)α(2.3.1.1)where ϵ > 0 \epsilon > 0 ϵ>0 is small, α ∈ [ 0 , 1 ] \alpha \in [0, 1] α∈[0,1].
Several other operators:
ψ ( t ) = t t + ϵ \psi(t) = \frac{t}{t + \epsilon} ψ(t)=t+ϵt
ψ ( t ) = l o g ( t + ϵ ) \psi(t) = log(t + \epsilon) ψ(t)=log(t+ϵ)
However, we process contract modification through local algrithm, which have better effect contract to above global algrithm.
To realize that, there are several steps as follow, which is referred in the above reference of algrithm introduction.
B
e
g
i
n
Begin
Begin
1
s
t
1st
1st, recompose the logarithm of
f
v
f_v
fv, and standarlize it in
[
0
,
1
]
[0, 1]
[0,1].
F
v
′
=
l
o
g
(
f
v
+
ϵ
)
F_{v}' = log(f_{v} + \epsilon)
Fv′=log(fv+ϵ)Then standarlize
f
v
f_{v}
fv,
a
a
a、
b
b
b is the minimum and maximum of
F
v
′
F_{v}'
Fv′.
F
v
=
F
v
−
a
b
−
a
F_{v} = \frac{F_{v} - a}{b - a}
Fv=b−aFv−aIn all,
F
v
=
l
o
g
(
f
v
+
ϵ
)
−
α
b
−
a
(2.3.1.2)
F_{v} = \frac{log(f_{v} + \epsilon) - \alpha}{b - a} \tag{2.3.1.2}
Fv=b−alog(fv+ϵ)−α(2.3.1.2)
2
n
d
2nd
2nd, get the base layer
Give the function,
B
v
=
B
σ
x
,
σ
v
(
F
v
)
(2.2.1.3)
B_{v} = B_{\sigma_{x}, \sigma_{v}}(F_{v}) \tag{2.2.1.3}
Bv=Bσx,σv(Fv)(2.2.1.3)
3
r
d
3rd
3rd, get the tone map
f
~
V
=
e
(
γ
B
V
+
F
V
−
B
v
)
(
b
−
a
)
+
a
−
ϵ
\tilde{f}_{V}=e^{\left(\gamma B_{V}+F_{V}-\right.B_{v})(b - a) + a} - \epsilon
f~V=e(γBV+FV−Bv)(b−a)+a−ϵ
E
n
d
End
End
Transform into RGB gragh, then output.
2.3.2 2.3.2 2.3.2 code is here
% f- input hdr gragh
p = min(size(f,1),size(f,2));
f = rescale( f(1:p,1:p,:) );
f = rescale( clamp( image_resize(f,n,n) ) );
fhsv = rgb2hsv(f);
fV = fhsv(:,:,3);
color_recompose = @(fV)hsv2rgb( cat(3, fhsv(:,:,1:2), rescale(fV) ) );
epsilon = 1e-5;
FV = log(fV+epsilon);
a = min(FV(:)); b = max(FV(:));
FV = (FV-a)/(b-a);
sx = 5; sv = .02;
FV1 = bilateral_lin(FV, sx,sv);
remap = @(FV,FV1,gamma)exp( ( gamma*FV1 + FV-FV1 ) * (b-a) + a ) - epsilon;
gamma = .1;
FVmapped = remap(FV,FV1,gamma);
imageplot(color_recompose(FVmapped), ['\gamma=' num2str(gamma)]);
official matlab hsv2rgb.m
function [rout,g,b] = hsv2rgb(hin,s,v)
%HSV2RGB Convert hue-saturation-value colors to red-green-blue.
% M = HSV2RGB(H) converts an HSV color map to an RGB color map.
% Each map is a matrix with any number of rows, exactly three columns,
% and elements in the interval 0 to 1. The columns of the input matrix,
% H, represent hue, saturation and value, respectively. The columns of
% the resulting output matrix, M, represent intensity of red, blue and
% green, respectively.
%
% RGB = HSV2RGB(HSV) converts the HSV image HSV (3-D array) to the
% equivalent RGB image RGB (3-D array).
%
% As the hue varies from 0 to 1, the resulting color varies from
% red, through yellow, green, cyan, blue and magenta, back to red.
% When the saturation is 0, the colors are unsaturated; they are
% simply shades of gray. When the saturation is 1, the colors are
% fully saturated; they contain no white component. As the value
% varies from 0 to 1, the brightness increases.
%
% The colormap HSV is hsv2rgb([h s v]) where h is a linear ramp
% from 0 to 1 and both s and v are all 1's.
%
% CLASS SUPPORT
% -------------
% If the input is an HSV image, it can be of class logical, single, or
% double. The output image is single when the input is single. For all
% other input data types, the output image is double. If the input is a
% colormap, it can be of class logical, single, or double. The output
% colormap is single when the input is single. For all other input data
% types, the output colormap is double.
%
% See also RGB2HSV, COLORMAP, RGBPLOT.
% Undocumented syntaxes:
% [R,G,B] = HSV2RGB(H,S,V) converts the HSV image H,S,V to the
% equivalent RGB image R,G,B.
%
% RGB = HSV2RGB(H,S,V) converts the HSV image H,S,V to the
% equivalent RGB image stored in the 3-D array (RGB).
%
% [R,G,B] = HSV2RGB(HSV) converts the HSV image HSV (3-D array) to
% the equivalent RGB image R,G,B.
% See Alvy Ray Smith, Color Gamut Transform Pairs, SIGGRAPH '78.
% Copyright 1984-2011 The MathWorks, Inc.
threeD = ndims(hin)==3;
if nargin == 1 % HSV colormap
if threeD
if(size(hin,3) ~= 3)
error(message('MATLAB:hsv2rgb:invalidInputSizeHSV'));
end
validateattributes(hin, {'double', 'single', 'logical'}, {'real'}, mfilename, 'HSV', 1);
h = hin(:,:,1); s = hin(:,:,2); v = hin(:,:,3);
elseif ismatrix(hin)
if(size(hin,2) ~=3)
error(message('MATLAB:hsv2rgb:invalidSizeForColormap'));
end
validateattributes(hin, {'double', 'single', 'logical'}, {'real','nonempty','nonsparse'}, mfilename, 'H');
if((any(hin(:) < 0) || any(hin(:) > 1)))
error(message('MATLAB:hsv2rgb:badMapValues'));
end
h = hin(:,1); s = hin(:,2); v = hin(:,3);
else
error(message('MATLAB:hsv2rgb:invalidInputSize'));
end
elseif nargin == 3
validateattributes(hin, {'double', 'single', 'logical'}, {'real', '2d'}, mfilename, 'HIN', 1);
validateattributes(s, {'double', 'single', 'logical'}, {'real', '2d'}, mfilename, 'S', 2);
validateattributes(v, {'double', 'single', 'logical'}, {'real', '2d'}, mfilename, 'V', 3);
if ~isequal(size(hin),size(s),size(v)),
error(message('MATLAB:hsv2rgb:InputSizeMismatch'));
end
h = hin;
else
error(message('MATLAB:hsv2rgb:WrongInputNum'));
end
h = 6.*h;
k = floor(h);
p = h-k;
t = 1-s;
n = 1-s.*p;
p = 1-(s.*(1-p));
% Processing each value of k separately to avoid simultaneously storing
% many temporary matrices the same size as k in memory
kc = (k==0 | k==6);
r = kc;
g = kc.*p;
b = kc.*t;
kc = (k==1);
r = r + kc.*n;
g = g + kc;
b = b + kc.*t;
kc = (k==2);
r = r + kc.*t;
g = g + kc;
b = b + kc.*p;
kc = (k==3);
r = r + kc.*t;
g = g + kc.*n;
b = b + kc;
kc = (k==4);
r = r + kc.*p;
g = g + kc.*t;
b = b + kc;
kc = (k==5);
r = r + kc;
g = g + kc.*t;
b = b + kc.*n;
if nargout <= 1
if nargin == 3 || threeD
rout = cat(3,r,g,b);
else
rout = [r g b];
end
if isempty(rout)
return
else
rout = bsxfun(@times, v./max(rout(:)), rout);
end
else
if isempty(r) || isempty(g) || isempty(b)
rout = r;
return
else
f = v./max([max(r(:)); max(g(:)); max(b(:))]);
end
rout = f.*r;
g = f.*g;
b = f.*b;
end