算法简介

算法的核心思想为：使用 DoG 模型提取图像边缘信息并与原图相加来获得细节增强的效果[1]。

output(x,y)=input(x,y)+(input∗DoGσc,σs,k)(x,y)output(x,y) = input(x,y) + (input * DoG_{\sigma_c,\sigma_s,k})(x,y)

其中：

DoGσc,σs,k=max(0,12πσc2exp(−x2+y22σc2)−k×12πσs2exp(−x2+y22σs2))DoG_{\sigma_c,\sigma_s,k} = max(0,\frac{1}{2\pi\sigma_c^2}exp(-\frac{x^2+y^2}{2\sigma_c^2}) - k\times \frac{1}{2\pi\sigma_s^2}exp(-\frac{x^2+y^2}{2\sigma_s^2}))

为了能够在增强细节的同时不放大噪声，依据图像的梯度信息对 k 的取值进行自适应化。图像的梯度越大时，k 的取值越小。理论上可以实现对图像的每一个像素都使用不同 k 值的 DoG 模型进行运算，但这会使得运算的时间代价特别大。因此，我们对梯度图量化到 N ，这样只需要 N 个DoG 模型作用到原图像上，可有效减少计算的时间花费[2]。

首先，梯度图可以使用如下公式进行计算：

G(x)=Gh(x)2+Gv(x)2G(x) = \sqrt{G_h(x)^2 + G_v(x)^2}

其中，GhG_h 和 GvG_v 分别为输入图像使用 3×33 \times 3 的 Sobel 算子计算的结果。我们假设在一个局部区域内图像的细节丰富程度是相同的，因此，在对梯度图进行量化操作之前，首先对梯度图进行扩张（dilate）操作[3]。

我们通过对扩张后的梯度图按其强度所在的区间进行标记来实现量化操作，例如，当梯度图的强度在 [0 , 0.2] 范围内时，将该点标记为 1 ；当梯度图的强度在 [0.2 , 0.4] 范围内时，将其标记为 2 …… 以此类推。接下来对输入图像进行 DoG 滤波，当该像素点的标记为 1 时，此处 DoG 模型的 k 值 取 1 ；当该像素点的标记为 2 时 ，此处 DoG 模型的 k 值取 0.95 …… 当该像素点的标记为 5 时，此处的 DoG 模型的 k 值取 0.8。

最后，将自适应 DoG 模型提取的细节图与原图相加，在不放大噪声的情况下实现细节增强。

PS：对于像素值超过 1 的情况，可以使用线性拉伸，非线性拉伸，裁剪等操作将其缩放到 [0 , 1]

Algorithm Overview

The core idea of the algorithm is to use the DoG model to extract edge information from the image and add it to the original image to achieve detail enhancement[1].

output(x,y)=input(x,y)+(input∗DoGσc,σs,k)(x,y)output(x,y) = input(x,y) + (input * DoG_{\sigma_c,\sigma_s,k})(x,y)

where:

DoGσc,σs,k=max(0,12πσc2exp(−x2+y22σc2)−k×12πσs2exp(−x2+y22σs2))DoG_{\sigma_c,\sigma_s,k} = max(0,\frac{1}{2\pi\sigma_c^2}exp(-\frac{x^2+y^2}{2\sigma_c^2}) - k\times \frac{1}{2\pi\sigma_s^2}exp(-\frac{x^2+y^2}{2\sigma_s^2}))

In order to enhance details without amplifying noise, the value of k is adapted based on the gradient information of the image: the larger the image gradient, the smaller the value of k. In theory, a DoG model with a different k value can be applied to every pixel of the image, but this would be extremely time-consuming. Therefore, we quantize the gradient image to N levels, so that only N DoG models need to be applied to the original image, which effectively reduces the computational cost[2].

First, the gradient map can be computed with the following formula:

G(x)=Gh(x)2+Gv(x)2G(x) = \sqrt{G_h(x)^2 + G_v(x)^2}

where GhG_h and GvG_v are the results of applying the 3×33 \times 3 Sobel operator to the input image. We assume that the level of detail richness is the same within a local region, so before quantizing the gradient map, a dilation (dilate) operation is first applied to the gradient map[3].

We implement the quantization by labeling the dilated gradient map according to the interval in which its intensity falls. For example, when the gradient intensity is in the range [0 , 0.2], the point is labeled 1; when it is in the range [0.2 , 0.4], it is labeled 2 … and so on. Next, DoG filtering is applied to the input image: when the pixel is labeled 1, the k value of the DoG model here is 1; when the pixel is labeled 2, the k value is 0.95 … when the pixel is labeled 5, the k value is 0.8.

Finally, the detail map extracted by the adaptive DoG model is added to the original image to achieve detail enhancement without amplifying noise.

PS: For pixels whose values exceed 1, operations such as linear stretching, nonlinear stretching, and clipping can be used to scale them to [0 , 1]

Matlab实现

二维非对称高斯函数

Matlab Implementation

2D Asymmetric Gaussian Function

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
function gauss = GaussFun2D(xsigma,ysigma)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 二维非对称高斯函数
% 参数列表：
%    xsigma:水平方向标准差 
%    ysigma:竖直方向标准差（缺省值为xsigma）
% 高斯核的直径为标准差的 6 倍
% GaussFun2D(0.5) 等效于 fspecial('gaussian',[3,3],0.5)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if nargin == 1
    ysigma = xsigma;
end

xsize = xsigma * 6;
ysize = ysigma * 6;

if mod(round(xsize),2) == 0
    if xsize < round(xsize)
        xsize = floor(xsize);
    elseif xsize > round(xsize)
        xsize = ceil(xsize);
    else
        xsize = xsize + 1;
    end
else
    xsize = round(xsize);
end
if mod(round(ysize),2) == 0
    if ysize < round(ysize)
        ysize = floor(ysize);
    elseif ysize > round(ysize)
        ysize = ceil(ysize);
    else
        ysize = ysize + 1;
    end
else
    ysize = round(ysize);
end

gauss = zeros(xsize,ysize);

x_center = (xsize + 1)/2;
y_center = (ysize + 1)/2;
A = 1/(2*pi*xsigma*ysigma);
for i = 1:xsize
    for j = 1:ysize
        gauss(i,j) = A*exp(-0.5*( ((i-x_center)/xsigma).^2 + ((j-y_center)/ysigma).^2  ));
    end
end

gauss = gauss ./ sum(gauss(:));
end

DoG 模型

DoG Model

1
2
3
4
5
6
7
function output = MyDoG(input,sigma_c,sigma_s,k)
    gauss_c = MyGaussFun2D(sigma_c);
    gauss_s = MyGaussFun2D(sigma_s);
    c = imfilter(input,gauss_c);
    s = imfilter(input,gauss_s);
    output = max(0,c - k .* s);
end

梯度图计算

Gradient Map Computation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
function [Grad] = getGrad(I, win)
r0 = 0;
r1 = 1;
Strch = @(x) (x-min(x(:))).*((r1-r0)/(max(x(:))-min(x(:)))) + r0;
I = im2double(I);
gray = rgb2gray(I);
[Gmag, ~] = imgradient(gray, 'sobel');

radius = win;
se = strel('square', radius); % ???

imG = imdilate(Gmag, se); % ???
Grad = Strch(imfill(imG, 8, 'holes'));
end

量化操作

Quantization

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
function label = Quantisation(input,N)

[m,n] = size(input);
label = ones(m,n);
levels = [0:1/N:1];

for i = 1:N
    for j = 1:m*n
        if input(j) > levels(i) && input(j) <= levels(i+1)
            label(j) = i;
        end
    end
end
label = int32(label);
end

数据归一化

Data Normalization

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
function [outputs] = Normalize(arg,str)
if nargin == 1
    str = 'Line';
end
if strcmp(str,'Line')
MAX = max(arg(:));
MIN = min(arg(:));
outputs = (arg - MIN)./ (MAX - MIN);
end
if strcmp(str,'sigmoid')
    u = mean(arg(:));
    s = std(arg(:));
    tmp = (arg - u)./ s;
    outputs = 1 ./ (1 + exp(-tmp));  
end
if strcmp(str,'log')
    alpha = max(arg(:)) + 1;
    outputs = log(arg + 1) ./ log(alpha);
end
end

细节增强

Detail Enhancement

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
function output = DetailEnhancement(input)
%% 参数
N = 5;
k = 1:-0.05:0.8;
win = 15;

%%
I = im2double(input);
grad = GetGrad(I,win);
label = Quantisation(grad,N);

for i = 1:N
    dog{i} = MyDoG(I,0.5,1.5,k(i));
end
dog_out = zeros(size(I));
for i = 1:size(I,1)
    for j = 1:size(I,2)
        dog_out(i,j,:) = dog{label(i,j)}(i,j,:);
    end
end
output = I + dog_out;

output = Normalize(output,'sigmoid');
% output = Normalize(output,'Line');
% output = Normalize(output,'log');
% output = min(1,output);
end

实验结果

原图

扩张后的梯度图

量化标记结果

label

DoG 模型的结果图

其中，σc=0.5,σs=1.5\sigma_c = 0.5 , \sigma_s = 1.5 。

k = 1 时

k = 0.95 时

k = 0.9 时

k = 0.85 时

k = 0.8 时

k 值自适应

dog_out

最终结果图

output

嗯… 确实和原图看不出任何差距，但也许在客观评价上有用哦，参考[1].

我的想法

这是我融合了几篇论文得出来的想法，虽然在主观上看不出细节的增强，但是也许在客观评价上有用。整个框架看起来复杂程度也还算够，可以作为图像增强或去雾框架的后处理步骤。

目前关于水下图像增强框架的想法是：首先一个颜色校正，接一个基于成像模型的去雾，最后在加上这里的细节增强，应该可以发一篇水…平很高的论文吧😉

其实算法简介里关于这种做法的解释还不能保证有道理，比如这句 “不放大噪声的同时增强细节” ，我的做法是梯度越大就提取更多细节，但是噪声和梯度有关系吗？该想想怎么吹。

另外我只试了这一张图，其他图还需要再试试。

参考文献

[1] K. Yang, H. Li, H. Kuang, C. Li and Y. Li, “An Adaptive Method for Image Dynamic Range Adjustment,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 3, pp. 640-652, March 2019, doi: 10.1109/TCSVT.2018.2810212.

image-20210223172633885

image-20210223172709542

[2] S. Gao, M. Tan, Z. He and Y. Li, “Notice of Violation of IEEE Publication Principles: Tone Mapping Beyond the Classical Receptive Field,” in IEEE Transactions on Image Processing, vol. 29, pp. 4174-4187, 2020, doi: 10.1109/TIP.2020.2970541.

image-20210223172555767

[3] Y. Peng, K. Cao and P. C. Cosman, “Generalization of the Dark Channel Prior for Single Image Restoration,” in IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2856-2868, June 2018, doi: 10.1109/TIP.2018.2813092.

image-20210223172454926

Experimental Results

Original Image

Dilated Gradient Map

Quantization Label Result

label

DoG Model Results

where σc=0.5,σs=1.5\sigma_c = 0.5 , \sigma_s = 1.5 .

When k = 1

When k = 0.95

When k = 0.9

When k = 0.85

When k = 0.8

Adaptive k

dog_out

Final Result

output

Hmm… There is indeed no visible difference from the original image, but maybe it is useful in objective evaluation. See [1].

My Thoughts

This is an idea I came up with by fusing several papers. Although the detail enhancement cannot be seen subjectively, it may be useful in objective evaluation. The whole framework looks complex enough, and it can serve as a post-processing step for image enhancement or dehazing frameworks.

My current idea for an underwater image enhancement framework is: first a color correction, then an imaging-model-based dehazing, and finally the detail enhancement described here. That should be enough to publish a high-lev…el paper, right? 😉

Actually, the explanation of this approach in the algorithm overview is not guaranteed to be sound. For example, this phrase “enhance details without amplifying noise” — my approach extracts more details when the gradient is larger, but is there any relation between noise and gradient? I need to think about how to sell it.

Besides, I only tried this one image; the others still need to be tested.

References

[1] K. Yang, H. Li, H. Kuang, C. Li and Y. Li, “An Adaptive Method for Image Dynamic Range Adjustment,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 3, pp. 640-652, March 2019, doi: 10.1109/TCSVT.2018.2810212.

image-20210223172633885

image-20210223172709542

[2] S. Gao, M. Tan, Z. He and Y. Li, “Notice of Violation of IEEE Publication Principles: Tone Mapping Beyond the Classical Receptive Field,” in IEEE Transactions on Image Processing, vol. 29, pp. 4174-4187, 2020, doi: 10.1109/TIP.2020.2970541.

image-20210223172555767

[3] Y. Peng, K. Cao and P. C. Cosman, “Generalization of the Dark Channel Prior for Single Image Restoration,” in IEEE Transactions on Image Processing, vol. 27, no. 6, pp. 2856-2868, June 2018, doi: 10.1109/TIP.2018.2813092.

image-20210223172454926