前言
参考 具有直线结构保护的网格化图像拼接 这篇论文的方法,在NISwGSP算法框架上新增直线保持项,保护典型的直线结构不被弯曲。
He C, Zhou J . Mesh-based image stitching algorithm with linear structure protection[J]. Journal of Image and Graphics, 2018, 23(7): 973-983. [何川, 周军. 具有直线结构保护的网格化图像拼接[J]. 中国图象图形学报, 2018, 23(7): 973-983.][DOI: 10.11834/jig.170653]
代码地址 NISwGSP_line. 喜欢的朋友给个⭐哦
直线保持项
由于人眼对直线结构较为敏感,而NISwGSP网格优化算法的相似性变换会将直线弯曲,造成不佳的视觉效果,因此新增直线保持项,对典型的直线结构进行保护。新增的直线保持项如下:
E
l
(
V
)
=
∑
i
=
1
N
∑
l
j
i
∈
L
i
∑
l
k
∈
l
j
i
∥
v
~
(
l
k
)
−
(
(
1
−
a
)
v
~
(
l
u
)
+
a
v
~
(
l
v
)
)
∥
2
E_l({\rm V}) = \sum^N_{i=1}\sum_{l^i_j\in {\rm L}_i}\sum_{l_{k}\in {l}^{i}_j}\| \tilde{v}(l_k)-((1-a)\tilde{v}(l_u)+a\tilde{v}(l_v))\| ^2
El(V)=i=1∑Nlji∈Li∑lk∈lji∑∥v~(lk)−((1−a)v~(lu)+av~(lv))∥2其中,
L
i
{\rm L}_i
Li表示第
i
i
i张图片的所有直线集合,
l
j
i
{l}^i_j
lji表示
L
i
{\rm L}_i
Li中第
j
j
j条直线,
l
k
{l}_k
lk表示
l
j
i
{l}^i_j
lji上的采样点(除去起点
l
u
{l}_u
lu和终点
l
v
{l}_v
lv)。
v
~
(
l
)
\tilde{v}(l)
v~(l)表示采样点所在网格的四个顶点的双线性插值表达,和对齐项相同。
a
a
a为采样点在直线局部坐标系下的1维坐标。
关键代码
prepareLinePreservingTerm()
void MeshOptimization::prepareLinePreservingTerm(vector<Triplet<double> > & _triplets,
vector<pair<int, double> > & _b_vector) const {
if (line_preserving_equation.second) {
const vector<vector<vector<InterpolateVertex> > > & line_interpolation = multi_images->getLineMeshInterpolation();
const vector<vector<vector<double> > > & line_term_u = multi_images->getLineTermU();
const vector<int> & images_vertices_start_index = multi_images->getImagesVerticesStartIndex();
int eq_count = 0;
for (int i = 0; i < multi_images->images_data.size(); i++) { //每幅图片
const vector<Indices> & poly_vertices_index = multi_images->images_data[i].mesh_2d->getPolygonsIndices();
for (int j = 0; j < line_interpolation[i].size(); j++) { //每条线
for (int k = 1; k < line_interpolation[i][j].size() - 1; k++) { //采样点
for (int dim = 0; dim < DIMENSION_2D; ++dim) {
for (int p = 0; p < multi_images->images_data[i].mesh_2d->getPolygonVerticesCount(); ++p) {
_triplets.emplace_back(line_preserving_equation.first + eq_count + dim,
images_vertices_start_index[i] + dim + DIMENSION_2D * (poly_vertices_index[line_interpolation[i][j][k].polygon].indices[p]),
line_interpolation[i][j][k].weights[p] * line_preserving_weight);
_triplets.emplace_back(line_preserving_equation.first + eq_count + dim,
images_vertices_start_index[i] + dim + DIMENSION_2D * (poly_vertices_index[line_interpolation[i][j].front().polygon].indices[p]),
-(1 - line_term_u[i][j][k]) * line_interpolation[i][j].front().weights[p] * line_preserving_weight);
_triplets.emplace_back(line_preserving_equation.first + eq_count + dim,
images_vertices_start_index[i] + dim + DIMENSION_2D * (poly_vertices_index[line_interpolation[i][j].back().polygon].indices[p]),
-line_term_u[i][j][k] * line_interpolation[i][j].back().weights[p] * line_preserving_weight);
}
}
eq_count += DIMENSION_2D;
}
}
}
assert(eq_count == line_preserving_equation.second);
}
}
运行结果
基于网格优化算法,采样直线的长度应该大于 40 2 40\sqrt{2} 402。对于LSD直线检测的结果可能较为片段化,还提供了一种交互式方式,可由用户对需要保护的直线进行手动提取。
Turn on interactive mode? Press 'y'.
直线检测与采样结果:
|
|
运行结果:
|
|
|
|
|
|
|
|