简单说明
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
突发奇想(实则作死),想要通过视觉识别进行扫雷。放弃了赶课设,断断续三天时间进行测试,相对也能接受那么一点。采用测试的扫雷是win10商城里面的免费扫雷初级模式。视觉识别通过halcon,本来打算用python,后来果断放弃(目前我所了解的halcon看调参效果相对好一点,直观);导出的数据通过matlab进行矩阵操作求解。
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![b00a4fae6b361c9c7b37c528f3da7c1d.png](https://i-blog.csdnimg.cn/blog_migrate/e1893e9c330fd2f3cce23cf2f3847ba0.png)
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![5c26c5661ad09d0b9cb045c6685c0308.png](https://i-blog.csdnimg.cn/blog_migrate/01b2f95b5cb5f50c543b23f69668e91d.png)
halcon视觉处理
****************************************halcon代码识别不好 带*全注释 长***为分段便于阅读*********初级*********扫雷***************read_image (Image, 'G:/python/saolei.png')*read_image (Image, 'G:/python/data/ceshi.png')read_image (Image, 'G:/python/data/ceshi30.png')get_image_size (Image, Width, Height)dev_close_window ()dev_open_window (0, 0, 540, 540*Height/Width, 'black', WindowHandle)dev_display (Image)**********整体区域划分***********decompose3 (Image, R, G, B)rgb1_to_gray (Image, GrayImage)threshold (GrayImage, Regions, 132, 255)fill_up (Regions, RegionFillUp)connection (RegionFillUp, ConnectedRegions)shape_trans (ConnectedRegions, RegionTrans1, 'rectangle1')select_shape (RegionTrans1, SelectedRegions, 'area', 'and', 3766.61, 12950.9)select_shape (SelectedRegions, SelectedRegions, 'row', 'and', 98.2, 1000)shape_trans (SelectedRegions, RegionTrans, 'rectangle1')sort_region (RegionTrans, SortedRegions, 'first_point', 'true', 'row')count_obj (RegionTrans, Number)****************************************获取未点击区域*********threshold (GrayImage, Regions_uclick, 134, 169)fill_up (Regions_uclick, RegionFillUp_uclick)connection (RegionFillUp_uclick, ConnectedRegions_uclick)*取出该区域select_shape (ConnectedRegions_uclick, SelectedRegions_uclick, 'rectangularity', 'and', 0.61683, 0.9567)select_shape (SelectedRegions_uclick, SelectedRegions1_uclick, ['rectangularity','row1','column1'], 'and', [0.60985,113.89,515.35], [1,852.18,1294.02])*矩形化每一个小区域 进一步并集为一个大区域shape_trans (SelectedRegions1_uclick, RegionTrans_uclick, 'rectangle1')select_shape (RegionTrans_uclick, SelectedRegions2_uclick, 'area', 'and', 5365.23, 7284.2)union1 (SelectedRegions2_uclick, RegionUnion_uclick)*****************************************训练识别文件*************123数字 -1为已经安全区域 -2为红旗 0为未知区域FontFile := 'G:/python/data/num.omc'* num := ['1', '2', '3','-2','-1','0'] * TrainFile := 'G:/python/data/num.trf'*添加文件夹下训练图片,一一对应* for Index := 1 to 6 by 1* read_image (test_img, 'G:/python/data/'+Index+'.png')* rgb1_to_gray (test_img, test_GrayImage)* append_ocr_trainf (test_img, test_GrayImage, num[Index-1]+'', TrainFile) * endfor*进行训练 此参数效果相对较好* read_ocr_trainf_names (TrainFile, CharacterNames, CharacterCount)* create_ocr_class_mlp (10, 10, 'bilinear', 'default', CharacterNames, 80, 'none', 10, 42, OCRHandle)* trainf_ocr_class_mlp (OCRHandle, TrainFile, 200, 1, 0.01, Error, ErrorLog)* write_ocr_class_mlp (OCRHandle, FontFile)* clear_ocr_class_mlp (OCRHandle)**********************************读取训练模型,创建数据存储矩阵9*9read_ocr_class_mlp (FontFile, OCRHandle) create_matrix (9, 9, 0, M)open_file ('G:/python/data/res.txt', 'output', FileHandle)*按区域进行识别数字,并分类结果存到Class 进而存储在矩阵for Index := 1 to Number by 1 select_obj (SortedRegions, ObjectSelected, Index) area_center (ObjectSelected, Area, Row, Column) reduce_domain (GrayImage, ObjectSelected, ImageReduced) do_ocr_multi_class_mlp (ObjectSelected, ImageReduced, OCRHandle, Class, Confidence) tuple_number (Class, Class) *无数字区域Confidence 0.5左右 最大0.7 if (Confidence < 0.9) Class := -1 *判断该区域中心是否在未点击区域内* test_region_point (RegionUnion_uclick, Row, Column, IsInside)* if (IsInside = 1) *在未点击区域内* Class := 0 * endif endif set_value_matrix (M, int((Index-1)/9), (Index-1)%9, Class) *数据导出到xls if (Index%9 = 0) fwrite_string (FileHandle, Class) fwrite_string (FileHandle, '\n') else fwrite_string (FileHandle, Class) fwrite_string (FileHandle, '\t') endif endforclose_file (FileHandle)**********这些效果不好,不能用***********************骨架轮廓方法* threshold (GrayImage, Regions1, 0, 98)* connection (Regions1, ConnectedRegions1)* select_shape (ConnectedRegions1, SelectedRegions3, 'rectangularity', 'and', 0, 0.11389)*骨架* skeleton (SelectedRegions3, Skeleton)* gen_contours_skeleton_xld (Skeleton, Contours, 1, 'filter')* junctions_skeleton (Skeleton, EndPoints, JuncPoints)*******分区域方法* threshold (GrayImage, Regions, 216, 234)*未点击区域* threshold (GrayImage, Regions_uclick, 134, 169)* fill_up (Regions_uclick, RegionFillUp_uclick)* connection (RegionFillUp_uclick, ConnectedRegions_uclick)*取出已点开区域* select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 5808.82, 7426.47)*取出未点击区域* select_shape (ConnectedRegions_uclick, SelectedRegions_uclick, 'rectangularity', 'and', 0.61683, 0.9567)*后面没区别* select_shape (SelectedRegions_uclick, SelectedRegions1_uclick, ['rectangularity','row1','column1'], 'and', [0.60985,113.89,515.35], [1,852.18,1294.02])* shape_trans (SelectedRegions1_uclick, RegionTrans_uclick, 'rectangle1')* select_shape (RegionTrans_uclick, SelectedRegions2_uclick, 'area', 'and', 5365.23, 7284.2)* union1 (SelectedRegions2_uclick, RegionUnion_uclick)* test_region_point (RegionUnion_uclick, 600, 600, IsInside)
游戏界面
![e510fde5124fbd1e0811345391b5cd23.png](https://i-blog.csdnimg.cn/blog_migrate/b6e76c75ec2d5fcf0d9bb22c30f46963.png)
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
识别结果
![a52883c1d0aa85a831d2da3ae418f4cb.png](https://i-blog.csdnimg.cn/blog_migrate/8bbfb2c161ac90e8be0d4d973357edf7.png)
![5c26c5661ad09d0b9cb045c6685c0308.png](https://i-blog.csdnimg.cn/blog_migrate/01b2f95b5cb5f50c543b23f69668e91d.png)
SUMMER
1° 规定数字123识别结果123;红旗为-2;未点击的方块为0;无法点击的空方块为-1。
2° 由于只要对初级进行测试(好划分区域),更换其他图片需要进行调参数。而且如果截图只保留扫雷区域效果会更好。为了方便没这样做。
3° 更改其他图片需要进行重新训练数字,训练图片相应进行更换,效果会好。训练好运行中训练过程就可以注释了。导出为文本形式,xls不知为什么导入matlab很玄学。
4° 图片那个Number参数,由于是9*9地图,测参数差不多就是看是不是为81,很崩溃。
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![5c26c5661ad09d0b9cb045c6685c0308.png](https://i-blog.csdnimg.cn/blog_migrate/01b2f95b5cb5f50c543b23f69668e91d.png)
matlab数据处理
cleardata = importdata('G:/python/data/res.txt');[m, n] = size(data);%9*9变10*10 扩充的元素为-5,不影响识别随便是几new_data(1:m+2,1:n+2) = -5;new_data(2:m+1,2:n+1) = data;%储存雷的 行列; 储存安全区域的行列lei_row = [];lei_col = [];safe_row = [];safe_col = [];while (1) %从2开始为有效区域 for i = 2:m+1 for j = 2:n+1 if (new_data(i,j)<=0) %不用管了 else %找雷 around_point = new_data(i-1:i+1,j-1:j+1);%遍历点 其周围元素 u_click = sum(around_point(:) == 0); %未点击区域的数目 flag = sum(around_point(:) == -2); %已经插红旗区域 if u_click + flag == new_data(i,j) [row,col] = find(around_point==0); %在3*3矩阵里找到雷 lei_row = [lei_row;(i -2 + row-1)]; %在原图中位置 lei_col = [lei_col;(j -2 + col-1)]; new_data((i -2 + row),(j -2 + col)) = -2; %变更数据 找到的雷插旗 end %找安全区域 类似上意思一样 flag = sum(around_point(:) == -2); if flag == new_data(i,j) [row_,col_] = find(around_point==0); safe_row = [safe_row;(i -2 + row_-1)]; safe_col = [safe_col;(j -2 + col_-1)]; new_data((i -2 + row_),(j -2 + col_)) = -1; end end end end if (isempty(row) && isempty(row_)) %循环跳出条件 当找不到雷且安全区域 跳出 %这个循环不好 每次变更数据需要从头来一遍 break endend% function special_point(i, j, data)% % end% % function return_data = general_point(i, j, data)% % return_data = data;% end
示例
![cca9c277c85ee7e19ccadde1b4d41981.png](https://i-blog.csdnimg.cn/blog_migrate/ffd99536a8c12c74b574cf31146d4d0e.png)
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
识别结果
![8d1d66149bb98dccac1911a8c5b5da91.png](https://i-blog.csdnimg.cn/blog_migrate/254291641ac4614e3855174353583373.png)
matlab处理
![504042403c3d3ef01b0cf75348c16cff.png](https://i-blog.csdnimg.cn/blog_migrate/bb670ea544f165d097452ecae2eb2aac.png)
%%这个示例貌似没找好 雷已经插旗了,只找到了安全区域 而且由于代码原因,明明很容易判断的位置 计算机要几步才可以。也没多想。。。
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![5c26c5661ad09d0b9cb045c6685c0308.png](https://i-blog.csdnimg.cn/blog_migrate/01b2f95b5cb5f50c543b23f69668e91d.png)
几点说明
SUMMER
1° 由于9*9在遍历每一个点周围元素时候,四个角落和四个边很特殊,所以想了想采用10*10矩阵,边上不管了随便填一些不影响计算的数据,中间9*9为有效的点
2° 这个计算逻辑只有两个最简单的逻辑,一看就明白的;那个需要计算机自己进行逻辑推理才能得到位置的没写,现在没想出来,也正是这个原因才选取初级难度,一般不需要逻辑推理位置。
3° 计算结果行列分开储存的,到这就有结果了,也就没进行后续在原图画出来位置这些操作。
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![5c26c5661ad09d0b9cb045c6685c0308.png](https://i-blog.csdnimg.cn/blog_migrate/01b2f95b5cb5f50c543b23f69668e91d.png)
总结
这次还是有所收获的,最起码又熟悉了一下halcon的操作,好久没玩过了。
但愿以后有机会进一步完善:
进程视频取代照片;
结果显示直观化;
训练找更好的方式,不在mlp上一直训练了,最后才想到换其他方式。。。;
添加更多的逻辑,简化循环;等等
赶课设走起!!!
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
整体数据
![7fa921060b44439d68a30823fe8c4409.png](https://i-blog.csdnimg.cn/blog_migrate/c5b97910bebf58c4d95a2c842f859892.png)
训练图片就是直接从扫雷里面截的小方格 80*80像素,训练文件还是有点少,不过毕竟好区分也还好。
![76b728f2629acff779f9d41c04669bf2.png](https://i-blog.csdnimg.cn/blog_migrate/831f6b5b6947eda14f7bf7bca36765c4.png)
![5c26c5661ad09d0b9cb045c6685c0308.png](https://i-blog.csdnimg.cn/blog_migrate/01b2f95b5cb5f50c543b23f69668e91d.png)
谢谢