OpenCV Python vs. Swift

1. 作业项目：将客户端的 Python + OpenCV 程式移植到 Swift + OpenCV Native

2. OpenCV Native 安裝

   1. Install 参考 https://vovkos.github.io/doxyrest-showcase/opencv/sphinx_rtd_theme/page_tutorial_ios_install.html
   2. Xcode-select 需先安装
   3. brew 需先安装
   4. brew install cmake
   5. sudo xcode-select -s /Applications/Xcode.app/Contents/Developer (optional)
   6. 安装完成后会有一个 opencv2.framework，将它加入 swift project 中

3. Python OpenCV 与 OpenCV Native 使用上的不同点

   1. 在 Python 一般很常用 Numpy，但在 OpenCV Native 中要用 Mat 代替一部分工作
   2. Syntax, Python 基本没有类型，Swift 强制类型，转换起来很烦躁
   3. Swift -> Objective-C -> C++ 可能会需要共且引用时在资料传送上很困难
   4. 当用到 OpenCV 以外的 package (如skimage, PIL) 时要想法找替代方案
   5. Matrix 的运算问题
   6. 好处是 Swift 语法很大程度上与 Python 很接近 ex: print 基本可以直接套用

4. 使用 OpenCV Native, 在 Python 统一在 cv2 这个套件, OpenCV Native 分散在不同套件中

   1. import opencv2
   2. Mat
   3. Scalar
   4. Imgproc
   5. Core
5. 需自行实践的功能
   1. from skimage.metrics import structural_similarity

      使用 Imgproc.matchTemplate 替代

   2. 裁剪

      static func cropped(frame: Mat, point: Point, w: Int32) -> Mat {     let py = Point(x: point.x - w, y: point.y - w)     let px = Point(x: point.x + w, y: point.y + w)     let rect = Rect(point: py, point: px)     return Mat(mat: frame, rect: rect).clone() }

   3. 存档 Mat, 首先要将 Mat 转化为 UIImage
                  来源 https://vovkos.github.io/doxyrest-showcase/opencv/sphinx_rtd_theme/page_tutorial_image_manipulation.html
      从 C++ 转为 Swift

      static func UIImageFromCVMat(mat: Mat) -> UIImage {     let colorSpace: CGColorSpace     if (mat.elemSize() == 1) {             colorSpace = CGColorSpaceCreateDeviceGray()         } else {             colorSpace = CGColorSpaceCreateDeviceRGB()         }     guard let providerRef = CGDataProvider(data: NSData(bytes: mat.dataPointer(), length: mat.elemSize()*mat.total())) else {         debugPrint("mat.empty()")         return UIImage()     }     let mapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.none.rawValue)     let imageRef = CGImage.init(         width: Int(mat.cols()),         height: Int(mat.rows()),         bitsPerComponent: 8,         bitsPerPixel: 8 * mat.elemSize(),         bytesPerRow: Int(mat.step1()),         space: colorSpace,         bitmapInfo: mapInfo,         provider: providerRef,         decode: nil,         shouldInterpolate: true,         intent: .defaultIntent             )     let finalImage = UIImage(cgImage: imageRef!)          return finalImage; }

   4. sum()

      func sum(array1: Array<Int>, p: Int) -> Int {     let x = p     if x >= array1.count {         return array1.reduce(0, +)     } else if x==0 {         return array1.reduce(0, +)     }     let slice: ArraySlice<Int> = array1[0...x]     return slice.reduce(0, +) }

   5. 计算三个点形成的角度
      来源 https://stackoverflow.com/questions/3486172/angle-between-3-points
      从 C++ 转为 Swift

      func get_angle(a: Point , b: Point, c: Point) -> Int32 {     let ab: Point = Point(x: b.x - a.x, y: b.y - a.y)     let cb: Point = Point(x: b.x - c.x, y: b.y - c.y)          let dot: Double = Double(ab.x * cb.x + ab.y * cb.y)     let cross: Double = Double(ab.x * cb.y - ab.y * cb.x)          let alpha: Double = atan2(cross, dot);          let ret = Int32(floor(alpha * 180.0 / Double.pi + 0.5))     if c.y < b.y {         return 360 - ret     }          return ret  }

   6. 求最远的点
      来源 https://www.geeksforgeeks.org/maximum-distance-between-two-points-in-coordinate-plane-using-rotating-calipers-method/
      从 C++ 转 Swift

      private func dist(p1: Point, p2: Point) -> Int32 {     let x0: Int32 = p1.x - p2.x     let y0: Int32 = p1.y - p2.y     return x0 * x0 + y0 * y0 }   // Function to find the maximum // distance between any two points // src : https://www.geeksforgeeks.org/maximum-distance-between-two-points-in-coordinate-plane-using-rotating-calipers-method/ func furthest_node(p: Point, array: Array<Point>) -> Point {     var largest: Int = 0;     var arr: Dictionary<Int32, Int> = [:]       if array.count == 0 {         debugPrint("empty array", array.count)         return Point(x: -1, y: -1)     }     // Iterate over all possible pairs     for i in 0...array.count-1 {         // Update max         let x = dist(p1: p, p2: array[i])         arr.updateValue(i, forKey: x)         largest = max(largest, Int(x))     }     largest = arr[Int32(largest)] ?? 0     return array[largest] }

6. 使用到的 OpenCV function

   Core.bitwise_and()	计算两个数组的按位连接 (dst = src1 & src2) 计算 two arrays or an array and a scalar
   Core.bitwise_or()	计算 two arrays or an array and a scalar
   Core.bitwise_xor()	在two arrays or an array and a scalar上计算每个元素的按位“异或”运算。
   Core.findNonZero()	返回非零像素的位置列表
   Mat.zeros()	返回零像素的 Mat
   Imgproc.ellipse()	Mat.zeros()  返回零像素的 Mat Imgproc.ellipse()
   Imgproc.cvtColor()	将图像从一种颜色转换为另一种颜色。
   Imgproc.threshold()	对每个数组元素应用固定级别的阈值。
   Imgproc.findContours()	在二值图像中查找轮廓。
   Imgproc.erode()	通过使用特定的结构元素腐蚀图像。
   Imgproc.resize()	调整图像大小。
   Imgproc.HoughCircles()	使用霍夫变换在灰度图像中查找圆。

参考资料

        opencv2 Reference : http://xtravision.stars.ne.jp/opencv44/docs/index.html

        Installation in iOS : Installation in iOS — OpenCV Documentation

        OpenCV iOS Image Processing : OpenCV iOS - Image Processing — OpenCV Documentation

        Crop Mat image in OpenCV : Crop Mat image in OpenCV 2.4.3 (iOS) - Stack Overflow

        OpenCV的基本矩阵操作与示例 : https://www.itread01.com/content/1548511039.html

        Finding sum of elements in Swift array : Finding sum of elements in Swift array - Stack Overflow

        Maximum distance between two points : Maximum distance between two points in coordinate plane using Rotating Caliper's Method - GeeksforGeeks

        线性代数（3）矩阵与向量的乘积的两种理解 : 线性代数（3）矩阵与向量的乘积的两种理解_洪流之源-CSDN博客_矩阵乘向量

        Angle between 3 points? : https://stackoverflow.com/questions/3486172/angle-between-3-points