Changchang Wu-SiftGPU-Windows

API文档：

http://ccwu.me/

http://ccwu.me/vsfm/

https://www.youtube.com/user/changchangwu

https://github.com/mattjr/structured/tree/master/vsfm

http://docs.ros.org/fuerte/api/siftgpu/html/annotated.html

0.5.370：

https://roboticsclub.org/redmine/projects/quadrotor/repository/revisions/9240aaa3a4a7b485cc52b9bc21036ad83821ceee/show/rgbdslam/external/siftgpu

0.5.400

https://github.com/pitzer/SiftGPU 

 

http://www.nonrelevant.net/2011/07/siftgpu-a-gpu-implementation-of-scale-invariant-feature-transform-sift/

https://groups.google.com/forum/#!topic/vsfm/AusO1QjQ-Gk

 

Windows10+vs2013 编译SiftGPU动态库

https://blog.csdn.net/baidu_40840693/article/details/84020737

SiftGPU一些细节整理

https://blog.csdn.net/baidu_40840693/article/details/84503083

SiftGPU关于增加匹配对的数目问题

https://blog.csdn.net/baidu_40840693/article/details/84887054

ASIFT和摄像机仿射模型

https://blog.csdn.net/baidu_40840693/article/details/95185395

关于SiftGPU匹配显示的问题

https://blog.csdn.net/baidu_40840693/article/details/84553011

 

MAC-SFM：

https://github.com/iromu/vsfm-osx

https://github.com/TheFrenchLeaf/Bundler

https://github.com/pitzer/cmvs

https://github.com/pitzer/openMVG

https://github.com/pitzer/SFMToolkit

关于windows，不同的github版本上传的不一样：

https://github.com/pitzer/SiftGPU 没有第三方库

 

https://github.com/pitzer/SiftGPU/tree/9afd00b8c128bc1b8842a147f08a46bacc2b6d80

含有第三方库：对于windows而言用这个比较好：

或者用其他人的：

https://github.com/scanner-research/SiftGPU

https://github.com/stnoh/SiftGPU

https://github.com/miryamch/MarsImg

 

 

再或者：

直接用没有lib的，自己去用VS2015 VS2017编译这些lib

使用vcpkg，比如：

devil                1.8.0-4

.\vcpkg install devil[libpng,tiff,libjpeg,openexr,jasper,lcms]:x86-windows

.\vcpkg install devil[libpng,tiff,libjpeg,openexr,jasper,lcms]:x64-windows

 .\vcpkg install glfw3:x86-windows-static

 .\vcpkg install glfw3:x64-windows-static

glew                 2.1.0-6

.\vcpkg install glew

freeglut             3.0.0-7

.\vcpkg install freeglut

glfw3                3.3-3

.\vcpkg install glfw3

这里说明一下：

https://blog.csdn.net/misayaaaaa/article/details/73087464

由于GLUT的作者已经很久没更新过了（最后更新于2000年！= =），所以其他人另外做了一个接口兼容GLUT的freeglut，开源而且一直在维护中。可以理解为glut的替代库。

vcpkg编译opengl的时候出了点问题，安装VS2015没有安装SDK window10

https://github.com/microsoft/vcpkg/issues/9145

期间遇到错误：

Starting package 2/4: opengl:x64-windows
Building package opengl[core]:x64-windows...
CMake Error at ports/opengl/portfile.cmake:16 (message):
  Portfile not yet configured for Windows SDK with version:
Call Stack (most recent call first):
  scripts/ports.cmake:94 (include)


Error: Building package opengl:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: opengl:x64-windows
  Vcpkg version: 2019.09.12-nohash

关于这个错误

看到一个博客：

https://www.cnblogs.com/X-Jun/p/9028764.html

看来，我安装的时候没有选择SDK选项

vcpkg/ports/opengl/portfile.cmake

进行包源设置：

https://blog.csdn.net/cjmqas/article/details/79282847#41-%E4%BB%80%E4%B9%88%E6%98%AF%E9%9B%86%E6%88%90

原本的siftgpu是：

opengl32.lib
glu32.lib
winmm.lib
glew64.lib
glew64s.lib

现在我们的是

winmm.lib

因为vcpkg会自动帮我们寻找 

然后改一个代码中的绝对路径：

注释掉即可

然后就可以编译成功：

成功运行，但会报错

Expresion vector subscript out of range

越界访问错误：

这程序已经千锤百炼了

不可能出错，肯定我哪里有问题

果然：

路径问题///

    if(sift->RunSIFT("D:/ubuntu16.04/vcpkg/vcpkg/SiftGPU-master/data/800-1.jpg"))

其实这个程序还是有一定问题，就是最后显存释放那一块

看我前面的博客

SiftGPU一些细节整理

https://blog.csdn.net/baidu_40840693/article/details/84503083

关于SiftGPU，还有一些内存释放的问题：

我加载dll后，使用FreeLibrary无法实现实时的释放，等待很久都无果，参考网上教程，发现SiftGPU有一些内存释放的问题

参考文档：

SiftGPU (Cg/GLSL/CUDA) for Matlab：

http://pkmital.com/home/2009/08/27/siftgpu-cgglslcuda-for-matlab/

SiftGPU不自动释放GL context的bug及其修正方法：

http://blog.sina.com.cn/s/blog_4298002e01019gle.html

---

---

---

---

如果使用的是：

SiftGPU_CUDA_Enabled

那么修改

 SiftGPU_CUDA_Enabled.vcxproj中nvcc的路径：

SiftGPU_CUDA_Enabled

361行：

这里设置的是相对路径

那么我们在自己的path下设置一个：

CUDA_BIN_PATH

CUDA_LIB_PATH

CUDA_INC_PATH

 

TestWinGlut项目：

-il D:/ubuntu16.04/vcpkg/vcpkg/SiftGPU-master/data/list640.txt  

---

GPU-SIFT封装：

https://www.cnblogs.com/wangguchangqing/p/10132052.html

 

#include <SiftGPU.h>

#include <iostream>
#include <vector>

#include <opencv2/opencv.hpp>
#include <opencv2/xfeatures2d.hpp>
#include <opencv2/xfeatures2d/nonfree.hpp>

#include <GL/gl.h>

using namespace std;
using namespace cv;

class GpuFeatureDetector{

    enum InitStatus{
        INIT_OK,
        INIT_IS_NOT_SUPPORT,
        INIT_VERIFY_FAILED
    };

public:
    GpuFeatureDetector() = default;
    ~GpuFeatureDetector() {
        if(m_siftGpuDetector) delete m_siftGpuDetector;
        if(m_siftGpuMatcher)  delete m_siftGpuMatcher;
    }
    InitStatus create(){
        m_siftGpuDetector = new SiftGPU();

        char* myargv[4] = {"-fo","-1","-v","1"};
        m_siftGpuDetector->ParseParam(4,myargv);
        // Set edge threshold, dog threshold

        if(m_siftGpuDetector->CreateContextGL() != SiftGPU::SIFTGPU_FULL_SUPPORTED){
            cerr << "SiftGPU is not supported!" << endl;
            return InitStatus::INIT_IS_NOT_SUPPORT;
        }

        m_siftGpuMatcher = new SiftMatchGPU();
        m_siftGpuMatcher->VerifyContextGL();

        m_maxMatch = 4096;

        return INIT_OK;
    }

    void detectAndCompute(const Mat &img,Mat &descriptors,vector<KeyPoint> &kpts){

        assert(img.channels() == 3); // RGB

        m_siftGpuDetector->RunSIFT(img.cols,img.rows,img.data,GL_RGB,GL_UNSIGNED_BYTE);
        auto num1 = m_siftGpuDetector->GetFeatureNum();

        vector<float> des(128 * num1);
        vector<SiftGPU::SiftKeypoint> keypoints(num1);
        m_siftGpuDetector->GetFeatureVector(&keypoints[0],&des[0]);

        // Trans to Mat
        Mat m(des);
        descriptors = m.reshape(1,num1).clone();

        for(const SiftGPU::SiftKeypoint &kp : keypoints){
            KeyPoint t(kp.x,kp.y,kp.s,kp.o);
            kpts.push_back(t);
        }     
    }

    void transToRootSift(const cv::Mat &siftFeature,cv::Mat &rootSiftFeature){
        for(int i = 0; i < siftFeature.rows; i ++){
            // Conver to float type
            Mat f;
            siftFeature.row(i).convertTo(f,CV_32FC1);

            normalize(f,f,1,0,NORM_L1); // l1 normalize
            sqrt(f,f); // sqrt-root  root-sift
            rootSiftFeature.push_back(f);
        }
    }

    int gpuMatch(const Mat &des1,const Mat &des2){
        
        m_siftGpuMatcher->SetDescriptors(0,des1.rows,des1.data);
        m_siftGpuMatcher->SetDescriptors(1,des2.rows,des2.data);

        int (*match_buf)[2] = new int[m_maxMatch][2];
        
        auto matchNum = m_siftGpuMatcher->GetSiftMatch(m_maxMatch,match_buf);
        
        delete[] match_buf;
        
        return matchNum;
    }

    int gpuMatch(const Mat &des1,const Mat &des2,vector<DMatch>& matches){
        m_siftGpuMatcher->SetDescriptors(0,des1.rows,(float*)des1.data);
        m_siftGpuMatcher->SetDescriptors(1,des2.rows,(float*)des2.data);

        int (*match_buf)[2] = new int[m_maxMatch][2];
        
        auto matchNum = m_siftGpuMatcher->GetSiftMatch(m_maxMatch,match_buf);

        for(int i = 0 ;i  < matchNum; i ++) {
            DMatch dm(match_buf[i][0],match_buf[i][1],0);
            matches.push_back(dm);
        }

        delete[] match_buf;
        return matchNum;
    }
private:
    SiftGPU *m_siftGpuDetector;
    SiftMatchGPU *m_siftGpuMatcher;

    int m_maxMatch;
};

int main()
{

    /
    ///
    /// Opencv extract sift
    ///
    ///

    // Read image  
    auto detector = cv::xfeatures2d::SIFT::create();

    Mat des;
    vector<KeyPoint> kpts;

    string file1 = "/home/liqiang/Documents/shared/8.jpg";
    auto t = getTickCount();
    auto img = imread(file1);
    detector->detectAndCompute(img,noArray(),kpts,des);
    auto end = static_cast<double>(getTickCount() - t) / getTickFrequency();
    cout << "OpenCV get sift consume:" << end << endl;
    cout << "count:" << kpts.size() << endl;


    
    ///
    /// SiftGPU extract sift
    ///
    ///
    // Declare sift and initlize
    SiftGPU sift;
    char* myargv[4] = {"-fo","-1","-v","0"};
    //char* myargv[5] = { "-m", "-s", "-unpa", "1"};
    //char* myargv[4] = {"-fo", "-1", "-cuda", "0"};
    sift.ParseParam(4,myargv);

    // Check hardware is support siftGPU
    int support = sift.CreateContextGL();
    if(support != SiftGPU::SIFTGPU_FULL_SUPPORTED){
        cerr << "SiftGPU is not supported!" << endl;
        return 2;
    }

    auto img1 = imread("/home/liqiang/Documents/shared/3.jpg");
    auto img2 = imread("/home/liqiang/Documents/shared/4.jpg");
    auto img3 = imread("/home/liqiang/Documents/shared/5.jpg");
    auto img4 = imread("/home/liqiang/Documents/shared/6.jpg");
    auto img5 = imread("/home/liqiang/Documents/shared/7.jpg");

    auto f = [&sift](Mat &img,vector<float> &des,vector<SiftGPU::SiftKeypoint> &kpts){
        
        auto t = getTickCount();
        sift.RunSIFT(img.cols,img.rows,img.data,GL_RGB,GL_UNSIGNED_BYTE);
        auto num1 = sift.GetFeatureNum();
        
        des.resize(128 * num1);
        kpts.resize(num1);
        sift.GetFeatureVector(&kpts[0],&des[0]);
        cout << "=======================================" << endl;
        cout << "width x height : " << img.cols << "x" << img.rows << endl;
        cout << "Features count:" << num1 << endl;
        cout << "Extract features,consume:" << static_cast<double>(getTickCount() - t) / getTickFrequency() << endl;
    };


    vector<float> des1,des2,des3,des4,des5;
    vector<SiftGPU::SiftKeypoint> kpts1,kpts2,kpts3,kpts4,kpts5;

    f(img1,des1,kpts1);
    f(img2,des2,kpts2);
    f(img3,des3,kpts3);
    f(img4,des4,kpts4);
    f(img5,des5,kpts5);


    SiftMatchGPU matcher;
    matcher.VerifyContextGL();

    matcher.SetDescriptors(0,kpts1.size(),&des1[0]);
    matcher.SetDescriptors(1,kpts2.size(),&des2[0]);

    int (*match_buf)[2] = new int[kpts1.size()][2];
    t = getTickCount();
    int num_match = matcher.GetSiftMatch(kpts1.size(), match_buf);
    cout << "====================================" << endl;
    cout << "Match keypoints count:" << num_match << endl;
    end = static_cast<double>(getTickCount() - t) / getTickFrequency();

    cout << "Match,consume:" << end << endl;


    
    ///
    /// Test class GpuFeatureDetector
    ///
    ///

    GpuFeatureDetector fp;
    fp.create();

    Mat des11,des22;
    vector<KeyPoint> kpts11,kpts22;

    fp.detectAndCompute(img1,des11,kpts11);
    fp.detectAndCompute(img2,des22,kpts22);

    vector<DMatch> matches;
    t = getTickCount();
    auto matcheNum = fp.gpuMatch(des11,des22,matches);
    cout << "gpu matche:" <<  static_cast<double>(getTickCount() - t) / getTickFrequency() << endl;
    cout << "gpu match count:" << matcheNum << endl;

    Mat matchImg;
    drawMatches(img1,kpts11,img2,kpts22,matches,matchImg);
    imshow("gpu matches",matchImg);
    

    //
    ///
    /// OpenCV extract sift and match
    ///
    
    Mat des111,des222;
    vector<KeyPoint> kpts111,kpts222;
    detector->detectAndCompute(img1,noArray(),kpts111,des111);
    detector->detectAndCompute(img2,noArray(),kpts222,des222);

    auto ov_matcher = DescriptorMatcher::create("FlannBased");
    const float minRatio = 0.8;
    const int k = 2;

    vector<vector<DMatch>> knnMatches;
    vector<DMatch> betterMatches;

    t = getTickCount();
    ov_matcher->knnMatch(des111, des222, knnMatches, k);

    for (size_t i = 0; i < knnMatches.size(); i++) {
        const DMatch &bestMatch = knnMatches[i][0];
        const DMatch &betterMatch = knnMatches[i][1];

        float distanceRatio = bestMatch.distance / betterMatch.distance;
        if (distanceRatio < minRatio)
            betterMatches.push_back(bestMatch);
    }
    cout << "Opencv Match:" << static_cast<double>(getTickCount() - t) / getTickFrequency() << endl;

    Mat ovMatchImg;
    drawMatches(img1,kpts111,img2,kpts222,betterMatches,ovMatchImg);

    imshow("opencv matches",ovMatchImg);

    waitKey();


    return 0;
}