SURF and OpenSURF : Usage of OpenSURF library

As I said before OpenSURF is an open source implementation of SURF library, which is being used for feature extraction and matching. In my last post I have provided the results of my experimentation. This post is for description of the methods and how to do it all by yourself.

Well, first of all you need to configure your IDE for using openCV dlls and libs. Then add the openSURF library to your project(else for testing you can create inside openSURF project e.g. see main.cpp). All you need is to include “surflib.h” from the openSURF library which contains all the main function you need for the implementation.

The main functions are :

//! Library function builds vector of described interest points  (from surflib.h)
inline void surfDetDes(IplImage *img,  /* image to find Ipoints in */
std::vector<Ipoint> &ipts, /* reference to vector of Ipoints */
bool upright = false, /* run in rotation invariant mode? */
int octaves = OCTAVES, /* number of octaves to calculate */
int intervals = INTERVALS, /* number of intervals per octave */
int init_sample = INIT_SAMPLE, /* initial sampling step */
float thres = THRES /* blob response threshold */)

//To get matches between two Ipoints (from ipoints.h)
void getMatches(IpVec &ipts1, IpVec &ipts2, IpPairVec &matches);

//! Draw all the Ipoints in the provided vector (from utils.h)
void drawIpoints(IplImage *img, std::vector<Ipoint> &ipts, int tailSize = 0);

//! Save the SURF features to file (from utils.h)
void saveSurf(char *filename, std::vector<Ipoint> &ipts);

//! Load the SURF features from file ( from utils.h)
void loadSurf(char *filename, std::vector<Ipoint> &ipts);

So as the very first function says, it calculates all the ipoints(Interest Points) from the image and stores them in the ipts vector.

As indicated by comments, the drawIpoints(…) method draws the interest points found by the previous method, saveSurf(…) saves the ipoints to a file and loadSurf(…) loads the ipoints from a file to a variable.

So once all interest points of the image are found, you can match them to the interest points of another image by the getMatches(), which stores all the matched points to the matches variable. The definition of IpPairVec is

typedef std::vector<std::pair<Ipoint, Ipoint> > IpPairVec;

So in case you want to connect all the match points by cvLine you can use the following code snippet (see the 5th procedure in main.cpp)

  for (unsigned int i = 0; i < matches.size(); ++i)
  {
    drawPoint(img1,matches[i].first);
    drawPoint(img2,matches[i].second);

    const int & w = img1->width;
    cvLine(img1,cvPoint(matches[i].first.x,matches[i].first.y),cvPoint(matches[i].second.x+w,matches[i].second.y), cvScalar(255,255,255),1);
    cvLine(img2,cvPoint(matches[i].first.x-w,matches[i].first.y),cvPoint(matches[i].second.x,matches[i].second.y), cvScalar(255,255,255),1);
  }

So this is all for now, I will keep you updated about my work !

So as the very first function says, it calculates all the ipoints(Interest Points) from the image and stores them in the ipts vector.

SURF and OpenSURF

GSoC 2010 work has already started. As the very first phase, I have to choose a good algorithm to implement the pattern matching technique.Initially I am concentrating on the pattern matching on images and after this I will be concentrating upon integrating into CCV. The application may need to handle multiple image matching in real-time, therefore a good algorithm is needed. So as my mentor Pawel suggested, I looked around for different algorithms on feature extraction. Features of images remain same on different condition of images. Therefore it seems like a good idea.

The very first thing that I got regarding this was SIFT (Scale Invariant Feature Transform). Then I came across SURF (Speeded Up Robust Features) which is inspired by SIFT and has better performance. Also there are some implementations of SURF around internet which are listed in the wiki page. Even SURF has been integrated into openCV.  But I have come across another open-source implementation named OpenSURF. It was quite nicely documented and had this Notes on openSURF library which has awesome explanation. But you should really check out the source as that has nice examples. Also each header files contain the use of each function.

One more good feature in SURF is that the Interest Points(IPoints) can be saved to a file and loaded for use which takes very less time than calculating interest points everytime. There are some specific parameters of iPoints which need to be saved/loaded. Those are scale,x,y,laplacian and the 64 descriptors acquainted with it.

I had carried out this experiment on 4 set of images. The results can be downloaded here. Each image features are extracted to a file of the same name and with “.txt” extension.In this post I will be analyzing one of the categories in details. You can see the result of all the categories by downloading the link I provided.

The Categories are :(Click on the Image to see it in actual size)

I will be analyzing the last category in which I got good results.

Here are the results in the image form. The matches are joined together with lines.

If you see the third image, out of the 3 matches, only one match is correct. Similarly I have found from my work that, if the images are very much similar then SURF works very well. But in case there is a major viewpoint change or major difference, then SURF fails to identify the features. With the Samples I have provided, see the Ferrari Images [c1.jpg-c5.jpg], there were no matches found in them (1 wrong match found).

The time taken for calculation depends upon the size of the image and also how the image is . In the process of my work, I have seen images with 2 Ipoints to 5k iPoints.As the number of iPoints increase, the time taken is more.For me some images took ~25 seconds and some tool 0.015 seconds for the iPoints calculation.

So this algorithm is not as good as the one used in Google Similar Images. But yes, if the images are quite similar (with rotation/change in lightening), then SURF seems to work awesome !