GSoC 2010 : CCV Template Matching

Hello everyone , its a little long since my last post but yes, I had been busy coding and thought of posting after I got some results. Now that I am getting some results, I thought of posting an article about my work and plans.

Also I want to tell you that there was a slight deviation of my plan/road-map from my last post. In that post I mentioned regarding the two parts, one for template selection and one for template matching. Well, I have changed the plan a little bit. Instead of having 2 tools, one for selection and one for tracking, I integrated selection and tracking into one.So it will be just modification of CCV, which will do both template selection and tracking. Initially I thought it will be a bit slow, but it turned out to be effective. You can follow my work on http://code.google.com/p/ccv-tt.

Here is a small preview of what I did.

As you can see, in the GUI I have added  a “Track” section where you specify what to track (fingers/objects/fiducial). Also I added a “Template Area” panel , which decided the blue and green rectangle you see in the figure. Also I modified the settings section to have Save templates and Load templates.

Also I have included the TemplateUtils class in the sourcecode which takes care of all the template works. The templates are saved in “templates.xml” file in “data” folder.The ID will be auto assigned. In case you want to change it , you can change in the xml file and load the xml again while the CCV is running.

Right now , I am able to track objects(I am able to differentiate them from finger blob). Next what I will be doing is :

• Template ID assignment
• x,y points conversion of the templates.
• Drawing of the tracked objects
• TUIO message sending
• Better Template matching method (Right now it is very simple)
• CCV shortcut keys for templates
• Fiducial integration

In next week, I will be implementing some of these functions and testing. Once it is ready for the release, I will have a release package for alpha testers and have a video showing the work. Also then I will be posting a step by step documentation soon how to work with this. Look for the posts here in near future.

CCV-Template Matching Tool : Roadmap

Hello people ! My GSoC work has been going on. In this post I will be briefing more about my project. For the working convenience and time splitting, I have divided my project into 4 major parts ! I will be describing each of them in detail below.

• ALGORITHM : This is the baseline of the project. This part of the project decides what is the template selection and matching algorithm is to be . I am almost done with this part. You can see the tests/results in the googlecode svn. Finally I have decided to do the contour bounding rectangle matching. Also I have planned to include a button for more accurate matching which will do the contour matching test. With this algorithm it will be a good for shape matching.

• TEMPLATE SELECTION TOOL : This will be acting as addon to the existing CCV project. With the use of this tool, you can select the particular templates which you can use for matching . For better options, I will include sliders for minimum and maximum area of the contour to filter out the best contours. Also you will have the option to see the binary video also. You click-and-drag over the contour which you want to select and it saves the selected contour in an XML file. The XML file can be modified to change the id also.

• TEMPLATE MATCHING TOOL :  This part of the application will be handling the main tracking part. This will be a modification to CCV. It will load the xml data generated by the template selection tool and do all the calculations and do the TUIO sending Part. I will work hard on this to optimize the performance.

• APPLICATION : Once the previous part is done , I will work on this part. The applications will use the TUIO sent from the Template Matching Tool. I am planning to do this part with the help of PyMT/ TUIO AS3 API. Will decide based on the time remaining and complications which arise. This will be the last part of the project.

In this way, I have divided the work which I am doing. More detailed posts with instruction will be there once I have crossed the milestones. Right now I am working on the “Template Selection Tool”. Its high time to shift the gear to maximum. I will keep updating

Template Matching Invariant of Rotation

The first week of GSoC 2010 is over.In the mean time, I created a project on code.google.com . All my updates will be in that google-code svn repository, so if you have svn command line or svn client such as Totoise SVN you can checkout the source. I also have committed my openSURF experimentation source-code there in /branch/test folder. All my experimentation will be going into that folder.

As of now I have worked on different algorithms. I am planning to make CCV-TT modular so that any algorithm can be fit into it easily, so that it will be easy for future developments also. I will be writing the structure of the project in my next posts when I start coding for the Template Selection Tool.

So as the topic says I had experimented on contour matching. The source-code can be found on the googlecode repository. It includes the images in Images folder. The test images are :

The first image is the main image where the matching has to be done (say a frame from video/camera). The second is the template provided by the user. I have tried using SURF to find out the feature points and got only single feature point among them.Which was quite slow and will not be good to track (feature point was not at the center). Therefore I moved on to contour matching. Then I realized that contour matching may give very good results [cvMatchShapes(...) returns 0 ] when tried on the same template [the one cropped and transformed from the main pic]. But when tried on different picture’s templates [like the images shown] it gave quite bad results. You can see the difference in the drawing of the contour.

After seeing the contours [drawn in white in both ], the difference can be seen, it can give bad results or say can match to arbitrary curves on the surface/image. Therefore I thought of new plan, say contour-bounding-rectangle matching. The advantages and disadvantages are :

Advantages :

• Faster – It will involve very basic level math calculation.
• Saving into files/Loading from files, as these are of CvBox2D data type, it can be saved in files/loaded from files. This will also fasten the process a lot, reducing the time for finding contour for every image saved and calculation.
• minArea, maxArea for each contour- which will make it even more accurate.
• Detection and tracking of angles- from the rectangle that has been matched.

Disadvantages :

• It will not be very accurate.
• It will be more like shape matching.

As I had done experiments on this, You can see this clearly from results. Or you can checkout the google-code repository and perform the experiment by yourself.

As you can see the result, the actual contour bounding rectangle is matched. In case you want to know the contour matching results, just include the line

cvMatchShape(contours,tempContours,CV_CONTOUR_MATCH_I1);

in the for loop where the draw_box(…) is used. This will return a double value. For the best detection it will return 0.This code basically has 2 funtions-to match the images, or to see the drawn contours. You can see the match(…) and check(…) for more details as it is documented. For match(…) you have to define PROCEDURE as 1 and for check(…) , you have to define PROCEDURE as 2. It is better explained in source.

Also for the linux users, just copy the main.cpp and you can compile it with g++. But you need to have openCV2.0 installed.

This is all for now. Next I will be coding the template matching tool on OpenFrameworks. I will be coding it as CCV mod. For the progress of the work, watch out this space or the googlecode project.