CCV – Things to do !

Initially I thought, roadmap will be a better word , but naah! This is just a post on what “may be” in CCV in future releases ! Also people who want to contribute into CCV code base and have no idea what they can do , these are some features that can be worked on ! Some of them are pretty easy, so can be nice place to start !

  • Multicam (up to 16 PS3 cameras supported by CLEye Driver)
  • WM_TOUCH messages along with TUIO messages
  • TUIO on multiple ports(done in CCV 1.4 final)
  • Debug mode improvements (say -p arguments prints number of blobs and so on. Right now only -d option is supported for logging )
  • Run directly as service. Once all parameters are set and saved , you don’t need any GUI !
  • Porting to Linux – Now it’s about time to port CCV to Linux and do cross platform development !
  • Object Tracking improvements by use of ROI.
  • GUI improvements – as suggested by Patrick.
  • Tracking performance improvements and ability to adjust the ccv settings in single steps(suggested by the_goat)
  • Improve fiducial tracking (may be using direct reactivision library instead of addon ?) (suggested by Rogier Mars)
  • un-Distortion filters in CCV (https://amitsarangi.wordpress.com/2010/09/02/ccv-1-4/#comment-874)

This is not all , this is just a list of what comes to my mind right now ! If you want to add/modify any feature, just let me know , and I will add it up !

I have tried some of these and got success in some (like- running as service,debug mode improvements, multiple port TUIO) but the code is not mature enough to be merged into the repo. So if any of you want to work on these features, you can ask me/anyone in dev team for help !

You think you are not the “code-guy” and still want to contribute ? You are always welcome to help in documentation ! CCV is nicely documented, also the code is written so that it is easy to follow ! But better is always good 😉 !

Community Core Vision v 1.4

Finally the summer is over. Google Summer of Code made this summer one of the most amazing summers I have had. There are many people I need to thank for the successful completion of my project including Pawel Solyga(Mentor), Christian Moore, Jimmy Hertz, Sharath Patali and many other users who took time to test and give feedback for the test version of CCV-COT.

Its a great pleasure for me to announce that my work will be integrated into CCV mainstream development and will be released with this release (1.4).

Note : There has been some major changes in this release, so please read this fully before trying CCV-1.4a. Also if you have not tried out CCV-COT , you may want to read this post prior to reading this.

Changelog (After CCV-COT):

  • Complete Fiducial Support (Integration of ofxFiducialFinder addon by Alain Ramos).
  • TUIO XML messages bug solved.
  • CCV Debug mode
  • Different Filter chain and controls for Fiducial and Object/Fingers.
  • Fiducial settings on config.xml file.
  • Object acceleration calculations included.
  • removal and inclusion of some keyboard controls.
  • Closing of application from the “X” button,minimizing the application.
  • Changes in GUI
  • Blob counter for Finger, Fiducials and Objects respectively.
  • Sample AS3 app to test Custom Object Tracker(COT) added.

Community Core Vision v 1.4a

  • For Fingers + Objects – You can follow the post regarding CCV-COT and get started. The blob counter in the information window shows the count of finger blobs, object blobs and fiducial blobs respectively.

  • For Fingers + Fiducial –
  1. Start the application. Configure(Adjust the filter and calibrate) it for Finger. Enable Fiducials mode in the “Track” Panel.
  2. Press “i”. This will take you to the Fiducial Control Mode. “i” is the filter toggle key. In the debug window you will see The “Filter” tag changing to “Fiducial” from “Finger/Object” (See the screenshot below).
  3. Now you can adjust the filters so that you can see clear fiducials on the Binary Image. Note : If Fiducial mode is not enabled in the “Track” panel , you will see either a blank image or a still image.
  4. For better understanding , think of it like this. The camera image is copied into two images, one of them is analyzed for Finger/Objects and other for fiducials. Initially (when Filter is “Finger/Object”), all the sliders and controls are used to control the image that is analyzed for “Finger/Object”. When you press “i”, now all the sliders and controls are used to control the image that is analyzed for fiducials.
  5. The blob counter in the information window shows the count of finger blobs, object blobs and fiducial blobs respectively.
  • For Fingers+Objects+Fiducials- (**Not recommended for all**) This was something which was disabled initially because Fiducial tracking solves the problem of Object tracking. But again , we decided to give the user the freedom to choose what they have to track.  In this mode, most likely Objects and Fiducial will be confused (e.g. the image below)

But of course you need to be very lucky to get this mode working. A lot of adjustment will be required. But it is worth trying ;).

Tests :

Thanks to jimmy hertz, he has tested this version of CCV with his DI setup. Here are some images from his test.

In case of any doubts/suggestions get back to me by mail/comment/twitter/Facebook or by whatever mean you can. :-). It is now released on NUIGroup. Grab your own copy here :

DOWNLOAD : CCV-1.4.0-win-bin-preview.zip

DOWNLOAD : CCV-1.4.0-win-bin-final.zip

DOWNLOAD : CCV-1.4.0-lin-32-bin.tar.gz

DOWNLOAD CCV 1.4.0 For your own Platform

Or

Compile it yourself from LATEST SOURCE CODE

CCV-COT : Update 2

Hi everyone , was a bit busy with work on CCV-COT. There were a few bugs slipped in which took most of the time. Here is the summary of some works which I did after my last post.

  • Fiducial addon integratoin completed. Tried with paper fiducials and they work good with that. Used 2 different filter chains , one for finger+objects and other for fiducials. Both have the same settings. Also Modified the GUI to allow either Objects or Fiducial at once (so that there is no confusion between Fiducial’s part and Finger/Objects). Fiducial TUIO sending done.
  • In the preview release , people pointed out a bug with the TUIO XML part, which was due to an extra [/TCP] in the TCP messages. The [/TCP] was added by the openFrameworks network addon to distinguish between different TCP messages. I had to remove this tag manually.
  • One more TUIO bug was to use different “fseq” tag for same messages which I fixed by using same “fseq” tags. But again that caused another problem. Jerky response to the Object touch events. That was because I was sending 2 different messages for Objects and Fingers. Now I have combined them to a single message before sending.
  • Object calculation of dx,dy and acceleration added.
  • Fiducial Crash bug solved. And also in the TUIO UDP data there were some corrections (change in the format)

Somethings which are added to the roadmap are :

  • Releasing the CCV-COT 1.3.2
  • Sample Application (AS3) which can distinguish Objects and Fingers.
  • Use of 2Dobj messages and include all the calculations for it (angle, angle acceleration etc.)

CCV-COT : Update

In the past weeks, I have been working on my Last Post’s TO DO List, and this post is to update you with my work.

CCV Debug Mode :

As the CCV users may be knowing , CCV logging was removed in CCV 1.3. This was due to several users reporting a bug  that the logs become very big size(sometimes in GBs). Logs help developers to visualize a crash situation, so its quite an important part. Therefore I have tried a workaround for that. Instead of logging every time, we make the logging optional. This is called CCV Debugging mode. You just have to pass a command line argument -d for that e.g.

"Community Core Vision.exe" -d

And CCV Debugging mode is activated. Also , to make it simpler , in the next release a shortcut to CCV Debug mode will be included with the binaries. As usual this saves the Logs into the “logs” folder with the timestamp as the filename.

CCV Fiducial:

I know, this has been much awaited and many people have tried to do this before. Here is Stefan Schlupek’s repo where he managed to do it properly. I have integrated Fiducial support in CCV-COT.

Things TBD :

  • First of all , I have to test the Fiducial additions ASAP and do all the corrections.
  • Sending 2DObj messages for Objects (right now it is 2DCur) also for Fiducials.
  • The TUIO bug , that has been many reports of TUIO not working properly. Have to look into that.
  • Better Object Tracking algorithm (after all the integrations are done)

Release :

I am not releasing this time, as I have not tested them well. So in next few days, there will be changes going on. I will release them after testing it out. It must be soon though ! In case you are in a hurry to try it out , you can do it yourself by checking out the repository and compiling the code yourself. In case you run into any problems, just let me know via comments/mail.

Also for my previous version , slugmobile(noah) has compiled binaries for Mac OSX . In case you want to try them out, check here.

CCV-COT : Getting Started

CCV-COT or CCV Custom Object Tracker is the modified CCV to do the object tracking. For a getting started guide to CCV, refer here : Getting Started with CCV . This post assumes that you have a basic understanding of CCV , as I will be explaining only the difference my work has.

DOWNLOAD : CCV-Custom Object Tracker

SOURCECODE : Googlecode SVN

Source-code Structure Changes :

  • Moved to OpenFrameworks 0.61.Using  OpenFrameworks 0.61 precompiled library instead of source.
  • A new module in the sourcecode named “Templates” in ofxNCore.
  • A very simple tracking process now, which will be changed in next releases.
  • Modularized tracking of fingers/objects/fiducials.
  • Calculation time minimized, even in finger tracking by removing some extra calculation.

Changes in GUI :

This diagram only mentions the difference it has from CCV 1.3 . For a complete Description refer – http://wiki.nuigroup.com/Getting_Started_with_tbeta#Overview_Diagram

  1. Camera/Video toggle – Instead of having 2 buttons one button controls it. And yes, the crash on multiple switching between camera and video is fixed.
  2. Template Area Panel – The use will be described later in this post. This may change according to the algorithm used.
  3. Track Panel – Now you have a panel to choose what you want to track. Whether finger/objects/fiducial. Fiducial tracking is not yet integrated.
  4. Save/Load Template – These are two more buttons in the Settings Panel. As it says SAVE TEMPLATES saves template data stored, and LOAD TEMPLATES load the templates. The file used is “templates.xml”.

ID Assignment Rules :

Configuration & Calibration :

The configuration of this is pretty much similar to the configuration of CCV 1.3 : http://wiki.nuigroup.com/Getting_Started_with_tbeta#Configuration . While configuring for fingers switch on the Finger Tracking, to be able to see id/outlines of fingers detected.

There are some changes in the XML structure in config.xml, which end users need not worry about.

The use camera button [1] in the figure, is used for camera and video also.

The Calibration is same as the calibration of CCV 1.3 : http://wiki.nuigroup.com/Getting_Started_with_tbeta#Calibration

You should first configure your CCV to track finger blobs and then you should do the calibration with finger blobs. The calibration can be done by the Object tracking also, but that would not be precise.

Changing the settings in config.xml file:  http://wiki.nuigroup.com/Getting_Started_with_tbeta#Changing_Input_Resolution.2C_Frame_Rate.2C_Communication.2C_Video.2C_and_Blob_Settings

Till now is what mostly same as in CCV 1.3.

Process :

  • Configure and Calibrate.
  • Check that “template.xml” is present. Which is empty (if you have no template saved), or has template data in the following format.
<TEMPLATE>
    <WIDTH>36.000000</WIDTH>
    <HEIGHT>55.000000</HEIGHT>
    <MINWIDTH>25.520044</MINWIDTH>
    <MINHEIGHT>38.988960</MINHEIGHT>
    <MAXWIDTH>46.833553</MAXWIDTH>
    <MAXHEIGHT>71.551262</MAXHEIGHT>
    <<strong>TRUEID</strong>>0</<strong>TRUEID</strong>>
    <ID>180</ID>
</TEMPLATE>
  • In case you want to assign a particular ID to this blob, change the  “TRUEID” tag to 1 and “ID” tag to the particular ID you want to give this template. Say in the previous example, it would look like
<TEMPLATE>
    <WIDTH>36.000000</WIDTH>
    <HEIGHT>55.000000</HEIGHT>
    <MINWIDTH>25.520044</MINWIDTH>
    <MINHEIGHT>38.988960</MINHEIGHT>
    <MAXWIDTH>46.833553</MAXWIDTH>
    <MAXHEIGHT>71.551262</MAXHEIGHT>
    <TRUEID>1</TRUEID>
    <ID>189</ID>
</TEMPLATE>
  • MAKE SURE THAT THE ID ASSIGNED IS ACCORDING TO THE RULES . Or else this will cause problems as object ID/Finger ID/ Fiducial ID will be confused.
  • If you want to clear all templates, you have to delete all the lines from “template.xml” manually.
  • For adding a new template, make sure that tracking “Objects” in “Track” panel is on. Put the object which you want to track on the surface. You should be able to see a binary image on the “Tracked” image panel (Right big one). Draw an rectangle with mouse drag from upper-left corner to bottom-right corner , surrounding the object (The red rectangle in the image). Be as close as possible. Then adjust the “Template Area” panel’s minimum Area and maximum Area sliders to select the maximum and minimum variation of the contour (blue and green rectangle respectively).

  • Then press “Enter” to add the template.
  • Check the video for more details –
  • If you want to change the ID of a template while running, then first Add the template using the previous instructions. Save Templates. Open the templates.xml file. Change the ID to the wanted ID number and TRUEID to 1 for the template. And then Load Templates.

Coming Soon :

  • Fiducial Integration.
  • Better Tracking method (The current one is very very crude).
  • CCV in Debug mode (As logging is not there).

GSoC 2010 : CCV Template Matching

Hello everyone , its a little long since my last post but yes, I had been busy coding and thought of posting after I got some results. Now that I am getting some results, I thought of posting an article about my work and plans.

Also I want to tell you that there was a slight deviation of my plan/road-map from my last post. In that post I mentioned regarding the two parts, one for template selection and one for template matching. Well, I have changed the plan a little bit. Instead of having 2 tools, one for selection and one for tracking, I integrated selection and tracking into one.So it will be just modification of CCV, which will do both template selection and tracking. Initially I thought it will be a bit slow, but it turned out to be effective. You can follow my work on http://code.google.com/p/ccv-tt.

Here is a small preview of what I did.

As you can see, in the GUI I have added  a “Track” section where you specify what to track (fingers/objects/fiducial). Also I added a “Template Area” panel , which decided the blue and green rectangle you see in the figure. Also I modified the settings section to have Save templates and Load templates.

Also I have included the TemplateUtils class in the sourcecode which takes care of all the template works. The templates are saved in “templates.xml” file in “data” folder.The ID will be auto assigned. In case you want to change it , you can change in the xml file and load the xml again while the CCV is running.

Right now , I am able to track objects(I am able to differentiate them from finger blob). Next what I will be doing is :

  • Template ID assignment
  • x,y points conversion of the templates.
  • Drawing of the tracked objects
  • TUIO message sending
  • Better Template matching method (Right now it is very simple)
  • CCV shortcut keys for templates
  • Fiducial integration

In next week, I will be implementing some of these functions and testing. Once it is ready for the release, I will have a release package for alpha testers and have a video showing the work. Also then I will be posting a step by step documentation soon how to work with this. Look for the posts here in near future.