Recently I’ve been running a number of experiments in Matlab using SIFT and attempting to get various roadsigns recognised by using a cropped training set for a particular sign and then matching it against keypoint descriptors found in a test image.

I’ve been gathering statistical data and dumping this out to a file for analysis and further work but I’ve also been generating match images for each roadsign tested against the most favorable image from the training set. To give people an idea with how this is progressing I include an example of a very successful result on a STOP sign below.

Its not all rosy though and I’ve been given food for thought after my interim report and some failures of the SIFT algorithm in recognising certian signs.

In my initial experimentation with actual roadsigns I decided to go for a simple sign as this would be less likely to be susceptable to noise. This was in fact quite an unwise decision as SIFT works by finding particularly unique points and I had effectively removed the possiblity of it finding such points by using a simple sign. I include below a matches image for a No Entry sign.

As you can see, its actually quite noisy. This is a problem which got me thinking about how I can make the whole process more robust and forced me to return to some of my early research on object recognition.

If I can use SIFT as a pre-processor then I can identify signs quickly and easily that have many descriptors such as STOP signs and then use a more basic system such a a template or colour match to identify the simpler signs such as the No Entry sign. I could also do it the other way around and will need to perform tests to decide what the best order is.

I’m currently working on an idea that SIFT could be used to detect the presence of any sign (not which one it is, but where it is) and then further tests could perform the recognition. In addition I’m researching examples of where SIFT has been modified to be used in colour and also how best to display experimental results. I’m also in the process of writing code to fully automate my training and testing process in Matlab so I should be able to run batch jobs and get results and test theories quicker.

Watch this space for more updates - its all go!

Since first using the implementation I have been able to demonstrate how to find objects that have previously been seen by the system in a completely new image, and have also managed to make use of the SIFT algorithm in a more generic fashion by applying it to one of my coursework tasks - to find faces and cars in a set of images.

The power and potential in this algorithm becomes immediately apparent after using it on test image for an hour or so and indeed my reading since first use of it in Matlab indicates to me that the use of this algorithm in the kind of task I am researching is highly recommended by a number of academics. I certainly intend to use my findings here as an important component in my interim project report.

I welcome any and all comments on my work and if you have something more substantial to say I encourage you to drop me an e-mail.