So, to recap, I promised Arkwood that I’d write some Python code on my Raspberry Pi computer, which would detect guitars on a TV screen. That way he can be alerted whenever a rock ‘n’ roll band are onstage during the Glastonbury Festival TV coverage. And when rap and pop acts are performing, he can wash the dishes or polish the car. ‘I will create my own OpenCV haar cascade classifier for detecting guitars,’ I told him proudly, ‘It will detect electric, bass and acoustic guitars on the TV screen, using a webcam attached to my Pi.’ It seemed perfectly feasible to me. Only problem was, as my initial post demonstrated, it did not detect a guitar but instead a baseball bat. Clearly the system was not fit for purpose.
Redrafting the plan, I decided that I would only detect electric guitars – bass and acoustic guitars can have their own classifiers. Also, I would concentrate on Stratocaster-style electric guitars for now (that is, the Fender Stratocaster and all the imitation brands). In time I will consider specifics such as fingers on fretboard, left-handed guitars, etc.
I needed more positive images too – 60 in total. There would be 30 photographs of my red Fender Stratocaster, with different background and lighting. Plus another 30 of my grey Harmony imitation Stratocaster (the first guitar I bought, many moons ago). I would aim to add a further 60 online snaps of Stratocaster-style electric guitars after the first round of testing.
I also increased the number of negative images to 325, but fear that I could still do with more. We’ll see how it goes.
So the process of preparing and training my guitar classifier was similar to the initial post. My 60 postitive .png images were cropped and reduced to a 160px width variable height, producing 500 samples (-w 60 -h 24). My 325 negative .png images were reduced to a 480px width 360px height. I configured 19 stages of training:
opencv_haartraining -data guitarcascade -vec samples.vec -bg negatives.dat -nstages 19 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -npos 500 -nneg 325 -w 60 -h 24 -nonsym -mem 2048 -mode ALL
Now, the plan is to detect guitars on the TV screen. But for testing, let’s set up a nice tableau:
We have Gorilla & Son in our living room, to model our guitars whilst we trial our guitar classifier!
We will use the OpenCV convert_cascade application to output an intermediate guitar classifier xml file (as our training process still has a few hours to run):
convert_cascade --size="60x24" guitarcascade guitarcascade-inter.xml
Note: to get the OpenCV convert_cascade application working, I had to download dependant dlls msvcr71.dll and cv100.dll from http://www.dllme.com.
Brilliant. Now we can point the webcam, which is attached to our Raspberry Pi, at those menacing gorillas. Our Python program, running on our Pi, will try to find the guitars in the webcam shots:
Woo-hoo! Guitar detection is working, albeit there are still a fair number of false-positives. But this is only at stage 8 of 19, so hopefully we will get better accuracy once the training has completed. Plus, I will add to my positive and negative images (optimising image sizes), and tweak my settings, so it’s all looking rather rosy.
I retested my guitar classifier after 16 stages of training, and now get a successful guitar detection on pretty much every webcam snap, with absolutely no false-positives.
Both my red Fender Stratocaster and my grey Harmony imitation Stratocaster are being detected. I also exercised the guitar classifier with my bass and acoustic guitars and, not unsurprisingly, they did not get recognised (although I did get the occasional match on a red child guitar, similar in shape to a Strat).
I am greatly encouraged by these results. Next up I want to increase the negative image size and amount. I’ll also add more positive images to the mix, using online snaps of Stratocaster-style electric guitars. It will be important to detect Strat guitars that do not feature in our postitive image collection. Glastonbury Festival 2014, here we come!