Monita, my Hispanic maid, loves her rock ‘n’ roll. ‘Oh, that singer of Bon Jovi is a sexy boy. What’s his name again?’ Gritting my teeth, I replied, ‘Jon Bon Jovi. And he’s not a boy anymore.’ Whilst Monita clutched the laundry basket with dreamy eyes, I drew up a plan to win her affections. I would create a rock band using Lego figures, so as to serenade my beautiful maid with drums and guitar.
I will attach a webcam to my Raspberry Pi computer, which will take snaps of the stage that my Lego figures will be performing on. If the Python code on my Raspberry Pi detects a Lego policeman in the webcam photo it will play some drum sounds through a set of speakers. If it detects a Lego soldier it will play guitar. Monita will be thrilled.
But first I will need to create some OpenCV haar cascade classifiers – the xml files that our Python code will use to detect Lego figures. Thankfully I have already created a classifier for Lego policemen, so I followed the exact same process to create a classifier for Lego soldiers.
Now for the Python Webcam class, which will use my classifier files to detect Lego policemen and soldiers in each webcam photo:
import cv2 from datetime import datetime class Webcam(object): WINDOW_NAME = "Lego Detection System" # constructor def __init__(self): self.webcam = cv2.VideoCapture(0) # save image to disk def _save_image(self, path, image): filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg' cv2.imwrite(path + filename, image) # detect lego in webcam def detect_lego(self, cascade_path): # get image from webcam img = self.webcam.read() # do lego detection lego_cascade = cv2.CascadeClassifier(cascade_path) gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY) lego = lego_cascade.detectMultiScale(gray, scaleFactor=1.5, minNeighbors=5, minSize=(120, 190)) for (x,y,w,h) in lego: cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2) # save image to disk self._save_image('WebCam/Detection/', img) # show image in window cv2.imshow(self.WINDOW_NAME, img) cv2.waitKey(2000) cv2.destroyAllWindows() # indicate whether lego detected if len(lego) > 0: return True return False
And here’s the main Python program, which will play drums and guitar depending on whether any Lego policemen or soldiers have been detected via the webcam:
from webcam import Webcam import pygame webcam = Webcam() # set up instruments pygame.mixer.init() pygame.mixer.set_num_channels(2) drums = pygame.mixer.Channel(0) guitar = pygame.mixer.Channel(1) drums.set_volume(0.5) guitar.set_volume(0.3) drumsFile = pygame.mixer.Sound("Drums.WAV") guitarFile = pygame.mixer.Sound("Guitar.WAV") # play rock forever! while True: # play drums if lego policeman detected if webcam.detect_lego('haarcascade_policeman.xml') == True: print "lego policeman detected so play drums" if drums.get_busy() == False: drums.play(drumsFile) else: print "lego policeman not detected so stop drums" drums.stop() # play guitar if lego soldier detected if webcam.detect_lego('haarcascade_soldier.xml') == True: print "lego solider detected so play guitar" if guitar.get_busy() == False: guitar.play(guitarFile) else: print "lego solider not detected so stop guitar" guitar.stop()
Okay, so let’s see the Lego rock band in action…
It is pre-gig, and the stage is empty – there are no drum or guitar sounds coming through the speakers attached to my Raspberry Pi.
Suddenly a lego policeman takes to the stage and the drums begin to play.
Now the policeman’s drunken bandmate struts onto the stage, dressed in his military uniform, and the crowd go wild. The guitar chords begin to play on top of the drums.
Note: the Lego policeman and soldier on-stage are not the same figures that were used for training the classifiers.
Now our percussionist policeman has left the stage, so only the guitar rings out.
The post-gig party is in full flow, and the stage is empty. There are no more sounds coming through the speakers.
The program output on my Raspberry Pi shows how the gig unfolded (I have used a browser to display the last webcam snap):
Perhaps I will create classifiers for other Lego figures, such as spacemen and those crappy sorts without movable arms and legs – then I will have a complete band with bass guitar and vocals.
I let my Lego rock band perform for Monita, waiting at the side of the stage to receive my congratulatory kiss on the cheek.
‘Oh, darling, that funny,’ she said, ‘But where is the gorgeous Jon Bon Jovi?’
I was seething. My stomach was boiling with a furious anger. Nevertheless, as all sad fools of love are wont to do, I made a slight alteration to keep my maid happy: