Tags

, , , , , , , , , , , ,

Monita, my Hispanic maid, loves her rock ‘n’ roll. ‘Oh, that singer of Bon Jovi is a sexy boy. What’s his name again?’ Gritting my teeth, I replied, ‘Jon Bon Jovi. And he’s not a boy anymore.’ Whilst Monita clutched the laundry basket with dreamy eyes, I drew up a plan to win her affections. I would create a rock band using Lego figures, so as to serenade my beautiful maid with drums and guitar.

I will attach a webcam to my Raspberry Pi computer, which will take snaps of the stage that my Lego figures will be performing on. If the Python code on my Raspberry Pi detects a Lego policeman in the webcam photo it will play some drum sounds through a set of speakers. If it detects a Lego soldier it will play guitar. Monita will be thrilled.

But first I will need to create some OpenCV haar cascade classifiers – the xml files that our Python code will use to detect Lego figures. Thankfully I have already created a classifier for Lego policemen, so I followed the exact same process to create a classifier for Lego soldiers.

Now for the Python Webcam class, which will use my classifier files to detect Lego policemen and soldiers in each webcam photo:

import cv2
from datetime import datetime

class Webcam(object):

    WINDOW_NAME = "Lego Detection System"

    # constructor
    def __init__(self):
        self.webcam = cv2.VideoCapture(0)        
    # save image to disk
    def _save_image(self, path, image):
        filename = datetime.now().strftime('%Y%m%d_%Hh%Mm%Ss%f') + '.jpg'
        cv2.imwrite(path + filename, image)

    # detect lego in webcam
    def detect_lego(self, cascade_path):

        # get image from webcam
        img = self.webcam.read()[1]
        
        # do lego detection
        lego_cascade = cv2.CascadeClassifier(cascade_path)

        gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
        lego = lego_cascade.detectMultiScale(gray, scaleFactor=1.5, minNeighbors=5, minSize=(120, 190))

        for (x,y,w,h) in lego:
            cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)

        # save image to disk
        self._save_image('WebCam/Detection/', img)

        # show image in window
        cv2.imshow(self.WINDOW_NAME, img)
        cv2.waitKey(2000)
        cv2.destroyAllWindows()
        
        # indicate whether lego detected 
        if len(lego) > 0:
            return True

        return False

And here’s the main Python program, which will play drums and guitar depending on whether any Lego policemen or soldiers have been detected via the webcam:

from webcam import Webcam
import pygame

webcam = Webcam()

# set up instruments
pygame.mixer.init()
pygame.mixer.set_num_channels(2)
drums = pygame.mixer.Channel(0)
guitar = pygame.mixer.Channel(1)
drums.set_volume(0.5)
guitar.set_volume(0.3)
drumsFile = pygame.mixer.Sound("Drums.WAV")
guitarFile = pygame.mixer.Sound("Guitar.WAV")

# play rock forever!
while True:

    # play drums if lego policeman detected
    if webcam.detect_lego('haarcascade_policeman.xml') == True:
        print "lego policeman detected so play drums"
        if drums.get_busy() == False:
            drums.play(drumsFile)
    else:
        print "lego policeman not detected so stop drums"
        drums.stop()

    # play guitar if lego soldier detected    
    if webcam.detect_lego('haarcascade_soldier.xml') == True:
        print "lego solider detected so play guitar"
        if guitar.get_busy() == False:
            guitar.play(guitarFile)
    else:
        print "lego solider not detected so stop guitar"
        guitar.stop()

I am using Pygame to play the guitar and drums sound files. I created these sound files myself with Magix Music Maker.

Okay, so let’s see the Lego rock band in action…

legorockband_pregig

It is pre-gig, and the stage is empty – there are no drum or guitar sounds coming through the speakers attached to my Raspberry Pi.

legorockband_policeman

Suddenly a lego policeman takes to the stage and the drums begin to play.

legorockband_policemanandsoldier

Now the policeman’s drunken bandmate struts onto the stage, dressed in his military uniform, and the crowd go wild. The guitar chords begin to play on top of the drums.

Note: the Lego policeman and soldier on-stage are not the same figures that were used for training the classifiers.

legorockband_soldier

Now our percussionist policeman has left the stage, so only the guitar rings out.

legorockband_postgig

The post-gig party is in full flow, and the stage is empty. There are no more sounds coming through the speakers.

The program output on my Raspberry Pi shows how the gig unfolded (I have used a browser to display the last webcam snap):

legorockband_pi

Perhaps I will create classifiers for other Lego figures, such as spacemen and those crappy sorts without movable arms and legs – then I will have a complete band with bass guitar and vocals.

I let my Lego rock band perform for Monita, waiting at the side of the stage to receive my congratulatory kiss on the cheek.

‘Oh, darling, that funny,’ she said, ‘But where is the gorgeous Jon Bon Jovi?’

I was seething. My stomach was boiling with a furious anger. Nevertheless, as all sad fools of love are wont to do, I made a slight alteration to keep my maid happy:

legorockband_jonbonjovi

Advertisements