, , , , , , , , ,

Arkwood was in the kitchen, shaving his legs with a potato peeler. A sure sign that he was in trouble.

‘Put the peeler down and tell me what the matter is,’ I said.

Instead, he burst into tears. After a cup of cocoa, Arkwood lamented, ‘I want to be a rock star, but I can’t afford an instrument.’

Well, my little Belgian buddy was in luck! For it just so happens that I have created a piano using Python code and OpenCV computer vision. And some paper.

Here’s the paper piano:


It has seven keys: C, D, E, F, G, A, B.

All I need do is point my webcam at the piano and split the resultant image into seven cells:


Then I’ll use OpenCV motion detection to work out which cell my finger is pressing:


There’s my finger – the white blob in the bottom right of the image! The Python code can now use winsound to beep the musical note for that cell.

Here’s the main program (supporting classes at foot of post):

from webcam import Webcam
from detection import Detection
import winsound

# musical notes (C, D, E, F, G, A, B)
NOTES = [262, 294, 330, 350, 393, 441, 494]

# initialise webcam and start thread
webcam = Webcam()

# initialise detection with first webcam frame
image = webcam.get_current_frame()
detection = Detection(image) 

# initialise switch
switch = True

while True:

    # get current frame from webcam
    image = webcam.get_current_frame()
    # use motion detection to get active cell
    cell = detection.get_active_cell(image)
    if cell == None: continue

    # if switch on, play note
    if switch:
        winsound.Beep(NOTES[cell], 1000)
    # alternate switch    
    switch = not switch

We only play a note on alternate cycles through the while loop. I don’t want the note to play twice; once when my finger is detected and once when my finger disappears!

Time for a demo of Arkwood playing “Anarchy in the U.K.” by the Sex Pistols:

‘What do you think?’ I asked him, my eyes eager with anticipation.

He simply stated, ‘It’s shite.’ Charming! Looks like I’ll have to cancel the gig I booked him at the White Horse Inn. Another quiet night in with The Inscrutable Diaries Of Rodger Saltwash for us.



I ran the code on my Windows 7 PC using Python Tools for Visual Studio.

Rosettacode.org helped with the musical scale.

Here’s the Detection class:

import cv2
import numpy as np

class Detection(object):

    THRESHOLD = 1500

    def __init__(self, image):
        self.previous_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

    def get_active_cell(self, image):
        # obtain motion between previous and current image
        current_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        delta = cv2.absdiff(self.previous_gray, current_gray)
        threshold_image = cv2.threshold(delta, 25, 255, cv2.THRESH_BINARY)[1]

        # debug
        cv2.imshow('OpenCV Detection', image)

        # store current image
        self.previous_gray = current_gray

        # set cell width
        height, width = threshold_image.shape[:2]
        cell_width = width/7

        # store motion level for each cell
        cells = np.array([0, 0, 0, 0, 0, 0, 0])
        cells[0] = cv2.countNonZero(threshold_image[0:height, 0:cell_width])
        cells[1] = cv2.countNonZero(threshold_image[0:height, cell_width:cell_width*2])
        cells[2] = cv2.countNonZero(threshold_image[0:height, cell_width*2:cell_width*3])
        cells[3] = cv2.countNonZero(threshold_image[0:height, cell_width*3:cell_width*4])
        cells[4] = cv2.countNonZero(threshold_image[0:height, cell_width*4:cell_width*5])
        cells[5] = cv2.countNonZero(threshold_image[0:height, cell_width*5:cell_width*6])
        cells[6] = cv2.countNonZero(threshold_image[0:height, cell_width*6:width])

        # obtain the most active cell
        top_cell =  np.argmax(cells)

        # return the most active cell, if threshold met
        if(cells[top_cell] >= self.THRESHOLD):
            return top_cell
            return None

And the Webcam class:

import cv2
from threading import Thread
class Webcam:
    def __init__(self):
        self.video_capture = cv2.VideoCapture(0)
        self.current_frame = self.video_capture.read()[1]
    # create thread for capturing images
    def start(self):
        Thread(target=self._update_frame, args=()).start()
    def _update_frame(self):
            self.current_frame = self.video_capture.read()[1]
    # get the current frame
    def get_current_frame(self):
        return self.current_frame

And my next project, Holidays in the Sun.