Arkwood was in the kitchen, shaving his legs with a potato peeler. A sure sign that he was in trouble.
‘Put the peeler down and tell me what the matter is,’ I said.
Instead, he burst into tears. After a cup of cocoa, Arkwood lamented, ‘I want to be a rock star, but I can’t afford an instrument.’
Well, my little Belgian buddy was in luck! For it just so happens that I have created a piano using Python code and OpenCV computer vision. And some paper.
Here’s the paper piano:
It has seven keys: C, D, E, F, G, A, B.
All I need do is point my webcam at the piano and split the resultant image into seven cells:
Then I’ll use OpenCV motion detection to work out which cell my finger is pressing:
There’s my finger – the white blob in the bottom right of the image! The Python code can now use winsound to beep the musical note for that cell.
Here’s the main program (supporting classes at foot of post):
from webcam import Webcam from detection import Detection import winsound # musical notes (C, D, E, F, G, A, B) NOTES = [262, 294, 330, 350, 393, 441, 494] # initialise webcam and start thread webcam = Webcam() webcam.start() # initialise detection with first webcam frame image = webcam.get_current_frame() detection = Detection(image) # initialise switch switch = True while True: # get current frame from webcam image = webcam.get_current_frame() # use motion detection to get active cell cell = detection.get_active_cell(image) if cell == None: continue # if switch on, play note if switch: winsound.Beep(NOTES[cell], 1000) # alternate switch switch = not switch
We only play a note on alternate cycles through the while loop. I don’t want the note to play twice; once when my finger is detected and once when my finger disappears!
Time for a demo of Arkwood playing “Anarchy in the U.K.” by the Sex Pistols:
‘What do you think?’ I asked him, my eyes eager with anticipation.
He simply stated, ‘It’s shite.’ Charming! Looks like I’ll have to cancel the gig I booked him at the White Horse Inn. Another quiet night in with The Inscrutable Diaries Of Rodger Saltwash for us.
I ran the code on my Windows 7 PC using Python Tools for Visual Studio.
Rosettacode.org helped with the musical scale.
Here’s the Detection class:
import cv2 import numpy as np class Detection(object): THRESHOLD = 1500 def __init__(self, image): self.previous_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) def get_active_cell(self, image): # obtain motion between previous and current image current_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) delta = cv2.absdiff(self.previous_gray, current_gray) threshold_image = cv2.threshold(delta, 25, 255, cv2.THRESH_BINARY) # debug cv2.imshow('OpenCV Detection', image) cv2.waitKey(10) # store current image self.previous_gray = current_gray # set cell width height, width = threshold_image.shape[:2] cell_width = width/7 # store motion level for each cell cells = np.array([0, 0, 0, 0, 0, 0, 0]) cells = cv2.countNonZero(threshold_image[0:height, 0:cell_width]) cells = cv2.countNonZero(threshold_image[0:height, cell_width:cell_width*2]) cells = cv2.countNonZero(threshold_image[0:height, cell_width*2:cell_width*3]) cells = cv2.countNonZero(threshold_image[0:height, cell_width*3:cell_width*4]) cells = cv2.countNonZero(threshold_image[0:height, cell_width*4:cell_width*5]) cells = cv2.countNonZero(threshold_image[0:height, cell_width*5:cell_width*6]) cells = cv2.countNonZero(threshold_image[0:height, cell_width*6:width]) # obtain the most active cell top_cell = np.argmax(cells) # return the most active cell, if threshold met if(cells[top_cell] >= self.THRESHOLD): return top_cell else: return None
And the Webcam class:
import cv2 from threading import Thread class Webcam: def __init__(self): self.video_capture = cv2.VideoCapture(0) self.current_frame = self.video_capture.read() # create thread for capturing images def start(self): Thread(target=self._update_frame, args=()).start() def _update_frame(self): while(True): self.current_frame = self.video_capture.read() # get the current frame def get_current_frame(self): return self.current_frame
And my next project, Holidays in the Sun.