, , , , , , , , , , , , ,

I once wrote some Python code that could replay a random stroll through Google Street View. All because my creepy Belgian buddy, Arkwood, wanted to walk down the same streets again and again, seeking perverse pleasure from each predatory step. It made me feel sick, and I told him that I would help no more.

But today he pleaded, ‘Please! Just make the program play some atmospheric music. I need to feel the moment.’ He was rubbing his groin as he spoke to me.

No!, I reasserted. But it did get me thinking. I have recently taken a gander at OpenCV Histograms, which can describe a photograph in terms of light and shade. What if I were able to obtain a screenshot of each Google Street View step in the random stroll, and use its calculated histogram to determine which music to play. For example, if the histogram tells me that the screenshot is dark, I play some funky jazz. If the screenshot is light, I play the sound of birds tweeting. Let’s give it a go!

Here’s my Python class, which I will hook into the code from the previous post:

import cv2
import numpy
import ImageGrab
import winsound
from storage import Storage

class Histogram(object):

    SCREENSHOT_FILE = 'gsv/streetview_screenshot{}.jpg'
    HISTOGRAM_FILE = 'gsv/streetview_histogram.txt'
    LOWERSOUND_FILE = 'gsv/73621__julezhaze__funk-git1.wav'
    UPPERSOUND_FILE = 'gsv/182502__swiftoid__birds-chirping-02.wav'
    THRESHOLD = 40

    # initialise
    def __init__(self):
        self.storage = Storage()
        self.screenshot_counter = 0

    # grab screenshot
    def _grab_screenshot(self):
        screenshot = ImageGrab.grab(bbox=(0,100,1500,850))
        screenshot = cv2.cvtColor(numpy.array(screenshot), cv2.COLOR_RGB2GRAY)
        cv2.imwrite(self.SCREENSHOT_FILE.format(self.screenshot_counter), screenshot)
        self.screenshot_counter += 1

        return screenshot

    # play sound
    def _play_sound(self, top_bin_index):
        if top_bin_index < self.THRESHOLD:
            winsound.PlaySound(self.LOWERSOUND_FILE, winsound.SND_FILENAME)
            winsound.PlaySound(self.UPPERSOUND_FILE, winsound.SND_FILENAME)

    # calculate
    def calculate(self):

        # grab screenshot
        img = self._grab_screenshot()

        # calculate histogram
        hist = cv2.calcHist([img],[0],None,[256],[0,256])
        self.storage.write_value(self.HISTOGRAM_FILE, str(hist))

        # get top bin index from histogram
        top_bin_index = numpy.argmax(hist)
        self.storage.write_value(self.HISTOGRAM_FILE, "Top bin index: {}".format(top_bin_index))

        # play sound based on top bin index

First, we define some file constants, as well as a threshold to determine which music to play.

We initialise our Histogram class with storage capability (the Storage class can be found in the previous post). We also set a screenshot counter, so as to provide a unique filename when saving each screenshot to disk.

We have a _grab_screenshot private method. It uses ImageGrab to take a screenshot of the current street view photograph rendered in our browser, converting it to grayscale.

We have a _play_sound private method. If the histogram of our screenshot peaks on or above the threshold, we play bird tweets appropriate to a sunny street view. If the histogram falls below the threshold, we play some dark funky music.

The calculate public method will be executed from the code in my previous post. After grabbing a screenshot, we calculate its histogram. The OpenCV Histogram documentation explains how the histogram of an image is split into bins (darker pixels in the lower bins and lighter pixels in the upper bins). Next we use numpy.argmax to fetch the index of the histogram bin with the highest value. Finally we play an sound depending on where the top bin sits in terms of light and shade.

Using a single bin value to determine the brightness of an image is crude to say the least. For example, the histogram could spike on a certain pixel tone which is not representative of the image as a whole. No matter. The code can be fine-tuned later.

Also, for the convenience of this post, I’ve put all the code in one class. The two private methods ought to be refactored into their own classes, switching the constants to parameters.

Here’s how we slot the Histogram class into the code from the previous post:

from storage import Storage
from histogram import Histogram
import webbrowser
from time import sleep
storage = Storage()
histogram = Histogram()

# retrieve all urls to replay
urls = storage.read_all_values("gsv/streetview_replay.conf")
#loop urls
for url in urls:
    # open web browser with next url

    # pause

    # calculate histogram

Easy. We call our Histogram’s calculate method after each Google Street View photograph has been rendered in the browser.

Okay, time for a demo:


Top bin index: 39

The darkness of this screenshot has put it below the threshold. The funky music plays out through the computer speakers.


Top bin index: 40

With a lighter screenshot, the threshold is met. Bird tweets fill the air, befitting of a polluted concrete hellhole.


Top bin index: 41

Above the threshold, the squawks of those creatures with feathery flying-pieces still violate your ears.

And that’s it! A basic foray into OpenCV Histograms. Just don’t tell Arkwood about the code, as it will only fuel his seedy mind.



I sourced the funky music and bird tweets from Freesound:

I used Python Tools for Visual Studio to run the Python code on a Windows 7 PC.

I used Internet Explorer 11 to render the Google Street View photographs of Hong Kong Island.