2.3: Color Classification

You can see colors and differentiate between them, but did you know computers can do this too? In this lesson, you will learn how to use a special algorithm to teach Zumi different colors. This is a very basic example of machine learning, which can be summarized in three steps:

• Gathering data
• Generating a model
• Making predictions

You will use Zumi’s camera to take many pictures of your favorite colors, and then run code that will use the color information to label each color. In the final step, you will be able to test your model and write code for Zumi to react to each color differently! For now, let’s teach Zumi about colors. You will need to have your activity color cards handy!

Before getting started with training Zumi to recognize colors, you need to learn how Zumi sees colors. It is very different than how you see colors!

An image is made up of an array of small dots called pixels (short for picture element). A pixel can be a single color or a combination of colors, and each of those colors is represented by a series of three numbers that tell you exactly how much red, green, and blue are in it. This is called the RGB value, which stands for red, green, and blue. For example, a beautiful shade of turquoise might look something like (27, 209, 197) since there isn’t a lot of red, but there is a lot of green and blue. Because each value of RGB can be between 0 and 255, there are 256 values to choose from for each color. That results in 256^3, or 16,777,216, different color combinations!

Since each pixel can be represented by numbers, a picture is a grid of numbers. This is where humans and computers start to see images a little differently. Humans see colors and shapes that we recognize as different objects, but computers only see these grids, also called matrices. Each number represents the RGB value of each pixel. They look a little like the image.

Your program will convert each RGB image to the HSV colorspace. HSV stands for hue, saturation, and value.

• Hue normally ranges from 0-360 and represents the color (in this application, however, it ranges from 0-180)
• Saturation is the color’s intensity
• Value is how light or dark the color is

In computer vision applications, it is better to use the HSV colorspace since it separates values for colors and intensity. This is important because shadows, reflections, and other factors may cause certain colors to look very different. The HSV colorspace takes this into account for more accurate results.

Head on over to the KNN Color Training Wizard in your Zumi dashboard and select three of your favorite colors from the packet of color cards in your Zumi box. In order to find the KNN Color Training Wizard from Zumi dashboard click the explore page from the main Zumi dashboard page

Once Inside of the Explore page click on the Learning Colors module

Once inside of the learning colors module follow the prompt to create your own color classifier. When you are done, don’t forget to save your model with a project name that is easy to remember. You will be using it in the next section, so make sure to write it down in addition to all of your label names.

The great thing about machine learning is that when you save a model, you can use it again! In the previous activity, you chose three of your favorite colors to teach to Zumi. Now you will use Python code to load the model and have Zumi react differently to each of the colors!

Import the following libraries to access the functions you need for driving, camera, and color classification.

from zumi.zumi import Zumi
from zumi.util.camera import Camera
from zumi.util.screen import Screen
from zumi.util.color_classifier import ColorClassifier
import time

camera = Camera()
screen = Screen()
zumi = Zumi()

To load the model, you will call a function from the ColorClassifier library: load_model(). This function needs a parameter, which is your project name. Remember what you named your project? Insert your user name AND the project name within the quotes below. Check that your spelling and punctuation is correct. If it doesn’t exist, you will get an error.

user_name = ''
demo_name = ''

knn = ColorClassifier(user_name=user_name)
train = knn.load_model(demo_name)
knn.fit("hsv")

Once your model has been successfully loaded, you can test it. Run the code below and pick a color card to show to Zumi. When you are ready, press enter to see the results on the screen. If you press “q”, the program will break out of the loop and turn off the camera.

camera.start_camera()

while True:
    user_input = input("Press 'enter' to predict or 'q to quit:  ") 

    if user_input == "q":
        break

    image = camera.capture()
    predict = knn.predict(image)
    screen.draw_text_center(predict)

camera.close()

You have trained Zumi to recognize the colors, but Zumi should probably make some decisions depending on the color she sees! Think about a traffic light. There are three colors on the traffic light. What do each of the colors mean?

Go back to the Color Training Wizard and train the labels “red”, “yellow”, and “green”. Next, save the model as “trafficlight”.

In this activity, you will make Zumi stop if she sees red, drive when she sees green, and drive at a slower speed if she sees yellow. Use the functions stop(), forward(speed=30), and forward(speed=70) for red, yellow, and green. For example:

if predict == "yellow":
    zumi forward(speed=30)

Load the “trafficlight” project here by providing the project name in the quotes.

knn = ColorClassifier(user_name=user_name)
train = knn.load_model("trafficlight")
knn.fit("hsv")

Load the “trafficlight” project here by providing the project name in the quotes.

knn = ColorClassifier(user_name=user_name)
train = knn.load_model("trafficlight")
knn.fit("hsv")

Fill in the code with the correct actions to go with the traffic light color. Then test it by showing Zumi a color card and pressing enter to see Zumi react!

camera.start_camera()

while True:
    user_input = input("Press 'enter' to predict or 'q to quit: ")

    if user_input == "q":
        break

    image = camera.capture()
    predict = knn.predict(image)
    screen.draw_text_center(predict) 

    # Add your if statements here!

    # Finish the code

camera.close()

Who said traffic lights always have to be red, yellow, and green? In the real world, these are universal symbols and we should keep them that way, but in Zumi world, you can create your own traffic lights that signal different behaviors. Maybe purple means “do a u-turn”, or blue means “go leh”. It’s your world so you get to decide!

Go to the Color Training Wizard in your Zumi dashboard and train three (or more!) colors for your new traffic light. You can go beyond simply stopping and driving. Check out the driving commands lessons or the Zumi documentation for more ideas! Remember your project name and labels because you will need to load the model below when you are done.

knn = ColorClassifier(user_name=user_name)
train = knn.load_model("PROJECT NAME HERE")
knn.fit("hsv")

Just like in the normal traffic light example, include if statements to tell Zumi what to do when she sees each of your colors. When you’re finished, build a small city and have someone be your traffic light by holding up the color cards you trained.

camera.start_camera()

while True:
    user_input = input("Press 'enter' to predict or 'q to quit: ")

    if user_input == "q":
        break

    image = camera.capture()
    predict = knn.predict(image)
    screen.draw_text_center(predict)

    # Add your if statements here!

camera.close()

Red Light Green Light

Have you ever played the game “Red Light Green Light”? Here are the rules: a referee calls out “red light” or “green light” to players who are trying to race to a finish line. If the referee calls out red light, players must stop and freeze in their places. If the referee calls “green light”, players race to the finish line. Whoever arrives first wins!

Use what you learned in this lesson to have Zumi go when it sees a green card and stop when it sees a red card. Play against your classmates — whoever’s Zumi gets to the finish line first wins!