3.8: Color Classifier

Goals:

By the end of this lesson, I will be able to:

1. Know the difference between HSV and RGB
2. Describe how the CoDrone EDU classifies colors
3. Calibrate my color sensor
4. Predict colors with the CoDrone EDU

Steps to Success:

1. Color Sensor Calibration
2. Color Sensor Prediction
3. Color Piano
4. Challenge 1: Multiple Samples
5. Challenge 2: Color Piano – Extended

You can see colors and differentiate between them, but did you know computers can do this too? In this lesson, you will learn how to use the CoDrone EDU’s color sensor to distinguish between different colors! The CoDrone EDU is equipped with two color sensors. When your CoDrone EDU is powered on you may have notice the two bright lights projected underneath the drone. Those lights are to help your color sensors see colors by illuminating the surface they are looking at.

In order to get our color sensors to be the most accurate we will first need to calibrate our color sensors. We need to calibrate them because different rooms have different brightness levels and shadows. If we want our sensors to be the most accurate then we should calibrate them in the room we plan on using the color sensor in!

Your program will convert each RGB image to the HSV colorspace. HSV stands for hue, saturation, and value.

• Hue normally ranges from 0-360 and represents the color (in this application, however, it ranges from 0-180)
• Saturation is the color’s intensity
• Value is how light or dark the color is

In computer vision applications, it is better to use the HSV colorspace since it separates values for colors and intensity. This is important because shadows, reflections, and other factors may cause certain colors to look very different. The HSV colorspace takes this into account for more accurate results.

Let’s start by finding how to access color data from the remote controller! First make sure that you are not in the “Link” state. To access color data from the controller we need to be in remote control flight mode. Press the power button to switch between the link state and the remote control state. Once in the remote control state press either the “P” or “S” button until you see the screen in the image below.  From this screen you can see HSV values (Hue, Saturation, Value) as well as the current color seen by the drone!

In order to calibrate our color sensor, we will need to write the code ourselves! Create a new file called calibrate_color to get started. We will start by using our usual imports, object creation, and pairing.

from codrone_edu.drone import *
import time

drone = Drone()

drone.pair()

For this example we will use a CoDrone EDU function to create a new directory that will hold all of the color data we are about to receive. We will make a variable called dataset and assign it to the string “color_data”. Eventually we will pass this variable into a CoDrone EDU function. We will also need to create a list holding all of our landing pad colors.

NOTE: When calibrating the color sensor there must be at least 3 colors in your data set. If there are less than 3 colors in the data set an error will occur. For reference, 8 colors are included into the data set below.

dataset = "color_data"

colors = ["green", "purple", "red", "lightblue", "blue", "yellow", "black", "white"]

We will now make a for loop for each color in our list. Each color will need to be calibrated so using a loop to get through all of our colors is the most efficient way to code this! Inside of our for loop we will also need to make an empty list for our color data called data .We will also make a variable called samples that will hold the number of samples we will take for each color. We will assign samples to 500.

for color in colors:

    data = []

    samples = 500

Now we will use the input function to have our program pause. Our program will wait for the user to press enter before continuing. We will need to use this pause to set our drone on top of our desired landing pad.

We will also be adding a loading bar to this program. This will help you see that the samples are being read and something is happening with the drone. Make sure not to move your drone while the loading bar is in progress!

    for i in range(1):
        print("Sample: ", i+1)
        next = input("Press enter to calibrate " + color)
        print("0% ", end="")

Earlier we said we wanted to take 500 samples for each color. We will use a nested for loop to take 500 samples for each color. As these samples are taken we want to append them to our empty data list. We will also have our progress bar slowly increment as we get through this for loop by using the modulo function! Once this loop finishes we print 100% to indicate that it is finished taking samples.

        for j in range(samples):

            color_data = drone.get_color_data()[0:9]

            data.append(color_data)

            time.sleep(0.005)

            if j % 10 == 0:

                print("-", end="")

        print(" 100%")

Once all of our color data has been gathered we need to send it to a CoDrone EDU function that will create a new color directory for us. This will give us more accurate color data to use once we start writing color detection programs! The new_color_data() function takes a label, data, and dataset. We want to add our data list under the color label in the dataset folder.

If you want to add to an existing directory instead of creating a new one you can use the append_color_data() function. This function takes the same parameters as new_color_data().

    drone.new_color_data(color, data, dataset)

print("Done calibrating.")

Now we have a finished color classifier program! After running this program a new directory called “color_data” will be created. Inside of this directory will be all of the text files of data for each color with hundreds of lines of data! Feel free to look at your data but make sure not to change anything! It could throw off your color detection in later programs. Here is the full code:

from codrone_edu.drone import *
import time
drone = Drone()
drone.pair()

dataset = "color_data"
colors = ["green", "purple", "red", "lightblue", "blue", "yellow", "black", "white"]

for color in colors:
    data = []
    samples = 500

    for i in range(1):
        print("Sample: ", i+1)
        next = input("Press enter to calibrate " + color)
        print("0% ", end="")

        for j in range(samples):
            color_data = drone.get_color_data()[0:9]
            data.append(color_data)
            time.sleep(0.005)
            if j % 10 == 0:
                print("-", end="")
        print(" 100%")

    drone.new_color_data(color, data, dataset)

print("Done calibrating.")

In our last example we learned how to calibrate our color sensor for accurate color detection. In this example we will begin predicting colors using our calibrated color sensor data! We will start with bringing in our imports, object creation, and pairing.

from codrone_edu.drone import *
import time

drone = Drone()

drone.pair()

Now that that’s out of the way, we will move on to loading our color classifier that we just created. To do this we will use the drone.load_classifier() function. When this function is called without a parameter it will load the default color data in the drone. The default data isn’t always the most accurate so instead we will use our directory that we created with our color calibration program as a parameter!

If you followed the last example then your directory should be called “color_data”. If your directory is named something else, make sure you use the name you used as a parameter to the load classifier function.

drone.load_classifier("color_data")

Now that our classifier is loaded all we have left to do is predict the color! This will be a two step process. First, we need to get the current color data being read into our sensors. Once we get that data we compare it to our calibrated color_data to predict the color! Since this is our first color prediction program we will use a for loop that will print out the color your drone currently sees.

for i in range(500):

    color_data = drone.get_color_data()

    color = drone.predict_colors(color_data)

    print(color)

    time.sleep(0.5)

drone.close()

Your colored landing pads should be on a flat surface. Make sure that you place the CoDrone EDU directly onto the landing pads. Holding the drone above them will result in inaccurate color readings. This program will run for a few minutes and constantly be printing colors into your console. Check all of the colors to see the color predictor work its magic!

Let’s make a piano out of our color predictor! In this example we will use 4 colors to make a simple color piano. Once our piano is finished we will be able to make music with our drone!

As always, start with imports, object creation, and pairing.

from codrone_edu.drone import *
import time
drone = Drone()
drone.pair()

From our last example we also learned that we need to load our calibrated color classifier. We will also need to do that here!

drone.load_classifier("color_data")

Next, we will create an infinite while loop so that our drone can continuously look for colors. Inside of this while loop we will also need to get our current color data and have the CoDrone EDU predict what color it is seeing like in our previous color prediction example.

while True:
    color_data = drone.get_color_data()
    color = drone.predict_colors(color_data)

The color variable that we created is actually a list! color[0] holds our front color sensors prediction while color[1] holds the back sensors prediction. For this example we will use color[0] to check for our predicted color. If our color is green, then we will play a B4 note for 1 second.

if color[0] == "green":
    drone.drone_buzzer(Note.B4, 1000)

Our main logic is finished, all we need to do now is add more sounds and colors. We will add an else statement that uses the buzzer but doesn’t make any sounds. Just in case!

elif color[0] == "red":
    drone.drone_buzzer(Note.A4, 1000)
elif color[0] == "yellow":
    drone.drone_buzzer(Note.C4, 1000)
elif color[0] == "black":
    drone.drone_buzzer(Note.G4, 1000)
else:
    drone.drone_buzzer(0, 0)

Lastly, we will add a print so that we can see what color is being predicted as well as a sleep so we don’t send too many signals to the color sensor at once.

print(color[0])
time.sleep(0.2)

That’s all for the color piano! Here is the completed code:

from codrone_edu.drone import *
import time
drone = Drone()
drone.pair()

drone.load_classifier("color_data")

while True:
    color_data = drone.get_color_data()
    color = drone.predict_colors(color_data)

    if color[0] == "green":
        drone.drone_buzzer(Note.B4, 1000)
    elif color[0] == "red":
        drone.drone_buzzer(Note.A4, 1000)
    elif color[0] == "yellow":
        drone.drone_buzzer(Note.C4, 1000)
    elif color[0] == "black":
        drone.drone_buzzer(Note.G4, 1000)
    else:
        drone.drone_buzzer(0, 0)

    print(color[0])
    time.sleep(0.2)

Challenge 1: In our calibrate_color program we took 500 samples one time for each color. Can you figure out how to change the code to take multiple samples of 500 for each color?
hint: Only one thing needs to be changed!

Challenge 2: We created a basic color piano with 4 keys. Add onto it and make a color piano that includes all of your colored landing pads! You can also choose notes that are in your favorite song and try to recreate it!

In this lesson, you learned how to use your CoDrone EDU’s color sensor to detect different colors. Using this data, you learned how to calibrate your drone to predict what color is being detected!