2.4: Robot Emotions

Take a look at the images below and see if you can identify each of the emotions.

How did you determine which emotion was which? There are many features that can be indicators, like eyes, mouth, eyebrows, and maybe gestures. How do we translate human emotions to a robot?

If you have seen the movie Cars, you may know that each of the cars has a personality. How was each car able to express emotions? Was it through movements? Sounds? Eyes?

To use personality functions, we need to import the Zumi, screen, and personality libraries.

from zumi.zumi import Zumi
from zumi.util.camera import Camera
from zumi.util.screen import Screen
from zumi.personality import Personality
from zumi.util.color_classifier import ColorClassifier

import time
••zumi = Zumi()
screen = Screen()
personality = Personality(zumi, screen)

Here are some functions you can call:

• happy()
• celebrate()
• angry()
• look_around()
• look_around_open()
• disoriented_leh()
• disoriented_right()
• awake()

For example, personality.happy() will make Zumi wiggle and make a sound!

Try testing out some of the personality functions to see what they do.

Zumi can play sounds to match her emotions! Sound can be measured in frequency and amplitude.

• Frequency is the number of pulses or vibrations per second, and is measured in hertz. The higher the frequency, the higher the pitch of the sound is.
• Amplitude is how loud or strong the sound is and is measured in decibels. The higher the amplitude, the louder the sound is.

Video: Sound: Wavelength, Frequency, and Amplitude

What does each emotion sound like? Is happy a low or high frequency? Is angry a low or high amplitude? How does this apply to Zumi?

You can use play_note() to play various notes. The first parameter is the note you want to play (anywhere from C2 to B6). The second parameter is optional and denotes the amount of time you want the note to play in milliseconds. The default value is set to 500ms, but you can change that by adding a second parameter like this:

play note(Note.GS3, 400)

This plays the note G sharp below middle C for 400 milliseconds. Try the code below to hear a scale and then compose your own music!

from zumi.protocol import Note
zumi.play_note(Note.C4)
zumi.play_note(Note.D4)
zumi.play_note(Note.E4)
zumi.play_note(Note.F4)
zumi.play_note(Note.G4)
zumi.play_note(Note.A4)
zumi.play_note(Note.B4)
zumi.play_note(Note.C5)

Code your own sounds for happy, sad, angry, or excited. Try out different melodies until you find your favorites.

Zumi personality also uses the OLED (organic LED) screen to display emotions. There are many different “eyes” Zumi has:

• close_eyes()
• sleepy_eyes()
• sleepy_left()
• sleepy_right()
• blink()
• look_around_open()
• sleeping()
• look_around()
• glimmer()
• sad()
• happy()
• hello()
• angry()

To use the screen, call the screen class with a function of your choice. Try this:

screen.sad()

Aside from drawing Zumi eyes, you can also have Zumi write messages on the screen! Use the draw_text() function to write a message like this:

screen.draw text("hello!")

If you want to automatically center the text on the screen, call this function instead:

screen.draw text center("hello!")

If you want to write text with numbers, you need to make sure everything is of the String data type.

number = 10
screen.draw_text("ten " + str(number))	# the str() functions turns the number into a string

You can even make Zumi display the time for you!

for i in range(0,50):
    screen.draw_text(time.ctime())
    time.sleep(0.1)

In this activity, Zumi will keep her eyes on you! You will write code so that Zumi eyes follow your face when it’s the middle, left, or right side of her line of sight. If Zumi can’t see you, Zumi will be sad! This activity uses code from the Face Detection lesson. Take a look at that lesson if you haven’t yet!

Dividing the frame into left, right, and middle

The first step is to figure out how many pixels wide Zumi’s camera images are. To do this, we can access the camera object’s properties like this:

In [ ]:

print(camera.image_width)

By default, the pixel width will always be 160. Although you can change this, we are going to use this value in our example. Now, divide this image frame into areas that you will consider the left side of the frame, the right side of the frame, and the middle. You can divide it equally or not. It’s up to you! Just remember that you only have 160 pixels to divide into three parts and you can’t have half of a pixel.

Now that you have learned to use sounds and screen to give Zumi a personality, have her react to a smile! In a previous lesson, you probably learned about using vision.find_face() to search for faces. What do you think the features are for a smile? Are the darker pixels of our mouths angled up or down? In this lesson, you will call a function to check if the face Zumi is detecting is smiling. If she sees a smile, she will be happy. If not, give Zumi some sad eyes.

The function you will call for this activity is vision.find_smile(). This function will return the pixel coordinates if Zumi finds a smile, or (-1,-1,-1,-1) if she doesn’t. For example, if you take a picture and save it in an image, you can check if Zumi found a smile or not:

(x,y,w,h) = vision.find_smile(image)
if (x,y,w,h) == (-1,-1,-1,-1):
    # Zumi did not find a smile :(
else:
    # Zumi sees a smile! :)

Now write some code that will take a picture with Zumi’s camera and look for a smile. Give Zumi some personality and match her emotions with yours! Fill in the code template below:

camera.start_camera()
try:
    for i in range(1000):
        # Finish the code!


finally:
    camera.close()

Frequency

Use tuners to identify the frequency, wavelength, and amplitude of different Zumi sounds. Which emotion has the highest or lowest frequency, wavelength, and amplitude?

Add personality to color classifier lesson!

Note! This activity requires the Color Training Wizard.

Go back to the Color Training Wizard in the Zumi dashboard and train Zumi on a variety of colors. Load the model below and set a happy reaction to her favorite color and sad or angry reactions to other colors. Have a partner show Zumi various colors and guess what Zumi’s favorite color is!

camera = Camera()
knn = ColorClassifier()
train = knn.load_model("PROJECT NAME HERE")
knn.fit("hsv")

camera.start_camera()

while True:
    user_input = input("Press 'enter' to predict or 'q to quit: ")

    if user_input == "q":
        break

    image = camera.capture()
    predict = knn.predict(image)
    screen.draw_text_center(predict)

    # Add your if statements here!

camera.close()