Artist Statement

As someone who has traveled a lot, I have come to face a lot of loneliness while being apart from family, friends and anything familiar. I created an interactive virtual friend to address the theme of loneliness. I will focus on friendship and how just being with someone you know can make you happy and less lonely. My main visual component is an alien-esque being who want to be your friend that will demonstrate this concept.

I programmed a monster named is Leroy. Leroy is just a kid who seems to have the same response for everything. Despite being an annoying little kid, Leroy will listen to whatever you say no matter what. He loves your company and he will become someone you would want to talk to and be friends with. Leroy can always be there to listen to your problems and you will soon forget that he’s just a computer-generated creature.

He works by reading the volume level in the room so Leroy will know when you are talking, to be quiet. He knows when you are quiet; he is allowed to talk. When you yell at him, he will yell back just like a kid throwing a temper tantrum. These reactions are all displayed using videos. On top of the videos, there is a live feed from a webcam so that there is a more realistic feel of an online video chat.

I’ve always wanted to be able to talk and interact with the alien/monster characters I create and share them with other people. Leroy will be the first one who gets this chance. Everyone will be able to share experiences with him and find a friend in him. He is not a Tamagachi or any other virtual pet that you may have seen before. He is on the same level as you: a friend, not a pet. You don’t feed or bathe him. He is you’re true-life virtual friend.

Final Product

This is the final test for Leroy. He works okay. I added the live video window in the bottom right corner so it appears more Skype like.
However, I still wish I could have gotten the word recognition library to work, I think the interaction would have been a lot more personal. I still have problems with the 'Invalid memory access' error a bit so I was afraid to add more video clips giving him a bit of variety. But I guess my idea of an annoying little kid character gets across better with only the one response. I also couldn't figure out how to make the full clip of the scream play without overlapping the others.

Final Video Clips

These are the final video clips that I will be using for Leroy.

Quiet Leroy



Talking Leroy


Angry Leroy

Problems and Script

I have been encountering a serious problem that I'm not sure how to fix.
I keep getting a 'Invalid memory' error in Processing if I run the program for a while. I expanded the memory in Processing's preferences as well as making sure my computer wasn't using a lot of memory in other applications while I was trying to run Leroy. It hasn't helped very much so far. I might try some smaller sized videos or something. I'm not sure. I'm going to look on some forum to see how other people overcome it.




On a brighter note, I have figured out my script for Leroy. I knew I wanted something silly that could circle in on itself, so I took a line from Tweedledee and Tweedledum 'if it was so it might be, and if it were so it would be, but it ain't so it isn't'

More Images

Leroy is responsive to the volume in the room. So, I had to make more videos of him reacting to the different scenarios. This shows my tests of Leroy with the added videos of being quiet yet animated, and when he is angry. I had no problem with the angry sound playing for my prototype, but for some reason, the video doesn't act the same way and won't play the whole clip of him screaming if the noise drops lower for a second. I'm hoping I'll be able to figure this out for the final. I also need to still figure out a new script for Leroy, something that will make some kind of 'sense' no matter what someone might say to him. This is proving to be the hardest part.

Post-Prototype - Moving Image

After some feedback from my prototype, I started working on more detailed images for Leroy.
Ones that actually move and look like he's talking. This is the first tests I did with the moving image. I made him in Photoshop originally as an animated gif, but processing wasn't matching the sounds I already had to the gif very well, so I morphed the audio and images into a QuickTime video file that Processing liked better. This is what I did with the rest of the visuals for the project.

Minim to Sonia / Prototype Status

I orignially start to program with the Minim sound library in Processing. However, the volume level numbers were really unstable and didn't seem to follow anything that I could measure. If that wasn't difficult enough to work with, after about 20 seconds of running the program, Processing would freeze and I would have to restart my whole computer. I tried to restrict the number using the map function, but it still didn't help much and my computer still would freeze after a short time.
Looking through the "Learning Processing" book, I decided to try the other sound library they mentioned. Sonia seemed to restrict the sound more and it was easier to read the numbers I need and have actions triggered by them than with Minim (I computer also didn't freeze, which was VERY helpful).
The only issue I have run into is that Sonia only plays .wav sound files and all the recording I did was in .mp3 format. I went online to find alien speech samples in .wav format so I could make sure that the code works properly before going through the process of converting the sound files.
Thankfully, everything is working now. Since the prototype is due today (Nov. 1) and today is also the day I managed to figure out all the code so that it responds properly, I am still using the sample files from online.
For the final piece, I will have my original recordings as well as more options when interacting with Leroy.

Final Communication Style

The idea I have is to make Leroy try to learn English.
If there is very little noise in the room, Leroy will prompt people to come talk to him by saying "Hello? Is someone there?"
If the level of noise in the room is a comfortable level, he will use a mix of broken English and his native language.
If it is really loud, Leroy will get irritated, and yell in his language and through a little hissy fit until get people to quiet down.

Code Update

I have gotten to a point in the code where I have the number for the amplitude.
It is just a system of writing 'if' statements to trigger the different recorded audio responses.

Language Idea

While talking about my project with a friend, I thought of an idea to have Leroy speak in his native tongue. Since he is an alien from a far off planet, he has his own language that is completely different and incomprehensible to humans.
Since my original idea is to present a character as a toy for kids, I feel that this could be really successful. I remember one of my favourite toys when I was little was an alien that would "talk" when you squeezed his stomach, pause for you to respond at random intervals. Leroy could work in a similar way. Instead of just talking and stopping at random times, he could what until you stop talking to him and responds to you in his language.
What he is meaning to say is up to interpretation by the user, and having the imaginative mind of a child explore that would make for a successful project.

Code Update

I have decided to give up on the word recognition coding and move into a noise reader.
I currently have Processing code that is reading the input from a microphone and turning it into a visual audio wave.
I need to figure out how to make this visual output into an audio output and I should have the practical part of Leroy done and just left to the visual presentation of him.

Voice Recognition Software

I've been trying to play around the Voce. I've been having some problems with it so I'm still trying to troubleshoot it but might end up not using it.

http://sourceforge.net/projects/voce/
http://processing.org/discourse/yabb2/YaBB.pl?num=1191403856/15
http://forum.processing.org/topic/voce-library-speech-recognition
http://processing.org/discourse/yabb2/YaBB.pl?num=1224681162

Character Sketch

Layout Sketches

A couple sketches to show the layout/design of the interaction with Leroy (Start and Active).

How Speech Recognition Works

For reference:

http://electronics.howstuffworks.com/gadgets/high-tech-gadgets/speech-recognition.htm

Milo

This is a project that is similar to my idea. However Milo has a budget and is WAY more advance than anything I could produce in one term. It isn't a little monster friend but a little boy based in reality. The way you interact with Milo is the same way I would like people to interact with Leroy.

http://youtu.be/CPIbGnBQcJY

Project Idea 2.0

As someone who has traveled a lot, I have come to face a lot of loneliness while being apart from family, friends and anything familiar. I am creating interactive virtual friend to address the theme of loneliness. I will focus on friendship and how just being with someone you know can make you happy and less lonely. My main visual component is an alien-esque being who want to be your friend that will demonstrate this concept.

Using cameras and some kind of voice recognition programming to make an interactive monster friend. His name is Leroy. During the start of the interaction, Leroy will be resting in the corner of the room. When you approach the screen Leroy will jump up from his resting position to greet you. After introducing yourself, Leroy will continue chatting with you and wait for your responses at appropriate times. He will become someone you would want to talk to and be friends with. He has outlandish stories of his home planet and always interested in what you have to say. No matter where you are, Leroy can always be there to listen to your problems and you will soon forget that he’s just a computer-generated creature.

I’ve always wanted to be able to talk and interact with the alien/monster characters I create and share them with other people. Leroy will be the first one who gets the chance to do this. Everyone will be able to share experiences with him and have him share his. He is not a Tamagachi or any other virtual pet that you may have seen before. He is on the same level as you: a friend, not a pet. You don’t feed or bathe him. He is you’re true-life virtual friend.

Changes

I've decided to change my project.
The concept will remain the same, monster friends, but it will be moved to a screen.
It will become an interactive friend. I haven't decided if he will respond to the user due to sensors or if it will be more of a digital ragdoll where the user will be able to manipulate the character.

Re-written proposal will be posted in the near future.

Animated Robot Reference

Instuctables on other moving creatures

Monkeys
Bird

Sensor Research

I've been doing some research on types of sensors (specifically types proximity sensors).
I have yet to choose a type of sensor but for future reference, these are the most useful ones I have come across:

• List of Sensors
• Proximity sensors
• Capacitive displacement sensors
• Inductive sensors

Sketches

First sketch I made had no starting point/idea. I just started to draw something using my favourite brainstorming material Crayola Washable Markers. From this I spawned my actual idea.
Since I now have the idea of alien friends, I went ahead and draw one example of what they could look like. They have the "matching yet different" aspect I mentioned in my proposal, similar to salt and pepper shakers.

Project Idea (OFFICAL)

As someone who has traveled a lot, I have come to face a lot of loneliness while being apart from family, friends and anything familiar. I am creating interactive robots to address the theme of loneliness. I will focus on friendship and how just being with someone you know can make you happy and less lonely. My main visual component is alien-esque beings who are friends and who will recognize each other to demonstrate my concept.

With these robot friends I am going to make them sad while apart. I will use sound to covey this, i.e. have them crying while apart. When moved close together they will become happy. I will show this again using sound, i.e. giggling sounds. I am going to make one robot a sensor and the other the senseé. I’m not sure which kind of sensor will be most efficient since I haven’t worked with sensors before. Between research and trail and error, I will find out which sensor would be best for this situation. They will be alien-esque creatures from a distant planet. Depending on which designs are chosen, they will be the same colour, shape, face or any other one defining feature to show that they belong together. They would not be much more than a few inches tall (small teddy bear size) and would be portable.

If marketed like toys, these alien friends would be the new age (and new media) friendship bracelets. Kids would not only have their own friend and companion but when brought together with their best friend’s matching creature, there is a new bond and connection on both levels. Sharing the two aliens between two friends and bringing them together when the friends are together can represent the joys of friendship, not only between the two aliens but actual people.

Brainstorming / Ideas

I wanted to make a character of some kind from the start. I want to present this character as an interactive robot.

After a lot of thinking, I’ve decided that they best way to approach this is through a made up alien-esque character. I’ve created a few before and found it enjoyable process.

Creating this particular character, I was thinking about where he would be from, why is he here, and how he feels about this. I’ve narrowed it down to him being homesick as his main trait. Now, going about how to make this trait show through an interactive robot without getting over my head is the hardest part of the brainstorming.

After sketching a little (actually photos of sketches soon to follow), I decided that the best way to approach this would be to actually have 2 aliens. While together, they are happy (giggly) and while they are apart, they are sad (crying). So, I guess I have moved from the idea of homesick, to more of a lonely aliens who just want to be together.