Voice Assistant for the Ghostbusters

A study in Voice Interface Design



Project Details

This is the final project for the course INST631: Fundamentals of HCI


Motivation

For our final project for the course, our professor Dr. Jennifer Golbeck instructed us to design a product for the Ghostbusters. The only limitation we had was that we couldn't assume anything about our target audience that didn't hold true in that cinematic universe.

After watching the movie series looking for problems to solve, I proposed that I would design a voice assistant for the Ghostbusters. This would be useful for both the novice as well as the experienced Ghostbusters as a way of quickly accessing information about ghosts as well as issuing an SOS, checking the PKE status and checking the containment facility status.


Users, Context and Environment

The Ghostbusters are a paranormal investigation and elimination service originally formed by 3 former scientists (2 real and 1 mostly a fake) in New York City. The Ghostbusters are summoned by clients who experience paranormal phenomena, who then investigate the haunted area, locate the paranormal entity, and subdue it to be transported back to the containment unit back at their headquarters. Their investigation often requires them to go take PKE readings, collect and analyze slime samples, and look up historical records on the Occult Reference Net to find out about the origin of entities and to figure out how to tackle them.

The Ghostbusters’ main weapon is the Proton Gun which requires the use of both hands which makes it hard to access a radio if they need to communicate or call for help when they are in danger. This Proton Gun is connected to a Proton Pack which they carry around on their back. Also, they have gloves on which makes accessing touchscreens almost impossible. They usually go back to their desktop computer at the headquarters to lookup information about ghosts and demons. A voice assistant like Tony Stark’s Jarvis would make their lives easy as well as safe.

via GIPHY

Pictured above, Venkman can’t blast the Slimer ghost with his Proton gun because he is calling for help on the radio. Also, note the thick rubber gloves.


via GIPHY

Pictured above, in Ghostbusters 2 when Vigo the Carpathian resists their attempt at neutralizing him with their Proton guns, he ends up completely immobilizing all four Ghostbusters. They are unable to move their limbs but are still able to talk. In such a scenario a voice-based assistant would be useful for them to send out an SOS for help.


Concerns

Some concerns were raised about the assistant’s ability to work in noisy environments. These will be addressed later in the Evaluation section.


Design process

Hardware

I used Google's Do-It-Yourself kit known as AIY Voice Kit to implement the voice assistant. The major drawback is that it runs on a low-power Raspberry Pi Zero due to which it is not possible to make it voice activated via "trigger words". As a result, for this project, the activation has to be done using a physical button at this stage. But hypothetically, the final version will have to be purely voice-activated.


Software

To create the customized voice assistant I wrote a program in Python and used Google Cloud Speech APIs to analyze the audio input and generate relevant audio output.


Google AIY Kit

The pieces eventually fall into place

Journey Mapping

I mapped the journey of our users, the Ghostbusters, to find pain points where the Voice Assistant would be a valuable addition. Ghostbusters receive a call, visit the site, and take PKE readings. If contact is made, they proceed to try to neutralize or capture the entity. In some cases, if the ghost is a rare entity or is exceptionally strong, they might have to do more research into the history of the entity as well as ways to neutralize it and get back to try again. In some cases, they might have to call other for help (if the team has split up).

Based on this, I came up with the features listed in the next section.


Feature specifications


  • Ghost Wiki: Ask the voice assistant for information about ghosts, demons, and other paranormal entities.


  • Send out an SOS: Starts a loud alarm for easy location of the ghostbuster who issued the SOS. Also broadcasts the location of that ghostbuster to the others.


  • Send out a silent SOS: This is when the ghostbuster wants to ask for help but without drawing the attention of any entities. Issues a silent alarm along with a location broadcast.


  • PKE Readings: Now there’s no need to hold a bulky PKE meter in your hand. Stay hands-free and ask your Voice Assistant for the status.


  • Containment capacity: Find out if your ghost containment unit is full.


Storyboarding


Final Prototype

Ghost wiki




SOS




Silent SOS




PKE Reading




Containment status




Bonus Feature : Party time!




Evaluation

During the coding phase as well as later during the evaluation phase, I encountered a number of usability issues :

  • High latencies: Due to the hardware limitations and network latencies, responses were not immediate.
  • High-noise environment: This was feedback provided by some users and the professor. This is a problem all the current Voice Assistants like Siri and Alexa face but constant improvements are happening in this area to come up with algorithms that can filter background noise better. In this scenario, I would propose that the Ghostbusters use headphones with a mic to interact with the Assistant.
  • Error rates: I had to try a number of different Question Phrases for each scenario and choose the ones with the lowest error rates.
  • Some phrases were discarded for being too long or too short.
  • Some phrases were discarded because, for some reason, Google Speech API was having a hard time identifying them even they had been added as “Expected phrases”.
  • In phrases such as “PKE reading”, users tend to trail off and not pronounce the last letter ‘E’ clearly and hence the assistant was only identifying it as “PK reading”. Had to account for those scenarios.
  • Identification problems due to users with different accents.
  • Similar sounding letters - “PKE” was sometimes identified as “BKE”

THE END