Overview

The Speech App is a mobile application developed in flutter that is used to capture speech from users and sends it to mic.pyfor processing.

The code for this project is in: ~/sawyer_ws/src/communication/speech/speech_app

Controls

• Speech Communication with Robot

Communicates with:

• Rospublisher 'chat' from itself

Topics Publishes to:

Chat

• name: 'chat',
• type: '/common_msgs/ChatMessage',

Speech Status

• name: 'speech_status',
• type: 'common_msgs/SpeechStatus',

Eyes Method

• name: 'eyes/method',
• type: 'common_msgs/LookAt',

Gaze

• name: 'gaze',
• type: 'common_msgs/GazeAt',

Topics Subscribed To:

Speech Status

• name: 'speech_status',
• type: 'common_msgs/SpeechStatus',

To run:

• Download the app from the .apk from the speech channel on slack.

• In order for this app to work, the phone must be connected to EDURoam with a 192.**.*** ip address. If the ip address is 10.**.*.** you will need to connect to the network with OpenVPN.

Pages

The app consists of 3 main pages, namely:

• Connection
• Version Choice
• Speech Attention

Connection

This page is used for choosing the computer with the backend running.

Version Choice

This page is used to choose which version of the app to use. There are two versions:

• with text feedback
• without text feedback

this determines whether the user can read what the phone thinks it heard as they are saying it or if that is hidden.

Speech Attention

This page is used for capturing the speech of the user. Users press and hold on the green button and release to send the audio. To cancel they swipe down. The audio is sent over the chat topic to mic.py

With feedback

Without feedback