Menu

IoT

com.webos.service.ai.voice

Note
This API has been available since API level 11.

API Summary

This service is used to recognize voice input.

It can use and customize the configurations defined by the configuration API (com.webos.service.config).

Overview of the API

In webOS OSE, before using the ai.voice service, you need to set up Google Assistant on your device. The steps are detailed here:

Step A:  Create credentials.json file.

  1. Create a Google account. 
  2. Go to the Actions Console and perform the following steps:
    1. Create a new project.
    2. Register your device model and download the OAuth-2.0 credentials file. (See instructions.)
      Note: It is recommended that you select the device type as Auto to avoid errors during the setup. 
  3. After downloading the file,
    1. Move it from your local PC to /etc/googleAssistant/client_secret.json on your device using the command:
      $ scp <downloaded json file> root@<your device ip>:/etc/googleAssistant/client_secret.json 
    2. From the device, run the following script:
      $ cd /etc/googleAssistant
      $ ./get_credentials.sh
  4. Go to the URL generated by the above script. It provides some code.
  5. Copy the code and paste it on the device terminal and run the code. It creates the 'credentials.json' file. 
  6. Enable Google Assistant API, and configure the consent screen and activity controls for your project/account. (See instructions.)

Step B: Register the device ID.

  1. Open device_id.json file. 
    $ cd /etc/googleAssistant
    $ vi ./device_id.json
  2. Modify the fields:
    • id - Input the device instance string. Typically, 'my_webos'.
    • model_id -  Input the device model ID registered in Step A.
  3. Register the device using the command:
    $ ./register_device_id.sh

Step C: Input the device model and device ID for AI service.

  1. Enter the command:
    $ vi /etc/systemd/system.conf.d/ai.conf
  2. Modify the fields:
    • GOOGLEAI_DEVICE_MODEL: Input the device model ID registered in Step A
    • GOOGLEAI_DEVICE_ID: Input the device instance string registered in Step B

Step D: Reboot device or restart service daemon.

  • Enter the command:
    $ reboot
    OR,
    $ systemctl restart ai

Step E: Launch the service using the start() method.

Note: Google assistant also supports custom device actions.

  • For more information on the custom actions file, refer to /etc/googleAssistant/action.en.json. 
  • Requires the gactions CLI tool.

Methods

getResponse

ACG: ai.voice
  • Added: API level 11

Description

Returns speech recognition results. This method is asynchronous.

Parameters

Name

Required

Type

Description

subscribeRequiredBoolean

Subscribe for notifications. Possible values are:

  • true - Get notifications.
  • false - Notifications are not required.

Default value: false

Call Returns

Name

Required

Type

Description

returnValueRequiredBoolean

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
subscribedOptionalBoolean

Indicates if subscribed to get notified. Possible values are:

  • true - Get notifications.
  • false - Notifications are not required.

Subscription Returns

Name

Required

Type

Description

providerOptionalString

Specifies the AI service provider that returns the results of speech recognition.

In this case, it will always return "googleassistant".

responseOptionalString

The results of speech recognition.

Different providers will provide results in different format.

Example

Example scenario

Sample for webOS OSE users (when using googleassistant)

# luna-send -i -f luna://com.webos.service.ai.voice/getResponse '{"subscribe": true}'

 

Response:

{  
   "returnValue":true
}

 

Googleassistant for webOS OSE case:

Example subscribe for Trigger Word Detection:

{  
   "provider":"googleassistant",
   "response":{  
      "keywordDetected":1
   }
}

 

Example subscribe for Automatic Speech Recognition:

{  
   "provider":"googleassistant",
   "response":{  
      "partial":"what"
   }
}
{  
   "provider":"googleassistant",
   "response":{  
      "partial":"what's"
   }
}
{  
   "provider":"googleassistant",
   "response":{  
      "partial":"what's the"
   }
}
{  
   "provider":"googleassistant",
   "response":{  
      "partial":"what's the weather"
   }
}
{  
   "provider":"googleassistant",
   "response":{  
      "displayText":"It looks pleasant outside Currently in Seoul it's 79 and partly cloudy. The forecast today is 78 and mostly cloudy. ( More on weather.com )"
   }
}

getState

ACG: ai.voice
  • Added: API level 11

Description

Gets the current state of the service. It can have the following states:

  • idle: Before start.
  • waitActivation: Wait for keyword activation.
  • recording: Uploading audio to the server for speech recognition.
  • thinking: Waiting for results from server.
  • answering: Answering to the request through the speaker.

com.webos.service.ai.voice state diagram

Parameters

Name

Required

Type

Description

subscribeOptionalBoolean

Subscribe for notifications. Possible values are:

  • true - Get notifications.
  • false - Notifications are not required.

Default value: false

Call Returns

Name

Required

Type

Description

returnValueRequiredString

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
subscribedOptionalBoolean

Indicates if subscribed to get notified. Possible values are:

  • true - Get notifications
  • false - Notifications are not required
stateRequiredString

Indicates the running state of the service. Possible values are:

  • idle
  • waitActivation
  • recording
  • thinking
  • answering
errorCodeOptionalNumber (int8_t)

The error code for the failed operation.

errorTextOptionalString

Indicates the reason for the failure of the operation. See the "API Error Codes Reference" section for details.

Example

Example scenario

# luna-send -n 1 -f luna://com.webos.service.ai.voice/getState '{}'

 

Response:

{  
   "returnValue":true,
   "state":"idle"
}

 

# luna-send -i -f luna://com.webos.service.ai.voice/getState '{"subscribe": true}'

 

Response:

{  
   "returnValue":true,
   "state":"idle"
}
{  
   "state":"waitActivation"
}
{  
   "state":"recording"
}
...

start

ACG: ai.management
  • Added: API level 11

Description

Starts the service based on the mode selected.

Note: This method is only for allowed clients.

Parameters

Name

Required

Type

Description

modeRequiredString

Select the service mode. Possible values are: 

  • continuous - The service will restart automatically when a conversation ends.
  • single - The service will be stopped when a conversation ends.
keywordDetectRequiredBoolean

Indicates if a keyword is to be used for activating speech recognition. Possible values are:

  • true: Activated by a keyword. The default keyword is "Snowboy". 
    To change the keyword model, refer snowboy.kitt.ai. 
    Download custom the keyword model and move it to /etc/googleAssistant/ on your device. Then, modify KEYWORD_DETECT_MODEL in /var/systemd/system/env/ai.env.
     
  • false: Activated by a button press.

Call Returns

Name

Required

Type

Description

returnValueRequiredBoolean

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
errorCodeOptionalNumber (int8_t)

The error code for the failed operation.

errorTextOptionalString

Indicates the reason for the failure of the operation. See the "API Error Codes Reference" section for details.

Example

Example scenario

# luna-send -n 1 -f luna://com.webos.service.ai.voice/start '{"mode": "continuous", "keywordDetect": true}'

 

Response:

{  
   "returnValue":true
}

stop

ACG: ai.management
  • Added: API level 11

Description

Stops the service. Make sure to call this method before starting another engine.

Note: This method is only for allowed clients.

Parameters

None

Call Returns

Name

Required

Type

Description

returnValueRequiredBoolean

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
errorCodeOptionalNumber (int8_t)

The error code for the failed operation.

errorTextOptionalString

Indicates the reason for the failure of the operation. See the "API Error Codes Reference" section for details.

Example

Example scenario

# luna-send -n 1 -f luna://com.webos.service.ai.voice/stop '{}'

Response:

{  
   "returnValue":true
}

API Error Codes Reference

Error Code

Error Text

Error Description

-11Camera error has occurred

Camera error has occurred.

-10Audio error has occurred

Audio error has occurred.

-9Microphone error has occurred

Microphone error has occurred.

-8Service had already started

Service had already started.

-7Service had not started

Service had not started.

-6Not supported

Not supported api.

-5Network error has occurred

Network error has occurred.

-3Invalid configurations

This error occurs when system missed the environment files.

-2Invalid parameters

This error occurs when api parameters are incorrectly.

-1Unknown error

Unknown error.

0Not error

Not error.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 and sample code is licensed under the Apache License 2.0.

Contents