Speech
temi's speech flow is comprised of four main components - wakeup, ASR (Automatic Speech Recognition), NLP (Natural language processing) and TTS (text to speech).
temi's SDK provides developers tools to utilize, customize and listen to any of the components.
API Overview
| Return | Method | Description |
|---|---|---|
| void | speak(TtsRequest ttsRequest) | Ask temi to speak(play TTS) |
| void | cancelAllTtsRequests() | cancel TTS request |
| void | wakeup() | Wake up temi |
| String | getWakeupWord() | Get current wake-up word |
| void | askQuestion(String question) | temi speak actively and wait for user's reply |
| void | finishConversation() | Finish a conversation(Stop recording for ASR) |
| void | startDefaultNlu(String text, SttLanguage sttLanguage) | Trigger default NLU service |
| boolean | setTtsVoice(TtsVoice ttsVoice) | Set TTS voice, speed, and pitch |
| TtsVoice | getTtsVoice() | Trigger default NLU service |
| int | setAsrLanguages() | Set system ASR languages |
| Interface | Description |
|---|---|
| TtsListener | TTS status listener |
| WakeupWordListener | Wake-up event listener |
| AsrListener | ASR result listener |
| ConversationViewAttachesListener | Conversation view attaches listener |
| OnConversationStatusChangedListener | Listener for status chagned of Conversation view |
| OnTtsVisualizerWaveFormDataChangedListener | Listener for wave form data changes of TTS audio visualizer |
| OnTtsVisualizerFftDataChangedListener | Listener for fft data changes of TTS audio visualizer |
| Model | Description |
|---|---|
| TtsRequest | TTS request instance |
| TtsVoice | TTS voice configuration |
| Gender | TTS voice gender |
| SttLanguage | ASR / STT languages |
| WakeupRequest | Wakeup behavior |
| WakeupOrigin | Wakeup Origin |
Methods
speak()
Use this method to let temi speak something that from the parameter ttsRequest of this method.
From 134 verison, if the TtsRequest.id is the same, the requested will be queued when there is an ongoing TTS.
Parameters
Parameter Type Description ttsRequest TtsRequest An object of type TtsRequest in this object you will add the text to be spoken. Prototype
javavoid speak(TtsRequest ttsRequest);Required permissions
None.
Support from
0.10.36
cancelAllTtsRequests()
Stops currently processed TTS request and empty the queue.
Prototype
javavoid cancelAllTtsRequests();Required permissions
None.
Support from
wakeup()
Use this method to trigger temi's wakeup programmatically.
In 132 version, this method can take an optional list to make temi listen to specific langauges. The languages set will temporarily override setAsrLanguages() for current wake up session. Max. 3 extra languages can be set.
In 133 version there is a new overload method wakeup(SttRequest sttRequest) In 135 version there is a new parameter WakeupRequest
Parameters
Parameter Type Description languages List<SttLanguage> Langauges for STT, empty list to use system language. Add in 1.132.0, works with 132 version wakeupRequest WakeupRequest Control behavior of wake up. Add in 1.135.1, works with 135 version Parameter Type Description sttRequest SttRequest Contains languages, and some experimental paramters, added in 133. wakeupRequest WakeupRequest Control behavior of wake up. Add in 1.135.1, works with 135 version Prototype
javavoid wakeup(List\<SttLanguege> languages, WakeupRequest wakeupRequest); void wakeup(SttRequest sttRequest, WakeupRequest wakeupRequest);Required permissions
None.
Support from
0.10.49
getWakeupWord()
Use this method to get temi's wake word assistant.
Return
Type Description String Wake-up word Prototype
javaString getWakeupWord();Required permissions
None.
Support from
0.10.49
askQuestion()
Use this method to let temi actively speak to the user and wait for the user to answer.
In 133 version, there is an overload method to pass TtsRequest, and SttRequest parameter.
Parameters
Parameter Type Description question String The text to be spoken Parameter Type Description question TtsRequest The text to be spoken, support all TtsRequest parameters added in 133 version. sttRequest SttRequest Control the speech in the STT session after the TTS, similiar to wakeup(SttRequest sttRequest), default is null, added in 133 version.Prototype
javavoid askQuestion(String question); void askQuestion(TtsReqeust question, SttRequest sttRequest);Required permissions
None.
Support from
0.10.63
Recommendation
A custom dialog flow can be realized by cooperating with AsrListener. For details, please refer to Sample code.
finishConversation()
Use this method to finish the conversation (Stop recording for ASR).
Prototype
javavoid finishConversation();Required permissions
None.
Support from
0.10.63
startDefaultNlu()
Use this method to trigger the system's default natural language understanding (NLU). If you want to directly trigger system skills such as weather or music in your skill, you can directly pass in "What's the wheather today" or "Play Music" as a parameter and invoke this method to achieve it.
In 1.133.0 version. It will allow to assign system langauge, to trigger an NLU recognition of target language.
Parameters
Parameter Type Description text String Natural language text to be processed sttLanguage SttLanguage Language of NLU, default as SttLangauge.SYSTEM, added in 1.133.0 Prototype
javavoid startDefaultNlu(String text, SttLanguage sttLanguage);Required permissions
Selected Kiosk
Support from
0.10.70
Note
This interface can only be called once every 5 seconds.
setTtsService()
Use this method to configure your own TTS service. After the correct configuration, temi's TTS function requirements will depend on this TTS service. If you want to use temi's original TTS service, call method speak() to do that.
Parameters
Parameter Type Description ttsService ITtsService The instance of class that implemented ITtsService. If the passed paramenter is null, the TTS service will be unbound.Prototype
javavoid setTtsService(ITtsService ttsService);Required permissions
com.robotemi.sdk.metadata.OVERRIDE_TTSdeclared in the manifest to override original TTSSupport from
0.10.77
publishTtsStatus()
Use this method to publish the TTS status from your TTS service to temi.
Parameters
Parameter Type Description ttsReqeust TtsRequest Current instance of TtsRequest (need to include status)) Prototype
javavoid setTtsService(TtsRequest ttsReqeust);Required permissions
com.robotemi.sdk.metadata.OVERRIDE_TTSdeclared in the manifest to override original TTSSupport from
0.10.77
setTtsVoice()
Set TTS voice, speed and pitch. Only available for temi Global version.
Parameters
Parameter Type Description ttsVoice TtsVoice TtsVoice configuration Return
Type Description boolean true if set is successful Prototype
javaboolean setTtsVoice(TtsVoice ttsVoice);Required permissions
SETTINGS
Support from
1.129.0
getTtsVoice()
Get TTS voice, speed and pitch. Only available for temi Global version.
Return
Type Description TtsVoice current TTS voice configuration Prototype
javaTtsVoice getTtsVoice();Support from
1.129.0
setAsrLanguages()
Change ASR langauges, this settings will persist when this kiosk app is running. Max. 3 extra languages can be set.
This languages can be temporarily overriden by wakeup()
Return
Type Description int 0 OK, -1 invalid, 403 no permission Prototype
javaint setAsrLanguages(List\<SttLanguage> languages);Required permissions
KIOSK
Support from
1.132.0
Interfaces
TtsListener
Set your context to implement this listener and add the override method to get TTS status changes.
Prototype
package com.robotemi.sdk;
interface Robot.TtsListener {}Abstract methods
Parameters
Parameter Type Description ttsRequest TtsRequest TTS request object that holds its text content and status Prototype
javavoid onTtsStatusChanged(TtsRequest ttsRequest);
Method for adding listener
Parameters
Parameter Type Description listener TtsListener An instance of a class that implements this interface Prototype
javavoid addTtsListener(TtsListener listener);
Method for removing listener
Parameters
Parameter Type Description listener TtsListener An instance of a class that implements this interface Prototype
javavoid removeTtsListener(TtsListener listener);Support from
0.10.36
WakeupWordListener
Set your context to implement this listener and add the override method to get wake word value when triggered by the user.
Prototype
package com.robotemi.sdk;
interface Robot.WakeupWordListener {}Abstract methods
Parameters
Parameter Type Description wakeupWord String The wakeup word used to trigger direction int - 0 - temi was triggered from the front
- 90 - temi was triggered from the left
- 180 - temi was triggered from the back
- 270 - temi was triggered from the right
- 555 - cannot detect wakeup direction
origin WakeupOrigin The origin of this wakeup event Prototype
javavoid onWakeupWord(String wakeupWord, int direction, WakeupOrigin origin);
Method for adding listener
Parameters
Parameter Type Description listener WakeupWordListener An instance of a class that implements this interface Prototype
javavoid addWakeupWordListener(WakeupWordListener listener);
Method for removing listener
Parameters
Parameter Type Description listener WakeupWordListener An instance of a class that implements this interface Prototype
javavoid removeWakeupWordListener(WakeupWordListener listener);Support from
0.10.36
AsrListener
Set your context to implement this listener and add the override method to get the ASR result.
From 132 version, both the text and language will be returned.
Prototype
package com.robotemi.sdk;
interface Robot.AsrListener {}Abstract methods
Parameters
Parameter Type Description asrResult String The ASR result sttLanguage SttLanguage The language of the ASR result (Added in 132) Prototype
javavoid onAsrResult(String asrResult, SttLanguage sttLanguage);
Method for adding listener
Parameters
Parameter Type Description listener AsrListener An instance of a class that implements this interface Prototype
javavoid addAsrListener(AsrListener listener);
Method for removing listener
Parameters
Parameter Type Description listener AsrListener An instance of a class that implements this interface Prototype
javavoid removeAsrListener(AsrListener listener);Support from
0.10.53
ConversationViewAttachesListener
Set your context to implement this listener and add the override method to listen if the conversation view attaches.
Prototype
package com.robotemi.sdk;
interface Robot.ConversationViewAttachesListener {}Abstract methods
Parameters
Parameter Type Description isAttached boolean truemeans the conversation view attaches,falseotherwisePrototype
javavoid onConversationAttaches(boolean isAttached);
Method for adding listener
Parameters
Parameter Type Description listener ConversationViewAttachesListener An instance of a class that implements this interface Prototype
javavoid addConversationViewAttachesListenerListener(ConversationViewAttachesListener listener);
Method for removing listener
Parameters
Parameter Type Description listener ConversationViewAttachesListener An instance of a class that implements this interface Prototype
javavoid removeConversationViewAttachesListenerListener(ConversationViewAttachesListener listener);Support from
0.10.36
OnConversationStatusChangedListener
Set your context to implement this interface and override its' abstract method to listen to the status and text changes of the Conversation layer.
Note: Only the selected Kiosk App declared in the AndroidManifest.xml file to override the original conversation layer UI can receive the related callback data.
Prototype
package com.robotemi.sdk.listeners;
interface OnConversationStatusChangedListener {}Static constants
All constants here are only for the status of Conversation layer.
| Constant | Type | Value | Description |
|---|---|---|---|
| IDLE | int | 0 | Idle, no useriteraction |
| LISTENING | int | 1 | Listening user's voice |
| THINKING | int | 2 | Doing NLP |
| SPEAKING | int | 3 | Playing TTS |
Abstract methods
Parameters
Parameter Type Description status int Status of Conversation layer text String Text of Conversation layer Prototype
javavoid onConversationStatusChanged(int status, String text);
Method for adding listener
Parameters
Parameter Type Description listener OnConversationStatusChangedListener An instance of a class that implements this interface Prototype
javavoid addOnConversationStatusChangedListener(OnConversationStatusChangedListener listener);
Method for removing listener
Parameters
Parameter Type Description listener OnConversationStatusChangedListener An instance of a class that implements this interface Prototype
javavoid removeOnConversationStatusChangedListener(OnConversationStatusChangedListener listener);Support from
0.10.72
OnTtsVisualizerWaveFormDataChangedListener
Set your context to implements this interface and override its' abstract method to listen to the wave form data changes of the TTS audio visualizer.
Prototype
package com.robotemi.sdk.listeners;
interface OnTtsVisualizerWaveFormDataChangedListener {}Abstract methods
Parameters
Parameter Type Description waveForm byte[] Wave form data Prototype
javavoid onTtsVisualizerWaveFormDataChanged(byte[] waveForm);
Method for adding listener
Parameters
Parameter Type Description listener OnTtsVisualizerWaveFormDataChangedListener An instance of a class that implements this interface Prototype
javavoid addOnTtsVisualizerWaveFormDataChangedListener(OnTtsVisualizerWaveFormDataChangedListener listener);
Method for removing listener
Parameters
Parameter Type Description listener OnTtsVisualizerWaveFormDataChangedListener An instance of a class that implements this interface Prototype
javavoid removeOnTtsVisualizerWaveFormDataChangedListener(OnTtsVisualizerWaveFormDataChangedListener listener);Support from
0.10.72
OnTtsVisualizerFftDataChangedListener
Set your context to implements this interface and override its' abstract method to listen to the fft data changes of the TTS audio visualizer.
Prototype
package com.robotemi.sdk.listeners;
interface OnTtsVisualizerFftDataChangedListener {}Abstract methods
Parameters
Parameter Type Description fft byte[] fft data Prototype
javavoid OnTtsVisualizerFftDataChangedListener(byte[] fft);
Method for adding listener
Parameters
Parameter Type Description listener OnTtsVisualizerFftDataChangedListener An instance of a class that implements this interface Prototype
javavoid addOnTtsVisualizerFftDataChangedListener(OnTtsVisualizerFftDataChangedListener listener);
Method for removing listener
Parameters
Parameter Type Description listener OnTtsVisualizerFftDataChangedListener An instance of a class that implements this interface Prototype
javavoid removeOnTtsVisualizerFftDataChangedListener(OnTtsVisualizerFftDataChangedListener listener);Support from
0.10.72
ITtsService
Implement this interface and override the abstract methods, and use method setTtsService() to bind the TTS service.
Prototype
package com.robotemi.sdk.voice;
interface ITtsService {}Abstract methods
speak()
temi will call this method indirectly to play TTS.
Parameters
Parameter Type Description ttsRequest TtsRequest Pending TtsRequest instance Prototype
javavoid speak(TtsRequest ttsRequest);
cancel()
temi will call this method indirectly to cancel(stop) current TTS.
Prototype
javavoid cancel();
pause()
temi will call this method indirectly to pause current TTS.
Prototype
javavoid pause();
resume()
temi will call this method indirectly to resume current TTS.
Prototype
javavoid resume();Support from
0.10.77
Models
TtsRequest
Request object passed to temi, which contains all the information temi needs to in order to speak and for the skill to track its' request.
Prototype
package com.robotemi.sdk;
class TtsRequest {}Subclass
Status
The status currently in use
Status description STARTED Start playing COMPLETED Finish playing ERROR Errors occurred while playing NOT_ALLOWED Play is not allowed Prototype
javaenum Status { PENDING, PROCESSING, STARTED, COMPLETED, ERROR, NOT_ALLOWED; }
Language
Current supported TTS language
Language Description SYSTEM(0) Follow system EN_US(1) English (United States) ZH_CN(2) Chinese (Mandarin, Simplified) ZH_HK(3) Chinese (Cantonese, Traditional) ZH_TW(4) Chinese (Taiwanese Mandarin) TH_TH(5) Thai (Thailand) HE_IL(6) Hebrew (Israel) KO_KR(7) Korean (Korea) JA_JP(8) Japanese (Japan) ID_ID(10) Indonesian (Indonesia) DE_DE(11) German (Germany) FR_FR(12) French (France) FR_CA(13) French (Canada) PT_BR(14) Portuguese (Brazil) AR_EG(15) Arabic (Egypt) RU_RU(18) Russian (Russia) IT_IT(19) Italian (Italy) PL_PL(20) Polish (Poland) ES_ES(21) Spanish (Spain) CA_ES(22) Catalan (Spain) (supported from 130 version) HI_IN(23) Hindi (supported from 130 version) ET_EE(24) Estonian (supported from 131 version) TR_TR(25) Turkish (supported from 131 version) EN_IN(26) English (India) (supported from 133 version) MS_MY(27) Malay (supported from 134 version) VI_VN(28) Vietnamese (supported from 134 version) EL_GR(29) Greek (supported from 134 version) Prototype
javaenum Language { SYSTEM(0), EN_US(1), ZH_CN(2), ZH_HK(3), ZH_TW(4), TH_TH(5), HE_IL(6), KO_KR(7), JA_JP(8), IN_ID(9), ID_ID(10), DE_DE(11), FR_FR(12), FR_CA(13), PT_BR(14), AR_EG(15), AR_AE(16), AR_XA(17), RU_RU(18), IT_IT(19), PL_PL(20), ES_ES(21), CA_ES(22), HI_IN(23), ET_EE(24), TR_TR(25), EN_IN(26), MS_MY(27), VI_VN(28), EL_GR(29); }
Attributes
| Attribute | Type | Description |
|---|---|---|
| id | UUID | Unique number that identifies each tts request |
| speech | String | The text to be spoken |
| packageName | String | Skill package name so that temi knows who made the request |
| status | Status | Status of the request |
| isShowOnConversationLayer | boolean | Should the conversation line be shown when temi speaks the text. Note: Only relevant for 'Hey temi' assistant skills |
| language | int | Language |
| showAnimationOnly | boolean | true if you want to show a face animation while the speech is ongoing. This only works if there is an assigned interaction animation in temi Settings, otherwise it will just display the text on screen without a face animation. Set this as true will override isShowOnConversationLayer if that value is set to false |
| cached | boolean | true if you want to have this tts cached. Default as false. If there is cache, it will be spoken offline. This is useful for TTS from some sentences you have in the strings.xml (Supported from 129 version) |
Static methods
Create a TtsRequest object and pass it to speak(TtsRequest ttsRequest) method to play TTS. Only speech is mandatory. The other parameters are optional.
Parameters
Parameter Type Description speech String The text to be spoken isShowOnConversationLayer boolean default as true language Language default as Language.SYSTEMshowAnimationOnly boolean default as false cached boolean default as false Return
Type Description TtsRequest TTS request object created by this method Prototype
javastatic TtsRequest create(String speech, boolean isShowOnConversationLayer);
TtsVoice
Tts voice configuration.
Prototype
package com.robotemi.sdk.voice.model;
class TtsVoice {}Attributes
| Attribute | Type | Description |
|---|---|---|
| gender | Gender | only female and male can be used as parameter |
| speed | float | 0.5 - 2.0, stepping by 0.1, default 1.0 |
| pitch | int | -10 - 10, stepping by 1, default 0 |
Gender
Tts voice gender.
Prototype
package com.robotemi.sdk.constants;
enum class Gender {
FEMALE, MALE, GIRL, BOY, UNKNOWN
}SttLanguage
Current supported ASR language
Language Description SYSTEM(0) Follow system EN_US(1) English (United States) ZH_CN(2) Chinese (Mandarin, Simplified) JA_JP(3) Japanese (Japan) KO_KR(4) Korean (Korea) ZH_HK(5) Chinese (Cantonese, Traditional) ZH_TW(6) Chinese (Taiwanese Mandarin) DE_DE(7) German (Germany) TH_TH(8) Thai (Thailand) IN_ID(9) Indonesian (Indonesia) PT_BR(10) Portuguese (Brazil) AR_EG(11) Arabic (Egypt) FR_CA(12) French (Canada) FR_FR(13) French (France) ES_ES(14) Spanish (Spain) CA_ES(15) Catalan (Spain) IW_IL(16) Hebrew (Israel) IT_IT(17) Italian (Italy) ET_EE(18) Estonian TR_TR(19) Turkish HI_IN(20) Hindi, added in 1.133.0 version EN_IN(21) English (India), added in 1.133.0 version MS_MY(22) Malay, added in 134 version VI_VN(23) Vietnamese, added in 134 version RU_RU(24) Russian, added in 134 version EL_GR(25) Greek, added in 134 version
Prototype
enum SttLanguage {
SYSTEM(0),
EN_US(1),
ZH_CN(2),
JA_JP(3),
KO_KR(4),
ZH_HK(5),
ZH_TW(6),
DE_DE(7),
TH_TH(8),
IN_ID(9),
PT_BR(10),
AR_EG(11),
FR_CA(12),
FR_FR(13),
ES_ES(14),
CA_ES(15),
IW_IL(16),
IT_IT(17),
ET_EE(18),
TR_TR(19),
HI_IN(20),
EN_IN(21),
MS_MY(22),
VI_VN(23),
RU_RU(24),
EL_GR(25);
}WakeupRequest
Wakeup request
Prototype
data class WakeupRequest(
val wakeupResponse: Boolean = false
)Attributes
| Attribute | Type | Description |
|---|---|---|
| wakeupResponse | Boolean | whether wakeup response will be triggered |
WakeupOrigin
Wakeup origin
Prototype
enum class WakeupOrigin {
ROBOX,
ANDROID,
TOP_BAR,
SDK,
ANALOG,
UNKNOWN,
}Attributes
| Attribute | Description |
|---|---|
| ROBOX | listening from Robox, apply for V3 |
| ANDROID | listening from Android, apply for temi GO and temi Platform |
| TOP_BAR | trigger by touch on top bar wake up button |
| SDK | trigger by SDK wakeup() method from an app |
| ANALOG | trigger by temi internal flow, like dynamic mode auto-listening, continuous conversation etc. |
| UNKNOWN | unknown origin under current SDK version |
Override original voice flow
Override the NLP
Steps
Add the following
<meta-data>s under the<application>element to AndroidManifest.xml file:java<!-- Kiosk mode is required --> <meta-data android:name="@string/metadata_kiosk" android:value="true" /> <meta-data android:name="@string/metadata_override_nlu" android:value="true" />Listen to the ASR and access your own NLP service in its callback method.
Operate in Launcher: Settings > Kiosk > Select your skill. Or request to be the selected Kiosk App by method
Robot.getInstance().requestToBeKioskApp().
Required permissions
Selected Kiosk
Support from
0.10.63
Override the ASR
Steps
Add the following
<meta-data>s under the<application>element to AndroidManifest.xml file:java<!-- Kiosk mode is required --> <meta-data android:name="@string/metadata_kiosk" android:value="true" /> <meta-data android:name="@string/metadata_override_stt" android:value="true" />Listen to the wake-up event and access your own ASR service in its callback method.
Operate in Launcher: Settings > Kiosk > Select your skill. Or request to be the selected Kiosk App by method
Robot.getInstance().requestToBeKioskApp().
Required permissions
Selected Kiosk
Support from
0.10.70
Override the Conversation layer
Steps
Add the following
<meta-data>s under the<application>element to AndroidManifest.xml file:java<!-- Kiosk mode is required --> <meta-data android:name="@string/metadata_kiosk" android:value="true" /> <meta-data android:name="@string/metadata_override_conversation_layer" android:value="true" />Listen to onConversationStatusChangedListener and draw the UI according to the data(status, text) from its callback method.
Operate in Launcher Settings > App > Kiosk > Select your skill. Or request to be the selected Kiosk App by method
Robot.getInstance().requestToBeKioskApp().
Required permissions
Selected Kiosk
Support from
0.10.72
Override the TTS
Steps
Add the following
<meta-data>s under the<application>element to AndroidManifest.xml file:java<!-- Kiosk mode is required --> <meta-data android:name="@string/metadata_kiosk" android:value="true" /> <meta-data android:name="@string/metadata_override_tts" android:value="true" />Implement interface ITtsService and override methods
speak(),cancel(),pause(),resume(). temi will call the corresponding method to achieve TTS requirements.Use method setTtsService() to bind the instance of the class that implemented ITtsService in the previous step.
[Very important] Use method publishTtsStatus() to publish the TTS status to temi, temi needs these status to respond things(UI, Sequence steps).
Operate in Launcher Settings > App > Kiosk > Select your skill. Or request to be the selected Kiosk App by method
Robot.getInstance().requestToBeKioskApp().Refer to Sample for details.
Required permissions
Selected Kiosk
Support from
0.10.77