Android SDK

ⅠPreparation

1 Overview

This document serves as the integration guide for the Embodia.ai Embodied Driving SDK Android version. It is intended to guide the usage of the SDK. It assumes that the reader is familiar with the basic use of IDEs (Eclipse or Android Studio) and possesses a foundational knowledge of Android programming.

2 Quick Demo Experience

APK File: The APK file included in the Android package is the installer for the digital human demo. You can install it directly on an Android phone to quickly experience its performance on your device. Access Links:

• GitHub: https://github.com/publicize0828/XmovLiteAvatarAndroidDemo
• Gitee: https://gitee.com/xmovmaster/XmovLiteAvatarAndroidDemo

Sample Project: The demo folder in the Android package contains the digital human sample project. Open the project with Android Studio and follow these steps to configure and run it:

1. Replace the appId and appSecret in demo_configs.json.

2. Configure the settings in demo_configs.json as needed.

3. MockAudioInputsData.json is a sample format for providing custom audio data inputs.

3 Development Environment Setup

(1) Copy SDK to Project:

Copy the .aar package from the libs directory of the SDK into your project's libs directory (create one if it doesn't exist). In the dependencies section of your app's build.gradle, add the dependency as follows:

implementation files('libs/xmovdigitalhuman-xxx.aar')

(2) Add External Third-Party Dependencies:

implementation "javax.vecmath:vecmath:1.5.2"
implementation "com.google.code.gson:gson:2.13.1"
implementation "com.squareup.okhttp3:okhttp:5.1.0"
implementation "org.msgpack:msgpack-core:0.9.3"
implementation "io.socket:socket.io-client:2.1.0"

(3) Configure AndroidManifest.xml:

Add necessary permissions within the <manifest> tag.

<uses-permission android:name="android.permission.INTERNET"/>

(4) ProGuard Rules:

-keep public class com.xmov.metahuman.sdk.data. {;}
-keep public class com.xmov.metahuman.sdk.impl.data. {;}
-keep public class com.xmov.metahuman.sdk.impl.transport.http. {;}
-keep public interface com.xmov.metahuman.sdk.IXmovAvatar {;}
-keep class com.xmov.metahuman.sdk.IXmovAvatar$Companion { ;}
-keep public interface com.xmov.metahuman.sdk.IAvatarListener {
    public protected <methods>;
}
-keep public interface com.xmov.metahuman.sdk.PreCacheListener {
    public protected ;
}

Ⅱ SDK Usage Instructions

1 Initialization

You must initialize the SDK before using any of its functions.

1.1 Method Prototype

fun init(
    context: Context,
    layout: ViewGroup,
    initConfig: InitConfig,
    listener: IAvatarListener?
)

Parameter Description

InitConfig Class Details

Initialization Sample Code

IXmovAvatar.get().init(this, mBinding.avatarLayout, initConfig.appId, initConfig.appSecret, initConfig.gatewayServer, object : IAvatarListener {
    override fun onInitEvent(code: Int, message: String?) {
        // Initialization result callback
        LogUtil.d("onInitEvent code: $code, message: $message")
    }

    override fun onWidgetEvent(widgetData: IRawEventFrameData?) {
        // Callback for widget events (e.g., subtitles)
        LogUtil.d("onWidgetEvent widgetData: $widgetData")
    }

    override fun onNetworkInfo(sdkNetworkInfo: SDKNetworkInfo?) {
        // Callback for network latency and speed
        LogUtil.d("onNetworkInfo $sdkNetworkInfo")
    }

    override fun onMessage(sdkMessage: SDKMessage?) {
        // General SDK message callback
        LogUtil.d("onMessage $sdkMessage")
    }

    override fun onStateChange(state: String?) {
        // Callback when digital human state changes
        LogUtil.d("onStateChange $state")
    }

    override fun onStatusChange(status: SDKStatus?) {
        // Callback when SDK connection status changes
        LogUtil.d("onStatusChange $status")
    }

    override fun onStateRenderChange(state: String?, duration: Long) {
        // Time taken from state change to rendering
        LogUtil.d("onStateRenderChange state: $state, duration: $duration")
    }

    override fun onVoiceStateChange(status: String?) {
        // Callback for audio playback status
        LogUtil.d("onVoiceStateChange status: $status")
        runOnUiThread {
            if ("voice_end" == status) {
                // Speech finished
            } else if ("voice_start" == status) {
                // Speech started
            }
        }
    }

    override fun onDebugInfo(debugInfo: JSONObject) {
        // Debug information callback
    }

    override fun onReconnectEvent(code: Int, message: String?) {
        // Callback for reconnection attempt
        toast("Reconnecting: code=$code message=$message")
    }

    override fun onOfflineEvent() {
        // Triggered when entering offline mode
        toast("Entered offline mode")
    }
})

1.2 Detailed Callback Descriptions

(1) Initialization Callback

onInitEvent(code: Int, message: String?)

The return parameters of the method are divided into the outer code and result, with the following meanings:

(2) Event Callback (Subtitle Callback)

onWidgetEvent(widgetData: IRawEventFrameData?)

IRawEventFrameData Entity Class

(3) Voice Broadcast Callback

onVoiceStateChange(status: String?)

(4) Reconnection Online Mode Callback

onReconnectEvent(code: Int, message: String?)

2 config Configuration

The config in InitConfig is a JSON string used for customization.

Configuration Example

{
  "input_audio": false,
  "output_audio": true,
  "resolution": {
    "width": 1080,
    "height": 1920
  },
  "init_events": [
    {
      "type": "SetCharacterCanvasAnchor",
      "x_location": 0,
      "y_location": 0,
      "width": 1,
      "height": 1
    }
  ]
}

3 Speak

Method Prototype

fun speak(ssml: String?, isStart: Boolean, isEnd: Boolean)

Parameter Description

Code Examples

Full Text Input:

IXmovAvatar.get().speak("Welcome to Embodia.ai", true, true)

Streaming Text Input:

// Start
IXmovAvatar.get().speak("msg segment 1", true, false)
// Middle
IXmovAvatar.get().speak("msg segment 2", false, false)
...
// End
IXmovAvatar.get().speak("msg final segment", false, true)

4 Reconnection

The SDK supports automatic switching to offline mode when the network is unstable. Reconnection starts automatically when the network is restored. Manual switching is also supported. Note: Reconnection can only be initiated from an offline state.

Method Prototype

fun switchModel(isOffline: Boolean)

Parameter Description

5 Destroy

Destroys the SDK instance and releases resources. Call this when the digital human exits.

IXmovAvatar.get().destroy()

Ⅲ Digital Human States

Common action/state switching for the digital human:

1. Listen: IXmovAvatar.get().listen()

2. Think: IXmovAvatar.get().think()

3. Interrupt: IXmovAvatar.get().interrupt()

Ⅳ Return Codes

These codes represent the internal status of the digital human SDK.