On-Screen Keyboard for Pixel Streamed Games

In pixel-streamed Unreal Engine games, you can define a field that accepts user input to show the on-screen keyboard. This feature enhances the gaming experience on mobile devices by allowing users to input text seamlessly.

To enable this feature, use Pixel Streaming messaging to detect the user's device. Based on the detected device, you can show or hide the on-screen keyboard accordingly. If you are not familiar with the basics of Pixel Streaming messaging, check this link.

You can also download Eagle's feature template from this link to see an example of how it is implemented.

Getting Device information from eagle

You can get device information at any time by sending a Pixel Streaming message with the following JSON payload: {"cmd":"sendDeviceInfo","value":""}. After sending this message, Eagle's system will respond with a message named "userDeviceInfo".

To parse the "userDeviceInfo'' message, first set up an event that will be triggered when the message is received. Use the image below to create your event, or copy the text from this link (compatible with UE5.2). Additionally, you can refer to "E3DSbpLib" Blueprint function library in Eagle's feature template for more details.

Event Handling

When Eagle responds with the "userDeviceInfo" message, you can get device information through the following JSON hierarchy:

{ "userDeviceInfo":{"os": { "name": "#OperatingSystemName" }}}

To get the name field, use the image below as a reference. You can also copy this code or refer to Eagle's feature template "Multiplyaer_PC" player controller Blueprint for more details.

Show on screen Keyboard

Based on the OS name, you can call an event or function to show the on-screen keyboard. However, since your application is running using Pixel Streaming, you will need to inform the front end to show the on-screen keyboard first and then receive the user input data from the front end. To do this, you need to send a Pixel Streaming message with the following JSON payload {"cmd":"lunchKeyBoard" , "id":"#TEXTID"}

#TEXTID can be replaced by any id.

in eagle feature template you can head over to OnScreenKeyboard Widget Blueprint to see our sample code. in this example we are checking if the text box has focus (meaning user has touched inside the textbox) immediately after this is done we send the message to front end to show the on screen keyboard.

After user has typed down their input string and confirmed it via the button on eagle’s front end, front end will send a message to Unreal with the following hierarchy

{ "type" : "TextCommitedForWidgetUsingE3dsOnscreenKeyboard",

"value": "#InputString" }

Where #InputString is the string that the user has typed down; now you can parse this string as your application demands.

In the eagle feature template an event is bound which fires when a pixel streaming message is received by unreal and if it is the type of "TextCommitedForWidgetUsingE3dsOnscreenKeyboard" the value will be read and directly set on the textbox.