- Install Unreal 4.25 via Epic Games Launcher:
- Install the Data Glove Unreal SDK zip:
- Unzip the SDK download and navigate to the DataGloveSDK folder
- In your Unreal project’s Plugins directory place the DataGloveSDK folder
- Tips: If your Unreal project has not created a Plugins folder then create a Plugins folder in the same directory as your .uproject file
- Create or open up your Unreal project
- Tips: If your project was open you may need to restart Unreal to load the plugin
- In the Settings/Plugins menu in the Unreal Editor make sure the plugin is loaded and enabled
- In order to see the Plugin Content, in the Content Browser (in the bottom right corner) select “View Options” and check “Show Plugin Content”
- After this you should see the following directories in the Content Broswer
- DataGloveSDK Content
- DataGloveSDK C++ Classes
2. Using the Hand Blueprints
Within the Content/Blueprints folder, are 2 SkeletalMesh Actor Blueprints:
Essentially, these Blueprints are the same, except for a boolean change to determine the “handedness” of the Data Glove, and a re-oriented model.
3. Required Parameters for “Create Data Glove” BluePrint Function
- Is Right Hand (boolean)
- Is Using Bluetooth Device Name
- Check to Use a Bluetooth Device Name to scan for
- Bluetooth Device Name (FName)
- Name of the Data Glove you want to connect to (example: Forte R H100)
- Controller Platform (Enum)
- Oculus Controller
- Used with Rift S and Quest
- SteamVR Tracker
- Used with SteamVR/Vive
- No Tracker
- Used for applications that don’t need VR
- Oculus Controller
- Posable Mesh
- Knuckle Joint Names Array (FName array)
- these names need to match the knuckle finger bones
- Middle Joint Names Array (FName array)
- these names need to match the middle finger bones
- Tip Joint Names Array (FName array)
- these names need to match the tip finger bones
- Knuckle Capsule Colliders (Capsule Collision array)
- Middle Capsule Colliders (Capsule Collision array)
- Tip Capsule Colliders (Capsule Collision array)
4. Setting the Joint Name Arrays
The Joint Name Array parameters in the “Create Data Glove” function are determined by the name of the bones/joints in your model. The Unreal DataGlove SDK uses the SteamVR hand model for the RightHand and LeftHand Blueprints.
The SteamVR hand model utilizes it’s custom joint-naming convention, and thus another model would needs a new set of names to function. At BeBop Sensors, some of our custom hand models use the same naming convention, so only the model needs to be switched out and the Blueprint visual scripting nodes can remain the same.
Below is a picture showcasing the Macro used to set simple FName arrays. (The generic MakeArray node can also be used):
5. Setting Up Finger/Palm Collision and the Capsule Collider Arrays
In order for the Hand Blueprints to interact with the other Interactable Blueprints, colliders are required by some of the C++ modules (Forte_Interactor, Forte_Interactable, Forte_PickUpObject, and Forte_TurnKnob). The Capsule collider on the fingers are set using the CreateDataGlove function node in the Hand Blueprints. Since hand models may differ below is a set of steps required for the current interaction system.
1. Each hand model will require hand-tuning the capsule colliders (and single palm box collider) to fit the 3D mesh appropriately. There are other solutions in Unreal (such as creating custom collision components), but we found capsule colliders simple solutions for both Unreal Engine and Unity3D. Below is an example of the Hand Blueprint colliders on the SteamVR hand model:
2. There are 14 Capsule Collision components for the fingers on the Right/Left Hand Blueprints, with 3 colliders per finger except the thumb has 2. (The last box collider is for the palm)
3. In the Details panel of each collider set the Parent Socket to the appropriate finger bone/joint of your model. This insures the capsule is parented and moves with the correct finger segment.
4. In the Details panel, change the Collision component to match the following collision settings. This insures the appropriate behavior with the Interactable Blueprint objects the glove can interact with:
5. Use the provided Set Collider Array Macro (or the generic MakeArray node) to create arrays for knuckle, middle, tip colliders, which are then used with the Create Data Glove function node.
6. Swapping out a hand 3D model in the Blueprint
Swapping out a hand model is very simple. Under the components tab of the Blueprint (in the top left corner of the Editor), there are 2 Mesh components:
Each component has a mesh field, so simply replace each mesh with the new desired mesh. Make sure that the joint names of the new mesh match the ones being sent via the CreateDataGlove function.
7. Detailed Look at the GloveController Component and exposed Blueprint functions
The GloveController component is a C++ class located in the C++ Classes/DataGlove folder. Here’s an in-depth list of some the features the GloveController component offers:
- The GloveController acts as bridge to handle incoming data from the Data Glove into Unreal and for sending commands to the Data Glove
- The GloveController also handles VR platform differences for Vive and Oculus (since different libraries are required for each platform and the glove uses different positional/rotational offsets)
- The GloveController exposes API functions to the Blueprints to utilize for setup, haptics, VR platforms, calibration, sensor/imu data, device information, and triggers.
- The GloveController handles multi-platform communication (Windows/Android) utilizing the Windows DLL and Android JAR file. The DLL and JAR file act as plugins for the Data Glove API on each platform.
All the Data Glove API functions are exposed for use for the Hand Blueprints GloveController Component.
Notes on Windows DLL:
- The Windows DLL is dynamically loaded.
- Function typedefs are created for the core functions of the Data Glove API.
- Function definitions are declared and defined at runtime.
- The Build.cs file copies the DLL to the local Plugins folder.
Notes on the Android JAR:
- The GloveController uses JNI to find the core Data Glove API functions from the DatagloveAPL.xml
- DatagloveAPL.xml acts a bridge between the C++ and Java functions in the JAR
- The Build.cs file copies the JAR to the Intermediate/Android/APK/libs folder.
The following image showcases the 6 modules the GloveController offers:
- Data Glove and Sensor Information
- Data Glove Calibration
- Data Glove Device Info
- Data Glove Haptic Library
- Contains sub modules for the different types of haptic functions
- Note: New Synth commands are for Data Gloves with firmware version v5.6.3 or greater. The rest of the commands work with any firmware version.
- Data Glove Sensor Information
- Data Glove Setup
- Contains some sub modules with more functions during the setup process
8. Setting up the GloveController
- The setup module of the DataGlove API contains functions for setting up a Data Glove connection, setting default parameters, setting up VR controller offsets, and toggling various components.
- The Data Glove IMU data is utilized when the Platform Controller is set to No Tracker in the “Create Data Glove” function. By default, since VR platforms (Vive/Oculus) support both rotation/translation with their trackers/controllers the Data Glove IMU quaternion data is not needed and only the tracker/controller’s rotational data is utilized.
- The VR Controller positional and rotational offset functions are a way to programmatically set the offset, so the hierarchy of the Blueprints does not need to be changed for every new model that is integrated. The positional and rotational offsets can also be hard-coded in the C++ GloveController as well.
- Position and Rotation offsets are required to correct and fine-tune the mounted orientation of the Vive tracker and Oculus controller setups on the Data Glove.
- Since 3D models are exported with various X/Y/Z forward settings, this can make these offsets mandatory with different models.
9. Calibration Function Nodes
The Calibration module contains functions for homing the glove and calibrating the finger sensors. The homing and calibration commands are mapped to button press events in the Right and Left Hand Blueprints.
Note: With VR (Oculus/Vive), IMU homing is not needed since the tracker/controller’s rotation data is being used instead of the Data Glove IMU data. For non-VR apps, the imu homing command is way to orient the glove, so it correctly matches the users hand orientation.
Below is the calibration function list:
10. Calibration Process
Press the X-button on the Oculus controller or the space bar to initiate the calibration UI process (which utilizes the Blueprint CalibrationVR). This button will walk you through the setup to calibrate the glove, which is 3 different poses while wearing the glove.
Note: Oculus X-button and space bar functionality is defined in the CalibrationVR Blueprint. Other calibration functions for the Oculus controller are defined in the TableTopInteractions Level Blueprint.
Flat Calibration: All fingers are out straight
Fist Calibration: All fingers are in, except thumb
Thumb Calibration: Thumb curled in, while the other fingers are out
There are also hotkeys for calibrating:
F: Flat Calibration
K: Fist Calibration
T: Thumb Calibration
Space Bar: Full Calibration UI menu from the CalibrationVR Blueprint
A Button: Flat Calibration
B Button: Fist Calibration
Right Trigger: Thumb Calibration
X Button: Full Calibration UI menu from the CalibrationVR Blueprint
Up to 16 unique haptic sound files can be resident on the glove with new files rapidly uploaded over Bluetooth or USB. The glove’s custom haptic actuators are non-resonant with a 4+ octave frequency response of 100 Hz to 2000 Hz. The 6 actuators are located on each of the 5 fingertips and palm, with the palm haptic conveying possession of an object. The haptic signals can be pitch shifted and volume adjusted in real-time. Signals can loop continuously or operate in a single play mode. Our API includes easy commands to control the haptics.
The Data Glove has the ability to play back waveforms with looping granular control. A grain is simply a portion of a sound file that plays back in a loop. The loop size can be increased to maximum so that the entire sound file loops, or can be decreased so small that the loop covers a minute portion of the larger sound file.
Complex sounds can be created on any given actuator with creative use of the following parameters: Pitch – Also known as playback speed, this is the rate at which the loop is played. Grain Volume – The playback loudness. A new note with a volume of zero is a note off message. Grain Location – This is the loop start location point within the waveform where the loop will begin. Grain Size – The length of the loop. Grain Fade – The length of the fade in and out of each grain. Waveform Selection – The current selected sound file.
12. Haptic Function Nodes and Good Haptic Practice
The Haptic functions are specified in the haptics sub module of the DataGlove API for visual Blueprint scripting with a GloveController component.
GOOD PRACTICE: Use the haptic function in a state-based manner versus constantly sending commands to the Data Glove in an update loop. For instance, when an object is touched the granular settings could be set and played. Avoid “haptic spamming” in an update loop by sending multiple commands .
- The Base and Granular Synthesis commands are the base elements of playing a waveform and tuning the granular parameters to get specific haptic feedback.
- The Silence, Loop and One Shot modules are helper functions for playing or stopping waveforms on specific fingers, or all the fingers, or all actuators (including the palm).
- The Pulse submodule focuses on sending a haptic for a specific amount of time utilizing Timers. For instance, on touch a user may want to pulse the haptics for 0.2 seconds utilizing a HapticSettings struct (which is just the granular paramters) versus programming that manually.
- The New Synth submodule is for Data Gloves with firmware v5.6.3 or higher. This firmware introduces a selection of new features including LFOs, ADSRs, filters, and an increased selection of granular commands.
13. Device Information Map
The DeviceInformation Map is a scene to showcase the DeviceInformationVR Blueprint, which displays the Data Glove hardware and firmware versioning information. The DeviceInformation Widget utilizes the DataGloveSDK API firmware functions to get this device information.
The DeviceInformation Map can be found DataGloveSDK Content\Maps folder starting in v2.0.2 of the Unreal DataGlove SDK.
The display should look like this: