SuperVision App

Using the SuperVision App & SDK with emteqPRO

Prerequisites

To use the emteqPRO software you will need:

  1. The SuperVision app.
  2. The EmteqSDK executable for either VR or Open-face configuration.
Note

In order to record and monitor data, you will need to connect the mask to a computer running the Emteq Unity SDK. This can be either using the pre-made demos available on our downloads page, or by creating your own custom Unity app using the emteqVR Unity SDK plugin.

The following sections will explain how to use the SuperVision app and the EmteqSDK to to monitor signals from the emteqPRO mask, and to record a session.

The SuperVision App

SuperVision allows researchers to monitor mask fit, data quality and the session in progress, and displays the contact state of sensors together with real-time signals from each. In this section we describe how to install the app onto your computer and connect to the emteqPRO mask, as well as how to navigate and select sensor displays and filters within the app.

Installation Guide

Please download SuperVision from the downloads section of this website. Next extract and run the downloaded executable from the zip file to install the application. Release Notes are also available here.

Tip

The following guide applies to the emteqPRO system both in ‘VR mode’ (connected to an HTC VIVE Pro) and in ‘Open-Face mode’ (standalone, looking out at the real world!) sometimes referred to as ‘desktop’ as it runs as a normal app on your computer. Please select the appropriate version of the EmteqSDK demo from the download site and follow the instructions below.

SuperVision Install in progress

When the installation is completed it should launch automatically. You can also launch it by clicking the desktop shortcut for the SuperVision application.

It is not required to keep the downloaded zip file, you may delete this file once the SuperVision app is successfully installed.

The SuperVision app will automatically check for new versions and run updates in the background. You will see a Windows notification whenever a new version becomes available.

Removing the Application

To uninstall the SuperVision app from your computer simply remove it as you would any other Windows 10 application via the control panel. Instructions can be found here.

Remove SuperVision app

Installing and Using the emteqPRO Unity SDK

Warning

To stream data from the mask to the SuperVision application you must use the provided SDK.

There are two options to use SDK:

a) Download and simply run one of the compiled unity executables (see downloads page).

b) Download and develop a new Unity application using the emteqPRO SDK (see the SDK the installation guide here.

Once the emteqPRO SDK unity application is running, follow the steps in the next section Connecting the Emteq device to the SuperVision App to connect the emteqPRO mask to the SuperVision.

Connecting the emteqPRO mask and the SuperVision App

Once the emteqPRO SDK unity application is downloaded and running, follow the steps below to connect the emteqPRO mask to the SuperVision application.

Warning

Please make sure that the mask is connected to the HTC VIVE Pro via the USB cable and that the HTC VIVE Pro is on and operating normally. If you are using the Open-face system, please ensure the USB cable is connected to the PC.

Open the SuperVision App on your computer

  • Upon first open the app will show a login dialogue. Login Dialogue
    Opening the SuperVision app for the first time.
  • You can select to login as a ‘Guest' or login with your credentials, as Guest in this example.
    • When logged in as Guest you will not be able to upload data insights, only monitor real-time sessions.
  • You may also ‘signup', or use the forgot password utility, both will link to an external site in your browser of choice.
  • Once logged in this is the intial screen you will see. Post-Login
    SuperVision app the post-login.
  • Note the warning message on the top of the screen and the red connection icon. This is because the mask is not yet connected. We need to connect the mask to the SuperVision app. Click the settings icon on the left of the SuperVision app as shown below, and follow the instructions in the next sections, SuperVision Settings
    The SuperVision settings icon.
  • If no error message is shown, and the connection symbol on the top right of the app is blue, then the mask is successfully connected to the SuperVision. You can test the demo and look at the real-time signals in section Running the SDK Demo Scene.

Otherwise, please follow the steps below on how to connect the emteqPRO mask to the SuperVision.

Connecting to the SuperVision App

Currently the SuperVision can be connected to a mask under the same local area network (LAN). Therefore, it is possible to run the application on a PC other than the one connected to the emteqPRO device and running VR experience.

Click Edit, as per below. Edit button

The SuperVision settings icon.
Running SuperVision on the same PC running the emteqPRO device

If you are connecting the SuperVision app to the emteqPRO mask on the same PC you are currently using, you can simply use the IP address: 127.0.0.1 or type localhost into the SuperVision app. IP Address

Connection tab of the SuperVision app, after pressing Edit. The IP address is connecting to the same PC.
Running SuperVision on a different PC in the same LAN

If you are running the SuperVision on a different PC that the mask is connected to (we will call that the ‘VR PC'), you will need to locate the IP address of the VR PC and type it in the SuperVision. To find the IP address on the Windows VR PC (which is running the VIVE Pro Eye and the emteqPRO mask) follow the steps listed below.

  1. On the Windows PC running the EmteqVR SDK, follow these instructions to find your IP.
  2. A simpler way might be to use the Command Prompt or PowerShell to get the IP using the command ipconfig. Windows Key + X then press i to open PowerShell. SuperVision IP
  3. Type the IP address into the SuperVision app. You can optionally enter the port here too. SuperVision IP
Note

If you do not enter a port number along with the IP (in the above example 127.0.0.1:1883) the SuperVision app will use the default value of 1883.

Running the SDK Demo Scene

  1. Open and run the sample unity scene (Go to Downloads) for the relevant software. or a unity executable that contains the emteqSDK. See below for screen shot of the sample unity scene. Unity demo scene
The Unity demo scene.
  1. Once the SDK is running and mask is connected to the SuperVision, the connection icon on the upper right will show full (light blue), see image below. This indicates the SuperVision app is receiving data from the mask. Connection icon
The connection icon on the SuperVision app when the mask is connected.
  1. Next, the sensors will change from orange to green when in contact with the skin, explained in section Ensuring good signal. The connection icon should now show as blue. If that does not happen, please double-check the IP address and ensure the USB cable connecting the emteqPRO mask to the VR headset and from the VR headset to the computer are securely connected. If the issue persists, please see section ‘Customer support'. Good Connection
Good skin contact of emteqPRO sensors will be shown on the mask view of the SuperVision.

To collect data using the system, you can simply use the emteqSDK scene provided (the data are automatically saved) or use the Unity SDK to trigger data recording. For more information see EmteqVR Unity SDK.

Using the EmteqSDK Unity demo scene

The Emteq Unity demo scene allows you to try some of the demos and start data recordings with the emteqPRO system. The main menu is shown below. EmteqSDK Main menu

EmteqSDK main menu, showing all the different demos available.

All demos and functionalities are explained in EmteqVR Unity SDK.

In this section, we will show the Calibration example. By clicking on the Calibration button, a new view will appear and data will start recording. This section is available in the VR Demo under Affective Videos which contains a calibration section before 2 videos chosen to evoke different emotional responses.

  • A mask will appear each time the sensor contact is not optimal.
  • On the upper right corner, you will see a simplified mask interface, similar to the one seen in the SuperVision App. Sensors that are not touching will show as grey, any sensors struggling to maintain contact will be blue. When all sensors are connected and touching the skin the they will be shown as solid green.
  • If the sensors are connected but you are struggling to achieve good sensor contact refer to section Ensuring good signal. If the sensors are not connected, follow the steps in section Getting started with emteqPRO. Once all sensors have good contact you are ready to begin the calibration. SDK Demo Calibration
    emteqPRO Unity demo SDK, showing the calibration scene when not all sensors are touching the skin. An alert will tell you to adjust the headset until all sensors are green and then you can begin.

    SDK Demo Mask Warning

    Once all sensors are touching the skin you are ready to begin recording the calibration. The top right corner shows a diagram of the emteqPRO mask which allows you to monitor the EMG signals throughout the calibration scene.
  • Follow the on-screen instructions to carry out the expression calibration.
  • Exit the demo.
  • The outputted data file will be saved locally, in the Documents folder on your PC, for example: (C:\Users\$USER\Documents\EmteqLabs\EmteqVR SDK\Upload).

For more information on the data format, see section Understanding the data.

Ensuring Good Signal

Preparation

In order to have successful data collection there needs to be good contact with the sensors. This can be done in a few ways:

  • Prepare the skin. For best results, it is recommended that you remove any skin products or make-up before applying the sensors on the skin. The emteqPRO can operate fully without the need for skin preparation of any kind, however for optimal quality we recommend cleaning the face and remove any products or natural oils from skin by applying a cleanser or a mild abrasive gel.
  • Ensure all hair is pushed off the face, and especially the areas under the mask. The below image demonstrates how to push back any hair covering the face before data collection (e.g. fringes or any loose hairs) to avoid any noisy signals from the sensors Hair
    All hair must be pushed away from the face before using the emteqPRO device to avoid any noisy signals.
  • Before placing your mask, please view the following section the EMG sensors describing the correct positions for the sensors.
  • Ensure you are using the right side inserts for the user's face shape. The side inserts can be removed and swapped as shown below. Two sizes of inserts are provided, the Wide (WL,WR) and Narrow (NL, NR). You can identify the size using the engravings on the backs, as shown below. Mask insert
    The side inserts can be removed and swapped based on the user's face shape. Wide left (top left), wide right (top right), narrow left (bottom left) and narrow right (bottom left).

    Inserts wides with engravings

    The two sets of sensor side inserts. Wide left (top left), wide right (top right), narrow left (bottom left) and narrow right (bottom left).

The EMG sensors

The mask, shown below this list, contains 7 electromyographic (EMG) sensors. These namely are:

  • Frontalis: Left and right side of the forehead
  • Orbicularis: Left and right side of the eyes.
  • Zygomaticus: Left and right side of the cheeks.
  • Corrugator: Measures the corrugator and procerus in the midline.

Sensor locations on mask

Image of the Emteq insert with the different sensors highlighted, showing the frontalis (in red), the orbicularis (in green), the zygomaticus (in blue) and the corrugator (in purple).

Sensor locations on face

Image demonstrating sensor location, with colours corresponding to the above image.

The zygomaticus and orbicularis sensors are part of adjustable sides which can be inserted into the mask. Ensuring the right size-side insert for your face will greatly improve the sensor signal.

The orbicularis sensors should be placed as close to the end of the eye as possible, as long as they are not affecting normal eye movements or comfort. Note, on some faces using the wide inserts may mean the orbicularis sensors placed further away from the eye and thus give weaker signal.

Problems with sensor contact

The SuperVision app uses a traffic light system to help provide a quick indication of sensor contact. The mask icon displayed in the application shows sensors in three different states: grey (no connection), orange (bad connection) and green (good connection). If you are struggling to find a good connection with the sensors after readjusting you can do a few things to help.

After you fit the device, try leaving it on the face for a brief period of time before starting any data collection. We recommend 15-20 seconds. This duration may vary depending on the conditions of the recording. After this step, check the mask diagram on the SuperVision app, if all sensors are green then you have good contact. Generally sensor conductivity may improve over time.

SuperVision sensor contact status

If any sensors are orange it means they are struggling to maintain a strong connection to the skin and the mask needs adjusting. Once all sensors are fully in contact with the skin they will remain solid green.

Grey sensors indicate no contact with the skin.

Orange sensors indicate low/bad contact.

Sensors that stay solid Green indicate a good connection with the skin.

See all possible colour indications in the above image.

To achieve stable contact with the skin, try repositioning the device, tightening the head strap, or changing the sensor inserts (wide or narrow inserts for the eye sensors).

It is important to ensure all sensors have good contact with the skin before recording any data.

Please check the sensor placement (see The EMG Sensors). If any of the sensors are still not green please try the following steps:

  • try lowing the device slightly, to avoid the sensors being placed on the curvature before the hair line. Different face shapes and may need extra attention for the brow sensor set up.
  • Remove any hair or products that might be trapped under the sensor in that area. You can use alcohol swabs to clean the area under the sensor and the sensors themselves. For researchers, it is advisable that all participants cleanse the skin beforehand.
  • To minimise electric potential interference, move to an area which has less electrical cables (e.g., laptop chargers, multi-plugs). You can also try switching off additional monitors or other devices in close proximity. The effect should be instantly visible on the streaming impedance signals on the Terminal app.
  • Press the device slightly against the skin as this sometimes creates long-lasting contact between the skin and sensors. Contact Status poor vs good
    The tab at the top indicates the status contact.

If any sensors are not touching or showing as orange, the tab will say ‘contact status poor' and the mask needs readjusting. Once all sensors have good contact with the skin, the tab will show as ‘status good', then you are able to continue with data collection.

Using SuperVision

  • Once connected to the emteqPRO device, you will see live signals coming through from each of the sensors. To see the signals, click the second tab on the left with the graph icon. Graph Tab
    The Graph Icon takes you to the signal view. In this example a user smiled 4 times which is visible on the activation of the left and right zygomaticus EMG sensors.
  • You are able to view EMG, PPG, IMU accelerometer, magnetometer, gyroscope and HR in beats per minute (BPM), the graphs are shown below. PPG Graph
    The signal view contain all sensors real-time readings. You also manually select and configure the graphing properties.
  • There is also a graph for viewing the event markers the user is experiencing in real time, these events are sent over from the Unity SDK. Event Markers
    Real-time event-markers can be used to annotate important moments during the data-collection.
  • On each graph you are able to select which channels you want to view and the position of the graph on the screen. EMG Graph
    On the right of each graph you are able to select the channels you want to view and relocate the graphs position on the screen.
  • You are also able to edit the graphs using the configure button in the top right corner. This feature allows you to select which channels you want to view, change the amplitude range and apply filters to the signals. Configuration Button
    The configure tab allows you to select which channels you want to view, the filter you want to use on the signal and the amplitude range of the graph.
  • Additionally, you are able to carry out live recordings using the record feature. Pressing start opens up a session management tab allowing you to enter some key features about the data collection. This recording will be saved within Documents on the Users PC, for example C:\Users\[user name]\Documents\EmteqLabs\[application name]\Upload.
  • The connection icon on the top right of the SuperVision shows the connection with the emteqPRO mask. If connection is lost, and the mask is not recognised by the SuperVision, then the icon will turn from blue to red, as shown in. Mask Connection Status
    Connection icon showing when the mask has disconnected.

Data Insights

The Data Insights section allows you to upload your data to the cloud where the data to be processed and returned as a batch of standard CSV files in a compressed zip format. You can access the Data Insights section at any time from the tab on the left of the SuperVision app, even when not connected to the mask.

Data Insights Tab

Once you open the Data Insights section you will be greeted with the option to upload new insights at the top, along with a list of the uploads you have made from the current computer.

Note

This functionality is not available to ‘guests’. Users must be logged in to be able to access upload previous uploads and create new ones. If you are not logged in click on the ‘guest’ button the bottom left of the window. Uploaded files are also only accessible from the same computer they were uploaded from currently.

Uploading New Insights

  • Begin by clicking the [New Insight] button highlighted in the image below. New Upload
  • Specific insight algorithms, and versions of each insight algorithm can be selected prior to upload.
    • These selections will impact the content of the returned zip file once processed.
  • Following the instructions provided on screen select the .dab files you wish to upload into the corresponding fields using the [Browse] buttons. Data Insights Upload Screen
  • Once you have selected your files click the [Start Upload] button to begin uploading the files selected. Upload In Progress
  • Once the upload completed you will be returned to the catalogue screen showing all of your uploads along with their status. Upload In Progress
  • Depending on the size of the data files you uploaded the time taken to process them will vary.
  • Once completed a [download] button will allow you to retrieve the zipped CSV file(s).

The CSV Specification is available to help you understand the data shown in this files.

Upload Status

Each item you upload will have a status:

  • pending - The upload has been received and is being processed.
  • complete - The upload has been successfully processed and a corresponding [download] button will allow you to retrieve the zipped CSV file(s).
  • failed - In the event the files are not processed these will be shown as ‘failed', please contact support for assistance.

Remote Viewer

You can also view a video stream of what the participant sees within the application. This includes audio from the Unity App along with the ability view the video in a floating window.

Using the Remote Viewer Service

Note

You must first connect the SuperVision and Unity apps with sensor data streaming to the SuperVision app (i.e. graph data displayed) before starting the Remote Viewer Service. This enables an automatic connection to the video stream once you start the service.

  • Assuming you are already running both Unity Application with Core Services and the SuperVision app you can start by selecting the Remote Viewing tab on the left. Remote Viewer
  • Click on [Start Service] to being streaming.
  • Once the SuperVision app is running you should see the video stream appear in the top left. Remote Viewer Pop-out
  • You can use the controls below the video thumbnail to: Close, Mute/Unmute, Pop-out and Full Screen the video stream.
  • Once in a separate window the stream can be resized and moved as you wish, the controls for audio remain in the left hand nav of the main SuperVision window unless running in full screen mode.
  • If you are having issues with the stream or are seeing a black screen you can [Restart Viewing Service] to resolve the issue.
  • The [Configure] option allows you to view the currently active port used for stream or make changes based on your network requirements. Remote Viewer Configuration
  • Clicking on [Edit Port] will expand this section, here you can: Reset to default (port 8085) Auto Scan or enter a manual port number, required to be between 8000-9000.
  • Clicking on [Auto Scan] will open a new pop-up window with the option to scan for available port within the range listed above. Remote Viewer Port Scan