Using Expression Controls
Overview and Getting Started
Expression controls let you use your face, body, and head to send commands to your controller. It uses the camera on your phone or tablet to process your facial features and expressions offline, track changes, and use those changes to trigger commands. When using expression controls, your video and audio is never sent to any third parties and is processed entirely offline on your device to ensure your complete privacy.
To get started using expression controls with your Enabled Play controllers, go to your controller’s page and tap the “Start Expression Controls” button
This will pop up a page that uses your camera. If it’s your first time, you’ll need to give permission for Enabled Play to access your camera first.
You can then keep the camera in front of your face in a stable place (be sure your entire head is in the frame of the camera) and start changing your head position and expressions to send commands.
Expression controls can detect a few different motions and changes including:
– Head position changes (Tilting Up and down, Tilting left and right, Turning left and right)
– Smiling changes (Start or stop smiling)
– Leaning changes (Lean left, Lean right, Lean forward, Lean back, Rise up, Duck down)
– Open and close mouth
– Eyebrow raising
Note: availability of these commands varies on different devices. Certain older versions of iOS cannot support turning heads or smile detections, for example.
Below, we’ll dive deeper into each of these types of commands you can trigger, then show you how to add them to your profiles. Be sure to checkout the “Changing Your Expression Baseline” and “Best Practices and Tips” sections as well for some pointers on making expression controls easier, personalizing it for you, or help solving problems with expression detection.
Head Position Commands
Head position commands allow you to trigger a command by tilting or turning your head in different directions. For example if you…
– look up or down,
– look left or right,
– or tilt your head left or right
The default baseline for detecting these changes and sending commands is as if your nose was in the center of the camera and your head was perfectly straight. This means that if you don’t normally sit or stand in this position, or if you want to position your phone or tablet off-center, make sure to update your baseline (more on that below).
Head position commands are a great tool for controlling character movement in games, navigating documents, or running any quick macro!
Smile Change Commands
Smiling commands let your device detect when you either start or stop smiling (or both!). This feature is usable as long as your mouth is entirely within the view of the camera.
You can choose to create commands for when you start smiling or when you stop smiling. These are great commands for very quick actions (such as jumping, attacking, or moving a character in a game, or saving a document, adding an article to your favorites, and more!). Combining both in one profile is great for “back and forth” or quick and repetitive types of actions in games and apps such as starting to fly in Minecraft, playing certain minigames that need button mashing, or finally closing all your browser tabs at the end of the day!
Mouth Open Commands
Smilar to Smiling commands, you can choose to trigger commands by either opening your moth, closing your mouth, or both! Unlike the smile commands, mouth commands are based on your baseline. This means that if your resting position has your mouth open, then you can open your mouth more to trigger the command.
Note: Mouth commands can conflict with smiling commands if you use them both at the same time.
Leaning commands let you use your body position to trigger commands! These commands include:
These commands are excellent tools for gaming to create some fun and immersive controls! They are also great for productivity such as leaning ducking to scroll down on a page or leaning left and right to move your cursor in a text editor.
You can also trigger a command by raising your eyebrows! This is a great tool to use for quick commands that you might do often or at the same time as other commands such as leaning or head position changes such as jumping in a game, or saving your file.
Updating Your Expression Baseline
The Expression controls work best when your phone or tablet camera is directly in front of your face, your whole face fits in the frame, and your posture is straight. However, this isn’t everyone’s default position, and it can often be hard to get the camera level with your face while you’re using a computer or console. Because of this, you can update your baseline at any point during the expression controls by simply tapping the “Update baseline” button at the bottom.
This will change how Enabled Play calculates your face and head position in order to detect commands. When you tap “update baseline”, it will now use your current position as the baseline for calculating changes. This means if your head naturally tilts left, tilting it so that it is straight up will trigger the “tilt left” command. Or if your camera is now below your face line, looking directly at the camera will trigger the “tilt down” command.
We recommend tapping this button whenever you’re ready to get started and have your phone or tablet in your preferred steady position.
Changing Your Expression Control Sensitivity
Expression sensitivity settings let you fine tune how far you need to move your body, heady, or face in order to trigger commands. Each type of command has its own setting for sensitivity that you can set to have total control over what works best for you.
You can change your settings by going to your device page and then edit settings or from the expression controls page in the app.
To change each setting, drag the slider left or right to decrease or increase the sensitivity. The higher the sensitivity (the further right the slider is), the less you have to move to trigger that command. Inversely, the lower the sensitivity (the further left the slider is), the more you have to move to trigger the command.
Best Practices and Tips
Expression controls are a great tool to help you automate tasks, play games more easily, and multi-task faster. Here are some of our favorite tips and tricks for using expression controls:
– Combine expression controls with other commands like voice or hotkeys as well to do more than you ever could with just two hands
– Combine multiple head movement commands into tasks! You can tilt your head back while also looking to the right, and if you do, it will fire both commands.
– Try to keep your face and body generally centered on the camera view for best results and accuracy.
– If you get up and move around while using expression controls, some unexpected things might happen (such as turning around in your chair at your desk – it may trigger turning left or right because you are actually turning left or right)
– Be careful using winking, smiling, or eyebrow commands for critical outputs if you’re going to use it while talking. People tend to have different facial expressions while talking with others (and also while playing games) that may trigger different commands when you don’t expect it.
– If commands don’t seem to be firing when you expect them to, try these steps:
○ Make sure your entire head is in the frame of the camera view
○ Make sure you have enough lighting so that your whole face is clearly visible in the camera view
○ Make sure nothing is blocking your eyes or mouth (like hair, hands, clothing, etc)
○ Update your baseline
○ If that doesn’t work, restart expression controls by going back to your device page
○ If that doesn’t work, restart your app