Harnessing Device Sensors with the Generic Sensor API: A Comprehensive Guide

Enhance web applications with device sensors using the Generic Sensor API. Access accelerometer, gyroscope, and magnetometer data for immersive experiences. Let's explore how to leverage this API for enhanced user engagement and functionality.

Introducing the Generic Sensor API

The Generic Sensor API bridges the gap between platform-specific and web applications. It allows web developers to access sensor data previously limited to native apps. Use cases include immersive gaming, fitness tracking, and augmented/virtual reality experiences.

Alex Shalamov

Mikhail Pozdnyakov

Thomas Steiner

Streamlining Sensor Access for Web Applications

The Generic Sensor API is a set of interfaces exposing sensor devices to the web platform using a base Sensor interface. Concrete sensor classes extend this base, simplifying implementation. A Gyroscope class, for example, extends the base interface with angular velocity attributes.

Low-Level vs. Fusion Sensors

Low-level sensors interface with hardware like accelerometers and gyroscopes. Fusion sensors, like AbsoluteOrientation, merge data from multiple low-level sensors. This offers ready-to-use data, such as rotation matrices, which would otherwise require complex calculations.

Why Use the Generic Sensor API?

While the web platform offers sensor data via DeviceMotion and DeviceOrientation events, the Generic Sensor API offers significant advantages:

• Extensibility: Easily extendable with new sensor classes while maintaining a generic interface.
• Configurability: Allows configuration of sensor parameters like sampling frequency.
• Availability Detection: Detect sensor availability on the platform.
• High-Precision Timestamps: Provides high-precision timestamps for better data synchronization.
• Clearly Defined Data Models: Ensures interoperable solutions with clearly defined data models and coordinate systems.
• DOM Independence: Not bound to the DOM, enabling use in service workers or headless JavaScript runtimes.
• Enhanced Security: Top priority for security with integration with the Permissions API.
• Automatic Screen Coordinate Synchronization: Available for Accelerometer, Gyroscope, and others.

Available Sensor APIs

Currently, available sensors include:

Motion Sensors:

• Accelerometer
• Gyroscope
• LinearAccelerationSensor
• AbsoluteOrientationSensor
• RelativeOrientationSensor
• GravitySensor

Environmental Sensors:

• AmbientLightSensor (Behind a flag in Chromium)
• Magnetometer (Behind a flag in Chromium)

Feature Detection Techniques

Detecting hardware APIs involves checking browser support and device sensor availability. Here's how to check for Accelerometer support:

if ('Accelerometer' in window) {
 // The `Accelerometer` interface is supported by the browser.
 // Does the device have an accelerometer, though?
}

To ensure meaningful feature detection, attempt to connect to the sensor:

let accelerometer = null;
try {
 accelerometer = new Accelerometer({ frequency: 10 });
 accelerometer.onerror = (event) => {
 // Handle runtime errors.
 if (event.error.name === 'NotAllowedError') {
 console.log('Permission to access sensor was denied.');
 } else if (event.error.name === 'NotReadableError') {
 console.log('Cannot connect to the sensor.');
 }
 };
 accelerometer.onreading = (e) => {
 console.log(e);
 };
 accelerometer.start();
} catch (error) {
 // Handle construction errors.
 if (error.name === 'SecurityError') {
 console.log('Sensor construction was blocked by the Permissions Policy.');
 } else if (error.name === 'ReferenceError') {
 console.log('Sensor is not supported by the User Agent.');
 } else {
 throw error;
 }
}

Utilizing a Polyfill

A polyfill is available for browsers lacking Generic Sensor API support. It allows loading specific sensor implementation as needed:

// Import the objects you need.
import { Gyroscope, AbsoluteOrientationSensor } from './src/motion-sensors.js';

// And they're ready for use!
const gyroscope = new Gyroscope({ frequency: 15 });
const orientation = new AbsoluteOrientationSensor({ frequency: 60 });

Deep Dive into Available Sensors

Let's explore each supported sensor in detail.

Accelerometer and Linear Acceleration Sensor

The Accelerometer measures device acceleration on three axes (X, Y, Z). It's an inertial sensor; in free fall, total measured acceleration is 0 m/s². When flat on a table, upward acceleration (Z axis) equals Earth's gravity (g ≈ +9.8 m/s²).

Accelerometers find use in step counting, motion sensing, and device orientation. Data is often combined with other sources to create fusion sensors.

The LinearAccelerationSensor measures acceleration applied to the device, excluding gravity. At rest, it measures ≈ 0 m/s² on all three axes.

Gravity Sensor

The GravitySensor returns the effect of acceleration along the device's X, Y, and Z axes due to gravity. It is more efficient and accurate than deriving gravity readings manually.

Gyroscope

The Gyroscope sensor measures angular velocity in radians per second around the device's local X, Y, and Z axes. MEMS gyroscopes are prone to drift and consume more power compared to other sensors.

Orientation Sensors

The AbsoluteOrientationSensor measures device rotation relative to the Earth's coordinate system. The RelativeOrientationSensor provides data on device rotation relative to a stationary reference coordinate system.

All modern 3D JavaScript frameworks support quaternions and rotation matrices. OrientationSensor has a quaternion property and a populateMatrix() method.

three.js

let torusGeometry = new THREE.TorusGeometry(7, 1.6, 4, 3, 6.3);
let material = new THREE.MeshBasicMaterial({ color: 0x0071c5 });
let torus = new THREE.Mesh(torusGeometry, material);
scene.add(torus);

// Update mesh rotation using quaternion.
const sensorAbs = new AbsoluteOrientationSensor();
sensorAbs.onreading = () => torus.quaternion.fromArray(sensorAbs.quaternion);
sensorAbs.start();

// Update mesh rotation using rotation matrix.
const sensorRel = new RelativeOrientationSensor();
let rotationMatrix = new Float32Array(16);
sensor_rel.onreading = () => {
 sensorRel.populateMatrix(rotationMatrix);
 torus.matrix.fromArray(rotationMatrix);
};
sensorRel.start();

BABYLON

const mesh = new BABYLON.Mesh.CreateCylinder('mesh', 0.9, 0.3, 0.6, 9, 1, scene);
const sensorRel = new RelativeOrientationSensor({ frequency: 30 });
sensorRel.onreading = () => mesh.rotationQuaternion.FromArray(sensorRel.quaternion);
sensorRel.start();

WebGL

// Initialize sensor and update model matrix when new reading is available.
let modMatrix = new Float32Array([1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]);
const sensorAbs = new AbsoluteOrientationSensor({ frequency: 60 });
sensorAbs.onreading = () => sensorAbs.populateMatrix(modMatrix);
sensorAbs.start();

// Somewhere in rendering code, update vertex shader attribute for the model
gl.uniformMatrix4fv(modMatrixAttr, false, modMatrix);

Orientation sensors enable immersive gaming and augmented/virtual reality experiences.

Screen Coordinate Synchronization

By default, spatial sensors' readings are resolved in a device-bound local coordinate system.

Device coordinate system

For use cases like games or AR/VR, sensor readings need to be resolved in a screen orientation-bound coordinate system.

Screen coordinate system

Previously, remapping required JavaScript implementation. The Generic Sensor API simplifies this by allowing configuration of the local coordinate system for spatial sensor classes like Accelerometer, Gyroscope, and others. Use the referenceFrame option in the sensor constructor to specify device or screen coordinates.

// Sensor readings are resolved in the Device coordinate system by default.
// Alternatively, could be RelativeOrientationSensor({referenceFrame: "device"}).
const sensorRelDevice = new RelativeOrientationSensor();

// Sensor readings are resolved in the Screen coordinate system. No manual remapping is required!
const sensorRelScreen = new RelativeOrientationSensor({ referenceFrame: 'screen' });

Hands-On Coding with the API

The Generic Sensor API is simple to use. The Sensor interface has start() and stop() methods. Event handlers notify about sensor activation, errors, and new readings. Concrete sensor classes add specific reading attributes.

Development Environment