Data collection libraries

The data collection from IMU sensors built-in Android smartphones or Wear OS smartwatches for HAR (and other applications) should be reliable, i.e., reduce the presence of missing data.

However, as indicated previously, it has been documented that a broad amount of passive sensing applications used in published research suffer up to 50% of missing data (Boonstra et al. 2018). As pointed out by González-Pérez et al. (2022), the cause of this missingness comes from the energy consumption restrictions imposed by the OS, which tends to reduce resources and even kill long-running processes (e.g., data collection).

To mitigate these energy restrictions, González-Pérez et al. (2022) propose a set of development guidelines that must be applied to reduce the amount of missing data, showing only a 1% of data missingness in long-running executions.

This section describes the developed libraries implementing these guidelines for data collection in Android smartphones and WearOS smartwatches, addressing research objective RO1.

Background Sensors

The Background Sensors library is an Android library developed in Java that implements a reliable passive collection of IMU sensors (i.e., accelerometer, gyroscope and magnetometer), and, therefore, it can be used to develop native Android applications that require to collect samples from such sensors. The library has been developed to support devices running from Andriod 5.0 (API level 21) to Android 13 (API level 33).

Availability

The version of the library at the moment of writing this thesis is v1.3.0. The library is available in:

• Maven Central Repository
• GitHub
• Zenodo (Matey-Sanz, Casteleyn, and Granell 2024a)

Figure 3.1 includes the simplified architecture of the Background Sensors library.

Figure 3.1: Conceptual architecture of the Background Sensors library.

Library documentation

The full documentation of the library and its components can be found in GitHub.

Sample usage

Tip

Tap on the numbers at the end of the lines to obtain insights about the code.

public class ExampleActivity extends Activity {

    private SensorManager sensorManager;
    private ServiceManager serviceManager;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        sensorManager = new SensorManager(this);
        List<Sensor> sensors = sensorManager.availableSensors(BaseSensor.values());

        serviceManager = new ServiceManager(this, BaseSensorRecordingService.class);
    }

    public void start() {
        Sensor sensor = BaseSensor.ACCELEROMETER;

        CollectionConfiguration config = new CollectionConfiguration(
            sensor,
            10000, // sensorDelay (microseconds)
            100    // batchSize
        );

        RecordCallback callback = new RecordCallback {
            public void onRecordsCollected(List<Record> records) {
                // Accelerometer samples received here
            }
        };

        serviceManager.startCollection(config, callback);
    }

    public void stop() {
        serviceManager.stopCollection(BaseSensor.ACCELEROMETER);
    }
}

1

Create an instance of SensorManager and call availableSensors(...) to determine which sensors are available in the device.

2

Create a ServiceManager instance with a SensorRecordingService.

3

Create a CollectionConfiguration indicating the sensor, the sensorDelay(microseconds) and the batchSize.

4

Implement the RecordCallback. The samples will be received here.

5

Start the data collection calling ServiceManager#startCollection(...).

6

Stop the data collection calling ServiceManager#stopCollection(...).

WearOS Sensors & NativeScript WearOS Sensors

The WearOS Sensors library is a Wear OS library written in Java that can be used to develop native Wear OS applications and, like Background Sensors, implements a reliable passive collection of IMU sensors (i.e., accelerometer, gyroscope and magnetometer). In addition, it supports collecting data from the heart rate sensor and the location services (i.e., GNSS).

On the other hand, NativeScript WearOS Sensors is a library written in TypeScript that can be used to build smartphone applications complementary to WearOS Sensors developed applications. Unlike the other libraries, this one cannot be used to develop native applications but applications built with the NativeScript development framework, which uses web technologies (e.g., JavaScript, TypeScript) to build Android and iOS applications.

The following features are available when using applications developed with these libraries:

• WearOS Sensors:
  • Start and stop from the smartwatch the data collection of the available sensors in the device.
  • Store collected data in the smartwatch.
• WearOS Sensors + Nativescript WearOS Sensors:
  • Start and stop from the smartwatch the data collection of the available sensors in the device.
  • Start and stop from the paired smartphone the data collection of the available sensors in the smartwatch.
  • Store the collected data in the smartwatch or in the smartphone.

The WearOS Sensors and NativeScript WearOS Senosrs libraries have been developed to support devices running from Wear OS 1.0 ( level 23) to Wear OS 4 ( level 33) and from Android 6 ( level 23) to Android 13 ( level 33), respectively.

Availability

The version of WearOS Sensors and NativeScript WearOS Sensors at the moment of writing this thesis are v1.2.1 and v1.3.0. The libraries are available in:

• WearOS Sensors:
  • Maven Central Repository
  • GitHub
  • Zenodo (Matey-Sanz, Casteleyn, and Granell 2024c)
• NativeScript WearOS Sensors:
  • Node Package Manager
  • GitHub
  • Zenodo (Matey-Sanz, Casteleyn, and Granell 2024b)

Figure 3.2 includes the simplified architecture of the WearOS Sensors and NativeScript WearOS Sensors libraries.

Figure 3.2: Conceptual architecture of the Wear OS Sensors and NativeScript WearOS Sensors libraries.

Library documentation

The full documentation of the libraries and their components can be found in their respective repositories: WearOS Sensors and NativeScript WearOS Sensors.

Sample usage

Managing collection process and storing data in smartwatch.

Since WearOS Sensors is built on top of Background Sensors the same code as the one presented previously can be employed for this purpose.

Managing collection process in smartwatch, storing data in smartphone.

public class ExampleActivity extends Activity {
    private CommandClient commandClient;

    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        commandClient = new CommandClient(this);
    }

    public void start() {
        Sensor sensor = WearSensor.ACCELEROMETER;

        CollectionConfiguration config = new CollectionConfiguration(
            sensor,
            10000, // sensorDelay (microseconds)
            100 // batchSize
        );

        commandClient.sendStartCommand(config);
    }

    public void stop() {
        commandClient.sendStopCommand(WearSensor.ACCELEROMETER);
    }
}

1

Create a CommandClient instance.

2

Create a CollectionConfiguration including the sensor, sensorDelay (microseconds) and batchSize.

3

Start the data collection using the CommandClient#sendStartCommand(...).

4

Stop the data collection using the CommandClient#sendStopCommand(...).

function registerListener() {
    const collectorManager = getCollectorManager();

    collectorManager.addSensorListener((sensorRecord: SensorRecord<any>) => {
        // Accelerometer samples received here.
    });
}

5

Obtain a CollectorManager instance using the getCollectorManager() function.

6

Register a listener with the CollectorManager#addSensorListener(...).

Managing collection process and storing data in smartphone.

let collectorManager;
function registerListener() {
    const collectorManager = getCollectorManager();

    collectorManager.addSensorListener((sensorRecord: SensorRecord<any>) => {
        // Accelerometer samples received here.
    });
}

let smartwatch;
async function getConnectedSmartwatch() {
    const nodesDiscovered = await getNodeDiscoverer().getConnectedNodes();
    smartwatch = nodesDiscovered[0];
}

function start() {
    const config: CollectionConfiguration = {sensorInterval: 10000, batchSize: 100};
    
    collectorManager.startCollecting(smartwatch, SensorType.ACCELEROMETER, config);
}

function stop() {
    collectorManager.stopCollecting(smartwatch, SensorType.ACCELEROMETER);
}

1

Obtain a CollectorManager instance using the getCollectorManager() function.

2

Register a listener with the CollectorManager#addSensorListener(...).

3

Use NodeDiscoverer#getConnectedNodes() to get a reference to the connected smartwatches.

4

Create a CollectionConfiguration including the sensorInterval (microseconds) and batchSize.

5

Start the data collection on the selected smartwatch calling CollectorManager#startCollecting(...).

6

Stop the data collection on the selected smartwatch calling CollectorManager#stopCollecting(...).

Integration in the AwarNS Framework

The AwarNS Framework, implemented by a colleague researcher, is an Android-based development framework aimed to facilitate the development of context-aware smartphone applications which require systematic data acquisition, on-device analyses and perform actions based on these analyses (González-Pérez et al. 2023). The framework is built on top of the NativeScript Task Dispatcher, which allows the definition and execution of reactive workflows using its building blocks: Tasks (code units), Events (execution drivers) and TaskGraphs (allow the definition of reactive workflows, i.e., Tasks triggered by Events) (González-Pérez et al. 2022).

This dissertation contributes to the AwarNS Framework by wrapping the previously described data collection tools in it. This integration would allow the combination of already existing features in the framework (e.g., geolocation) with the features of the developed libraries (e.g., IMU data for HAR) to implement context-aware applications (e.g., recognition of activities in specific contexts).

In particular, two new packages are developed and added into the framework: the Phone Sensors and the Wear OS packages. These packages wrap up the Background Sensors and NativeScript WearOS Sensors libraries to integrate their functionality into the AwarNS Framework.

Note

For a more detailed description of the AwarNS Framework, check its GitHub repository and the research paper (González-Pérez et al. 2023)

UML diagram of the integration of Background Sensors and NativeScript WearOS Sensors in AwarNS.

Library documentation

The full documentation of the libraries and their components can be found in the AwarNS Framework repository: Phone Sensors and WearOS Sensors.

Boonstra, Tjeerd W, Jennifer Nicholas, Quincy JJ Wong, Frances Shaw, Samuel Townsend, and Helen Christensen. 2018. “Using Mobile Phone Sensor Technology for Mental Health Research: Integrated Analysis to Identify Hidden Challenges and Potential Solutions.” J. Med. Internet Res. 20 (7): e10131. https://doi.org/10.2196/10131.

González-Pérez, Alberto, Miguel Matey-Sanz, Carlos Granell, and Sven Casteleyn. 2022. “Using Mobile Devices as Scientific Measurement Instruments: Reliable Android Task Scheduling.” Pervasive and Mobile Computing 81: 101550. https://doi.org/10.1016/j.pmcj.2022.101550.

González-Pérez, Alberto, Miguel Matey-Sanz, Carlos Granell, Laura Díaz-Sanahuja, Juana Bretón-López, and Sven Casteleyn. 2023. “AwarNS: A Framework for Developing Context-Aware Reactive Mobile Applications for Health and Mental Health.” Journal of Biomedical Informatics, 104359. https://doi.org/10.1016/j.jbi.2023.104359.

Matey-Sanz, Miguel, Sven Casteleyn, and Carlos Granell. 2024a. “Background Sensors.” Zenodo. https://doi.org/10.5281/zenodo.10635734.

———. 2024b. “NativeScript WearOS Sensors.” Zenodo. https://doi.org/10.5281/zenodo.10640461.

———. 2024c. “WearOS Sensors.” Zenodo. https://doi.org/10.5281/zenodo.10640429.