Approach

The Open Wearables Initiative will develop a community-based resource to access high-quality algorithms for use in the development of sensor-generated digital measures of health measures through two main functions:

Community hub for digital health algorithms: To facilitate the process of finding and using algorithms for analysis of digital health sensor data, we will create an open central hub to index algorithms and solicit depositions from across the community. A standard meta-data schema will be developed to support developers with a common mechanism to describe, index, and discover digital health algorithms. They will be expected to link their source codes, APIs, and other documentation from GitHub or other public software locations. A uniform licensing agreement will be recommended to support reuse. This resource will facilitate the process of finding and using such algorithms with the goal of creating commonly accepted de facto standards. As the index matures, we expect to promote peer review and independent evaluation of content through collaboration tools, requirements for evidence in support of quality evaluation, and standards for algorithm development. The community hub will be seeded with algorithms arising from two sources: (A) the benchmarking program described below and (B) the Shimmer researcher community.

Community hub for digital health data: To promote transparency and to facilitate the evaluation of algorithms, OWEAR will distribute digital health data. Developers will be encouraged to submit data used in validation of algorithms to foster transparency in algorithm assessment. These data assets may also be (re)used by other researchers in algorithm development. In addition, OWEAR will seek to develop gold-standard data sets for the independent evaluation of algorithms.

Benchmarking program to identify performant algorithms: OWEAR members will work together to identify performant algorithms for digital measures of joint interest by conducting a set of algorithm benchmarking programs. In each case, OWEAR will gather or develop data sets to serve in the evaluation of algorithm performance and then invite analysts to submit their algorithms for performance evaluation. This approach is designed to provide an objective evaluation of the state of the field and to identify best in class algorithms. Because digital measurement endpoint data is typically collected across a broad population in an uncontrolled and unmonitored setting, this effort will also focus on establishing standards for the evaluating performance across a broad range of conditions. Performance metrics will be publicly exposed for all submissions, even in the event that the underlying algorithm remains proprietary. Evaluation queues will remain open to be used as a mechanism to evaluate new algorithms as they are developed relative to existing ones.