OONI Probe Android 2.0.0 Incident Report

The first public release of OONI Probe Android 2.0.0 (since 2.0.0-alpha.7) included a very serious bug: we lost measurements and collected measurements that we shouldn’t have. As soon as the problem was identified, we quickly released a fix (OONI Probe Android 2.0.1). All measurements that should not have been uploaded have been deleted and we are working on a new OONI Probe release (OONI Probe Mobile 2.1.0) that will allow you to manually re-upload measurements. No other OONI Probe platforms (iOS, macOS, Linux, Lepidopter) were affected.

As part of our commitment to transparency, we are publishing this report to share details about the bug, what we did to fix it, and what we are doing to reduce the possibility of similar bugs recurring in the future.

We first discovered the bug after a user reported to us that they were not seeing the report_id inside of the raw data of measurements. Looking at the stats for OONI Probe Android, we noticed that there were significantly fewer submitted measurements from Android users than expected, which made us understand that there was something wrong in how measurements were submitted from Android.

The bug itself was actually pretty simple. We were interpreting the meaning of the setting “Share results” opposite to the intended value. In practice, this meant that Android users who wanted to share their measurements, were not sharing them. But more importantly, it meant that users who didn’t want to share them (because they had opted-out), did.

As soon as we noticed the bug, we swiftly worked on resolving the incident. Specifically, we:

For a full timeline of the incident, see this incident ticket. The reason why we are only disclosing this incident publicly now is because we were still working on ensuring that all the affected data had been deleted and that people who had copies of the data had done the same.

We will soon release another version of OONI Probe (2.1.0) which will allow you to manually upload measurements. This means that if you were an OONI Probe Android 2.0.0 user and you wanted to share your data, you will be able to re-upload your measurements by enabling the “Manual measurement upload” feature in the Sharing Settings.

To limit the possibility of such bugs recurring in the future, we are working on improving our development practices. To this end, we are:

The OONI team sincerely apologizes to the OONI community for this incident. We would never intentionally harm our users, we value and respect user choice, and we take seriously the trust our users have placed in us. We do our best to give you as much control over your data and how you use OONI Probe, but sometimes we make mistakes. This has been a learning experience for us, an opportunity to make improvements to our software development practices and to make our tools more safe and useful to our community.

If you have any questions related to this incident or anything else OONI related, please do not hesitate to reach out to us at: contact@openobservatory.org.