In my previous post, I discussed the high-level idea of contact tracing and the potential to build technology to aid healthcare workers. In this post, I will delve deeper into the trade-offs that the different technological solutions represent. I have spent the better part of the last decade researching how wireless signals relate to proximity and location. I will distill some of the insights that I have learned over the years into this post. Specifically, I will try to answer a simple question — what are the metrics that we should care about when evaluating any solution for digital contact tracing? In my next post, I will discuss in depth the possible solutions that are being tried out today and how do they fare on these metrics of interest.
Metrics of Interest
Given the importance of digital contact tracing, you will soon hear (or are hearing) about lots of different solutions to enable contact tracing using smartphones. If you are a policymaker, or a retail store manager, or a city administrator, or just someone concerned about COVID-19, what questions should you ask about the contact tracing apps that the technologists are selling to you?
Accuracy
Accuracy is, of course, one of the most important parameters, i.e. any solution that we use should be able to correctly identify contact. Generally, a contact is defined as being closer than 6 feet for more than 10 minutes. According to a recent survey by Microsoft Research, around 85% of the surveyed population was willing to install contact tracing apps that were perfectly accurate. In terms of measuring accuracy, there are two kinds of errors that we care about:
False positives: The number of people falsely identified as being in contact, i.e. people who were never in contact with a COVID-positive patient, but were identified as a contact.
False negatives: The number of people falsely identified as not being in contact, i.e. people who were actually in contact with a COVID-positive patient, but were not identified as one.
As you can imagine, both false positives and false negatives are important numbers. If the number of false positives is high, the app will recommend a lot more tests than needed, thereby causing false alarms, reducing trust in the app, and overwhelming the healthcare resources. On the other hand, if the number of false negatives is high, it will miss out on high-risk individuals and increase the risk to the general population.
The number of false positives and false negatives is intricately linked to the accuracy of the underlying technology being used to measure the distance between individuals. If the technology can identify the difference between three feet, six feet, and nine feet, then it will perform well on this task. On the other hand, if two feet and twenty feet are interchangeable, then the technology will lead to a high number of false positives and/or false negatives.
Privacy
When it comes to apps that track our proximity to others and have access to our health information, privacy is a very important metric. In fact, in the Microsoft research study I quoted above, privacy concerns were almost as high on people’s minds as accuracy. The graph below is a summary of their results.
Source: How good is good enough for COVID19 apps? The influence of benefits, accuracy, and privacy on willingness to adopt. Kaptchuk et al. https://arxiv.org/pdf/2005.04343.pdf There are varying levels of information that these apps can leak about you:
Personal Information: Your name, address, health history, COVID-19 status
Location: Your fine-grained location data, what places you visit, when you go, who do you go there with?
Associations: Who do you travel with, what people do you meet, how frequently do you meet them?
Not all apps collect all of this information. The least privacy-invasive apps will just collect association data and not expose it to anyone but you. The most privacy-invasive apps will collect all of this data and access this data externally. Before you adopt an app or endorse it, be sure to look out for the data it collects and remember to check who can access that data.
Ubiquity
Public health officials believe that any contact tracing app needs to be used by 60 percent of the population for it to be effective. Achieving this density of adoption imposes two requirements:
Universal availability: The technology used for proximity detection has to be universal: it must be compatible with smartphones using outdated software. To understand why this is important, consider the chart of Android users below. Even if one restricts the app to Android 8.0 that was launched in mid-2017, about 25% of global smartphone users won’t be able to use it. Therefore, any apps that use capabilities available only in modern mobile operating systems won’t be widely accessible.
Source: Android Version Market Share worldwide. https://gs.statcounter.com/android-version-market-share/mobile-tablet/worldwide
Scalable design: The app must be able to identify contact not only in homes where few smartphones talk to each other and identify their distance, but also in crowded retail stores, subways, and supermarkets, that have hundreds of people walking around. The design of the apps must be able to handle such large traffic and measure accurate proximity within this large group of people. The other aspect of scalable design is the ability to not crash when tens of millions of people end up using the app — this is the scale that most internet applications dream of. For example, AarogyaSetu, the official Indian government application for digital contact tracing recently crossed 100 million downloads. Apps operating at such a scale need to follow skillful engineering practices.
In summary, if you are a decision-maker that needs to choose an app for yourself, your business, county, city, or country, here is a list of questions to ask:
How accurate is the technology? What is the percentage of false positives and false negatives?
What data does it collect? What data can be accessed on the cloud? Who has access to this data?
What is the minimum requirement for a device that can install and run this app?
How many people can it track in a single location?
How many simultaneous users can it handle?
For completeness, I will wrap up this series of posts by talking about the different technological solutions for proximity detection. Please feel free to leave feedback as a comment.
Originally published at https://www.linkedin.com.
Watch this space for part three.