Since dawn From the iPhone, most of the smarts in smartphones have come from elsewhere: the corporate computers known as the cloud. Mobile apps sent user data to the cloud for useful tasks like transcribing speeches or suggesting replies to messages. Today, Apple and Google claim that smartphones are smart enough to perform some crucial and sensitive machine learning tasks on their own.
At Apple’s WWDC event this month, the company said its virtual assistant Siri will transcribe the speech without using the cloud in certain languages on recent and future iPhones and iPads. At its own I / O developer event last month, Google said that its latest version of its Android operating system has a feature dedicated to handling sensitive data securely on the device, called Private Compute. Core. Its initial uses include powering the company’s version of the Smart Reply feature built into its mobile keypad that can suggest replies to incoming messages.
Apple and Google both claim that machine learning on the device offers more privacy and faster apps. Not transmitting personal data reduces the risk of exposure and saves time waiting for data to cross the Internet. At the same time, the retention of data on devices aligns with the long-term interest of tech giants in keeping consumers connected to their ecosystems. People who hear that their data can be treated more privately might become more willing to agree to share more data.
The recent promotion by companies of on-device machine learning comes after years of working on the technology to restrict the data their clouds can “see”.
In 2014, Google began collecting data on Chrome browser usage through a technique called differential privacy, which adds noise to the data collected in a way that narrows down what these samples reveal about individuals. Apple has used the technique on data collected from phones to inform emoji and typing predictions and for web browsing data.
More recently, the two companies have adopted a technology called federated learning. It allows a cloud-based machine learning system to be updated without recovering raw data; instead, individual devices process data locally and only share digested updates. As with differential privacy, companies have only discussed the use of federated learning in limited cases. Google used this technique to keep its mobile typing predictions up to date with language trends; Apple has published research on its use to update speech recognition models.
Rachel Cummings, an assistant professor at Columbia who was previously a privacy consultant for Apple, says the rapid shift to machine learning on phones has been striking. “It’s incredibly rare to see something go from first conception to full scale deployment in such a few years,” she says.
These advancements have required not only advancements in computing, but also enterprises to address the practical challenges of processing data on consumer-owned devices. Google said its federated learning system only taps users’ devices when they’re plugged in, inactive, and on a free internet connection. The technique was made possible in part by improving the power of mobile processors.
Mobile hardware Beefier also contributed to Google’s announcement in 2019 that voice recognition for its virtual assistant on Pixel devices would be entirely on the device, without the cloud kickstand. Apple’s new on-device voice recognition for Siri, announced at WWDC this month, will use the “neural engine” the company has added to its mobile processors to power machine learning algorithms.
The technical prowess is impressive. It is questionable to what extent they will significantly change the relationship of users with the tech giants.
Apple’s WWDC presenters said Siri’s new design was a “major privacy update” that addressed the risk associated with accidental streaming of audio to the cloud, saying it was was of the utmost privacy concern of users regarding voice assistants. Some Siri commands, such as setting timers, can be recognized entirely locally, allowing for quick response. Yet in many cases, commands transcribed to Siri, possibly including from accidental recordings, will be sent to Apple’s servers for the software to decode and respond. Siri voice transcription will still be cloud-based for HomePod smart speakers commonly installed in bedrooms and kitchens, where accidental recording may be of more concern.
Google is also promoting the handling of data on the device as a privacy victory and has indicated that it will expand the practice. The company expects partners like Samsung who use its Android operating system to embrace the new Privacy Compute Core and use it for features that rely on sensitive data.
Google has also made local analysis of browsing data a feature of its proposal to reinvent online ad targeting, dubbed FLoC and claimed to be more private. Academics and some rival tech companies have said the design will likely help Google consolidate its dominance of online ads by making targeting more difficult for other companies.