2 Controversial Privacy Features Are Now In Beta For The Latest Apple Watches - Should You Worry?

watchOS 27.3 beta for developers is here, which means Apple is finally testing two controversial privacy features for the Apple Watch Series 12 and Apple Watch Ultra 4: Live Rewind and Siri Recap. These two functions are part of the Audio Intelligence capabilities offered by the new S11 SoC, Apple Watch's new processor, and Apple Intelligence, and, theoretically, you shouldn't worry about them. First announced during the iPhone Duo event, Apple has been holding up these functions, as it said it would require more time to introduce them — however, the truth is that these features can be really complicated due to the fact that the new Apple Watch models can be listening to everything 24/7, even though Apple says nothing is actually recorded.

According to the company, Live Rewind helps users recall what was just said in a conversation. For example, if you didn't understand what a waiter said, or you want to catch up on something that has been said during a meeting, you can double-press the Digital Crown, which will then show the previous 15 seconds of the conversation as a text snippet. Users can also ask Siri about the context of a text or save it to the Siri app to revisit later.

The other functionality is Siri Recap, which can create "high-level" summaries of a conversation. This way, users can get the main points of a conversation later in the Siri app, something many AI assistants already do.

While these new features may sound like a privacy nightmare at first, Apple is convinced it's offering them in the best way possible. Apple says that even though you can go back to any topic in the past 15 seconds, the Watch never records your voice. This is thanks to a hardware-isolated compartment called Secure Exclave, which is built into the S11 processor. Data processed in the Exclave, such as 24/7 voice recording, can't be accessed by watchOS, apps, the user, or Apple. It's only available as a text snippet of that conversation, and then it disappears for good.

For Siri Recap, the S11 chip recognizes a conversation has taken place, then captures the audio in your Watch's Secure Exclave before transmitting it to your iPhone's Secure Exclave. The iPhone decrypts the audio, and an on-device language model extracts the conversation highlights. After that, the raw audio is immediately and permanently deleted, making it impossible to be found anywhere on the devices. Private Cloud Compute summarizes the condensed text, so users can get more details about where the talk occurred, whether it's attached to a Calendar event, and so on. While there seems to be no reason to worry about your data leaking, Apple still hasn't said how others can know the Apple Watch is recording everything all the time.

Even though Apple proves how secure the new audio features are, those features still have fundamental issues. The first is that people in the conversation aren't aware they are being recorded—even if the recording isn't ultimately stored anywhere. This way, if you're in a restaurant and miss what the waiter said, the most common outcome would be, "Sorry, can you repeat that?" or asking the person in front of you, instead of double-tapping your Watch's Digital Crown.

In a meeting, conversation with a friend, or so on, Apple won't tell the other person in case the Apple Watch user has Siri Recap on, meaning that while the conversation isn't stored line by line, there will be a register, a summary, about what's been discussed. It does admittedly feel concerning that another person is being analyzed by an AI in the iPhone's Notes app, even if we're talking about a local model, without their consent.

While Apple tries to catch up in the AI race, the company seems to be aiming everywhere without considering whether it's actually solving a problem or creating a solution for something users don't want. More than that, these features seem to focus on solving issues that could be resolved with something as simple as asking a question again, showing empathy, and paying attention to what the other person is saying.