Deep Learning Advancements: From Privacy to Wearable Tech

This blog post addresses frequently asked questions about Fully Homomorphic Encryption (FHE). FHE allows computation on encrypted data without decryption, ensuring privacy. It uses cryptographic schemes where operations on ciphertexts yield results corresponding to operations on the underlying plaintext. The author discusses the practicality of FHE in various applications like facial recognition, its quantum-resistant security, and its potential in scenarios where privacy is paramount due to legal requirements, insider risks, or the need for novel services. The article also compares FHE with other privacy-enhancing technologies, like SGX, TEE, and CVMs.

Google Research explores using ultra-wideband (UWB) radar, common in mobile phones, for contactless heart rate monitoring. By applying transfer learning, a deep learning model trained on FMCW radar data was adapted to UWB radar, achieving a mean absolute error of 4.1 bpm. This demonstrates the potential for leveraging existing hardware in consumer devices for health monitoring, paving the way for wider adoption of contactless heart rate measurement in everyday settings and continuous health tracking using mobile devices.

The SensorLM family of sensor-language models bridges the gap between wearable sensor data and human language. Pre-trained on 60 million hours of data from over 103,000 individuals, SensorLM interprets and generates human-readable descriptions from high-dimensional wearable data. It achieves state-of-the-art results in zero-shot sensor understanding, sensor-text alignment, few-shot learning, and sensor caption generation. Experiments show SensorLM's performance consistently improves with more data and larger model sizes, paving the way for personalized insights and future applications in digital health and wellness.

Google introduces Graph Foundation Models (GFM) to leverage interconnected relational tables for improved machine learning. By transforming tables into heterogeneous graphs, GFM captures crucial signals from the graph structure, outperforming traditional tabular methods. A key finding is that models trained on how features interact with each other in diverse tasks leads to better generalization. GFM demonstrates significant performance boosts in internal classification tasks like spam detection in ads, achieving substantial gains compared to single-table baselines, marking a significant step forward in graph learning and tabular ML.

Existing mobile transcription apps struggle to differentiate speakers in group conversations. Google's SpeechCompass enhances mobile captioning using multi-microphone localization for speaker diarization and directional guidance. It employs a TDOA-based algorithm with GCC-PHAT to accurately localize sound, offering visual cues like color-coded text and directional arrows. User studies showed SpeechCompass accurately localizes sound and improves diarization, making group conversations more accessible. The approach has lower computational costs, reduces latency, and enhances privacy preservation compared to ML approaches.

The blog post introduces LSM-2 with Adaptive and Inherited Masking (AIM), a novel self-supervised learning approach that learns directly from incomplete wearable sensor data. AIM addresses the challenge of missing data in wearable sensor streams by treating missingness as a natural artifact, improving upon previous models by reducing reliance on imputation or data removal. LSM-2 demonstrates strong performance across classification, regression, and generative tasks, proving more robust and scalable than its predecessor, especially when handling sensor failures or incomplete data.