Advancements in Language Models, Deep Learning, and Earthquake Detection

Recent developments in large language models include architectural innovations, applications in privacy-preserving federated learning, and the introduction of Regression Language Models (RLMs) for predicting numerical outcomes from unstructured data. Open LLMs are incorporating advancements such as Multi-Head Latent Attention and Mixture-of-Experts. Additionally, new models are being developed for health AI, emphasizing efficiency, multimodality, and privacy. Research also focuses on enhancing reasoning abilities in LLMs through training strategies like reinforcement learning. In the realm of privacy, synthetic data is being used in federated learning to train LLMs, particularly for applications like mobile typing, while maintaining user privacy. RLMs are also being utilized to predict resource efficiency, adapting to new tasks with few-shot learning and offering uncertainty quantification. Finally, collections of open models for health AI are being developed, offering adaptability for specific tasks and addressing privacy concerns.

Recent progress in deep learning spans multiple domains. One area focuses on privacy, with exploration of Fully Homomorphic Encryption (FHE) and its applications. Another highlight is the development and review of SensorLM, a family of sensor-language models that translate wearable sensor data into human-readable language, and LSM-2, which learns from incomplete wearable sensor data. In addition, Google Research is applying deep learning to contactless heart rate monitoring using ultra-wideband radar and introducing Graph Foundation Models (GFM) to enhance machine learning using interconnected relational tables. Finally, improvements to mobile transcription have been made, with SpeechCompass using multi-microphone localization for speaker diarization.

Using accelerometers in Android phones, a system detects earthquakes and sends early warnings. Analyzing data from many phones, it estimates earthquake location and magnitude, alerting users in affected areas. It has detected over 18,000 earthquakes and delivered 790 million alerts across 98 countries, greatly expanding access to early warning systems. User feedback shows that 85% of people found the alerts helpful and took protective actions.