Embedded Files
I'm a Google / Verily AI researcher working in AI for Social Good. I'm focused on healthcare, accessibility, and speech.
Email / Google Scholar / Patents
Low-tech intervention to reduce anxiety for patients diagnosed with chronic disease, published in JMIR
AI models for gastroenterology: real-time polyp detectors in colonoscopies, medically-aware video compression [1, 2], improved transcription tools for physicians [1, 2]. Publications in United European Gastroenterology Week, Digestive Disease Week, Gastrointestinal Endoscopy, NeurIPS, ACL Anthology
Improved long-read DNA sequencing with transformers, NeurIPS paper and used by PacBio
Started project Euphonia, which developed personalized speech recognition and voice imitation text-to-speech to let people with medically-impacted speech communicate with their original voice [paper at Interspeech 2019, Google AI blog post, other publications].
Started COVID-19 Forecasting in Japan project, which was seen by millions of people and by the national government to shape their COVID-19 policy [blog, npj Digital Medicine paper. Released one of Big Tech's first AI Fairness reports in Japan for our Covid forecasting model.
Creator of TensorFlow GAN, the Google standard for training and evaluating Generative Adversarial Networks. Created a Google Machine learning crash course on GANs, and an ML tech talk.
Created a series of state-of-the-art speech embeddings for understanding emotion and medical conditions. Used by Google and others for sleep monitoring and cough detection. Publications in Interspeech and ICASSP [papers 1 2 3 4, blogs 1 2]
On the team that created the first indistinguishable-from-human text-to-speech system, Tacotron. Published in ICML
Image and video compression using neural networks
Joined Google through acquisition of targeted video search startup
Some projects:
Research on Inflammatory Bowel Disease
Developing product to detect and classify polyps in realtime
Custom video compression for preserving medically relevant video content (NeurIPS 2022 workshop paper: Medical imaging meets NeurIPS)
Superresolution and video compression patents
Exploring metabolomic data as a disease predictor
DeepConsensus sequence transformers for PacBio Circular Consensus Sequencing data
Co-first author on NeurIPS workshop paper describing an on-device, error-correcting, long-read gene sequence called DeepConsensus
Pacific Biosciences (PacBio) announced a new instrument called Revio which DeepConsensus onto the instrument. The Revio machine can sequence 1,300 genomes a year (a 15x increase) for $1,000 per genome (4x cheaper) at the highest quality possible for genome analysis [keyword blog, video describing the instrument, marketing material]
Community reception is very positive: “DeepConsensus will run natively on the Revio instrument and will polish every read that comes off it. If you want a peek of the future, it's right here.” An industry expert highlighted DeepConsensus contributions, including enabling machine runtime to speed up from 30h to 24h.
Covid-19 Forecasting in Japan and the US
Launched a 28-day Covid19 public forecast for Japan. Used by top government policymakers to shape Japan’s Covid response. Seen by millions of people worldwide [paper, blog post]
Work won VLED Award for enhancing public good with Big Data, received “Feats of Engineering” award from Google Cloud, and I was recognized with a “Citizen Awards: Collaboration Category”
“Visiting Researcher” at Japan’s top-ranked research university, Keio University, in the school of Medicine, to investigate the reliability of large-scale surveys in helping model the spread of COVID-19 [announcement]
Non-text aspects of speech
Created many novel representations for paralinguistic speech [blog post, Interspeech papers 1, 2, code, models]
Uses include fake audio detection (patent), cough detection (patent), and disordered speech detection [paper], and tracking mental wellbeing (patents 2x). Many external academic and start-up collaborators
State-of-the-art paralinguistic representations, applied to better understanding emotion, user intent, and medically-impacted speakers [papers ex. 1, 2, 3, 4, 5], in ex. ICASSP, Interspeech, and IEEE Signal Processing [patents ex 1, 2, 3]
Work successfully ported to the open source Hugging Face (github)
Accessibility for speech
Co-create of TensorFlow GAN, an open-source library for easily training and evaluating Generative Adversarial Networks
[github, blog posts 1, 2, patent, public lecture]
4M downloads; papers using TF-GAN have thousands of citations; 170K students took TF-GAN course in 2020
Invited talks
(2022-10) Google research conference
(2022-04) Tech talk at Boston University’s Math & Statistics department
(2022-04) Tech talk at Boston University’s Mathematical Association of America
(2022-04) Tech talk at MIT’s Department of Electrical Engineering & Computer Science course 6.345/HST.728
(2022-03) Tech Talk at Boston University (link)
(2022-02) Co-presenter at the 4th Conference of the Japan Society of Colon Examination in Hokkaido [poster]
(2022-02) National Institute of Informations (NII), Japan
(2021-12) MIT CSAIL’s Spoken Language Systems Group
AI Blog posts
TRILLsson: Small, Universal Speech Representations for Paralinguistic Tasks
FRILL: On-Device Speech Representations using TensorFlow-Lite
Project Euphonia’s Personalized Speech Recognition for Non-Standard Speech
Improving Speech Representations and Personalized Models Using Self-Supervision
Introducing TF-GAN: A lightweight GAN library for TensorFlow 2.0
TFGAN: A Lightweight Library for Generative Adversarial Networks
Public courses
Education
Princeton BA in Mathematics
Manfred Memorial Manfred Pyka Memorial Physics Prize
Page updated
Google Sites
Report abuse