Joel Shor
I'm a Google / Verily AI researcher working in AI for Social Good. I'm focused on healthcare, accessibility, and speech.
Email / Google Scholar / Patents
Started two projects that launched internationally (speech recognition and voice recreation for people with medically-impacted speech, and Covid forecasting in Japan). I released one of Big Tech's first AI Fairness reports in Japan for our Covid forecasting model.
Believe in open sourcing (author of TensorFlow GAN and accompanying education materials, paralinguistic speech models and code).
Enjoy making my work understood and accessible (Google AI blog posts on GANs [1, 2, 3], speech [1, 2, 3], accessibility [1], Covid forecasting [1])
Some projects:
Research on Inflammatory Bowel Disease
Developing product to detect and classify polyps in realtime
Custom video compression for preserving medically relevant video content (NeurIPS 2022 workshop paper: Medical imaging meets NeurIPS)
Superresolution and video compression patents
Exploring metabolomic data as a disease predictor
DeepConsensus sequence transformers for PacBio Circular Consensus Sequencing data
Co-first author on NeurIPS workshop paper describing an on-device, error-correcting, long-read gene sequence called DeepConsensus
Pacific Biosciences (PacBio) announced a new instrument called Revio which DeepConsensus onto the instrument. The Revio machine can sequence 1,300 genomes a year (a 15x increase) for $1,000 per genome (4x cheaper) at the highest quality possible for genome analysis [keyword blog, video describing the instrument, marketing material]
Community reception is very positive: “DeepConsensus will run natively on the Revio instrument and will polish every read that comes off it. If you want a peek of the future, it's right here.” An industry expert highlighted DeepConsensus contributions, including enabling machine runtime to speed up from 30h to 24h.
Covid-19 Forecasting in Japan and the US
Launched a 28-day Covid19 public forecast for Japan. Used by top government policymakers to shape Japan’s Covid response. Seen by millions of people worldwide [paper, blog post]
Work won VLED Award for enhancing public good with Big Data, received “Feats of Engineering” award from Google Cloud, and I was recognized with a “Citizen Awards: Collaboration Category”
“Visiting Researcher” at Japan’s top-ranked research university, Keio University, in the school of Medicine, to investigate the reliability of large-scale surveys in helping model the spread of COVID-19 [announcement]
Non-text aspects of speech
Created many novel representations for paralinguistic speech [blog post, Interspeech papers 1, 2, code, models]
Uses include fake audio detection (patent), cough detection (patent), and disordered speech detection [paper], and tracking mental wellbeing (patents 2x). Many external academic and start-up collaborators
State-of-the-art paralinguistic representations, applied to better understanding emotion, user intent, and medically-impacted speakers [papers ex. 1, 2, 3, 4, 5], in ex. ICASSP, Interspeech, and IEEE Signal Processing [patents ex 1, 2, 3]
Work successfully ported to the open source Hugging Face (github)
Accessibility for speech
Co-create of TensorFlow GAN, an open-source library for easily training and evaluating Generative Adversarial Networks
[github, blog posts 1, 2, patent, public lecture]
4M downloads; papers using TF-GAN have thousands of citations; 170K students took TF-GAN course in 2020
Invited talks
(2022-10) Google research conference
(2022-04) Tech talk at Boston University’s Math & Statistics department
(2022-04) Tech talk at Boston University’s Mathematical Association of America
(2022-04) Tech talk at MIT’s Department of Electrical Engineering & Computer Science course 6.345/HST.728
(2022-03) Tech Talk at Boston University (link)
(2022-02) Co-presenter at the 4th Conference of the Japan Society of Colon Examination in Hokkaido [poster]
(2022-02) National Institute of Informations (NII), Japan
(2021-12) MIT CSAIL’s Spoken Language Systems Group
AI Blog posts
TRILLsson: Small, Universal Speech Representations for Paralinguistic Tasks
FRILL: On-Device Speech Representations using TensorFlow-Lite
Project Euphonia’s Personalized Speech Recognition for Non-Standard Speech
Improving Speech Representations and Personalized Models Using Self-Supervision
Introducing TF-GAN: A lightweight GAN library for TensorFlow 2.0
TFGAN: A Lightweight Library for Generative Adversarial Networks
Public courses
Education
Princeton BA in Mathematics
Manfred Memorial Manfred Pyka Memorial Physics Prize