Joel Shor

I'm a Google / Verily AI researcher working in AI for Social Good. I'm focused on healthcare, accessibility, and speech.

Started two projects that launched internationally (speech recognition and voice recreation for people with medically-impacted speech, and Covid forecasting in Japan). I released one of Big Tech's first AI Fairness reports in Japan for our Covid forecasting model.
Believe in open sourcing (author of TensorFlow GAN and accompanying education materials, paralinguistic speech models and code).
Enjoy making my work understood and accessible (Google AI blog posts on GANs [1, 2, 3], speech [1, 2, 3], accessibility [1], Covid forecasting [1])

Some projects:

Research on Inflammatory Bowel Disease

Developing product to detect and classify polyps in realtime
Custom video compression for preserving medically relevant video content (NeurIPS 2022 workshop paper: Medical imaging meets NeurIPS)
Superresolution and video compression patents
Exploring metabolomic data as a disease predictor

DeepConsensus sequence transformers for PacBio Circular Consensus Sequencing data

Co-first author on NeurIPS workshop paper describing an on-device, error-correcting, long-read gene sequence called DeepConsensus
Pacific Biosciences (PacBio) announced a new instrument called Revio which DeepConsensus onto the instrument. The Revio machine can sequence 1,300 genomes a year (a 15x increase) for $1,000 per genome (4x cheaper) at the highest quality possible for genome analysis [keyword blog, video describing the instrument, marketing material]
Community reception is very positive: “DeepConsensus will run natively on the Revio instrument and will polish every read that comes off it. If you want a peek of the future, it's right here.” An industry expert highlighted DeepConsensus contributions, including enabling machine runtime to speed up from 30h to 24h.

Covid-19 Forecasting in Japan and the US

Launched a 28-day Covid19 public forecast for Japan. Used by top government policymakers to shape Japan’s Covid response. Seen by millions of people worldwide [paper, blog post]
Work won VLED Award for enhancing public good with Big Data, received “Feats of Engineering” award from Google Cloud, and I was recognized with a “Citizen Awards: Collaboration Category”
“Visiting Researcher” at Japan’s top-ranked research university, Keio University, in the school of Medicine, to investigate the reliability of large-scale surveys in helping model the spread of COVID-19 [announcement]

Non-text aspects of speech

Created many novel representations for paralinguistic speech [blog post, Interspeech papers 1, 2, code, models]
Uses include fake audio detection (patent), cough detection (patent), and disordered speech detection [paper], and tracking mental wellbeing (patents 2x). Many external academic and start-up collaborators
State-of-the-art paralinguistic representations, applied to better understanding emotion, user intent, and medically-impacted speakers [papers ex. 1, 2, 3, 4, 5], in ex. ICASSP, Interspeech, and IEEE Signal Processing [patents ex 1, 2, 3]
Work successfully ported to the open source Hugging Face (github)

Accessibility for speech

Co-creator and former tech lead of project Euphonia, which gives patients who lost their voice back the ability to speak with their original voice
Announced at Google I/O 2018 [blog post, paper, Google I/O video, documentary video] and mentioned in Google’s 2018 “Letter to the Founders”

Co-create of TensorFlow GAN, an open-source library for easily training and evaluating Generative Adversarial Networks

[github, blog posts 1, 2, patent, public lecture]
4M downloads; papers using TF-GAN have thousands of citations; 170K students took TF-GAN course in 2020

Invited talks

(2022-10) Google research conference
(2022-04) Tech talk at Boston University’s Math & Statistics department
(2022-04) Tech talk at Boston University’s Mathematical Association of America
(2022-04) Tech talk at MIT’s Department of Electrical Engineering & Computer Science course 6.345/HST.728
(2022-03) Tech Talk at Boston University (link)
(2022-02) Co-presenter at the 4th Conference of the Japan Society of Colon Examination in Hokkaido [poster]
(2022-02) National Institute of Informations (NII), Japan
(2021-12) MIT CSAIL’s Spoken Language Systems Group

AI Blog posts

Public courses

Education