Emerging trends: Deep nets for poets

• Published in: Natural language engineering 2021-09, Vol.27 (5), p.631-645
• Main Authors: Church, Kenneth Ward, Yuan, Xiaopeng, Guo, Sheng, Wu, Zewu, Yang, Yehua, Chen, Zeyu
• Format: Article
• Language: English
• Subjects: Audiences; Data mining; Datasets; Emerging Trends; Hubs; Ideograph recognition; Image classification; Inference; Input output; Language; Linguistics; Machine translation; Object recognition; Operating systems; Optical character recognition; Programmers; Skills; Speech; Speech recognition; Training; User interface; Voice recognition
• ISSN: 1351-3249; 1469-8110
• DOI: 10.1017/S1351324921000231

Deep nets have done well with early adopters, but the future will soon depend on crossing the chasm. The goal of this paper is to make deep nets more accessible to a broader audience including people with little or no programming skills, and people with little interest in training new models. A github is provided with simple implementations of image classification, optical character recognition, sentiment analysis, named entity recognition, question answering (QA/SQuAD), machine translation, speech to text (SST), and speech recognition (STT). The emphasis is on instant gratification. Non-programmers should be able to install these programs and use them in 15 minutes or less (per program). Programs are short (10–100 lines each) and readable by users with modest programming skills. Much of the complexity is hidden behind abstractions such as pipelines and auto classes, and pretrained models and datasets provided by hubs: PaddleHub, PaddleNLP, HuggingFaceHub, and Fairseq. Hubs have different priorities than research. Research is training models from corpora and fine-tuning them for tasks. Users are already overwhelmed with an embarrassment of riches (13k models and 1k datasets). Do they want more? We believe the broader market is more interested in inference (how to run pretrained models on novel inputs) and less interested in training (how to create even more models).