newsletter.etymo

16th November - 29th November 2018

2000 new papers

In this newsletter from Etymo, you can find out the latest development in machine learning research, including the most popular datasets used, the most frequently appearing keywords, the important research papers associated with the keywords, and the most trending papers in the past two weeks.

If you and your friends like this newsletter, you can subscribe to our fortnightly newsletters here.

Fortnight Summary

Again computer vision (CV) is still strong, as reflected on the popularity of the CV datasets and the most trending papers. Very little change with the most popular datasets, mainly just a reshuffle of the ordering.

Generative Adversarial Networks (GANs) have been popular over the last two weeks with (GAN Dissection: Visualizing and Understanding Generative Adversarial Networks) developing a new framework to help visualise and understand GANs on a unit level, and (On the Implicit Assumptions of GANs) which explain the problems with GANs that apparently contradict theoretical guarantees and provide an alternative approach which sidestep these problems. For anyone unfamiliar with GANs, we recommend NIPS 2016 Tutorial: Generative Adversarial Networks by Ian Goodfellow.

We present the emerging interests in research under the "Trending Phrases" section. The papers in this section shows some cutting edge results, Incremental learning abstract discrete planning domains and mappings to continuous perceptions provides a new framework for planning (model based) reinforcement learning. They allow the agent to dynamically learn new states, map between states and real world, and update these mapping along the life of the agent. Recognizing Plans by Learning Embeddings from Observed Action Distributions also explores planning domain, where they apply Word2Vec on actions to construct plans. They develop a UDUP (Uncertain DUP|) algorithm, where DUP was originally developed in Discovering Underlying Plans Based on Distributed Representations of Actions. Deformable ConvNets v2: More Deformable, Better Results explores spatial support where they present a reformulation of Deformable ConvNets that improves its ability to focus on pertinent image regions. Finally Sequence-based Person Attribute Recognition with Joint CTC-Attention Model propose a joint CTC-Attention model, which maps attribute labels into sequences to learn the semantic relationship among attributes. Attribute recognition is widely used in many computer vision tasks.

The trending of the last two weeks are from Google and Deepmind, a short summary of each trending paper is given in the section below.

Other noticable mentions include ExpandNets: Exploiting Linear Redundancy to Train Small Networks, which introduces another alternative approach to training a small network. They propose to expand each linear layer of a small network into multiple linear layers, without adding any nonlinearity. Please Stop Explaining Black Box Models for High Stakes Decisions want authors to stop explaining black box models, and start creating models that are interpretable in the first place. And finally Dataset Distillation which tries to distill a large dataset into a smaller one. This idea is to form a smaller dataset using the the larger dataset and then train on the smaller dataset instead. They compare their method on the ImageNet and CIFAR-10 datasets.

Popular Datasets

Computer vision is still the main focus area of research. Remember, Yelp are still holding the 12th Yelp Dataset Challenge till Dec 31, 2018, including categories of photo classification, natural language processing and graph mining.

Name Type Number of Papers
MNIST Handwritten Digits 83
ImageNet Image Dataset 63
COCO Common Objects in Context 36
CIFAR-10 Tiny Image Dataset in 10 Classes 34
KITTI Autonomous Driving 21
CelebA Large-scale CelebFaces Attributes 21
RGB-D Scenes Dataset 17

Trending Phrases

In this section, we present a list of words/ phrases that appeared significantly more in this newsletter than the previous newsletters.

Etymo Trending

Presented below is a list of the most trending papers added in the last two weeks.

  • Scalable agent alignment via reward modeling: a research direction:
    The 30 page paper from Deepmind highlights a research direction to solve the agent alignment problem centered around reward modeling. The idea behind reward modelling is to construct a reward function based on user interaction, which can then be used during reinforcement learning. The paper provides a brief literature review and then explains many open problems in the subject.

  • 50 Years of Test (Un)fairness: Lessons for Machine Learning:
    This short paper from Google discusses the definition of `unfairness' in machine learning over the last 50 years. Each decade is given a short section and a detailed overview is provided at the end.

  • Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures:
    Again Deepmind and Google Brain explore variants of target-propagation (TP) and feedback alignment (FA) algorithms for deep neural nets. Results are presented on results on the MNIST, CIFAR-10, and ImageNet datasets. Then conclude that both approached perform significantly worse than backpropogation.

Frequent Words

"Learning", "Model", "Data" and "Training" are the most frequent words. The top two papers associated with each of the key words are:

Hope you have enjoyed this newsletter! If you have any comments or suggestions, please email ernest@etymo.io or steven@etymo.io.