
22nd February  7th March 2019
1941 new papers

In this newsletter from Etymo, you can find out the latest development in machine learning research, including the most popular datasets used, the most frequently appearing keywords, the important research papers associated with the keywords, and the most trending papers in the past two weeks.
If you and your friends like this newsletter, you can subscribe to our fortnightly newsletters here.

Fortnight Summary
There are 1941 papers published in the past two weeks. Computer vision (CV) is still a main research area, as reflected on the popularity of the CV datasets and the most trending papers.
We present the emerging interests in research under the "Trending Phrases" section. The papers in this section show some cutting edge results. There are four good papers, each of which is related to Process Mining, Time Parameter, and Causal Graph.
Other notable development in research includes the following:
The topology of weight evolution in neural networks: Topology of Learning in Artificial Neural Networks
A conceptually simple and effective transfer learning approach without pretraining or finetuning: An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models
A study on the quantum learnability of constantdepth classical circuits under the uniform distribution and in the distributionindependent framework of probably approximately correct learning: Quantum hardness of learning shallow classical circuits
A standard pruning technique to uncover subnetworks whose initializations made them capable of effective training: The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
To tackle the model update problem with the recurrent metalearning framework: Learning to Update for Object Tracking with Recurrent Metalearner
A new threat model by characterizing, developing and evaluating new attacks in the brokered learning setting, along with new defenses for these attacks: Dancing in the Dark: Private MultiParty Machine Learning in an Untrusted Setting
Probabilistic Modeling for Novelty Detection with Applications to Fraud Identification
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
A deep learning algorithm with Monte Carlo (MC) and QuasiMonte Carlo (QMC) methods to efficiently compute uncertainty propagation for nonlinear PDEs: Deep learning observables in computational fluid dynamics
An investigation on the model inversion problem in the adversarial settings: Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment
There is also one interesting paper discussing the most common myths in machine learning:
Seven Myths in Machine Learning Research

Popular Datasets
Computer vision is still the main focus area of research.
Name 
Type 
Number of Papers 
MNIST 
Handwritten Digits 
72 
ImageNet 
Image Dataset 
54 
CIFAR10 
Tiny Image Dataset in 10 Classes 
41 
COCO 
Common Objects in Context 
27 
CelebA 
Largescale CelebFaces Attributes 
19 
KITTI 
Autonomous Driving 
14 
Cityscapes 
Images from 50 different cities 
11 

Trending Phrases
In this section, we present a list of phrases that appeared significantly more in this newsletter than the previous newsletters.

Etymo Trending
Presented below is a list of the most trending papers added in the last two weeks.

Topology of Learning in Artificial Neural Networks:
The authors study the emergence of structure in the weights by applying methods from topological data analysis. They train simple feedforward neural networks on the MNIST dataset and monitor the evolution of the weights. When initialized to zero, the weights follow trajectories that branch off recurrently, thus generating trees that describe the growth of the effective capacity of each layer. When initialized to tiny random values, the weights evolve smoothly along twodimensional surfaces. They show that natural coordinates on these learning surfaces correspond to important factors of variation.

An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models:
This paper presents a conceptually simple and effective transfer learning approach that combines the taskspecific optimization function with an auxiliary language model objective, which is adjusted during the training process. This preserves language regularities captured by language models, while enabling sufficient adaptation for solving the target task. The method does not require pretraining or finetuning separate components of the network and models can be trained endtoend in a single step.

Seven Myths in Machine Learning Research:
In this research paper, the authors present seven myths commonly believed to be true in machine learning research:
 Myth 1: TensorFlow is a Tensor manipulation library
 Myth 2: Image datasets are representative of real images found in the wild
 Myth 3: Machine Learning researchers do not use the test set for validation
 Myth 4: Every datapoint is used in training a neural network
 Myth 5: We need (batch) normalization to train very deep residual networks
 Myth 6: Attention > Convolution
 Myth 7: Saliency maps are robust ways to interpret neural networks

Frequent Words
"Learning", "Model", "Data" and "Training" are the most frequent words. The top two papers associated with each of the key words are:

Hope you have enjoyed this newsletter! If you have any comments or suggestions, please email ernest@etymo.io or steven@etymo.io.
