21st September - 4th October 2018
In this newsletter from Etymo, you can find out the latest development in machine learning research, including the most popular datasets used, the most frequently appearing keywords and the important research papers associated with the keywords, and the most trending papers in the past two weeks.
If you and your friends like this newsletter, you can subscribe to our fortnightly newsletters here.
1366 new papers
Etymo added 1366 new papers published in the past two weeks. These newly published papers on average have 3.9 authors for each paper.
The bar diagram below indicates the number of papers published each day from some major sources, including arXiv, DeepMind, Facebook and etc. This diagram also indicates the pattern of publishing machine learning research papers.
Fortnight Summary
In this newsletter, we added a new section called "Trending Phrases", which show the phrases that appeared significantly more in this newsletter than the previous ones, and the research papers associated with these phrases. Please take a look.
There was still a strong focus on computer vision (CV) in research from the papers published in the last two weeks, as reflected on the popularity of the CV datasets used. The ranking of the datasets appearing in research papers stayed almost the same compared to the last newsletter. The only non-image based dataset is Twitter.
In other areas of machine learning, there were some interesting development in knowledge representation (in robotics and medicine), such as REBA: A Refinement-Based Architecture for Knowledge Representation and Reasoning in Robotics, and Rough set based lattice structure for knowledge representation in medical expert systems: low back pain management case study. Properties and Bayesian fitting of restricted Boltzmann machines explains how the parameter specification of a restricted Boltzmann machine (RBM) is related to model properties such as degeneracy, instability and uninterpretability. This gives us a better fundamental understanding of deep learning.
Unsupervised Hypergraph Feature Selection via a Novel Point-Weighting Framework and Low-Rank Representation presents a new feature selection method, which has low computation cost (much higher efficiency) but similar performance compared with state-of-the-art feature selection methods.
In the past two weeks, there was a great tool developed in R to audit machine learning models of any classes (auditor: an R Package for Model-Agnostic Visual Validation and Diagnostic). In addition, there were some good summaries and reviews to help better understand existing machine learning approaches and the current machine learning status. These summaries/reviews include A Survey of Learning Causality with Data: Problems and Methods, Neural Approaches to Conversational AI, and Computational and informatics advances for reproducible data analysis in neuroimaging. There was also a very popular textbook-like document, An Introduction to Probabilistic Programming, which is also included in the trending section of this newsletter.
The trending of the last two weeks was still skewed towards computer vision: a new state-of-the-art class-conditional image synthesis method using large scale GAN (Large Scale GAN Training for High Fidelity Natural Image Synthesis), and a new general approach using unsupervised learning, called Deep Graph Infomax (DGI), for learning node representations with graph-structured data (Deep Graph Infomax).
Popular Datasets
Computer vision is still the main focus area of research. The ranking of the datasets used has not changed much compared to the last newsletter. SVHN (Street View House Numbers Dataset) is near the top for the first time. It is also the first time we have seen CIFAR-100 near the top, which is the same dataset as CIFAR-10 but images are classified into 100 classes instead of 10.
| Name | Type | Number of Papers |
|---|---|---|
| MNIST | Handwritten Digits | 54 |
| ImageNet | Image Dataset | 42 |
| CIFAR-10 | Tiny Image Dataset in 10 Classes | 28 |
| COCO | Common Objects in Context | 16 |
| Cityscapes | Urban Street Scenes | 12 |
| KITTI | Autonomous Driving | 12 |
| Tweets | 12 | |
| CIFAR-100 | Tiny Image Dataset in 100 Classes | 9 |
| SVHN | The Street View House Numbers Dataset | 8 |
Frequent Words
"Learning", "Model", "Data" and "Set" are the most frequent words, yet again. The top two papers associated with each of the key words are:
- Model:
- an R Package for Model-Agnostic Visual Validation and Diagnostic
- Properties and Bayesian fitting of restricted Boltzmann machines
- A Survey of Learning Causality with Data: Problems and Methods
- Neural Approaches to Conversational AI
- Computational and informatics advances for reproducible data analysis in neuroimaging
- Unsupervised Hypergraph Feature Selection via a Novel Point-Weighting Framework and Low-Rank Representation
- Rough set based lattice structure for knowledge representation in medical expert systems: low back pain management case study
- REBA: A Refinement-Based Architecture for Knowledge Representation and Reasoning in Robotics
Learning:
Data:
Set:
Trending Phrases
We have started monitoring trending words/ phrases. Below are a list of words/ phrases that appeared significantly more in this newsletter than the previous newsletters.
- Probabilistic programming:
- Automated learning with a probabilistic programming language: Birch
- Scenic: Language-Based Scene Generation
- Deep, Skinny Neural Networks are not Universal Approximators
- The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems
- Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs
- Hierarchy-based Image Embeddings for Semantic Image Retrieval
Level sets:
Semantic embedding:
Etymo Trending
Presented below is a list of the most trending papers added in the last two weeks.
-
Large Scale GAN Training for High Fidelity Natural Image Synthesis:
This 29-page paper uses a large scale GAN (Generative Adversarial Networks) model, with orthogonal regularization to the generator, to achieve the new state-of-the-art class-conditional image synthesis. In this paper, the authors explored a fine control of the trade-off between sample fidelity and variety by truncating the latent space. -
An Introduction to Probabilistic Programming:
A textbook style document co-authored by field experts from the US, the UK, Canada and Korea. This document is desinged to introduce probabilistic programming to first-year graduate-level students, or anyone who has an undergraduate-level understading of idealy both probabilistic machine learning and programming languages. This document contains 218 pages. -
Deep Graph Infomax:
This 15-page paper presents Deep Graph Infomax (DGI), a general approach for learning node representations with graph-structured data using unsupervised learning. DGI does not rely on random walk, and is readily applicable to both transductive and inductive learning setups. Its performance is competitive on a variety of node classification benchmarks, and sometimes exceeding the performance of supervised learning.