22nd March - 4th April 2019
2078 new papers
In this newsletter from Etymo, you can find out the latest development in machine learning research, including the most popular datasets used, the most frequently appearing keywords, the important research papers associated with the keywords, and the most trending papers in the past two weeks.
If you and your friends like this newsletter, you can subscribe to our fortnightly newsletters here.
Fortnight Summary
There are 2078 papers published in the past two weeks, a 25% increase since the previous fortnight. Since the start of these Newsletters, Computer vision (CV) has been the main research area, as reflected on the popularity of the CV datasets and the most trending papers.
It is worth mentioning 'A Sober Look at Neural Network Initializations', a large 56 page paper exploring neural network initialisations. This paper discusses some consequences of commonly used initialization strategies for vanilla DNNs with ReLU activations. Based on these observations, they present a new method of neural net initialization. It would be interesting if someone invented an initialization procedure that interpolated some datapoints?
The datasets remain unchanged, just in a different order, along with the most frequently occuring words: model, learning, data and training. Yet again, this last two weeks include some interesting review/survey papers:
Popular Datasets
Computer vision is still the main focus area of research.
| Name | Type | Number of Papers |
|---|---|---|
| MNIST | Handwritten Digits | 79 |
| ImageNet | Image Dataset | 65 |
| COCO | Common Objects in Context | 34 |
| CIFAR-10 | Tiny Image Dataset in 10 Classes | 30 |
| KITTI | Autonomous Driving | 22 |
| Cityscapes | Images from 50 different cities | 20 |
Trending Phrases
In this section, we present a list of phrases that appeared significantly more in this newsletter than the previous newsletters.
- Reduced Basis:
- Data-Driven Microstructure Property Relations
- Data driven approximation of parametrized PDEs by Reduced Basis and Neural Networks
- Optimize TSK Fuzzy Systems for Big Data Regression Problems: Mini-Batch Gradient Descent with Regularization, DropRule and AdaBound (MBGD-RDA)
- Affect Estimation in 3D Space Using Multi-Task Active Learning for Regression
- Discriminative Supervised Hashing for Cross-Modal Retrieval
- Actor Conditioned Attention Maps for Video Action Detection
Fuzzy System:
Map Results:
Etymo Trending
Presented below is a list of the most trending papers added in the last two weeks.
-
Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation:
This paper proposes a novel geometry-aware 3D representation for the 3D human pose estimation problem. Comprehensive experiments on three popular benchmarks show that their model can significantly improve the performance of state-of-the-art methods by simply injecting the representation as a robust 3D prior. This relatively short 10 page paper comapres their model against existing models using a quantitative comparisons of Mean Per Joint Position Error. -
Scale-Adaptive Neural Dense Features: Learning via Hierarchical Context Aggregation:
This paper introduces SAND features, a dedicated deep learning solution to feature extraction capable of providing hierarchical context information. This method employs sparse relative labels indicating relationships of similarity/dissimilarity between image locations. They demonstrate how the selection of negative examples during training can be used to modify the feature space and vary it's properties. This paper comes with a github repository found here. -
Improving image classifiers for small datasets by learning rate adaptations:
This paper claims to achieve a two-fold to ten-fold speedup in nearing state of the art accuracy, over different model architectures, by dynamically tuning the learning rate. We find it especially beneficial in the case of a small dataset, where reliability of machine reasoning is lower. They validate their approach by comparing our method versus vanilla training on CIFAR-10. CIFAR-10 is a popular computer vision dataset appearing regularly in this newsletter.
Frequent Words
"Model", "Learning", "Data" and "Training" are the most frequent words. The top two papers associated with each of the key words are:
- Model:
- Model Slicing for Supporting Complex Analytics with Elastic Inference Cost and Resource Constraints
- Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools
- Hyperbox based machine learning algorithms: A comprehensive survey
- Deep Learning Techniques for Music Generation - A Survey
- UAFS: Uncertainty-Aware Feature Selection for Problems with Missing Data
- Hyperbox based machine learning algorithms: A comprehensive survey
- General Probabilistic Surface Optimization and Log Density Estimation
- Differentially Private Model Publishing for Deep Learning
Learning:
Data:
Training: