newsletter.etymo

11th January - 24th January 2019

1361 new papers

In this newsletter from Etymo, you can find out the latest development in machine learning research, including the most popular datasets used, the most frequently appearing keywords, the important research papers associated with the keywords, and the most trending papers in the past two weeks.

If you and your friends like this newsletter, you can subscribe to our fortnightly newsletters here.

Fortnight Summary

The number of papers published in the past two weeks is back to 1361, the level we have observed for most part of 2018. Computer vision (CV) is still a main research area, as reflected on the popularity of the CV datasets and the most trending papers.

We present the emerging interests in research under the "Trending Phrases" section. The papers in this section show some cutting edge results. There are two good papers related to Attribute Recognition, an automatically generated description of people's appearance. There is also a new and more consistent algorithm to learn high-quality Node Representation in social and information networks. A very good summary on techniques to tackle False Information is also included. Please read the "Trending Phrases" section for more details.

Other notable development in research includes the following:

  • A new Neural Architecture Search to include network level structure search for better semantic image segmentation: Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
  • A new Transformer network to learn language dependency beyond a fixed-length context: Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
  • An investigation on fundamental requirements for the application of classifier patching on neural networks, an approach for adapting neural network models to handle concept drift in nonstationary environments: Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
  • An answer to a cross-network node classification problem by leveraging the abundant labeled information from a source network to help classify unlabeled nodes in a target network: Network Embedding for Cross-network Node Classification

    • Some of the notable review papers include:
  • Deep Learning for Anomaly Detection: A Survey
  • Taking Human out of Learning Applications: A Survey on Automated Machine Learning
  • Evolving the pulmonary nodules diagnosis from classical approaches to deep learning aided decision support: three decades development course and future prospect

    • And some discussion on the direction of machine learning in general include:
  • A framework to increase the transparency of machine learning technology: Model Cards for Model Reporting
  • A new principle for exploring context in machine learning models: Context Aware Machine Learning
  • A framework to extract reproducible knowledge from data that can guide scientific hypothesis generation and experimental design: Three principles of data science: predictability, computability, and stability (PCS)
  • Discussion about the impact of data science on politics: Data Science as Political Action: Grounding Data Science in a Politics of Justice

    • Popular Datasets

    Computer vision is still the main focus area of research.

    Name Type Number of Papers
    MNIST Handwritten Digits 77
    ImageNet Image Dataset 49
    CIFAR-10 Tiny Image Dataset in 10 Classes 32
    COCO Common Objects in Context 20
    Cityscapes Images from 50 different cities 13
    KITTI Autonomous Driving 12
    CelebA Large-scale CelebFaces Attributes 11

    Trending Phrases

    In this section, we present a list of phrases that appeared significantly more in this newsletter than the previous newsletters.

    Etymo Trending

    Presented below is a list of the most trending papers added in the last two weeks.

    • Deep Learning for Anomaly Detection: A Survey:
      The 50-page paper groups state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted. Within each category, the authors outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior, as well as the advantages and limitations of each technique.

    • Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation:
      The author Neural Architecture Search (NAS) for semantic image segmentation, an important computer vision task that assigns a semantic label to every pixel in an image. The authors propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space. They present a network level search space that includes many popular designs, and develop a formulation that allows efficient gradient-based architecture search (3 P100 GPU days on Cityscapes images). They demonstrate the effectiveness of the proposed method on the challenging Cityscapes, PASCAL VOC 2012, and ADE20K datasets.

    • Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context:
      This paper proposes a novel neural architecture, Transformer-XL, that enables Transformer to learn dependency beyond a fixed length without disrupting temporal coherence. It consists of a segment-level recurrence mechanism and a novel positional encoding scheme. This method not only enables capturing longer-term dependency, but also resolves the problem of context fragmentation. The code, pretrained models, and hyperparameters are available in both Tensorflow and PyTorch.

    Frequent Words

    "Learning", "Model", "Data" and "Network" are the most frequent words. The top two papers associated with each of the key words are:

    Hope you have enjoyed this newsletter! If you have any comments or suggestions, please email ernest@etymo.io or steven@etymo.io.