# WeNet: Weighted Networks for Recurrent Network Architecture Search

Source: [WeNet: Weighted Networks for Recurrent Network Architecture Search](<https://www.sota2.com/research/paper/68fa65a11e209131cea0b6c4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T05:57:48.173Z

Authors: Zhiheng Huang, Bing Xiang

Published: 2019-04-08T03:35:07.000Z

## Abstract

In recent years, there has been increasing demand for automatic architecture search in deep learning. Numerous approaches have been proposed and led to state-of-the-art results in various applications, including image classification and language modeling. In this paper, we propose a novel way of architecture search by means of weighted networks (WeNet), which consist of a number of networks, with each assigned a weight. These weights are updated with back-propagation to reflect the importance of different networks. Such weighted networks bear similarity to mixture of experts. We conduct experiments on Penn Treebank and WikiText-2. We show that the proposed WeNet can find recurrent architectures which result in state-of-the-art performance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1904.03819>)

[PDF](<https://arxiv.org/pdf/1904.03819.pdf>)