# Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernel

Source: [Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernel](<https://www.sota2.com/research/paper/68fa72531e209131cea55a42>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:15:31.758Z

Authors: Ziyang Jiang, Tongshu Zheng, Yiling Liu, David Carlson

Published: 2022-05-15T21:32:44.000Z

## Abstract

It is challenging to guide neural network (NN) learning with prior knowledge. In contrast, many known properties, such as spatial smoothness or seasonality, are straightforward to model by choosing an appropriate kernel in a Gaussian process (GP). Many deep learning applications could be enhanced by modeling such known properties. For example, convolutional neural networks (CNNs) are frequently used in remote sensing, which is subject to strong seasonal effects. We propose to blend the strengths of deep learning and the clear modeling capabilities of GPs by using a composite kernel that combines a kernel implicitly defined by a neural network with a second kernel function chosen to model known properties (e.g., seasonality). We implement this idea by combining a deep network and an efficient mapping based on the Nystrom approximation, which we call Implicit Composite Kernel (ICK). We then adopt a sample-then-optimize approach to approximate the full GP posterior distribution. We demonstrate that ICK has superior performance and flexibility on both synthetic and real-world data sets. We believe that ICK framework can be used to include prior information into neural networks in many applications.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [PM2.5 forecasting on Remote Sensing Dataset (test)](<https://www.sota2.com/research/sota/pm2-5-forecasting-on-remote-sensing-dataset-test.md>): PM2.5 forecasting; Remote Sensing Dataset (test); Pearson R: 0.68.

- [Worker Productivity Prediction on UCI Worker Productivity (test)](<https://www.sota2.com/research/sota/worker-productivity-prediction-on-uci-worker-productivity-test.md>): Worker Productivity Prediction; UCI Worker Productivity (test); MSE: 310.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2205.07384>)

[PDF](<https://arxiv.org/pdf/2205.07384.pdf>)

[Implementation](<https://anonymous.4open.science/r/ICK-53EF/>)
