Who Cited It

Distilling the Knowledge in a Neural Network

2015 · arXiv (Cornell University) · 14,099 citations · 34 from inside this corpus

Geoffrey E. Hinton, Oriol Vinyals, Jay B. Dean

The source holds an abstract for this work, but its best open-access copy is under no open licence, which does not permit us to republish the text. Read it at the source below.

Distilling the Knowledge in a Neural Network (2015)Distilling the Knowledge in a…Dropout: a simple way to prevent neural networks from overfitting (2014)Dropout: a simple way to prev…Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Fou… (2012)Deep Neural Networks for Acou…Ensemble Methods in Machine Learning (2000)Ensemble Methods in Machine L…Improving neural networks by preventing co-adaptation of feature detectors (2012)Improving neural networks by …Adaptive Mixtures of Local Experts (1991)Adaptive Mixtures of Local Ex…Model compression (2006)Model compressionXception: Deep Learning with Depthwise Separable Convolutions (2017)Xception: Deep Learning with …Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe… (2019)Explainable Artificial Intell…Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018)Peeking Inside the Black-Box:…Emerging Properties in Self-Supervised Vision Transformers (2021)Emerging Properties in Self-S…DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter (2019)DistilBERT, a distilled versi…Membership Inference Attacks Against Machine Learning Models (2017)Membership Inference Attacks …Deep Reinforcement Learning: A Brief Survey (2017)Deep Reinforcement Learning: …Learning without Forgetting (2017)Learning without ForgettingExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019)Exploring the Limits of Trans…Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018)Unsupervised Feature Learning…Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review (2017)Deep Convolutional Neural Net…Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks (2018)Grad-CAM++: Generalized Gradi…Continual lifelong learning with neural networks: A review (2019)Continual lifelong learning w…Language Models are Few-Shot Learners (2020)Language Models are Few-Shot …On the Opportunities and Risks of Foundation Models (2021)On the Opportunities and Risk…Deep visual domain adaptation: A survey (2018)Deep visual domain adaptation…Ensemble deep learning: A review (2022)Ensemble deep learning: A rev…Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018)Threat of Adversarial Attacks…Machine Learning Interpretability: A Survey on Methods and Metrics (2019)Machine Learning Interpretabi…Adversarial Examples: Attacks and Defenses for Deep Learning (2019)Adversarial Examples: Attacks…ZOO (2017)ZOOOn Calibration of Modern Neural Networks (2017)TinyBERT: Distilling BERT for Natural Language Understanding (2020)AutoML: A survey of the state-of-the-art (2020)AutoML: A survey of the state…Text Data Augmentation for Deep Learning (2021)Text Data Augmentation for De…Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustw… (2023)Explainable Artificial Intell…Relational Knowledge Distillation (2019)Relational Knowledge Distilla…A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer … (2017)A Gift from Knowledge Distill…A continual learning survey: Defying forgetting in classification tasks (2021)A continual learning survey: …Unsupervised Data Augmentation for Consistency Training (2019)Unsupervised Data Augmentatio…
36 of 40 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.
this paper works it cites works citing it node size = global citations · hover for the full title

What this paper cites, inside the corpus

PaperYearCited
Dropout: a simple way to prevent neural networks from overfitting201434,236
Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Fou…201210,399
Ensemble Methods in Machine Learning20008,058
Improving neural networks by preventing co-adaptation of feature detectors20126,653
Adaptive Mixtures of Local Experts19915,234
Model compression20062,118

What cites it, inside the corpus

PaperYearCited
Xception: Deep Learning with Depthwise Separable Convolutions201719,457
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe…20199,796
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)20186,166
Emerging Properties in Self-Supervised Vision Transformers20215,394
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter20194,600
Membership Inference Attacks Against Machine Learning Models20174,490
Deep Reinforcement Learning: A Brief Survey20174,434
Learning without Forgetting20174,034
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer20193,698
Unsupervised Feature Learning via Non-parametric Instance Discrimination20183,640
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review20173,570
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks20183,219
Continual lifelong learning with neural networks: A review20193,145
Language Models are Few-Shot Learners20203,020
On the Opportunities and Risks of Foundation Models20212,265
Deep visual domain adaptation: A survey20182,232
Ensemble deep learning: A review20222,136
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey20182,090
Machine Learning Interpretability: A Survey on Methods and Metrics20191,822
Adversarial Examples: Attacks and Defenses for Deep Learning20191,786
ZOO20171,779
On Calibration of Modern Neural Networks20171,741
TinyBERT: Distilling BERT for Natural Language Understanding20201,706
AutoML: A survey of the state-of-the-art20201,705
Text Data Augmentation for Deep Learning20211,701
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustw…20231,679
Relational Knowledge Distillation20191,669
A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …20171,660
A continual learning survey: Defying forgetting in classification tasks20211,648
Unsupervised Data Augmentation for Consistency Training20191,624

Links

DOI · Open access copy · OpenAlex record

Topics

Neural Networks and ApplicationsComputer Science
Time Series Analysis and ForecastingComputer Science
Topic ModelingComputer Science

Is this record sound?

complete

Nothing in this record contradicts itself and no field we check is missing.

  • supports3 author record(s) attached.
  • supports9 reference(s) recorded.
  • neutralThe DOI carries no year to check against.
  • supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:40+00:00.

sha256 7e3d99a592f7f61f…