How could machines learn as efficiently as humans and animals? How could machines learn to reason and plan?

We explore the design of lifelong learning AI accelerators that are intended for deployment in untethered environments

Sparse expert models are a thirty-year old concept re-emerging as a popular architecture in deep learning...

We present MegaBlocks, a system for efficient Mixture-of-Experts (MoE) training on GPUs

Deep residual networks (ResNets) have significantly pushed forward the state-of-the-art on image classification...

Deploying TinyML models on low-cost IoT hardware is very challenging, due to limited device memory capacity...

The backpropagation of error algorithm used to train deep neural networks has been fundamental to the successes of deep learning...

Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks...

Training directed neural networks typically requires forward-propagating data through a computation graph..

We propose a new simple network architecture