Andrej Karpathy on Building the Software 2.0 Stack
A lot of our code is in the process of being transitioned from Software 1.0 (code written by humans) to Software 2.0 (code written by an optimization, commonly in the form of neural network training)...
A lot of our code is in the process of being transitioned from Software 1.0 (code written by humans) to Software 2.0 (code written by an optimization, commonly in the form of neural network training). In the new paradigm, much of the attention of a developer shifts from designing an explicit algorithm to curating large, varied, and clean datasets, which indirectly influence the code. I will provide a number of examples of this ongoing transition, cover the advantages and challenges of the new stack, and outline multiple opportunities for new tooling.
https://youtu.be/y57wwucbXR8
Other articles
Tenstorrent and Movellus Form Strategic Engagement for Next-Generation Chiplet-Based AI and HPC Solutions
Enabling Cross-Foundry IP for Power and Performance Optimization
Attention in SRAM on Tenstorrent Grayskull
When implementations of the Transformer's self-attention layer utilize SRAM instead of DRAM, they can achieve significant speedups.
Tenstorrent is Continuing its Contributions to the RISC-V Open Source Ecosystem
Today we are pleased to announce the release of our RISC-V Architectural Compatibility Suite, now available in our GitHub repository.