✦
Summary
Z1T is a family of sparse, Transformer-like research models designed around Extropic Z1 probabilistic hardware. It explores how neural inference can use the sparse connectivity of thermodynamic computing while retaining useful language-model behavior, addressing the energy demands of conventional dense Transformer execution.
Key Features
Z1T targets the sparse physical connectivity of Extropic Z1 probabilistic chips.
Sparse operations match the limited tunable couplings between physical probabilistic bits.
The proposed pipeline combines Z1 with conventional digital accelerators and FPGAs.
Statistical averages from probabilistic hardware represent classical numerical vectors.
Experiments vary model size, training compute, and sparsity to study scaling behavior.
The research focuses on improving the energy efficiency of Transformer-like inference.
Extropic links a public sparse-transformers code repository.
The architecture supports experiments in co-designing sparse models and probabilistic hardware.
About Z1T
The approach encodes neural computations using statistics of probabilistic bits and designs sparse operations around the physical coupling structure of Z1. Inference is divided across thermodynamic accelerators and conventional digital processors or FPGAs, assigning different computations to suitable hardware within a heterogeneous pipeline.
Z1T offers researchers a concrete architecture and scaling experiments for investigating hardware-software co-design. Its open sparse-transformer implementation is a research resource; the hardware efficiency discussion concerns the proposed Z1 deployment strategy and should not be read as a universally available inference service.