Existing accelerators · their own benchmarks
Their chip. Their benchmark. The gaps Atelier finds.
Atelier Studio takes an accelerator we did not design and finds where its time goes: what the hardware needs, what the design loses, what its software adds. It searches the vendor's own design space for a better configuration, proposes the design dimensions that space is missing, and builds the pick with the vendor's own generator, to run the vendor's own benchmark on the RTL.
Technical preview · Gemmini (UC Berkeley) and NVDLA (NVIDIA), both open source · every figure labelled measured, fitted, predicted or estimate
How
Find the gaps, then prove them on their benchmark.
Four steps in Atelier Studio, on the accelerator as its vendor ships it.
Where the time goes
Layers measured on the RTL, set against the bound the hardware's description allows, and split: what the hardware needs, what it loses (design gaps), what its software adds (tool gaps).
Their design space
Every configuration the vendor's generator builds, on the vendor's own benchmark, with its own software's mapping replayed for each (exact against the RTL's DMA counters). The gaps are ranked parameter by parameter.
Beyond their generator
Where no parameter closes a gap, Atelier proposes the design dimension that would, and costs it: a depthwise mode, an activation table on the output path.
Build and run
The pick is built with the vendor's generator and runs the vendor's benchmark unchanged on its RTL: same network, weights, batch and correctness check, its own cycle printout as the result.
Gemmini · UC Berkeley
MobileNet and ResNet-50, as Gemmini ships them.
Gemmini is an open-source systolic-array generator from UC Berkeley, part of Chipyard. Its own benchmarks are gemmini-rocc-tests' imagenet/mobilenet.c and resnet50.c: four images, int8, through its own library, gemmini.h. Stock design: GemminiRocketConfig, a 16×16 mesh, 256 KB scratchpad, 64 KB accumulator and 128-bit DMA at 500 MHz.
On the 19 layers measured on Gemmini's RTL, about 79% of the time is what its own software (gemmini.h) adds: rows padded to whole DMA transfers, outputs written in small requests, array passes it does not need. The hardware needs about 18%; it loses about 8% more (its writes, idle lanes, a scratchpad too small for some layers). Fitted split of measured time; the model is within 5% of the total. fitted
ResNet-50 (imagenet/resnet50.c, 16.4 GMAC (4 images))
Stock design: about 74% of the array's MACs (fitted).
Hardware gaps Explore finds in Gemmini's own design space, one generator parameter at a time from the stock configuration:
| Parameter | Stock → best | Speedup | Area |
|---|---|---|---|
| meshRows × meshColumns | 16 → 32 | 1.64× | 1.52× |
| dma_buswidth | 128 → 256 | 1.12× | 1.00× |
| acc_capacity_kib | 64 → 128 | 1.08× | 1.17× |
| sp_capacity_kib | 256 → 1024 | 1.07× | 2.98× |
Explore's pick, all parameters together: 32 × 32 mesh, 128 KiB scratchpad, 64 KiB accumulator, 256-bit DMA and system bus: 2.22× at 1.19× the area predicted. built with Gemmini's generator; their ResNet-50 running on its RTL running
Atelier's compiler on the same hardware: 1.18× (explicit tiles, fitted).
MobileNet (imagenet/mobilenet.c, 1.20 GMAC (4 images))
Stock design: about 4% of the array's MACs (fitted): 91% of the time is in 17 depthwise layers, which Gemmini runs one channel at a time.
Hardware gaps Explore finds in Gemmini's own design space, one generator parameter at a time from the stock configuration:
| Parameter | Stock → best | Speedup | Area |
|---|---|---|---|
| acc_capacity_kib | 64 → 256 | 1.04× | 1.50× |
| dma_buswidth | 128 → 256 | 1.01× | 1.00× |
| sp_capacity_kib | 256 → 512 | 1.00× | 1.66× |
| meshRows × meshColumns | 16 → 32 | 0.94× | 1.52× |
No configuration of Gemmini's generator gains more than 1.07× on MobileNet: its time is in a shape the array cannot use. That is a design gap outside the generator's space.
Beyond Gemmini's generator, Atelier proposes a depthwise mode (each array column takes its own channel) and costs it on the same model:
| Design | Speedup on MobileNet | Area vs stock |
|---|---|---|
| depthwise mode, on the stock design | 6.4× | 1.02× |
| depthwise mode, 16 × 16, 128 KiB scratchpad, 256-bit DMA | 6.8× | 0.69× |
| depthwise mode, 32 × 32, 512 KiB scratchpad, 256 KiB accumulator | 12.1× | 2.75× |
estimate A proposed dimension needs an RTL change, which Atelier generates; it is a claim only once built and measured.
Atelier's compiler on the stock hardware: 6.2× predicted (each depthwise layer as dense blocks of 16 channels); one layer measured on the RTL: 6.0×, output identical.
Measured so far
| What | Quantity | Result | Evidence | Note |
|---|---|---|---|---|
| Depthwise rewrite, 56×56×32, 3×3 (dw_32) | 639,210 → 107,242 cycles | 6.0× | measured on RTL | byte-identical to tiled_conv_dw_auto |
| 32 × 32 build, 19 layers | DMA bytes | exact | measured on RTL | the replay of Gemmini's software equals the RTL's DMA counters on every layer |
| 32 × 32 build, cycles predicted from the 16 × 16 | mean error | 18.5% | measured on RTL | 57% worst; depthwise layers slower on 32 × 32 (0.75-0.89×) |
| SiLU on the Rocket core, an int8 table | cycles an element | 10.15 | measured on RTL | 51,200 elements, 519,826 cycles |
running Their MobileNet and ResNet-50 on the stock GemminiRocketConfig and on Atelier's picks (built with Gemmini's own generator) are running on the RTL. This table fills in only with measured totals that pass their own check.
NVDLA · NVIDIA
nv_small, and the space around it.
NVDLA is NVIDIA's open-source deep-learning accelerator, run here as Chipyard integrates nv_small. Each layer is mapped as NVIDIA's own compiler maps it, replayed exactly. NVIDIA publishes its own performance estimates for AlexNet, GoogLeNet and ResNet-50; the measured comparison against them is next.
| What | Result | Evidence | Detail |
|---|---|---|---|
| Chipyard's NVDLA memory port | fixed | measured on RTL | the AXI to TileLink chain re-used a source ID and stopped every conv over a few hundred bytes; with the chain corrected, 8 of NVIDIA's 14 nv_small conv tests pass with its golden CRC, up to 390,000 cycles |
| NVIDIA's compiler, replayed | exact | measured on RTL | the split, banks and DMA of every layer equal the prebuilt nvdla_compiler on its four split cases and NVIDIA's nv_small AlexNet network |
| Convolutions from any shape | 13 / 13 | measured on RTL | registers generated from a shape equal NVIDIA's conv tests field by field; their golden outputs reproduced |
YOLOv8n across NVDLA's design space
| Configuration | MACs / cycle | SiLU unit | Frames / s | mm² | From |
|---|---|---|---|---|---|
| nv_small (stock) | 64 | no | 2.4 | 0.051 | NVIDIA's |
| nv_small_256 | 256 | no | 3.0 | 0.108 | NVIDIA's |
| nv_medium_512 | 512 | no | 3.3 | 0.217 | NVIDIA's |
| nv_medium_1024_full | 1,024 | yes | 24.7 | 0.258 | NVIDIA's |
| nv_large | 2,048 | yes | 44.1 | 0.331 | NVIDIA's |
| Atelier: atom 32 × 16, 16 banks × 256, 128-bit, SDP 16/cycle | 512 | yes | 29.1 | 0.087 | Atelier's pick |
| Atelier: nv_large's array, half its buffer | 2,048 | yes | 44.1 | 0.243 | Atelier's pick |
estimate YOLOv8n 640 at 500 MHz, estimates: each layer as NVIDIA's compiler maps it (replayed exactly), timed at the units' nominal rates; SiLU on the output processor's LUT where the configuration has one, else on the CPU at the measured rate. NVIDIA's compiler leaves little on the table (98% of nv_small's MACs used); the gaps are the hardware's: no SiLU unit below nv_medium_1024_full, channel rounding at 64-wide atoms (49% of nv_large's lanes idle on YOLOv8n), an output processor too narrow on nv_small_256.
Method
What every number stands on.
Measured on RTL: cycles from Verilator simulation of the RTL the vendor's generator built (Chipyard 1.15.0), with the benchmark's own check passing. Fitted: Atelier's cycle model of the vendor's software, checked on layers and configurations it was not fitted on. Predicted / estimate: the model before a build; area from ASAP7 7 nm macro and cell figures; proposed design dimensions until built. Running: measurements under way. Vendors' networks, weights and numerics are run unchanged; nothing is re-quantized.
Gemmini and NVDLA are named only to identify the open-source designs and benchmarks measured here; ZenEdgeX is not affiliated with or endorsed by UC Berkeley or NVIDIA.
Your accelerator
Bring your NPU and its benchmark.
Send us your accelerator's description and profiles of a few layers. Atelier Studio shows where its time goes, which design and tool gaps cost the most, and the next generation worth building.
hello@zenedgex.in