OSU STAR Lab · NSF CROSS Project · 2026
Automating hardware accelerator designfor sparse tensor contraction Sparse structure guides hardware specialization: two contractions, two MLIR-based lowering paths, controlled compiler improvements.
Anthony Kung STAR Lab Hardware Optimization Team
0:00–0:35 · 35 seconds Our question this year is whether sparse structure can guide a toolchain toward substantially better accelerator hardware, without changing the mathematical model. I will show two block-sparse contractions, two MLIR-based lowering paths, and controlled improvements of about ten to eleven times in HLS average-cycle estimates. The important result is not just the magnitude: different sparse properties require different transformations. I will also show the physical-design evidence and the CPU and GPU context, because compiler improvement and a competitive accelerator are distinct research questions.
Q · What is the contribution?
Q · Is ten times a board measurement?
Research trajectory
From a working toolchain to better generated hardware 2025 Algorithm-to-layout feasibility Transformer proxy; ASIC-flow barriers; FPGA evaluation; transition toward a commercial ASIC flow.
Original archived study 2026 Sparse optimization experiments Test which sparse properties improve generated hardware, at what resource cost, and against which baseline.
Results in this talk 2027 Integration and scale Compiler-native selection, broader coverage, and parallel sparse execution and memory systems.
Research programme Hypothesis: output independence, reuse, and ownership require different hardware transformations. Historical scope: original 2025 slides 5–8 and 15. Completion of the Catapult → Genus → Innovus chain was a stated next step.
0:35–1:25 · 50 seconds In 2025, the project studied algorithm-to-layout feasibility using a Transformer proxy. The original slides report that the Bambu and open-source ASIC path encountered incompatible RTL, followed by FPGA evaluation and a transition toward SystemC, Catapult, Genus, and Innovus. That work established accelerator feasibility through FPGA evaluation and motivated a commercial SystemC-to-ASIC flow; completing that ASIC flow remained a subsequent milestone. In 2026 we ask a narrower optimization question on MLIR-based FPGA toolchains: once a contraction is synthesizable, can sparse-specific transformations materially improve the generated architecture? Our hypothesis is that output independence, reuse, and ownership determine which transformation helps. The 2027 programme is to integrate, generalize, and scale what these experiments establish.
Q · Does this contradict the 2025 speedup claim?
Q · Why change toolchains?
Experimental inputs
Two block-sparse contractions, two different bottlenecks TIDES E2 fbag,bhai→fghi C[f,g,h,i] = Σ(b,a) A[f,b,a,g] · B[b,h,a,i]
Stored A / B values 3,133,605 / 119
Matching block pairs 8,500
Stored output values 2,840,776
Scalar products 5,024,105 Conservative output dependence serializes independent inner-loop work. TIDES S1 dae,fga→defg C[d,e,f,g] = Σa A[d,a,e] · B[f,g,a]
Stored A / B values 188,454 / 189,096
Matching block pairs 1,310
Stored output values 692,685
Scalar products 27,144,530 A contraction recurrence and repeated external-memory value reads limit the pipeline. TIDES canonical_formulation_symm_kysznf cases E2/S1. Upstream block structures; FP32 values from Xorshift32, seed 42. Exact shapes and reproduction details in backup.
1:25–2:30 · 65 seconds These are two frozen cases from the TIDES block-sparse benchmark suite, motivated by DMRG tensor-network computations. In E2, we sum over b and a; f and g come from the left operand, and h and i from the right. In S1, we sum over a and retain d, e, f, and g. Missing blocks contribute zero, while stored blocks contain dense FP32 values. E2 has about 3.13 million stored left values but only 119 right values. It generates 8,500 matching block pairs and 2.84 million stored output values. S1 has fewer pairs but longer reductions: roughly 27 million scalar products versus E2's five million. Sparsity metadata comes from the upstream cases; values are deterministically regenerated with seed 42. These two cases test different mechanisms, not general performance across the whole suite.
Q · What are the exact inputs?
Q · Are the values real application measurements?
Q · Why just two cases?
Experimental toolchain
MLIR-based lowering; sparse specialization of generated HLS HIDA path Hierarchical-dataflow lowering; a minimally enabled build provides the controlled baseline.
STAR Lab path Our independent implementation of StreamTensor's published methodology—not the original codebase.
ARASHI contribution Profile checks, dependence intent, first-write, bounded caches, banking, and combination sweeps.
Same contraction, runtime values, reduction order, AXI boundary, and U280 target. The generated architecture changes. Current implementation: research transformations around compiler-generated HLS. Common MLIR-native legality analysis and automatic selection are 2027 goals.
2:30–3:35 · 65 seconds · two clicks within this slide Here is exactly where the current experiment sits. The contraction enters a compatible frontend and an MLIR-based lowering path, which emits HLS C++. HIDA is one path. [Click.] The second is STAR Lab's independent implementation of StreamTensor's published methodology, not the original StreamTensor codebase. [Click.] ARASHI applies the experimental sparse transformations to the generated representation, using profiles and structural checks to choose candidates. Vitis then schedules and synthesizes those candidates. Today this is a research optimization layer around compiler-generated HLS, not a completed common MLIR pass. The model equation, runtime values, and numerical reduction order are preserved; the generated hardware organization changes. We normalize the AXI boundary and target settings, so the baseline and optimized variants differ in the transformations being tested, not in an easier workload or a different external interface.
Q · Why call this toolchain optimization rather than model optimization?
Q · Is the selection fully automatic?
Q · Does a profile prove safety for every future input?
Check injective output addressing within the selected inner loop. Emit dependence intent: scheduled II falls from 20 to 2. Use 5,632 contiguous output groups to eliminate the global zero-fill. 10.01× baseline / optimized HLS average cycles
20 → 2 scheduled initiation interval
12.01× derived best-tested OOC latency ratio
Vitis HLS 2023.2 average-cycle estimates; runtime-loop bounds, not measured execution. Selected E2: 57,154,011 / 57,154,009 cycles (HIDA / lab framework). Routed targets: 4.5 → 3.75 ns.
3:35–4:55 · 80 seconds For E2, the key fact is that adjacent iterations of the selected inner loop address distinct output locations. A conservative memory-dependence assumption therefore serializes work that can overlap. Supplying the valid independence fact reduces the scheduled initiation interval from twenty cycles to two. That alone gives about 9.54 times fewer estimated cycles. The second transformation uses the descriptor ordering: 8,500 pairs form 5,632 contiguous output groups. The first contribution initializes each group directly; later contributions retain their original accumulation order. This removes a separate global zero-fill. Alone, it saves only about half a percent. Combined with dependence disambiguation, the estimate falls from 572.14 million to 57.15 million cycles, a 10.01-times ratio. Both lowering paths reproduce that result. Separately, the selected IP meets a 3.75-nanosecond routed target, versus 4.5 nanoseconds for the baseline. Those tested clock points yield a 12.01-times derived latency ratio, not measured board speedup.
Q · Do all E2 outputs have only one contribution?
Q · What did this cost?
Q · Why not quote the 3.65 ns HLS estimate?
11.31× baseline / optimized HLS average cycles
56 → 4 scheduled initiation interval
6 → 109.5 routed BRAM tiles · HIDA
Cache bounded A/B value blocks on chip.Bank local storage two ways for concurrent access.Assign completed reductions directly when each pair uniquely owns its output block.E2's dependence transformation gives no S1 latency benefit. The winning combination trades memory and DSPs for throughput. HIDA: 144,140,318 → 12,748,931 estimated average cycles; 16 → 171 routed DSPs. BRAM is 5.43% of the reported device capacity. All 692,685 outputs match the generated-C++ oracle.
4:55–6:15 · 80 seconds S1 is the counterexample to a universal pragma recipe. The E2 dependence transformation gives no latency benefit because the limiting recurrence is real. Here the useful combination caches bounded value blocks on chip, supplies two memory banks, and exploits unique output ownership. Each of S1's 1,310 matching pairs owns its output block. That permits a direct final write instead of global initialization and read-modify-write accumulation; the within-block reduction remains intact. The average-cycle estimate falls from 144.14 million to 12.75 million, an 11.31-times ratio, while the scheduled II falls from fifty-six to four. This gain has a real hardware cost. Routed BRAM usage rises from six to 109.5 tiles and DSPs from sixteen to 171. LUTs and registers fall. Four and eight banks do not improve latency, so two is the useful point in this sweep. The conclusion is workload-specific architecture selection, with an explicit throughput-versus-resource trade-off.
Q · Are reductions reassociated?
Q · Is 109.5 BRAM the same as the HLS report's 230 BRAM?
Q · What if area matters more than latency?
Ablation evidence
Combinations help—but a fixed pragma recipe does not Transformation E2: fewer estimated cycles S1: fewer estimated cycles Dependence disambiguation 89.52% · 9.54× ratio No latency benefit Grouped first-write 0.49% Different ownership structure Unbanked value cache — 48.08% Value cache + two banks — 62.28% Cache + two banks + dependence — 90.67% Selected combination 90.01% · 10.01× ratio 91.16% · 11.31× ratio
Four/eight banks add logic without reducing S1 latency; forced inner pipelining can regress E2. Within-workload HIDA baselines; “—” means not part of this ablation, not zero effect. E2 selection: dependence + first-write. S1: value cache + two banks + unique ownership.
6:15–7:10 · 55 seconds The ablations show why the optimization belongs in a compiler rather than in a fixed pragma recipe. On E2, first-write alone barely helps, while the combination with dependence disambiguation crosses ten times. On S1, dependence disambiguation does nothing, an unbanked value cache is useful, and banking plus ownership gives the largest gain. Dependence intent also becomes useful after adding the cache and banks. More banks are not automatically better. Forced inner pipelining also produced regressions, and disabling flattening raised a related S1 cache-and-bank schedule's estimate by 1.89 times. The scientific claim is therefore conditional: sparse facts identify which architectural opportunity exists, and combinations must be evaluated rather than assumed additive. These experiments motivate a selector; they do not yet demonstrate that a learned or general automatic selector predicts the best design on unseen contractions.
Q · How exhaustive was the search?
Q · How much slower was forced pipelining?
Validation and transfer
Correct generated C++; timing-clean IP in both lowering paths HIDA E2 10.0104× S1 11.3061× Baseline / selected average-cycle estimates
≈
STAR Lab framework E2 10.0104× S1 11.3061× Same AXI boundary and Vitis backend
3.53M output values checked; zero error in selected C++ variants
5 ns all selected E2/S1 IP meets OOC timing
≤ 2 estimated-cycle difference between lowering paths
4 modes SPARTA regression: approximately 47% fewer estimated cycles
Generated-C++ equivalence and out-of-context place-and-route are separate checks. RTL co-simulation and U280 board execution remain open; backend agreement shares a common downstream compiler.
7:10–8:05 · 55 seconds There are three supporting checks. First, the selected generated C++ variants match every reference output: 2.84 million E2 values and 692,685 S1 values, with zero observed absolute or relative error. Second, the synthesis reports preserve latency bounds, initiation intervals, and resource estimates. Third, the selected IP is placed and routed out of context and meets the five-nanosecond target in both paths. The two lowerings agree within two estimated cycles. That supports transfer of the transformation, but it is not independent silicon replication: both ultimately use the same Vitis backend and a normalized interface. The earlier SPARTA study also exercises all four algorithm modes, with roughly forty-seven percent fewer estimated cycles. It remains a small correctness and compiler regression, not the scale evidence for this talk.
Q · Is RTL simulation complete?
Q · Are the latency averages measured over our tensor?
Conclusion
Sparse specialization is worth pursuing; competitiveness is the next test 2026 controlled result 10.01–11.31× baseline / selected HLS average-cycle estimates, with distinct mechanisms and timing-clean IP.
Complete the 2026 evaluation Runtime RTL / U280 board validation; matched CPU–GPU–FPGA system and energy measurements; broader workload evidence.
2027: integrate, generalize, scale MLIR-native automatic selection; more contractions, formats, and toolchains; parallel execution and memory systems.
Questions Backup evidence follows OSU STAR Lab · NSF CROSS
9:10–10:00 · 50 seconds · stop prepared talk here The 2026 result is that sparse structure exposes large, reproducible opportunities in generated hardware: about ten to eleven times fewer estimated cycles on two distinct contractions, with correctness checks and timing-clean IP in both lowering paths. The ablations tell us why the transformations work and why one recipe is insufficient. That is a concrete justification for further research, while the GPU comparison defines the remaining challenge. To complete the 2026 evaluation, we need runtime RTL or U280 board validation, matched CPU–GPU–FPGA system and energy measurements, and more workloads. The 2027 programme then has three directions: compiler-native sparse optimization and automatic selection; generalization across contractions, formats, and toolchains; and scaling parallel sparse execution and memory systems. The opportunity is established; the next task is to make the benefit general and competitive. Thank you.
Q · What would falsify the research direction?
Q · What should customers expect now?
Inner-loop injectivity Distinct output addresses within the transformed loop justify inter-iteration dependence intent. This does not remove reductions across pairs.
Contiguous output groups First-write requires each output group to be contiguous and covered. Later contributions preserve their original order.
Unique block ownership S1 checks one contributing pair per output block before replacing output accumulation with assignment.
Bounded caches Profiled extents bound local storage. Changed descriptors or larger blocks require revalidation and possibly regeneration.
Current: scripted generated-HLS transformations and profile checks. Future: common MLIR legality analysis, guards, and automatic selection. Backup B2. Mathematical identity, structural preconditions, and empirical correctness are separate forms of evidence. A passing oracle alone is not a proof for all future sparsity. Profiles are checked for the frozen workloads; a production implementation needs enforceable contracts or guarded fallbacks.
Q · Are values compiled into the circuit?
Held constant Allowed experimental change Equation, inputs, output layout, FP32 reduction order Generated storage and loop organization Compatible equivalence frontend and required fixes Dependence and scheduling directives AXI interface, part, tool version, 5 ns HLS request Caches, bank count, legal initialization/writeback Oracle and comparison procedure Individual and combined transformation candidates
The baseline is minimally enabled HIDA or the lab framework—not untouched upstream execution, nor the best expert-tuned implementation. Vitis HLS / Vivado 2023.2, xcu280-fsvh2892-2L-e. StreamTensor attribution refers to STAR Lab's implementation of the published methodology.
Backup B3. Enablement is necessary to obtain a synthesizable equivalent; it is not counted as optimization. Runtime equivalence code expresses the same contraction for the target toolchain. We have not shown that arbitrary upstream CUDA lowers unchanged.
Q · Could an HLS expert write these pragmas?
HIDA design HLS avg. cycles WNS (ns) LUT Registers BRAM tiles DSP E2 baseline 572,136,789 +0.496 8,647 14,066 6.5 15 E2 selected 57,154,011 +0.847 8,803 14,567 6.5 29 S1 baseline 144,140,318 +0.032 30,343 33,868 6 16 S1 selected 12,748,931 +0.558 22,258 28,486 109.5 171
S1 uses more dedicated memory and arithmetic but less LUT/register logic. This is a throughput trade-off, not an across-the-board area reduction. Resources from placed-IP reports; WNS from routed timing. E2 is the final first-write + dependence design, not the earlier dependence-only candidate. No board power claim.
Backup B4. WNS is worst setup slack; positive means the tested target is met. BRAM tiles are physical units, not the HLS BRAM_18K estimate. Both reporting stage and units matter. The target report lists 2,016 BRAM tiles; 109.5 is 5.43%.
Q · Do vectorless watts establish energy efficiency?
Case / lowering Baseline period Selected period Cycle ratio Derived latency ratio E2 / HIDA 4.50 ns 3.75 ns 10.0104× 12.0125× E2 / lab framework 4.50 ns 3.75 ns 10.0104× 12.0125× S1 / HIDA 4.75 ns 4.25 ns 11.3061× 12.6362× S1 / lab framework 4.25 ns 4.25 ns 11.3061× 11.3061×
Estimate (ms) = reported average cycles × timing-clean target period (ns) ÷ 10⁶. Same synthesized RTL reimplemented at tighter clocks; HLS was not rescheduled. Best tested points, not proven Fmax. Final E2 fails 3.50 ns: WNS −0.080 / −0.093 ns.
Backup B5. Vitis average latency on runtime-bounded loops depends on report tripcounts; it is not the average of executions of the frozen tensor. E2 baseline best/average/worst are 2,857,789 / 572,136,789 / 1,141,415,789 cycles. Board memory stalls and the shell are absent. This is why no FPGA-versus-CPU/GPU speedup is computed.
Q · Can frequency alone close the GPU gap?
Backup B6 · selectivity
Rejected candidates and alternative objectives Forced inner pipelining E2 gets slower +49.88% HIDA latency; +80.49% on the lab framework's native interface. Within-interface experiments.
More banks No added S1 benefit Four/eight banks increase logic without improving latency beyond two banks.
Disable flattening 1.89× latency S1 cache + bank2 + dependence: 13,445,548 → 25,353,448 cycles without flattening.
Metadata-only caching Area objective S1 HIDA HLS estimates: LUT −26.34%, FF −25.12%, with little latency change.
Backup B6. The percentages come from different interfaces and objectives, so they support targeted ablation conclusions rather than an exhaustive search or an evaluation of a general learned cost model.
Q · What about runtime-bound recovery?
Platform / implementation E2 median [range] S1 median [range] Orin · TIDES full-call median 14.17 ms 3.09 ms Orin · cuTENSOR full-call median 62.20 ms 10.00 ms RTX 4060 · TIDES median [range] 3.18 [3.09, 3.27] ms 0.55 [0.49, 0.57] ms RTX 4060 · cuTENSOR median [range] 17.77 [17.64, 18.65] ms 3.14 [3.10, 3.17] ms
Both platforms use the matched full-call wall timer. Orin is one 20-call process; RTX reports the median and range of five 100-call process medians. Orin: JetPack 7.2.1, CUDA 13.2, cuTENSOR 2.6.0, SM87, 3 warmups + 20 calls, clocks unlocked. RTX: CUDA 12.6, SM89, 10 + 100 calls/process. Verification passes on both.
Backup B7. On both systems, CSV cutlass_ms is pipeline_execute total_ms measured by host chrono, while cutensor_ms uses CUDA events around contraction; both capture only the final timed call. The separately logged wall timer covers each full invocation. Orin records one median of 20 calls after three warmups; RTX records five process medians of 100 calls after ten warmups. The shared full-call boundary is the appropriate within-platform comparison. Stock TIDES versus cuTENSOR is separate from ARASHI's FPGA contribution.
Q · Is this ideal cuTENSOR performance?
Backup B9 · validity
What each check establishes Generated-C++ oracle Full-output equivalence on frozen descriptors; selected E2/S1 variants have zero observed error.
HLS + OOC routing A synthesizable schedule and timing-clean IP. Not a full system, RTL trace, or measured memory behavior.
Two lowering paths Transformation transfer under a common interface. Shared Vitis is not independent downstream replication.
Complete the 2026 evaluation Runtime RTL/board correctness, matched system costs, stronger tuned baselines, held-out sparsity/workloads.
Backup B9. The 24-design controlled matrix comprises 16 SPARTA and eight earlier TIDES implementations; the final two E2 first-write designs have additional records. These records are implementation variants rather than independent workloads, and only a subset of the 58 TIDES synthesis reports was routed. Functional RTL co-simulation remains incomplete.
Q · Why another year?
Original 2025 archive What carries into this talk Slides 2–5: Transformer proxy; model-to-layout objective Automated accelerator generation remains the objective Slides 6–7: ASIC-flow barriers; FPGA fallback Toolchain compatibility and end-to-end validation matter Slides 8–14: SystemC / Catapult / Genus / Innovus Historical ASIC methodology, not today's MLIR/Vitis pipeline Slide 15: complete chain and add sparse kernels Commercial ASIC-chain completion remained a subsequent milestone
2026 claims use their own sparse-contraction artifacts—not inherited proxy speedups or energy claims. Backup B10. The original deck remains unmodified at /2025. Its slide 7 FPGA result uses a different workload and evidence chain from E2 and S1; the current sparse-contraction claims therefore rely on their own 2026 artifacts.
01 Compiler-Native Sparse Optimization and Automatic Design Selection MLIR-integrated analysis and selection.
02 Generalization Across More Contractions, Formats, and Toolchains More contractions, formats, and backends.
03 Scaling Parallel Sparse Execution and Memory Systems Parallel execution and scalable memory.
Backup B11. These are the official programme titles. The first goal promotes validated transformations into a stable MLIR-facing capability. Broader experimental evidence and physical comparisons remain part of the 2026 evaluation; the 2027 roadmap begins from those evaluation requirements.
Backup B12 · terminology
Terms and attribution MLIR / HLS Multi-Level Intermediate Representation / High-Level Synthesis. Today's sparse transforms are not yet common MLIR-native passes.
II Initiation interval: cycles between loop-iteration launches; not complete-contraction latency.
OOC / WNS Out-of-context IP implementation / worst setup slack. Excludes board shell and runtime memory system.
TIDES / DMRG Block-sparse tensor-contraction implementation and benchmark context / Density Matrix Renormalization Group.
STAR Lab framework Independent implementation of StreamTensor's published methodology; not original source or a fork.
ARASHI Architecture Reconfiguration for Accelerating Sparse-tensor Hardware Implementations. Sparse optimization research within NSF CROSS.
Backup B12. This slide defines the compiler, scheduling, and physical-design terms used throughout the evidence.
