cyb/honeycrisp/.claude/plans/m4_upgrade.md

π 0.0% 1950 words · 10 min

honeycrisp M4 upgrade — SME, SME2 LUTI, Streaming SVE

owner: agent (drive autonomously per ~/cyber/cyber/root/cyberia/midao/dev.md) host: M4 Max (12P+4E, SVL=512 bits), galapot verified: 2026-05-22 via sysctl hw.optional.arm.FEAT_*

1. why this exists

The acpu crate already drives Apple's undocumented AMX matrix coprocessor and the NEON vector engine. M4 silicon adds three genuinely new compute paths that no existing acpu module reaches:

primitive pre-M4? M4 unlocks gated by
SME outer product no ISA-standard matrix unit FEAT_SME
SME2 LUTI lookup no 2-bit and 4-bit indexed TBL FEAT_SME2
Streaming SVE (SSVE) no predicated vector ops, Z/P regs FEAT_SME (sm mode)
SME f64 outer prod no 8×8 f64 ZA outer product FEAT_SME_F64F64
SME i16→i64 no wide integer accumulation FEAT_SME_I16I64

Three primitives that are NOT new on M4 and stay where they are: AMX (kept intact in acpu::matrix), NEON (kept in acpu::vector), Accelerate cblas_sgemm (not a honeycrisp dependency, only a benchmark reference).

2. correction to the original framing

The user's table said "sve predicated → new ISA features (SVE2)". sysctl on M4 Max shows no FEAT_SVE or FEAT_SVE2 flag exists — those oids are not present on darwin. Apple skipped standalone SVE/SVE2 entirely. The SVE-style register file (Z0–Z31, P0–P15) exists only inside SME streaming mode (SSVE). Every SSVE instruction must execute between SMSTART SM and SMSTOP SM (or inside a function annotated __arm_streaming). This is enforced by hardware — outside streaming mode the Z/P registers are not addressable.

Practical consequence: there is no separate sve module. The streaming bracket lives in streaming/ and exposes a typed handle (analogous to Matrix for AMX) that wraps SMSTART/SMSTOP and gives access to SSVE vector ops plus SME ZA-array outer products.

3. silicon facts (M4 Max)

SVL (streaming vector length)         = 512 bits   = 64 bytes
SSVE Z register count                 = 32
SSVE P (predicate) register count     = 16
ZA matrix tile width (single-vector)  = SVL bits   = 64 bytes
ZA accumulator total                  = SVL × SVL  = 4096 bytes (full ZA)
SME f32 outer product MOPA tile       = 16 × 16 f32 per slice (4 slices stacked)
SME f64 outer product MOPA tile       = 8 × 8  f64 per slice (8 slices stacked)
SME2 LUTI2.4S                         = 4 lookups × 4-bit index → 32-bit
SME2 LUTI4.4S                         = 4 lookups × 4-bit index → 32-bit (larger table)

The same physical block implements AMX and SME. They cannot run concurrently on the same thread. Per-thread Matrix (AMX) and per-thread Stream (SME) must be mutually exclusive. Different threads can run different units.

4. scope and module layout

Three new directories under acpu/src/, mirroring the existing organ pattern. One spec file per organ under acpu/specs/. One benchmark file per organ under acpu/bench/. File budget: 500 lines per source file (CLAUDE.md rule).

acpu/
  src/
    probe/
      mod.rs               (EXTEND) add has_sme, has_sme2, has_sme_f64f64,
                           has_sme_i16i64, svl_bytes fields
    streaming/             (NEW)
      mod.rs               Stream lifecycle (smstart/smstop), Send/Sync rules
      asm.rs               SMSTART_SM / SMSTOP_SM / SMSTART_ZA / SMSTOP_ZA
      ssve.rs              typed Z0–Z31 / P0–P15 handles, predicated ops
    sme/                   (NEW)
      mod.rs               ZA tile lifecycle, public matmul entry points
      asm.rs               FMOPA, BFMOPA, F64 MOPA, SMOPA, MOVA Z↔ZA encodings
      tile.rs              16×16 f32 / 8×8 f64 microkernel wrappers
      gemm.rs              cache-blocked SME f32 GEMM (mirrors acpu/src/gemm)
    lut/                   (NEW)
      mod.rs               public LUTI2/LUTI4 wrappers
      asm.rs               LUTI2/LUTI4 instruction encodings
      permute.rs           gather, table-permute, sbox-style apply
  specs/
    sme.md                 (NEW) source of truth for sme + streaming + lut
                           (or split: sme.md / streaming.md / lut.md)
  bench/
    sme.rs                 (NEW) SME GEMM vs Accelerate vs AMX (M4 only)
    lut.rs                 (NEW) LUTI vs NEON TBL (permute, gather)

Re-exports added in acpu/src/lib.rs: Stream, Za, lut, plus matmul_f32_sme, matmul_f64_sme, optionally a unified dispatch matmul_f32 upgrade that prefers SME on M4+, AMX on M3-, NEON otherwise.

5. probe upgrades (phase 0)

Extend acpu::probe::Features with:

  • has_sme: bool — FEAT_SME
  • has_sme2: bool — FEAT_SME2
  • has_sme_f64f64: bool — FEAT_SME_F64F64
  • has_sme_i16i64: bool — FEAT_SME_I16I64
  • svl_bytes: u16 — from RDSVL (only valid in streaming mode); cache via one-shot streaming probe at init

Extend Feature enum with Sme, Sme2, SmeF64F64, SmeI16I64.

Update Chip::amx_version table: M4 now exposes BOTH AMX (legacy, ver 2 for backward compat) AND SME. No change to the AMX path.

Add to probe/main.rs: print SME flags + SVL.

6. streaming module (phase 1)

Pattern follows matrix::Matrix exactly:

pub struct Stream {
    _not_send_sync: PhantomData<*const ()>,
}
impl Stream {
    pub fn new() -> Result<Self> { /* SMSTART SM + ZA */ Ok(...) }
}
impl Drop for Stream { fn drop(&mut self) { /* SMSTOP ZA + SM */ } }

Encodings: SMSTART SM = 0xD503437F, SMSTART ZA = 0xD503457F, SMSTOP SM = 0xD503427F, SMSTOP ZA = 0xD503447F. These are NOP-equivalent on chips without SME, so the Stream::new() precondition is purely a probe-guard (features.has_sme).

ssve.rs exposes typed Z/P register handles and a small set of SSVE ops needed to implement matmul packing and Tip5/Goldilocks kernels:

  • svld1_z, svst1_z — predicated load / store
  • svptrue, svwhilelt — predicate construction (tail-free loops)
  • svmla_z — predicated FMA
  • svadd_z, svsub_z — predicated add/sub
  • svmul_z — predicated multiply
  • svdup_z — broadcast
  • svrev_z — bit-reversal (for FFT/NTT)
  • svtbl_z / SME2 svluti2/svluti4 — table lookup

Each is one .word inline asm (no LLVM intrinsic guarantees on stable Rust yet for SME; fall back to raw encoding like AMX does).

Exit criterion: cargo run -p acpu --example sme_smoke prints non-zero SVL, returns from streaming mode cleanly, no SIGILL.

7. SME matmul (phase 2)

Two layers:

Layer A — microkernel (sme/tile.rs): one ZA tile-slice register holds a 16×16 f32 partial product. The MOPA instruction does Z[a] ⊗ Z[b] → ZA tile in one cycle. For 16×16 fp32 GEMM with k accumulations: load 16 floats from A into Za, 16 floats from B into Zb, FMOPA into ZA0; repeat for each k step; finally MOVA ZA0 → Z regs → store to C.

Layer B — cache-blocked GEMM (sme/gemm.rs): adapt the existing GEBP / microkernel structure from acpu/src/gemm/mod.rs. Same MR=16, NR=16 (matches ZA tile geometry), MC and KC retuned for M4's L1 (192 KB) and L2 (32 MB shared P-cluster).

matmul_f32_sme(a, b, c, m, n, k): same signature as matmul_f32. Inside the function: let _s = Stream::new()?; then run the cache-blocked GEMM. Multi-thread: each P-core thread gets its own Stream (per-thread state).

The existing matmul_f32 keeps the AMX path. Add a dispatch shim in gemm/mod.rs that, on M4+, calls the SME path for sizes where it wins (TBD after benchmark).

Exit criteria:

  • correctness vs naive scalar reference, all M/N/K in {7, 16, 33, 64, 127, 256, 1024}
  • on M4 Max: SME path ≥ 1.05× AMX path for at least one size bucket in the 64–4096 range (single-thread and multi-thread separately)
  • on M4 Max: SME path within 0.95× of cblas_sgemm at 1024×1024 single-thread

8. LUTI primitives (phase 3)

SME2 LUTI2 / LUTI4: indexed table lookup with 2- or 4-bit indices, single-pass permutation of up to 16 lanes from a 16-entry table (LUTI2) or 4 lanes from a larger table (LUTI4). Compared to NEON TBL which is limited to 16 bytes of table per instruction, LUTI works on the full streaming-mode Z register.

Public API:

  • lut::permute_u8_sme(table: &[u8], idx: &[u8], out: &mut [u8])
  • lut::permute_u32_sme(table: &[u32], idx: &[u8], out: &mut [u32])
  • lut::gather_u32_sme(table: &[u32], idx: &[u32], out: &mut [u32])

Each opens a Stream (or accepts an outer one), loads the table into Z registers, then runs LUTI in a tight loop with predicated tail.

Reference target: NEON TBL (single-register), TBL2/TBL3/TBL4 (multi-reg). Exit criterion: LUTI path ≥ 2× NEON TBL on 256-element u8→u8 permutation.

9. SSVE numerical kernels (phase 4)

Predicated vector kernels using SSVE where the tail-handling pays off:

  • field::gold_mul_ssve (Goldilocks 64-bit field multiply, batched)
  • field::tip5_round_ssve (Tip5 hash round-function, batched)

These are nika hot paths (per project_nika_hashrate). The win condition: the predicated tail saves the scalar epilogue NEON kernels currently spend on sub-vector remainders, and the wider effective vector (SVL=512 vs NEON 128) gives 4× the work per loop iteration.

Exit criterion: gold_mul_ssve ≥ 1.5× existing acpu::field::gold_mul on 4096-element batches; Tip5 hash ≥ 1.3× existing NEON Tip5 on 1024-element batches.

10. benchmarks (phase 5)

New bench files mirror existing ones. They MUST run on M4 only (gate via probe::scan().has_sme); on M1–M3 print a clean "skip: SME not present" and exit zero so the workspace bench still passes.

bench/sme.rs         SME GEMM spectrum (sizes 8..4096), vs AMX, vs Accelerate
bench/lut.rs         LUTI vs NEON TBL (permute, gather)
bench/ssve.rs        SSVE vs NEON (gold_mul, tip5)
bench/summary.rs     (EXTEND) include SME/LUTI/SSVE rows in summary table

Update .claude/plans/scoreboard.md with M4-only sections.

11. documentation (phase 6, in parallel with each previous phase)

For each new organ, write its spec section first (per midao/documentation.md "spec before code"). Specs live in acpu/specs/sme.md (or split). README gets a new row in the category table. The crate-level CLAUDE.md gets a new section under "architecture" describing the streaming/sme/lut organs.

12. order of execution

Hard ordering (later phases depend on earlier ones):

  1. Phase 0 — probe extension. ~1 pomodoro. Unblocks everything else by giving runtime detection.
  2. Phase 1 — streaming module + smoke example. ~2 pomodoros. Validates that SMSTART/SMSTOP work and that we can read SVL.
  3. Phase 2 — SME matmul. ~2 sessions. Largest single piece; the win condition (beat AMX or Accelerate at some size) is the long pole.
  4. Phase 3 — LUTI primitives. ~1 session. Independent of phase 2.
  5. Phase 4 — SSVE numerical kernels. ~1 session. Depends on phase 1.
  6. Phase 5 — benchmarks. ~1 pomodoro per primitive, can interleave.
  7. Phase 6 — documentation. Continuous; spec written before each phase's implementation per midao/documentation.md rule.

Phases 2, 3, 4 are independent and can be done by parallel agents partitioned by directory (sme/, lut/, streaming-extensions in ssve/). Each agent owns its non-overlapping file scope per midao/dev.md "parallel agents".

13. exit criteria (overall, what "done" means)

Done = all of:

  • workspace cargo build --release clean, zero warnings
  • workspace cargo test --workspace green on M4 Max
  • cargo run --release -p acpu --example bench_summary shows SME, LUTI, SSVE rows with WIN status against the chosen references
  • specs/sme.md (and friends) match the implemented API verbatim
  • README.md benchmark table includes the new M4-only category
  • scoreboard.md updated with M4-only section
  • commit history: one atomic commit per primitive (per CLAUDE.md git rules), conventional prefixes (feat:/test:/bench:/docs:), no Co-Authored-By lines

14. risk register

risk mitigation
LLVM/clang on stable Rust does not emit SME instructions use raw .word encoding (same trick the existing AMX path uses)
SMSTART faults if FEAT_SME absent probe gate before constructing Stream
SME shares physical block with AMX enforce !Send Stream + Matrix, document mutual exclusion per thread
ZA state must be zeroed at SMSTART ZA encoding handled in Stream::new
LUTI table layout differs from NEON TBL byte-order unit-test against scalar reference
Accelerate beat-target slips because cblas_sgemm already uses AMX acceptable: the WIN condition is "beat AMX path at some size bucket"; Accelerate parity is fine when AMX itself is at the ceiling
Streaming-mode entry cost (~30–80 cycles SMSTART + SMSTOP) per-thread Stream stays open across multiple matmul calls; document amortization

15. references

  • ARM ARM (DDI 0487Lc) chapter on SME, A64.SME (SMSTART, SMSTOP, MOPA, MOVA)
  • ARM SME2 supplement (LUTI2, LUTI4)
  • Apple's open-source corsix/amx repo — pattern for .word encoding
  • existing acpu/src/matrix — implementation template for the new Stream/ZA wrappers
  • existing acpu/src/gemm — cache-blocked GEMM template to port to SME tiles

Graph