Mocarabe is a CGRA (Coarse-Grained Reconfigurable Array) architecture generator and a fully-custom toolchain. The implementation and results are discussed in Mocarabe: High-Performance Time-Multiplexed Overlays for FPGAs, published at FCCM 2021. This work was done by Frederick Tombs, Alireza Mellat, and Prof. Nachiket Kapre.
To get started, jump to Toolchain setup.
The architecture consists of a grid of building blocks connected by a unidirectional torus network, as shown in the figure below. Each block contains a processing element (PE) to execute operations on incoming data and a set of NoC routers to control data movement.
PEs store incoming operands in shift registers and select the relevant stored operands as inputs to their ALU at each cycle, as seen below.
The architecture is designed for statically-scheduled, time-multiplexed sharing of both routing and compute resources, with a repeating context window of length II (for initiation interval). II is the number of cycles in the modulo schedule found by the compiler. This makes the architecture periodic in both space (toroidal topology) and time (modulo schedule).
The toolchain:
- Compiles a C kernel into a DFG
- Generates a right-sized architecture by allocating PEs, or maps onto a fixed device you describe
- Partitions nodes into processing elements
- Places these PEs in the grid to minimize communication cost (wirelength)
- Schedules routing and computation
- Generates both synthesizable RTL and simulation artifacts.
Placement is done with simulated annealing and scheduling has dual strategies: an integer linear programming formulation (default) and a temporal-spatial PathFinder implementation.
python3 -m venv .venv
source .venv/bin/activate
pip install -e .[dev]
# install icarus verilog. e.g. on Ubuntu:
sudo apt install iverilog
make -C llvm_pass # build the LLVM plugin
Here is a C kernel we can map:
// int_adder_chain.c
int int_adder_chain(int x, int y, int z, int a, int* ret) {
*ret = x+y+z+a;
}Compile to a DFG, then map and simulate:
# Compile C kernel to hypergraph DFG
./llvm-with-clang.sh int_adder_chain.c hgr/
# Map to a Mocarabe architecture
# -II 1 : initiation interval (schedule length in cycles)
# -C 2 : physical channels per NoC link to start with
# --max_C 8 : if scheduling fails, raise C up to 8
# -iod 1 : IO-node packing density (1 IO node per PE)
# -ard 1 : arithmetic-node packing density (1 op per PE)
mocarabe -dfg hgr/int_adder_chain -II 1 -C 2 --max_C 8 -iod 1 -ard 1
# Visualize the placed-and-routed design
mocarabe-viz --proj proj/int_adder_chain_*/
# Simulate the generated RTL (path printed by the command above)
cd proj/int_adder_chain_*/rtl/
iverilog -g2012 -o sim.vvp mocarabe.sv pe_2_input.sv torus_switch.sv \
pe_srl.v pe_mux_2_input.sv pe_mux_3_input.sv pe_mux_n_input.sv \
cfg_loader.sv cfg_row.sv pe_cfg.sv SRL16E.v SRLC32E.v SRL64.v mocarabe_tb.sv
vvp sim.vvp- Channels. The scheduling ILP grows with the channel count, so a large
-Coften runs out of solver time (--sched_time, default 30 s) rather than being infeasible. Start low and let--max_Craise it; most benchmarks schedule at C=2–4. - Fixed device.
--device devices/tt_3x3_1mul.jsonmaps onto a device you describe instead of a right-sized one. The file draws the PE grid top row first, with+(adder),*(multiplier) andIOcells:A PE runs at most II nodes per schedule, so e.g. two multiplies on one{"name": "tt_3x3_1mul", "grid": ["IO + IO", "+ * + ", "IO + + "], "channels": 2}*PE need-II 2; the tool checks this before placement. - Hard or soft bitstream. Every project carries both. The hard bitstream
bakes the schedule into the RTL with
$readmem(smallest, one kernel per chip). The soft bitstream loads it at run time: holdcfg_enlow for a clock, then high, and sendbitstream.hex, one--cfg_width-bit word per clock oncfg_data(socfg_width + 1pins), then pulserstand run. Its memories hold--ii_maxschedule slots (or the device file'sii_max) and the bitstream sets the II actually used. Pick the default with--bitstream soft, or per simulation withiverilog -DBITSTREAM_SOFT=1. - Wide input fanout. Each PE input buffers operands in a 64-deep FIFO. If one
input feeds consumers more than 63 cycles apart (e.g. Horner-form polynomials),
--split_input_fanoutgives each consumer its own copy of that input.
If you use any part of the code or data from this repository for academic work, please cite the associated paper as follows:
@INPROCEEDINGS{9444076,
author={Tombs, Frederick and Mellat, Alireza and Kapre, Nachiket},
booktitle={2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)},
title={Mocarabe: High-Performance Time-Multiplexed Overlays for FPGAs},
year={2021},
doi={10.1109/FCCM51124.2021.00021}
}

