Skip to content
gigereskPublic

About

CGRA overlay toolchain for Xilinx FPGAs (ILP scheduler, placer, RTL generator)

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

Mocarabe

CI

Mocarabe is a CGRA (Coarse-Grained Reconfigurable Array) architecture generator and a fully-custom toolchain. The implementation and results are discussed in Mocarabe: High-Performance Time-Multiplexed Overlays for FPGAs, published at FCCM 2021. This work was done by Frederick Tombs, Alireza Mellat, and Prof. Nachiket Kapre.

To get started, jump to Toolchain setup.

Architecture

The architecture consists of a grid of building blocks connected by a unidirectional torus network, as shown in the figure below. Each block contains a processing element (PE) to execute operations on incoming data and a set of NoC routers to control data movement.

PEs store incoming operands in shift registers and select the relevant stored operands as inputs to their ALU at each cycle, as seen below.

The architecture is designed for statically-scheduled, time-multiplexed sharing of both routing and compute resources, with a repeating context window of length II (for initiation interval). II is the number of cycles in the modulo schedule found by the compiler. This makes the architecture periodic in both space (toroidal topology) and time (modulo schedule).

Toolchain

The toolchain:

  • Compiles a C kernel into a DFG
  • Generates a right-sized architecture by allocating PEs, or maps onto a fixed device you describe
  • Partitions nodes into processing elements
  • Places these PEs in the grid to minimize communication cost (wirelength)
  • Schedules routing and computation
  • Generates both synthesizable RTL and simulation artifacts.

Placement is done with simulated annealing and scheduling has dual strategies: an integer linear programming formulation (default) and a temporal-spatial PathFinder implementation.

Mocarabe architecture and EDA toolchain flow

Toolchain setup

python3 -m venv .venv
source .venv/bin/activate
pip install -e .[dev]

# install icarus verilog. e.g. on Ubuntu:
sudo apt install iverilog

make -C llvm_pass  # build the LLVM plugin

Quick example

Here is a C kernel we can map:

// int_adder_chain.c

int int_adder_chain(int x, int y, int z, int a, int* ret) {
	*ret = x+y+z+a;
}

Compile to a DFG, then map and simulate:

# Compile C kernel to hypergraph DFG
./llvm-with-clang.sh int_adder_chain.c hgr/

# Map to a Mocarabe architecture
  #   -II 1      : initiation interval (schedule length in cycles)
  #   -C 2       : physical channels per NoC link to start with
  #   --max_C 8  : if scheduling fails, raise C up to 8
  #   -iod 1     : IO-node packing density (1 IO node per PE)
  #   -ard 1     : arithmetic-node packing density (1 op per PE)
mocarabe -dfg hgr/int_adder_chain -II 1 -C 2 --max_C 8 -iod 1 -ard 1

# Visualize the placed-and-routed design
mocarabe-viz --proj proj/int_adder_chain_*/

# Simulate the generated RTL (path printed by the command above)
cd proj/int_adder_chain_*/rtl/
iverilog -g2012 -o sim.vvp mocarabe.sv pe_2_input.sv torus_switch.sv \
          pe_srl.v pe_mux_2_input.sv pe_mux_3_input.sv pe_mux_n_input.sv \
          cfg_loader.sv cfg_row.sv pe_cfg.sv SRL16E.v SRLC32E.v SRL64.v mocarabe_tb.sv
vvp sim.vvp

Mapping options

  • Channels. The scheduling ILP grows with the channel count, so a large -C often runs out of solver time (--sched_time, default 30 s) rather than being infeasible. Start low and let --max_C raise it; most benchmarks schedule at C=2–4.
  • Fixed device. --device devices/tt_3x3_1mul.json maps onto a device you describe instead of a right-sized one. The file draws the PE grid top row first, with + (adder), * (multiplier) and IO cells:
    {"name": "tt_3x3_1mul",
     "grid": ["IO  +   IO",
              "+   *   + ",
              "IO  +   + "],
     "channels": 2}
    A PE runs at most II nodes per schedule, so e.g. two multiplies on one * PE need -II 2; the tool checks this before placement.
  • Hard or soft bitstream. Every project carries both. The hard bitstream bakes the schedule into the RTL with $readmem (smallest, one kernel per chip). The soft bitstream loads it at run time: hold cfg_en low for a clock, then high, and send bitstream.hex, one --cfg_width-bit word per clock on cfg_data (so cfg_width + 1 pins), then pulse rst and run. Its memories hold --ii_max schedule slots (or the device file's ii_max) and the bitstream sets the II actually used. Pick the default with --bitstream soft, or per simulation with iverilog -DBITSTREAM_SOFT=1.
  • Wide input fanout. Each PE input buffers operands in a 64-deep FIFO. If one input feeds consumers more than 63 cycles apart (e.g. Horner-form polynomials), --split_input_fanout gives each consumer its own copy of that input.

How to Cite

If you use any part of the code or data from this repository for academic work, please cite the associated paper as follows:

@INPROCEEDINGS{9444076,
  author={Tombs, Frederick and Mellat, Alireza and Kapre, Nachiket},
  booktitle={2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)},
  title={Mocarabe: High-Performance Time-Multiplexed Overlays for FPGAs},
  year={2021},
  doi={10.1109/FCCM51124.2021.00021}
}

About

CGRA overlay toolchain for Xilinx FPGAs (ILP scheduler, placer, RTL generator)

Resources

Stars

1 star

Watchers

0 watching

Forks

Contributors

Languages