This month, our primary focus was on defining the interfaces and interactions between the components and finalizing the microarchitecture specifications for the various blocks.

• Control Subsystem
  • Draft a detailed system workflow that outlines the runtime's management of Processing Elements (PEs) within the compute grid, encompassing scheduling mechanisms and workload monitoring for efficient execution.
  • Specify the file format for the Top-Level Loadable, which includes the essential network information and serves as the runtime's input.
  • Implement the first runtime prototype that demonstrates the implementation of parsing the initial set of subgraphs and event handling.
     
• Interconnect
  • Design the Data-Link Layer Protocol for messaging and data transactions on Network-on-Chip (NoCs).
  • Prepare the programming interface and microarchitecture specifications for all three Network Interface Controllers (NICs) within the system. Additionally, establish a clear and comprehensive understanding of how the NICs interact with each other.
  • Design and implement the NoC routers capable of supporting unicast and multicast functionalities for the two NoCs in the compute grid.
     
• PE Tile
  • Specify the address mapping and establish interfaces between all blocks within the PE tile, ensuring seamless communication and coordination.
  • Define PE-Level Loadable's file format, which contains the essential information of the dispatched model subgraphs.
  • Finalize the programming interface and detailed specification for the DMA.

This month, our primary focus was finalizing all blocks' specifications and ensuring compatibility and consistency. Additionally, we identified several system-level integration issues and proposed viable solutions.

• Control Subsystem
  • Finish the Network Interface Controller (NIC) specification in the Control Subsystem.
• Cache Clusters
  • Finish the specification of the Global Cache Subsystem, whose cache banks are designed as low-order interleaving to enhance the effective data throughput.
• Interconnect
  • Improve the flexibility of the Data-Link Layer Protocol to support software-defined communications between different subsystems.
  • Finish the Transaction Layer Protocol overlay on top of the Data Link Layer Protocol to facilitate communication among application processes running on different CPU runtimes.
• PE Tile
  • Generate the Programmable Compute Unit (PCU), the system bus, and the Private Memory with third-party open-source projects.
  • Finish the specification of the Shared Tensor Memory Controller for packing outbound tensors and unpacking inbound tensors.
• System Design Issues
  • Large-Size Tensor Data Movement
    With the Global Cache Subsystem being banked as a low-order interleaving design, the exchange and retrieval of data with cache clusters cause a significant challenge for PE-Tiles. Thus, we need to find a solution for PE-Tiles to send the correct parts of the whole tensors into corresponding cache banks.
  • Completion Hints Support for Tensor Write
    The Control Subsystem manages the dependency of tensors between PE-Tiles. It thus needs to know if the tensor can be fetched from the cache for later computation purposes. The completion hint supports Cache Clusters to notify the Control Subsystem of such information.
  • Tensor ID Assignment
    Our system uses Tensor ID for various purposes, such as resource management, communication, and data exchange. The finite bit length of the Tensor ID field defined in the hardware programming interfaces limits the number of on-the-fly tensors. Thus, we carefully design the Tensor ID assignment rule to prevent ID shortage.

• Verifying communication types without data exchange in Data Link Layer Protocol
  • PE-Tile: Implement the Network Interface Controller (NIC), the Direct Memory Access (DMA), and a debug module acting as a PCU to finish communication procedures.
  • Control subsystem: Implement a baremetal runtime running on the processor in the Processing System of our FPGA board.
  • Verify hardware correctness with RTL simulation.
  • Prototype the hardware design on our FPGA board and co-simulate with the runtime to monitor the process of each target communication types.
• Verifying communication types with data exchange in Data Link Layer Protocol
  • PE-Tile
    • Enhance the Network Interface Controller (PE-NIC), the Direct Memory Access (DMA), and the debug module acting as a PCU.
    • Implement the Shared Memory Controller (SMC) to handle memory access requests from PE-Tile system bus and data copy operations through NoC interconnects.
  • Control subsystem: Implement a baremetal runtime running on the processor in the Processing System of our FPGA board.
  • Global cache memory: Implement the Cache Network Interface Controller (Cache-NIC) and cache controller to accept requests from components attached on NoCs and return corresponding response.
  • The RTL verification and FPGA prototyping will be finished in the next milestone.