This project aims to design and implement a high capacity memory subsystem for Arm A series processor based SoC designs.  The current focus of the project is the design and implementation of a Memory Controller for DDR4 memory. 

Introduction

The efficient and cost effective use of memory is critical to SoC design. Significant design and intellectual property ("IP") blocks work in coordination within the SoC to deliver system level performance at a reasonable cost. Traditional SoC designs optimise memory use via a memory hierarchy. Within the processor, high speed registers are accessed directly by physical control signals for various compute actions. They can be grouped together into Register Files but these have limits to their physical addressing. A SoC usually contains small memory caches of Static Random Access Memory (SRAM) close to the processor core, these are accessed via the Memory Management Unit ("MMU"), which provides address and memory attribute translation allowing the core processor instructions to use a large Virtual Memory space with addresses that are mapped (translated) to the Physical Memory locations in the caches. Other IP blocks enable the efficient delivery of data through the caches to keep the processor running efficiently. There are cost and area limits to how much on chip memory the SoC can hold. Data not immediately required by the compute fabric of the SoC can be saved in more dense and cost effective external DRAM memory devices. Access to off chip DRAM is appreciably slower than on-chip SRAM and it's greater storage density comes with the downside that data needs to be actively refreshed periodically. The fabrication process for DRAM is different from on-chip SRAM and the suppliers of memory chips differ from the SoC foundries. To enable the industry to provide cost effective interfaces there are JEDEC standards for how external memory interfaces to the SoC. 

Through the Arm AAA access academics can get all the Arm IP for on-chip memory access. A memory controller IP for external DDR4 memory is not available.  There are a small number of DDR3 open source projects for Memory Controller IP blocks, those that have been silicon proven are even more limited in number. This project aims to design and implementation of a Memory Controller for DDR4 memory and to verify it within a physical fabrication of an Arm A series processor based SoC design. 

The design of the Memory Controller for DDR4 memory needs to sit within the classical memory hierarchy for Arm A series processor based SoC designs that may utilise complex multi-tasking operating systems. It also needs to support large data movements for custom acceleration compute for workloads such as machine learning and inference. These uses of data require clear control over the complete memory system.

Basic Principle of Operation

DRAM is a type of memory that as well as supporting the obvious read and write operations for data needs to refresh and data stored at specific time periods specified by the manufacturer of the memory in order for the data not to be corrupted. The external DRAM memory devices require time to undertake to any read, write, refresh and other memory operations. As DRAM memory devices can come from different manufacturers and can be connected via different physical interconnects, the JEDEC specification allows for timing parameters to be set as part of the DRAM Controller configuration. The timing of operations must be agreed between the on chip DDR memory controller and the external off chip memory devices. The specification also defines the various  memory operations as a set of standardised commands. These DDR commands are defined by the state of the signals of the physical layer ("PHY") interconnect at the rising edge of the clock. 

DRAM memory is organized into a number of banks which are further divided into rows and columns and the data is stored densely at the intersection of the rows and columns. The read process in DRAM is destructive. Sense amplifiers temporarily hold the data allowing it to rewritten as part of the read cycle. Data is also passed to the output buffers for transmission to the SoC. The read process requires an ACTIVE DDR command, (with row address) to activate a row in a particular bank, then a READ command (with column address) to access the data. This staged read process makes DRAM slower than SRAM but the benefit is improved memory density. CAS latency is the delay between the time at which the column address and the column address strobe signal ("CAS") are presented to the memory device and the time at which the corresponding data is made available. The write process requires and additional write enable signal ("WE") to be presented to the memory device. The PRECHARGE command deactivates active rows, which a then unavailable for a specific time, and must be  reactivated before any additional READ or WRITE requests.

Read cycles refresh specific memory locations but cannot be relied on to cover all memory locations within the necessary time. Modern DRAM devices have their own integral refresh logic and can be issued refresh commands. Otherwise the memory control within the SoC needs to implement refresh counters that increment to step through the rows to be refreshed. Refresh scheduling strategies range from a mode where normal memory access is suspended while all rows are refreshed in a single shot within the memory device time limit, to more complex modes where refresh cycles are interlaced with memory accesses. 

The basis of commands are defined by the three signals, row address strobe, column address strobe and write enable.

There are variations and additional operations within the full JEDEC specification defined in a Command Truth Table. 

Optimising Performance

One of the advantages of modern DRAM memory devices is that they can optimise the use of time by pipelining, accepting new commands, while existing commands are in process.  The clock signal controls the stepping of a finite-state machine that accepts the incoming commands. Other parallelism involves issuing commands to each bank simultaneously.  A write command can be immediately followed by another without waiting for the data to be written. For read requests, the data is available a fixed number of clock cycles (latency) after the read request, during which additional commands can be sent.

Configuration

The JEDEC standard for Serial Presence Detect (SPD) provides methods to store and access information about memory modules. Memory device manufacturers instantiate SPD data within their devices which includes timing parameters, serial number and other useful information about the memory. As part of the SoC boot sequence this information needs to be accessed and used to configure the memory controller. The DDR4 SDRAM "Annex L" standard for SPD defines the layout of the SPD information for DDR4 SDRAM within the 512 bytes of addressable memory depending on the type of the memory module.  It is usually accessed by the i2c based serial System Management Bus of the system board. The arm processor during boot sequence will need to access the SPD information which should be mapped into the address space and this information passed to the DDR memory controller via the APB interface.