There has been much request within the SoC Labs community for an Arm A-Class SoC that can support a full operating system platform, undertake more complex compute tasks and enable more complicated software loads. The Cortex-A53 is Arm's most widely deployed 64-bit Armv8-A processor and can provide these capabilities with power efficiency

This project mirrors the 'Arm Cortex-M0 microcontroller' project in establishing a baseline capability for the Cortex-A53 processor. It will lay the foundation for a megaSoC reference design in the same way the Arm Cortex-M0 microcontroller project laid the foundation for the nanoSoC reference design. Both foundational projects develop the core capabilities around the processor core including:

• Establishing the critical boot system
• The system debug environment
• An easy design transition from the FPGA prototyping flow to the full ASIC flow

The later will involve replicating the resources available in the Zynq FPGA processing system to allow the seamless transition between FPGA and ASIC.

towards megaSoC

Using these foundations work will be undertaken to develop a more complete megaSoC reference design. This system will have:

• Cortex A53 processor
• High bandwidth data bus (NIC400)
• High capacity memory (DDR)
• NVM storage for deploy-ability

 It will maintain resource for high data loads with configurable hardware accelerators working in combination with CPU cores under a complex software environment.

Milestone 1 - Minimum bootable subsystem

To start this project, we will begin with a minimum bootable subsystem in simulation. The idea for this subsystem is to be able to run code on the Cortex A53 and interact with the outer testbench (via UART)

From the above image you can see the basic architecture. This has a Cortex A53 connected to a NIC-400 bus. A ROM that acts as boot rom, XiP QSPI controller for instruction memory, and SRAM as data memory. The UART is included to allow for printf statements in the C code.

The XiP QSPI will currently be used as the entire instruction memory space, however in future this may be used only as a BIOS for a linux system. The rest of the linux software will be installed on external storage (either SD card or SATA capable device)

Boot Sequence

The boot sequence for the Arm Cortex A53 is a bit more complex than the Cortex M0 and requires some more care. The full information on this can be found from Arm here, which describes how to boot a Armv8-a processor.

Above is a simplified boot sequence that we are using for this subsystem. The boot-code (in ROM) is responsible for starting the A53 in a clean state (i.e. registers are initialised) and enables the UART communication and caches (L2 and Flash Cache) before enabling the XiP QSPI controller and then setting the execution to the Flash region.

The expected output from the ROM boot-code is as follows: 

SoCLabs MegaSoC
Flash enabled... Excecuting

This first stage booting process works as expected. However currently, our hello_world testcode looks like it is going straight into an exception. The code is as follows:

#include "host_chassis_control.h"
#include "system.h"
#include "system_level_functions.h"
#include "uart_stdout.h"

int main(void) {
  uint32_t errors = 0;

  printf("Hello SoCLabs MegaSoC\n");

}

However we don't see a print message. So further debugging is required.

A53 debugging in simulation

Debugging a complex processor like the A53 using the simulation GUI is difficult. That is why Arm processors are delivered with a Tarmac unit. These units are used in simulation to show exactly what the processor is executing, and can be very useful when debugging. The ca53_univent_follower module has to be included in the vc filelist, and the ca53_tarmac_dpi.so object must be loaded by the simulator. Exact details on how to do this are included in the README_tarmac.txt in the ca53univent directory.

The tarmac requires some libraries to be installed before use, most notable the libprotobuf.so.7 from protobuf 2.4.1. To install you can follow the steps outlined below (taken from here):

wget https://github.com/protocolbuffers/protobuf/releases/download/v2.4.1/protobuf-2.4.1.tar.bz2
tar -xvjf protobuf-2.4.1.tar.bz2
cd protobuf-2.4.1/
./configure --with-zlib --prefix=<prefix> CXX='g++ -m32 -std=c++98'
make install

WARNING

Currently this is not working for us, we get the message:

univent follower module: megasoc_tb.u_megasoc_chip_pads.u_megasoc_chip.u_megasoc_system.u_megasoc_tech_wrapper.u_megasoc_cpu_ss.u_cortexa53.g_ca53_cpu[0].u_ca53_cpu.u_ca53_noram.u_follower
/home/dwn1c21/SoC-Labs/megasoc_project/megasoc_tech/logical/CortexA53_1/logical/ca53univent/build_x86_32/bin/ca53_tarmac_decode: symbol lookup error: /home/dwn1c21/SoC-Labs/megasoc_project/megasoc_tech/logical/CortexA53_1/logical/ca53univent/build_x86_32/bin/ca53_tarmac_decode: undefined symbol: _ZN6google8protobuf8internal12kEmptyStringE
Error writing trace: Broken pipe

We are awaiting from Arm on a way to fix this issues and this page will be updated with the fix once we have it

UPDATE

We have received support from Arm on this issue with the tarmac module. It seems like its an issue of building the protobuf library on RHEL 8, they have supplied us with a compiled protobuf library to use (which I believe they compiled in RHEL 6) which now works. So back to debugging...

Now with the working A53 tarmac module, it could be seen that the issue was that when it changed to executing from the bootrom to the QSPI flash, there was an abort exception. After reviewing the simulation in the GUI, I could see there was some issue with our AHB QSPI module. As this module is not fully verified yet, we have decided to remove it for now and replace this with an AHB SRAM module (we've kept the same AHB port and range so that it is easy to re-instantiate the QSPI module once it is working)

The simplified testbench structure can be seen above. When using this we get a successful boot. The main bootcode now looks like below:

boot.s

.section SECURE_ROM_BOOT, "ax"
            .balign 8
            .global    Image$$ARM_LIB_STACK$$ZI$$Limit
            .global    __main
            .global   monitor_vectors    
            .global   __stack_multi_cpu_init 

            .weak monitor_vectors
// ------------------------------------------------------------------------------
// Core initialisation from reset state
// ------------------------------------------------------------------------------
            .global app_bl1_entry
            .type app_bl1_entry, @function

app_bl1_entry:
            MOV     x0,#0x0
            MOV     x1,x0
            MOV     x2,x0
            MOV     x3,x0
            MOV     x4,x0
            MOV     x5,x0
            MOV     x6,x0
            MOV     x7,x0
            MOV     x8,x0
            MOV     x9,x0
            MOV     x10,x0
            MOV     x11,x0
            MOV     x12,x0
            MOV     x13,x0
            MOV     x14,x0
            MOV     x15,x0
            MOV     x16,x0
            MOV     x17,x0
            MOV     x18,x0
            MOV     x19,x0
            MOV     x20,x0
            MOV     x21,x0
            MOV     x22,x0
            MOV     x23,x0
            MOV     x24,x0
            MOV     x25,x0
            MOV     x26,x0
            MOV     x27,x0
            MOV     x28,x0
            MOV     x29,x0
            MOV     x30,x0
            
            MSR     SP_EL0,x0
            MSR     SP_EL1,x0
            MSR     SP_EL2,x0
            MOV     sp,x0
            MSR     ELR_EL1,x0
            MSR     ELR_EL2,x0
            MSR     ELR_EL3,x0
            MSR     SPSR_EL1,x0
            MSR     SPSR_EL2,x0
            MSR     SPSR_EL3,x0

//===================================================================
// Set Vector Base Address Register (VBAR) to point to this application's vector table
//===================================================================
            LDR x0, =monitor_vectors       
            MSR VBAR_EL3, x0             // EL3 sets vector base address

//===================================================================
// Clear the PSTATE.A fpr enabling SError Aborts (Posion Error)   
//===================================================================
            MSR     DAIFCLR, #0x4
            ISB
            
//===================================================================
// Enable NEON and initialize the register bank
//===================================================================
            MRS     x0, ID_AA64PFR0_EL1
            SBFX    x5, x0, #16, #4         // Extract the floating-point field

            MOV     x1, #(0x3 << 20)
            MSR     cpacr_el1, x1
            MRS     x1, cptr_el3

            BIC     x1, x1, #(0x1 << 10)      // Ensure that CPTR_EL3.TFP is clear
            MSR     cptr_el3, x1
            ISB     sy
#ifndef NOFP
            FMOV    d0,  xzr
            FMOV    d1,  xzr
            FMOV    d2,  xzr
            FMOV    d3,  xzr
            FMOV    d4,  xzr
            FMOV    d5,  xzr
            FMOV    d6,  xzr
            FMOV    d7,  xzr
            FMOV    d8,  xzr
            FMOV    d9,  xzr
            FMOV    d10, xzr
            FMOV    d11, xzr
            FMOV    d12, xzr
            FMOV    d13, xzr
            FMOV    d14, xzr
            FMOV    d15, xzr
            FMOV    d16, xzr
            FMOV    d17, xzr
            FMOV    d18, xzr
            FMOV    d19, xzr
            FMOV    d20, xzr
            FMOV    d21, xzr
            FMOV    d22, xzr
            FMOV    d23, xzr
            FMOV    d24, xzr
            FMOV    d25, xzr
            FMOV    d26, xzr
            FMOV    d27, xzr
            FMOV    d28, xzr
            FMOV    d29, xzr
            FMOV    d30, xzr
            FMOV    d31, xzr
#endif            
            B   __main

bootloader.c 

#include "system.h"
#include "qspi_flash.h"
#include "cpu_asm_codes.h"
#include "uart_stdout.h"
#include <stdio.h>
int main(void) {
  uint32_t errors = 0;
  UartStdOutInit();

  printf("SoCLabs MegaSoC\n");
  enable_caches();
  enable_caches_el1();
  printf("Flash Enabled...Booting\n");

  void (*main_code)(void) = (void (*)())0x00400000;
  main_code();
}

hello_world.c

#include "uart_stdout.h"
#include <stdio.h>

int main(void) {
  uint32_t errors = 0;
  UartStdOutInit();

  printf("Hello SoCLabs MegaSoC\n");
  UartEndSimulation();
}

And the output we get form simulation is:

uartcapture: Generating output file logs/uart.log using MCD 00000003 @ megasoc_tb.u_uart_capture
univent follower module: megasoc_tb.u_megasoc_chip_pads.u_megasoc_chip.u_megasoc_system.u_megasoc_tech_wrapper.u_megasoc_cpu_ss.u_cortexa53.g_ca53_cpu[0].u_ca53_cpu.u_ca53_noram.u_follower
[M] 119692088.0 ()  Constructing static cortexa53 follower
[M] 119692088.0 ()  Tracing 'megasoc_tb.u_megasoc_chip_pads.u_megasoc_chip.u_megasoc_system.u_megasoc_tech_wrapper.u_megasoc_cpu_ss.u_cortexa53.g_ca53_cpu[0].u_ca53_cpu.u_ca53_noram.u_follower' to 'ca53_tarmac.0.0.0.log'
SoCLabs MegaSoC
Flash Enabled...Booting
Hello SoCLabs MegaSoC
Test Ended
$stop at time 753955 ns Scope: megasoc_tb.u_uart_capture.p_sim_end File: /home/dwn1c21/SoC-Labs/megasoc_project/verif/trace/megasoc_uart_capture.v Line: 244
xterm-256color is Not a valid terminal...
ucli% ca53_finish called
           V C S   S i m u l a t i o n   R e p o r t