During last years Arm released some specifications in an effort to help organise standardise their market. We got Server Base System Architecture (SBSA), then Base System Architecture (BSA) and finally PC Base System Architecture (PC-BSA) one. Both SBSA and PC-BSA extend BSA (SBSA 7.0 was rebased on top of BSA).
Did these documents changed the market? That’s a discussion for another time.
The checklist
Each of mentioned specifications comes with a compliance checklist referencing document sections such as B_PE_11 which states:
Each PE must implement a minimum of six breakpoints, two of which must be able to match virtual address, contextID or VMID.
To visualize these checklists I created the Arm BSA/PC-BSA/SBSA checklist page where this data is presented as a table. How it is generated will be explained below.
Running ACS
Following the checklist manually is a difficult task so Arm released Architecture Compliance Suites (ACS in short) for BSA and SBSA specifications which run tests and tell whether your hardware is compliant. PC-BSA so far does not have own ACS yet.
NOTE: I ran compliance suites only on UEFI+ACPI systems so do not know how BSA ACS behaves on DeviceTree based ones.
At the start, ACPI tables are parsed and hardware details are checked, such as GIC, SMMU, PCIe etc. You then get a summary of the information:
Creating Platform Information Tables
PE_INFO: Number of PE detected : 4
GIC INFO: GIC version : v3
GIC_INFO: Number of GICD : 1
GIC_INFO: Number of GICR RD : 1
GIC_INFO: Number of ITS : 1
TIMER_INFO: System Counter frequency : 1000 MHz
TIMER_INFO: Number of system timers : 0
WATCHDOG_INFO: Number of Watchdogs : 1
PCIE_INFO: Number of ECAM regions : 1
PCIE_INFO: Number of BDFs found : 4
PCIE_INFO: Number of RCiEP : 2
PCIE_INFO: Number of RCEC : 0
PCIE_INFO: Number of EP : 1
PCIE_INFO: Number of RP : 1
PCIE_INFO: Number of iEP_EP : 0
PCIE_INFO: Number of iEP_RP : 0
PCIE_INFO: Number of UP of switch : 0
PCIE_INFO: Number of DP of switch : 0
PCIE_INFO: Number of PCI/PCIe Bridge : 0
PCIE_INFO: Number of PCIe/PCI Bridge : 0
SMMU_INFO: Number of SMMU CTRL : 1
SMMU_INFO: SMMU index 00 version : v3.1
Peripheral: Num of USB controllers : 1
Peripheral: Num of SATA controllers : 1
Peripheral: Num of UART controllers : 1
The format may differ a bit between both BSA ACS and SBSA ACS but data is there.
Then tests run in groups (PE, Memory map, GIC, SMMU, PCIe etc.). Each test can return one of 3 values: PASS, FAIL or SKIPPED (usually when the requirements to run the test are not met):
28 : Check Fine Grain Trap Support
Failed on PE - 0
S_L7PE_01
Checkpoint -- 1 : Result: FAIL
29 : Check for ECV support
Failed on PE - 0
S_L7PE_02
Checkpoint -- 1 : Result: FAIL
30 : Check for AMU Support
Failed on PE - 0
S_L7PE_03
Checkpoint -- 1 : Result: FAIL
31 : Checks ASIMD Int8 matrix multiplc : Result: PASS
32 : Check for BFLOAT16 extension : Result: PASS
33 : Check PAuth2, FPAC & FPACCOMBINE : Result: PASS
34 : Check for SVE Int8 matrix multiplc : Result: PASS
35 : Check for data gathering hint : Result: PASS
36 : Check WFE Fine tune delay feature
Recommened WFE fine-tuning delay feature not implemented
S_L7PE_09
Checkpoint -- 2 : Result: SKIPPED
As you can see when test does not pass you get a tag (like S_L7PE_01) pointing to relevant specification.
Verbose mode
Each ACS can be run in verbose mode by adding “-v 1” to the command line. Amount of detail is increased to the level that logging output is highly recommended. You can compare my older logs:
Back to table stuff
In my sbsa-ref-status repository I have scripts that gather and parse data for QEMU’s SBSA Reference Platform (sbsa-ref in short). The result is set of YAML files (status-bsa.yml and status-sbsa.yml) that contain information how the tests went:
31:
level: '7'
status:
cortex-a57: FAIL
cortex-a72: FAIL
max: PASS
neoverse-n1: FAIL
neoverse-n2: PASS
neoverse-v1: PASS
tags: S_L7PE_04
title: Checks ASIMD Int8 matrix multiplc
As you see test for S_L7PE_04 passed on some cpu core models and failed on old ones. This pattern continues for other tests and tags. Several entries have only SKIPPED values because the hardware lacked something required to run them.
Those scripts should work with results from other hardware.
Test coverage
Some tags do not have coverage in the ACS. Some tests check for things that are not present in checklists present in the specifications. In these cases it is important to look into ACS documentation:
Both of these expand on the checklists from the specifications with additional information. Which SBSA level test was written for, is it tested in the UEFI or Linux environment, is additional exerciser card required etc.
I used both to expand status-(s)bsa.yml files to ensure all tested entries are listed.
Generation of checklist page
To generate a page with checklist table I use another YAML file: xbsa-checklist.yml. This file maps tags from the specifications into groups and subgroups and keeps information on whether tag required for BSA v1/v2, PC-BSA or SBSA levels. I wrote it by hand and it needs to be updated with every specification update.
Next generate-xbsa.py needs to be run which generates HTML page with the table.
Caveats
Changes to ACS may alter the test numbers or tags used. I reported several issues against both BSA ACS and SBSA ACS about it and they were handled. At this moment all tests have tags assigned.
If the generated page lists “ACS only tests” entries it means one of status-*.yml file needs to be updated because some unhandled tag was used. Or ACS change had error in tag name.
Tests may be renamed — in such cases status files will get new ones. When test numbers change (which is rare) then manual checking may be required.
PC-BSA ACS?
According to Arm developers (BSA ACS issue #395) there will be PC-BSA ACS in first week of December. Once it is released, a new parser will need to be written (like one for BSA ACS or for SBSA ACS) and the page generator updated to use this information.
Future?
This page is one of projects I plan to abandon in 2025. It was a useful tool for checking what needed to be done for the SBSA Reference Platform, either on virtual hardware (QEMU) or in firmware (TF-A + EDK2).
None of Arm hardware I use at home is SBSA compliant. Running BSA ACS on some of those causes them to hang. I do not expect this situation to change in the coming months.
The current page will remain online but I do not plan to invest time in updating it.