U-Boot and generic distro boot – Marcin Juszkiewicz

This post is part 4 of the "Standards in Arm Space" series:

Small board computers (SBC) usually come with U-Boot as firmware. There could be some more components like Arm Trusted Firmware, OPTEE etc but what user interact with is the U-Boot itself.

Since 2016 there is the CONFIG_DISTRO_DEFAULTS option in U-Boot configuration. It selects defaults suitable for booting general purpose Linux distributions. Thanks to it board is able to boot most of OS installers out of the box without any user interaction.

How?

How does it know how to do that? There are several scripts and variables involved. Run “printenv” command in U-Boot shell and there you should see some of them named like “boot_*, bootcmd_* scan_dev_for_*”.

In my example I would use environment from RockPro64 running U-Boot 2021.01 version.

I will prettify all scripts for readability. Script contents may be expanded — in such case I will give name as a comment and then it’s content.

Let’s boot

First variable used by U-Boot is “bootcmd”. It reads it to know how to boot operating system on the board.

In out case this variable has “run distro_bootcmd” in it. So what is there on RockPro64 SBC:

setenv nvme_need_init
for target in ${boot_targets}
do 
    run bootcmd_${target}
done

It says that on-board NVME needs some initialization and then goes through set of scripts using order from “boot_targets” variable. On RockPro64 this variable sets “mmc0 mmc1 nvme0 usb0 pxe dhcp sf0” order which means:

eMMC
MicroSD
NVME
USB storage
PXE
DHCP
SPI flash

Both eMMC and MicroSD look similar: ‘devnum=X; run mmc_boot’ — set MMC number and then try to boot by running ‘mmc_boot’ script:

if mmc dev ${devnum}; then 
    devtype=mmc; 
    run scan_dev_for_boot_part; 
fi

NVME one initialize PCIe subsystem (via “boot_pci_enum”), then scans for NVME devices (via “nvme_init”) and do the similar stuff (here with expanded scripts):

# boot_pci_enum
pci enum

# nvme_init
if ${nvme_need_init}; then 
    setenv nvme_need_init false;
    nvme scan;
fi

if nvme dev ${devnum}; then 
    devtype=nvme; 
    run scan_dev_for_boot_part; 
fi

USB booting goes with “usb_boot”:

usb start;
if usb dev ${devnum}; then 
    devtype=usb; 
    run scan_dev_for_boot_part;
fi

PXE network boot? Initialize USB, scan PCI, get network configuration, do PXE boot:

# boot_net_usb_start
usb start

# boot_pci_enum
pci enum

dhcp; 
if pxe get; then 
    pxe boot; 
fi

DHCP method feels like last resort one (do not ask me for meaning of all those variables):

# boot_net_usb_start
usb start

# boot_pci_enum
pci enum

if dhcp ${scriptaddr} ${boot_script_dhcp}; then 
    source ${scriptaddr}; 
fi;

setenv efi_fdtfile ${fdtfile}; 
setenv efi_old_vci ${bootp_vci};
setenv efi_old_arch ${bootp_arch};
setenv bootp_vci PXEClient:Arch:00011:UNDI:003000;
setenv bootp_arch 0xb;

if dhcp ${kernel_addr_r}; then 
    tftpboot ${fdt_addr_r} dtb/${efi_fdtfile};

    if fdt addr ${fdt_addr_r}; then 
        bootefi ${kernel_addr_r} ${fdt_addr_r}; 
    else 
        bootefi ${kernel_addr_r} ${fdtcontroladdr};
    fi;
fi;

setenv bootp_vci ${efi_old_vci};
setenv bootp_arch ${efi_old_arch};
setenv efi_fdtfile;
setenv efi_old_arch;
setenv efi_old_vci;

And last method is SPI flash:

busnum=0

if sf probe ${busnum}; then
    devtype=sf;

    # run scan_sf_for_scripts; 
    ${devtype} read ${scriptaddr} ${script_offset_f} ${script_size_f}; 
    source ${scriptaddr}; 
    echo SCRIPT FAILED: continuing...
fi

Search for boot partition

Note how block devices end with one script: “scan_dev_for_boot_part”. What it does is quite simple:

part list ${devtype} ${devnum} -bootable devplist; 
env exists devplist || setenv devplist 1; 

for distro_bootpart in ${devplist}; do 
    if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then 
        run scan_dev_for_boot; 
    fi; 
done; 
setenv devplist

We know type and number of boot device from previous step so now we check for bootable partitions. Which means EFI System Partition for GPT disks and partitions marked as bootable in case of MBR. If none are present then first one is assumed to be bootable one.

Search for distribution boot information

Once we found boot partitions it is time to search for boot stuff with “scan_dev_for_boot” script:

echo Scanning ${devtype} ${devnum}:${distro_bootpart}...;
for prefix in ${boot_prefixes}; do 
    run scan_dev_for_extlinux; 
    run scan_dev_for_scripts; 
done;

run scan_dev_for_efi;

Old style OS configuration

First U-Boot checks for “extlinux/extlinux.conf” file, then go for old style “boot.scr” (in uimg and clear text formats). Both of them are checked in / and /boot/ directories of checked partition (those names are in “boot_prefixes” variable).

Let us look at it:

# scan_dev_for_extlinux
if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf};then 
    echo Found ${prefix}${boot_syslinux_conf}; 

    # run boot_extlinux; 
    sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}

    echo SCRIPT FAILED: continuing...; 
fi

# scan_dev_for_scripts
for script in ${boot_scripts}; do 
    if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then 
        echo Found U-Boot script ${prefix}${script}; 

        # run boot_a_script; 
        load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; 
        source ${scriptaddr}

        echo SCRIPT FAILED: continuing...; 
    fi; 
done

EFI compliant OS

And finally U-Boot checks for EFI style BootOrder variables and generic OS loader path:

# scan_dev_for_efi
setenv efi_fdtfile ${fdtfile};
for prefix in ${efi_dtb_prefixes}; do
    if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then 
        # run load_efi_dtb; 
        load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
    fi;
done;

# run boot_efi_bootmgr;
if fdt addr ${fdt_addr_r}; then 
    bootefi bootmgr ${fdt_addr_r};
else 
    bootefi bootmgr;
fi

if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootaa64.efi; then
    echo Found EFI removable media binary efi/boot/bootaa64.efi; 

    # run boot_efi_binary; 
    load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi; 
    if fdt addr ${fdt_addr_r}; then 
        bootefi ${kernel_addr_r} ${fdt_addr_r};
    else 
        bootefi ${kernel_addr_r} ${fdtcontroladdr};
    fi

    echo EFI LOAD FAILED: continuing...;
fi; 
setenv efi_fdtfile

Booted

At this moment board should be in either OS or in OS loader (being EFI binary).

Final words

All that work on searching for boot media, boot scripts, boot configuration files, OS loaders, EFI BootOrder entries etc is done without any user interaction. Every bootable media is checked and tried.

If I would add SATA controller support into U-Boot binary then all disks connected to such would also be checked. Without any code/environment changes from my side.

So if your SBC has some weird setup then consider moving to distro generic one. Boot fresh mainline U-Boot, store copy of your existing environment (“printenv” shows it) and then reset to generic one with “env default -a” command. Probably would need to set MAC adresses for network interfaces.