Marcin Juszkiewicz - debian

Arm laptops for normal users?

2024-08-16T17:33:00+02:00

There are discussions in development circles about Arm powered laptops since forever. But most of time they do not mention “normal” users. Like your parents, spouses, kids who are not developers. People who turn computer on (cold boot or from suspend does not matter) and expect them to “just work”.

My teenage daughter is one of them. Her current laptop is one of Thinkpad models, previous one was Thinkpad as well. Fedora Linux as operating system serves her needs just fine. But despite my 20 years of work with Arm architecture I am unable to get Arm based laptop for her.

Choices

There are several Arm powered laptops on a market:

Macbooks
Windows on Arm laptops
Chromebooks
SBCs with screen like Pinebook Pro

And all of them have issues when it comes to using Fedora Linux on them.

Macbooks

Thanks to Asahi Linux team we can run Fedora Linux on M1/M2 based Macbooks. Which means second hand market as Apple does not sell those models any more (unless 8GB of ram is enough for you).

There are many things which are not supported:

Thunderbolt
DisplayPort over Thunderbolt
HDMI audio (work in progress)

So you pay for hardware and have features which you cannot use. I use Macbook Pro 2021 (with M1 Pro cpu) for local development and stopped checking how work goes.

Windows on Arm laptops

Qualcomm managed to convince Microsoft to not offer licenses for other vendors which means all we can have are Snapdragon based laptops. Which may work nice under MS Windows but if you want to use Linux then “good luck” is all you can get from me.

Some things work, some do not. I was told that Thinkpad x13s is one of best supported models. Johan Hovold has a Thinkpad x13s status page which lists what works and what needs to be done to have some kind of working laptop.

Definitely not a system for daily use for normal Linux user.

Chromebooks

Laptop to run web browser and Android apps. If this is all you need then go for it. But avoid if you are “normal” user and want to run Linux.

Finding how to enable running anything other than ChromeOS may involve digging through Internet pages, finding how to override ‘write protection’ etc.

Just no. Also Arm ones are usually ram limited.

Pinebook Pro and other SBCs with screen

Those are systems for developers only. Normal users should avoid using them as those systems require someone who knows how to prepare them to work at all.

Find/build proper firmware, put it properly in device (SPI Flash or storage media), keeping things up-to-date may end with partially not working device etc.

For developers those are ‘issues’ to workaround/solve but for normal users it may be ‘update went in background and now all I have is black screen’.

And like with Chromebooks you may be limited by ram size (Pinebook Pro has only 4GB ram).

Conclusion

If you are a normal user who wants to run Linux on a laptop then maybe stay away from Arm powered ones. Leave them for developers and check once/twice per year to see how situation looks.

I had some fun with FriendlyELEC NanoPC-T6

2024-08-09T19:38:00+02:00

At work I spend most of time on SBSA Reference Platform. Especially in firmware part (Arm Trusted Firmware also known as TF-A and Tianocore EDK2 also known as UEFI). However, for some time, I have felt the need to experiment with some UEFI-related task on existing hardware.

I first searched for “affordable” SystemReady SR system. But options were either Ampere Altra or NVIDIA Grace, both prices at 3000 EUR or more.

So I looked at the budget market and bought a FriendlyELEC NanoPC-T6 SBC.

Some words about the hardware

The FriendlyELEC NanoPC-T6 is a SBC (Seriously Bad Computer) based on Rockchip RK3588 SoC. It has some interesting on-board features:

8 cpu cores (4x Cortex-A76, 4x Cortex-A55)
16GB of memory
32MB of SPI flash for storing firmware
64GB of eMMC (not visible under Linux)
USB 2 ports
USB 3 ports (one type A, one type C)
2 HDMI outputs
1 HDMI input
m.2 type M slot for 2280 card (such as NVMe)
m.2 type E slot for 2230 card (such as Wi-Fi)
2 2.5GbE network ports
USB-C serial console port (CH341 chip)
microSD card slot
Power, Reset, MaskROM buttons
audio jack

It comes with metal case which works also as a heatsink.

NanoPC-T6 in the case

Out of the box experience

As you know I expect a good “out of the box” experience. And NanoPC-T6 was like any other SBC I used in the past — unpleasant, horrible and frustrating.

The device came with a fork of U-Boot 2017.09, configured in such terrible way that it was incapable of booting any standard distro images I tried. I managed to boot the pre-installed Android 12 on the eMMC but quickly rebooted to avoid dealing with it.

I managed to boot Debian ‘testing’ manually but there was no networking available under 6.9.x kernel.

I Then moved on to other things as my schedule was quite busy.

UEFI experience

This week I reserved some time to get NanoPC-T6 running properly. I downloaded a Rockchip tool called “upgrade_tool” and used it to flash a UEFI image from the EDK2-RK3588 project.

Experience was much, much better. The firmware was now capable of booting distro images, allowed me to choose between ACPI or DeviceTree for hardware description and had proper EFI Shell — almost like a well-developed systems.

I went with ACPI mode and booted directly to Fedora ‘rawhide’ system stored on a USB drive. Linux 6.11-rc booted, found devices plugged into USB 3 ports, recognized both network interfaces (Realtek 8125 ones) and the NVME drive as well. There was video output on the HDMI screen (in a hardcoded 1080p resolution).

I then copied the system from the USB3 drive to the NVME, set the proper boot order and enjoyed a nicely working system.

DeviceTree?

But aren’t Seriously Bad Computers (SBCs) expected to run with DeviceTree? ACPI is for MS Windows, not for Linux or *BSD systems, right?

So I decided to boot into DT land. It took me a while as I had to remind myself how it works and ensure that UEFI firmware will use the 6.11-rc DTB instead of one for 5.10-rk or 6.1-rk vendor kernels.

Finally it booted — or rather, it “kind of” booted…

No USB, no PCIe == no NVME == boot into emergency mode because the root filesystem is not present…

Future plans

What will future bring? I am going to find out. I have ordered a Wi-Fi card for m.2 type E slot to see how it performs and I am going to spend some time around this EDK2 fork to make some experiments on real hardware.

“Ten” years at Linaro

2023-04-17T12:36:00+02:00

Some time ago was a day when I reached “ten” years at Linaro. Why “”? Because it was 3 + 7 rather than 10 years straight. First three years as Canonical contractor now seven years at Red Hat employee assigned as Member Engineer.

My first three years at Linaro

NewCo or NewCore? Or Ubuntu on ARM?

In 2010 I signed contract with Canonical as “Foundation OS Engineer”. Once there I signed another paper which moved me to NewCo project (also called NewCore but NewCo name is on paper I signed).

On 30th April 2010 I got “Welcome to Linaro” e-mail.

Then UDS-M happened where we were hiding under “Ubuntu on Arm” name (despite the fact that Ubuntu had such team).

Ah, it is Linaro now :D

On 3rd June 2010 Linaro was officially announced. No more hiding, we went public with name.

My team

I became a part of Developer Platform team and worked mostly on toolchain packages for Ubuntu and later Debian. There were funny moments at sprints/summits/connects when I joked that I have two managers to listen to (one from Toolchain Working Group and one from Developer Platform).

There were Debian/Ubuntu developers there and people from other environments.

At some moment it was renamed to “Base and Baselines”. Or “Bed and Breakfast” as most of time “BB” was used instead of full name.

AArch64 bring up

In 2012 I dusted off my OpenEmbedded knowledge and started working on getting AArch64 architecture bring up. Lot of not-yet-public patches was in use. The fun of seeing “Hello world!” message in emulator printed by OS image I built from scratch was something I hope to never forget.

Each time I am choosing mug for my coffee I see Pac-Man one I bought during Linaro/Arm AArch64 sprint we had in October 2012.

My AArch64 mug

The End

There was lot of noise in 2011 about deal between Canonical and Linaro. Several engineers at Linaro were from Canonical and there was some messy situation related with money.

It ended with retiring of people every quarter. Some moved back to Canonical, some changed job and got hired directly by Linaro. There were also people who moved to Linaro member companies and stayed with their Linaro position. Some people left both companies and went to other jobs.

I was supposed to leave Linaro in 2012 but it was postponed by half a year. So I left after about 37 months.

Second round

Time passed, I was working at Red Hat on getting AArch64 first class citizen in RHEL and Fedora Linux distributions. And one day my manager asked do I want to work at Linaro again.

I did some research, discussed with friends at Linaro and on 8th April 2016 I was back.

My team II

This time I became part of LEG: Linaro Enterprise Group. Servers, data centres etc. There were several teams to choose and I ended in SDI (Software Defined Infrastructure) one. We were behind LDC (Linaro Developer Cloud) project.

At some moment LEG became LDCG (Linaro Datacenter and Cloud Group). Some years later LDCG lost “and Cloud” part as AWS and other cloud providers started offering AArch64 systems so we did not had to deal with it any more.

OpenStack all over

First version of LDC was Debian based with OpenStack ‘liberty’. Then were weird months when we had to reinvent deployment several times. It was mess most of time.

So we abandoned own solutions and went with OpenStack Kolla. I quickly became one of core developers there. LDC moved to be container based.

In 2022 we ended working on OpenStack. LDC is now used only for internal projects.

Building stuff

With my “give me software and I will build it” mantra I ended also as kind of CI jobs developer for LEG teams. Apache Bigtop, Apache Arrow, TensorFlow, EDK2 and several other projects. Some used containers, some were running shell scripts, some Ansible.

SBSA Reference Platform

I was involved in some work around getting QEMU to emulate SBSA Reference Platform (“sbsa-ref” machine). Created some CI jobs to run test suites, build firmware images etc. After running test suites I created a bunch of issues in Jira so we can track how things go.

During recent months I became more involved. I am testing patches, running them through both SBSA and BSA Arm Compliance Suites and reporting results.

I have own set of scripts to handle logs to make it easier to track how things are now.

Summary

At last Linaro Connect events there was always a moment when they announced people who worked at Linaro for 5 (or later) 10 years. I have to admit — I felt envy several times.

And when I was 5 years straight at Linaro we had COVID-19 pandemic so there was no event.

My twenty plus years of IRC

2021-05-20T10:31:00+02:00

In 1996 I started studies at Białystok University of Technology. And one of early days I found that corridor with text terminals. Some time later I joined that crowd and started using HP-2623A term with SunOS account.

And one of first applications crowd shown me was ircII (other were bash, screen, pine and ncftp). So I am IRC user for over 24 years now and this XKCD comics can be about me:

Team chat

Clients

As wrote above I started with ircII. It was painful to use so quickly some scripts landed — Venom, Lice and some others. Tried Epic, Epic2000 and some other clients to finally end with Irssi. I have never been a fan of GUI based ones.

There were moments when people used CTCP VERSION to check what clients other users use. In old Amiga days I usually had it set to similar one as AmIRC one but with version bumped above whatever was released. Simple trolling for all those curious people. Nowadays it simply replies with “telnet” and there was a day when I logged to IRC server and exchanged some messages using just telnet :D

Networks

For years I was user of IRCnet. It was popular in Poland so why bother with checking other networks. But as time passed and I became more involved in FOSS projects there was a need to start using Freenode, then OFTC, Mozilla etc.

Checked how old my accounts are as it nicely show when I started using which network.

IRCnet

IRCnet was my first IRC network. Stopped using it few years ago as all channels I was on went quiet or migrated elsewhere (mostly Freenode).

For years I was visible on Amiga channels #amisia, #amigapl where I met several friends and I am in contact with many of them still.

Freenode

The journey started on 15th May 2004. This was time when I started playing with OpenEmbedded and knew that this is a project where I will spend some of my free time (it became hobby, then job).

It was a place where CentOS, Fedora, Linaro, OpenStack and several other projects were present.

NickServ- Information on Hrw (account hrw):
NickServ- Registered : Mar 15 10:59:47 2004 (17y 9w 6d ago)
NickServ- Last addr  : ~hrw@redhat/hrw
NickServ- vHost      : redhat/hrw
NickServ- Last seen  : now
NickServ- Flags      : HideMail
NickServ- *** End of Info ***

OFTC

For me OFTC means Debian. Later also virtualization stuff as QEMU and libvirt folks sit there.

NickServ- Nickname information for hrw (Marcin Juszkiewicz)
NickServ- hrw is currently online
NickServ-   Time registered: Fri 10 Jun 2011 17:43:55 +0000 (9y 11m 10d 14:47:05 ago)
NickServ- Account last quit: Tue 18 May 2021 09:07:55 +0000 (1d 23:23:05 ago)
NickServ- Last quit message: Remote host closed the connection
NickServ-         Last host: 00019652.user.oftc.net
NickServ-               URL: http://marcin.juszkiewicz.com.pl/
NickServ-      Cloak string: Not set
NickServ-          Language: English (0)
NickServ-           ENFORCE: ON
NickServ-            SECURE: OFF
NickServ-           PRIVATE: ON
NickServ-             CLOAK: ON
NickServ-          VERIFIED: YES

Libera

And since yesterday I am on Libera as well.

NickServ- Information on hrw (account hrw):
NickServ- Registered : May 19 15:26:18 2021 +0000 (17h 32m 35s ago)
NickServ- Last seen  : now
NickServ- Flags      : HideMail, Private
NickServ- *** End of Info ***

What next?

I saw and used several instant messaging platforms. All of them were younger than IRC. Many of them are no longer popular, several are no longer existing. IRC survived so I continue to use it.

U-Boot and generic distro boot

2021-03-14T10:14:00+01:00

Small board computers (SBC) usually come with U-Boot as firmware. There could be some more components like Arm Trusted Firmware, OPTEE etc but what user interact with is the U-Boot itself.

Since 2016 there is the CONFIG_DISTRO_DEFAULTS option in U-Boot configuration. It selects defaults suitable for booting general purpose Linux distributions. Thanks to it board is able to boot most of OS installers out of the box without any user interaction.

How?

How does it know how to do that? There are several scripts and variables involved. Run “printenv” command in U-Boot shell and there you should see some of them named like “boot_*, bootcmd_* scan_dev_for_*”.

In my example I would use environment from RockPro64 running U-Boot 2021.01 version.

I will prettify all scripts for readability. Script contents may be expanded — in such case I will give name as a comment and then it’s content.

Let’s boot

First variable used by U-Boot is “bootcmd”. It reads it to know how to boot operating system on the board.

In out case this variable has “run distro_bootcmd” in it. So what is there on RockPro64 SBC:

setenv nvme_need_init
for target in ${boot_targets}
do 
    run bootcmd_${target}
done

It says that on-board NVME needs some initialization and then goes through set of scripts using order from “boot_targets” variable. On RockPro64 this variable sets “mmc0 mmc1 nvme0 usb0 pxe dhcp sf0” order which means:

eMMC
MicroSD
NVME
USB storage
PXE
DHCP
SPI flash

Both eMMC and MicroSD look similar: ‘devnum=X; run mmc_boot’ — set MMC number and then try to boot by running ‘mmc_boot’ script:

if mmc dev ${devnum}; then 
    devtype=mmc; 
    run scan_dev_for_boot_part; 
fi

NVME one initialize PCIe subsystem (via “boot_pci_enum”), then scans for NVME devices (via “nvme_init”) and do the similar stuff (here with expanded scripts):

# boot_pci_enum
pci enum

# nvme_init
if ${nvme_need_init}; then 
    setenv nvme_need_init false;
    nvme scan;
fi

if nvme dev ${devnum}; then 
    devtype=nvme; 
    run scan_dev_for_boot_part; 
fi

USB booting goes with “usb_boot”:

usb start;
if usb dev ${devnum}; then 
    devtype=usb; 
    run scan_dev_for_boot_part;
fi

PXE network boot? Initialize USB, scan PCI, get network configuration, do PXE boot:

# boot_net_usb_start
usb start

# boot_pci_enum
pci enum

dhcp; 
if pxe get; then 
    pxe boot; 
fi

DHCP method feels like last resort one (do not ask me for meaning of all those variables):

# boot_net_usb_start
usb start

# boot_pci_enum
pci enum

if dhcp ${scriptaddr} ${boot_script_dhcp}; then 
    source ${scriptaddr}; 
fi;

setenv efi_fdtfile ${fdtfile}; 
setenv efi_old_vci ${bootp_vci};
setenv efi_old_arch ${bootp_arch};
setenv bootp_vci PXEClient:Arch:00011:UNDI:003000;
setenv bootp_arch 0xb;

if dhcp ${kernel_addr_r}; then 
    tftpboot ${fdt_addr_r} dtb/${efi_fdtfile};

    if fdt addr ${fdt_addr_r}; then 
        bootefi ${kernel_addr_r} ${fdt_addr_r}; 
    else 
        bootefi ${kernel_addr_r} ${fdtcontroladdr};
    fi;
fi;

setenv bootp_vci ${efi_old_vci};
setenv bootp_arch ${efi_old_arch};
setenv efi_fdtfile;
setenv efi_old_arch;
setenv efi_old_vci;

And last method is SPI flash:

busnum=0

if sf probe ${busnum}; then
    devtype=sf;

    # run scan_sf_for_scripts; 
    ${devtype} read ${scriptaddr} ${script_offset_f} ${script_size_f}; 
    source ${scriptaddr}; 
    echo SCRIPT FAILED: continuing...
fi

Search for boot partition

Note how block devices end with one script: “scan_dev_for_boot_part”. What it does is quite simple:

part list ${devtype} ${devnum} -bootable devplist; 
env exists devplist || setenv devplist 1; 

for distro_bootpart in ${devplist}; do 
    if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then 
        run scan_dev_for_boot; 
    fi; 
done; 
setenv devplist

We know type and number of boot device from previous step so now we check for bootable partitions. Which means EFI System Partition for GPT disks and partitions marked as bootable in case of MBR. If none are present then first one is assumed to be bootable one.

Search for distribution boot information

Once we found boot partitions it is time to search for boot stuff with “scan_dev_for_boot” script:

echo Scanning ${devtype} ${devnum}:${distro_bootpart}...;
for prefix in ${boot_prefixes}; do 
    run scan_dev_for_extlinux; 
    run scan_dev_for_scripts; 
done;

run scan_dev_for_efi;

Old style OS configuration

First U-Boot checks for “extlinux/extlinux.conf” file, then go for old style “boot.scr” (in uimg and clear text formats). Both of them are checked in / and /boot/ directories of checked partition (those names are in “boot_prefixes” variable).

Let us look at it:

# scan_dev_for_extlinux
if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf};then 
    echo Found ${prefix}${boot_syslinux_conf}; 

    # run boot_extlinux; 
    sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}

    echo SCRIPT FAILED: continuing...; 
fi

# scan_dev_for_scripts
for script in ${boot_scripts}; do 
    if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then 
        echo Found U-Boot script ${prefix}${script}; 

        # run boot_a_script; 
        load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; 
        source ${scriptaddr}

        echo SCRIPT FAILED: continuing...; 
    fi; 
done

EFI compliant OS

And finally U-Boot checks for EFI style BootOrder variables and generic OS loader path:

# scan_dev_for_efi
setenv efi_fdtfile ${fdtfile};
for prefix in ${efi_dtb_prefixes}; do
    if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then 
        # run load_efi_dtb; 
        load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
    fi;
done;

# run boot_efi_bootmgr;
if fdt addr ${fdt_addr_r}; then 
    bootefi bootmgr ${fdt_addr_r};
else 
    bootefi bootmgr;
fi

if test -e ${devtype} ${devnum}:${distro_bootpart} efi/boot/bootaa64.efi; then
    echo Found EFI removable media binary efi/boot/bootaa64.efi; 

    # run boot_efi_binary; 
    load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi; 
    if fdt addr ${fdt_addr_r}; then 
        bootefi ${kernel_addr_r} ${fdt_addr_r};
    else 
        bootefi ${kernel_addr_r} ${fdtcontroladdr};
    fi

    echo EFI LOAD FAILED: continuing...;
fi; 
setenv efi_fdtfile

Booted

At this moment board should be in either OS or in OS loader (being EFI binary).

Final words

All that work on searching for boot media, boot scripts, boot configuration files, OS loaders, EFI BootOrder entries etc is done without any user interaction. Every bootable media is checked and tried.

If I would add SATA controller support into U-Boot binary then all disks connected to such would also be checked. Without any code/environment changes from my side.

So if your SBC has some weird setup then consider moving to distro generic one. Boot fresh mainline U-Boot, store copy of your existing environment (“printenv” shows it) and then reset to generic one with “env default -a” command. Probably would need to set MAC adresses for network interfaces.

I got HoneyComb

2021-02-18T19:09:00+01:00

Few years ago SolidRun released MACCHIATObin board. Nice fast cpu, PCI Express slot, several network ports. I did not buy it because it supported only 16 GB of memory and I wanted to be able to run OpenStack.

Time has passed, HoneyComb LX2 system appeared on AArch64 market. More cores, more memory. Again I haven’t bought it — my Ryzen 5 upgrade costed less than HoneyComb price is.

And when someone asked me for some serious AArch64 system to buy I was suggesting HoneyComb.

Let us look at hardware

So what do we have here?

16 Cortex-A72 cores
2 SO-DIMM slots (up to 64GB ram in total)
USB 2.0 and 3.0 ports (as ports and/or headers)
standard ATX power socket (no 12V AUX needed)
3 fan connectors (one with PWM, two with 12V)
front panel connectors like on x86-64 motherboards
M.2 slot for NVME (pcie x4)
PCI Express slot (open x8 one so x16 card fits)
MicroSD slot (for firmware)
4 SFP+ ports for 10GbE networking
1 GbE port
4 SATA ports
serial console via microUSB port
power/reset buttons

HoneyComb board layout

Lot of networking and there is even version with 100GbE port added: ClearFog CX LX2.

So how I got it?

I wrote that I did not bought it, right? Jon Nettleton (from SolidRun) contacted me recently and asked:

Morning. do you have any interest in a HoneyComb? I have some old stock boards available to the community. I figured it may help you out with your UEFI Qemu work.

We discussed about SBSA/SBBR stuff and I sent him an email with address information and shipping notes.

Some days passed and board arrived. I added spare NVME and two sticks of Kingston HyperX 2933 CL17 memory and it was ready to go (microsd card keeps firmware):

HoneyComb board ready to go

Let’s run something

Debian ‘bullseye’ booted right away. Again I used pendrive from my EBBR compliant RockPro64. Started without problems.

Network ports issue

Ok, there was one problem — on-board ethernet ports do not work yet with mainline nor distribution kernels so I had to dig out my old USB based network card.

There are patches for Linux kernel to get all ports running. May get merged into 5.13 kernel if things go nicely.

Plans?

I plan few things for HoneyComb:

check several distributions how they handle AArch64 systems
improve SBSA ACS code as HoneyComb is almost SBSA level 3 compliant (there are some places where error/warning messages break output)
build, deploy and test OpenStack
test software
check how it works as AArch64 desktop (like I did with APM Mustang 6 years ago)

EBBR on EspressoBin

2021-02-15T21:53:00+01:00

SBBR or GTFO

Me.

Yeah, right. But world is not so nice and there are many cheap SBC on market which are not SBBR compliant and probably never will. And with small amount of work they can do EBBR (Embedded Base Boot Requirements).

NOTE: I have similar post about EBBR on RockPro64 board.

WTH is EBBR?

It is specification for devices which are not servers and do not pretend to be such. U-Boot is all they have and with properly configured one they have some subset of EFI Boot/Runtime Services to load distribution bootloader (grub-efi usually) like it is done on servers.

ACPI is not required but may be present. DeviceTree is perfectly fine. You may provide both or one of them.

Firmware can be stored wherever you wish. Even MBR partitioning is available if really needed.

Few words about board itself

EspressoBin has 4MB of SPI flash on board. Less than on RockPro64 but still enough for storing firmware (U-Boot takes less than 1MB).

This SBC is nothing new — first version was released in 2016. There were several revisions with different memory type, amount of ram chips, type of them (ddr3 or ddr4), CPU speed and some more changes.

I got EspressoBin revision 5 with 1GB ram of ddr3 in 2 chips. And 1GHz processor.

It may sound silly that I repeat that information but it matters when you start building firmware for that board.

So let us build fresh firmware

This is Marvell so abandon all hope for sanity.

Thanks to Arm Trusted Firmware authors there is good documentation on how to build firmware for EspressoBin which guides step by step and explains all arguments you need. For me it was several git clone calls and then two make calls:

make -C u-boot CROSS_COMPILE=aarch64-linux-gnu- \
mvebu_espressobin-88f3720_defconfig u-boot.bin -j12

make -C trusted-firmware-a CROSS_COMPILE=aarch64-linux-gnu- \
CROSS_CM3=arm-none-linux-gnueabihf- PLAT=a3700 \
CLOCKSPRESET=CPU_1000_DDR_800 DDR_TOPOLOGY=2 \
MV_DDR_PATH=$PWD/marvell/mv-ddr-marvell/ \
WTP=$PWD/marvell/A3700-utils-marvell/ \
CRYPTOPP_PATH=$PWD/marvell/cryptopp/ \
BL33=$PWD/u-boot/u-boot.bin \
mrvl_flash -j12

And I had to install cross toolchain for 32-bit arm because the one I had was for building kernels/bootloaders only.

Is your U-Boot friendly or not?

First you need to check which version of U-Boot and hardware you have. Then check does it recognize SPI flash or not:

Marvell>> sf probe
SF: unrecognized JEDEC id bytes: 9d, 70, 16
Failed to initialize SPI flash at 0:0 (error -2)

I had bad luck as my board used SPI chip not recognized by any official U-Boot build…

Armbian to the rescue

I asked in few places did anyone had some experiences with this board. One of them was #debian-arm IRC channel where I got hint from Xogium that Armbian may have U-Boot builds.

And they have whole page about EspressoBin. With information how to choose firmware files etc.

So I downloaded archive with proper files for UART recovery. The important thing to remember is that once you move jumpers and load all firmware files over serial they are not written in SPI flash so reset of board means you start over.

Quick check is SPI flash detected:

Marvell>> sf probe
SF: Detected is25wp032 with page size 256 Bytes, erase size 4 KiB, total 4 MiB

Yeah! Now can start USB and flash own firmware build:

Marvell>> bubt flash-image.bin spi usb
Burning U-Boot image "flash-image.bin" from "usb" to "spi"
Bus usb@58000: Register 2000104 NbrPorts 2
Starting the controller
USB XHCI 1.00
Bus usb@5e000: USB EHCI 1.00
scanning bus usb@58000 for devices... 1 USB Device(s) found
scanning bus usb@5e000 for devices... 2 USB Device(s) found
Image checksum...OK!
SF: Detected is25wp032 with page size 256 Bytes, erase size 4 KiB, total 4 MiB
Erasing 991232 bytes (242 blocks) at offset 0 ...Done!
Writing 990944 bytes from 0x6000000 to offset 0 ...Done!

Quick reset and board boots to fresh, mainline U-Boot:

TIM-1.0
WTMI-devel-18.12.1-1a13f2f
WTMI: system early-init
SVC REV: 5, CPU VDD voltage: 1.108V
NOTICE:  Booting Trusted Firmware
NOTICE:  BL1: v2.4(release):v2.4-345-g04c122310 (Marvell-devel-18.12.2)
NOTICE:  BL1: Built : 17:11:19, Feb 15 2021
NOTICE:  BL1: Booting BL2
NOTICE:  BL2: v2.4(release):v2.4-345-g04c122310 (Marvell-devel-18.12.2)
NOTICE:  BL2: Built : 17:11:20, Feb 15 2021
NOTICE:  BL1: Booting BL31
NOTICE:  BL31: v2.4(release):v2.4-345-g04c122310 (Marvell-devel-18.12.2)
NOTICE:  BL31: Built : 18:07:02, Feb 15 2021


U-Boot 2021.01 (Feb 15 2021 - 19:25:41 +0100)

DRAM:  1 GiB
Comphy-0: USB3_HOST0    5 Gbps    
Comphy-1: PEX0          2.5 Gbps  
Comphy-2: SATA0         5 Gbps    
SATA link 0 timeout.
AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
flags: ncq led only pmp fbss pio slum part sxs 
PCIE-0: Link down
MMC:   sdhci@d0000: 0, sdhci@d8000: 1
Loading Environment from SPIFlash... SF: Detected is25wp032 with page size 256 Bytes, erase size 4 KiB, total 4 MiB
OK
Model: Globalscale Marvell ESPRESSOBin Board
Card did not respond to voltage select! : -110
Net:   eth0: neta@30000
Hit any key to stop autoboot:  0

Final steps

OK, so SBC has fresh, mainline firmware. Nice. But still some stuff needs to be done.

First note MAC addresses of Ethernet ports. Use printenv command to check stored environment and note few variables:

eth1addr=f0:ad:4b:aa:97:01
eth2addr=f0:ad:4b:aa:97:02
eth3addr=f0:ad:4b:aa:97:03
ethaddr=f0:ad:4e:72:10:ef

Of course you may also skip that step and rely on random ones or choose own ones (I had router with C0:FF:EE:C0:FF:EE in past).

Then reset environment to default values stored in U-Boot binary and set those MAC addresses by hand:

=> env default -a -f
=> setenv eth1addr f0:ad:4b:aa:97:01
=> setenv eth2addr f0:ad:4b:aa:97:02
=> setenv eth3addr f0:ad:4b:aa:97:03
=> setenv ethaddr f0:ad:4e:72:10:ef
=> saveenv
Saving Environment to SPIFlash... Erasing SPI flash...Writing to SPI flash...done
OK
=>

What EBBR brings?

Now your board is ready to boot Debian, Fedora and several other distribution install media with two commands:

=> set boot_targets usb0
=> boot

It will find EFI bootloader and start it. Just like on any other boring SBBR/EBBR system.

Distributions with old style ‘boot.scr’ script (like OpenWRT for example) will also work so no functionality loss.

Standards are boring

2021-01-20T17:33:00+01:00

We have made Arm servers boring.

Jon Masters

Standards are boring. Satisfied users may not want to migrate to other boards the market tries to sell them.

So Arm market is flooded with piles of small board computers (SBC). Often they are compliant to standards only when it comes to connectors.

But our hardware is not standard

It is not a matter of ‘let produce UEFI ready hardware’ but rather ‘let write EDK2 firmware for boards we already have’.

Look at Raspberry/Pi then. It is shitty hardware but got popular. And group of people wrote UEFI firmware for it. Probably without vendor support even.

Start with EBBR

Each new board should be EBBR compliant at start. Which is easy — do ‘whatever hardware’ and put properly configured U-Boot on it. Upstreaming support for your small device should not be hard as you often base on some already existing hardware.

Add 16MB of SPI flash to store firmware. Your users will be able to boot ISO without wondering where on boot media they need to write bootloaders.

Then work on EDK2 for board. Do SMBIOS (easy) and keep your existing Device Tree. You are still EBBR. Remember about upstreaming your work — some people will complain, some will improve your code.

Add ACPI, go SBBR

Next step is moving from Device Tree to ACPI. May take some time to understand why there are so many tables and what ASL is. But as several other systems show it can be done.

And this brings you to SBBR compliance. Or SystemReady ES if you like marketing.

SBSA for future design

Doing new SoC tends to be “let us take previous one and improve a bit”. So this time change it a bit and make your next SoC compliant with SBSA level 3. All needed components are probably already included in your Arm license.

Grab EDK2 support you did for previous board. Look at QEMU SBSA Reference Platform support, look at other SBSA compliant hardware. Copy, reuse their drivers, their code.

Was it worth?

At the end you will have SBSA compliant hardware running SBBR compliant firmware.

Congratulations, your board is SystemReady SR compliant. Your marketing team may write that you are on same list as Ampere with their Altra server.

Users buy your hardware and can install whatever BSD, Linux distribution they want. Some will experiment with Microsoft Windows. Others may work on porting Haiku or other exotic operating system.

But none of them will have to think “how to get this shit running”. And they will tell friends that your device is as boring as it should be when it comes to running OS on it == more sales.