Marcin Juszkiewicz

When I started working for Red Hat I got a list of packages in RHEL 7.0 which did not built for AArch64. Some time later I worked on merging those fixes in Fedora and upstream. Red Hat Enterprise Linux 7.0 got released. Then 7.1 followed. Then CentOS developers added AArch64 target based on work we did in RHEL.

Yesterday Red Hat Enterprise Linux 7.2 got released. What makes this version special is one paragraph:

Red Hat is also making available Red Hat Enterprise Linux Server for ARM 7.2 Development Preview, which was first made available to partners and their customers in June 2015. This Development Preview enables new partner hardware and additional features for the ARM architecture.

Which means AArch64 port. Working out of box on SBSA/SBBR compliant hardware.

Firmware update usually ends well. Previous (1.15.19) firmware failed to boot on some of Mustangs at Red Hat but worked fine on one under my desk. Yesterday I got 1.15.22 plus slimpro update and managed to get machine into non-bootable state (firmware works fine on other machines).

So how to get APM Mustang back into working state?

• Get a SD card and connect it to an PC Linux machine with reader support.
• Download Mustang software from MyAPM (1.5.19 was latest available there).
• Unpack “mustang_sq_1.15.19.tar.xz” and then “mustang_binaries_1.15.19.tar.xz” tarballs.
• Write the boot loader firmware to the SD card: “dd if=tianocore_media.img of=/dev/SDCARD“.
• Take FAT formatted USB drive and put there some files from “mustang_binaries_1.15.19.tar.xz” archive (all into root directory):
  • potenza/apm_upgrade_tianocore.cmd
  • potenza/tianocore_media.img
  • potenza/UpgradeFirmware.efi
• Power off your Mustang
• Configure the Mustang to boot from SD card via these jumpers change:
  • Find HDR9 (close to HDR8, which is next to PCIe port)
  • Locate pin 11-12 and 17-18.
  • Connect 11-12 and 17-18 with jumpers
• Insert SD card to Mustang SD port
• Connect serial cable to Mustang and your PC.
• Run minicom/picocom/screen/other-preferred-serial-terminal and connect to Mustang serial port
• Power up Mustang and you should boot with SD UEFI firmware:

X-Gene Mustang Board
Boot firmware (version 1.1.0 built at 12:25:21 on Jun 22 2015)
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3021001 I0
TianoCore 1.1.0 UEFI 2.4.0 Jun 22 2015 12:24:25
CPU: APM ARM 64-bit Potenza Rev A3 1400MHz PCP 1400MHz
     32 KB ICACHE, 32 KB DCACHE
     SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Board: X-Gene Mustang Board
The default boot selection will start in   2 second

• Press any key to get into UEFI menu.
• Select “Shell” option and you will be greeted with a list of recognized block devices and filesystems. Check which is USB (“FS6” in my case).

Shell> fs6:
FS6:> ls
Directory of: FS6:
08/04/2015  00:28              39,328  UpgradeFirmware.efi
08/27/2015  19:20                  56  apm_upgrade_tianocore.cmd
08/27/2015  19:20           2,098,176  tianocore_media.img

• Flash firmware using “UpgradeFirmware.efi apm_upgrade_tianocore.cmd” command.
• Power off
• Change jumpers back to normal (11-12 and 17-18 to be open).
• Eject SD card from Mustang
• Power on

And your Mustang should be working again. You can also try to write other versions of firmware of course or grab files from internal hdd.

Week ago Fedora 23 got released. Also for ARM and AArch64 architectures. But it does not mean that it supports all possible hardware.

AArch64

There is the installation guide which lists two supported hardware platforms (besides QEMU):

• Applied Micro Mustang
• AMD Seattle

And then we got email from Clive Messer with question why we do not support 96Boards, ie. HiKey and Dragonboard as they are cheap and available.

I am not surprised with such question. It would be great to have support for both boards but their current state makes it quite hard. There is no support for them in mainline kernel, Dragonboard needs some firmware files which license forbids packaging it (note bolded part):

Distribution of the Redistributable Binary Code is subject to the following restrictions: (i) Redistributable Binary Code may only be distributed in binary format and may not be distributed in source code format: (ii) Redistributable Binary Code may not be distributed on a stand alone basis but may be redistributed in conjunction with and as a part of a software application created by you; (iii) the Redistributable Binary Code may only operate in conjunction with platforms incorporating Qualcomm Technologies, Inc. chipsets; (iv) redistribution of the Redistributable Binary Code must include the .txt file setting forth the terms and condition of this Agreement; (v) you may not use Qualcomm Technologies’ or its affiliates or subsidiaries name, logo or trademarks; and (vi) copyright, trademark, patent and any other notices that appear on the Materials may not be removed or obscured.

So even if we get mainline kernel working on it some things will not work without non-free files.

Then we have bootloaders. Hikey can be flashed with UEFI but (according to bootloader install documentation) you need to keep partitions in some magic way. Where is “there has to be one ef00 type partition formatted with FAT” as it is with other UEFI powered machines? Dragonboard 410c uses fastboot ;(

ARM

On ARM side there is common question about “most readily available and used board, with the most units sold and the biggest community” one. I think that developers from that community do not want their board supported in main distributions like Debian or Fedora.

Heresy? Do not think so. What needs to get board supported was told many times. Mainline kernel support, firmware blobs with redistribution license, drivers for graphics and sane bootloader (UEFI, U-Boot, maybe some other too).

Summary

So if you want to have your board supported then spend some time on mainlining your changes/drivers. And then come to us.

Each year you can hear “this is a year of Linux desktop” phrase. After few days with AArch64 desktop I know one thing: it is not a year of ARMv8 Linux desktop.

Web browsing

OK, I can be spoiled by speed of my i7-2600k desktop but situation when Firefox with less than 20 tabs open is unable to display characters I type into textarea fast enough shows that something is wrong (16GB ram machine). And tell me that this is not typical desktop use of web browser…

YouTube. Main source of any kind of videos. Sometimes it works, but most of time I lack patience to wait until it will start (VP9 and h264 codecs support present). And no way to watch “live hangouts”.

And say bye to music streaming services like Deezer or Spotify.

Gaming

I am not a game player. Installed Quake3 (which I never played before) and it worked, SuperTuxKart worked as well. But that does not prove anything as both those games have low requirements.

It probably never will be gaming platform on Linux desktop.

Development

For my style of development it is fine. But all I need is terminal and gVim ;D

Other hardware?

I think that results may be affected by a fact that all I have here is Applied Micro Mustang based on X-Gene 1 cpu. It is one of first ARMv8 processors in Linux world and it is optimized for server use rather than desktop.

One thing is sure: in next year I will try this experiment with other AArch64 hardware. Just hope it will be sooner than in a year from now (which is my feeling after lack of new aarch64 hardware announcements from Linaro members during this week Linaro Connect).

First day behind me. Had to disconnect DVD drive as X-Gene SATA controller still does not like it. And to resolve some issues with multimedia support. Got some questions too.

What is desktop class?

There were questions about desktop class hardware, links to Gigabyte mainboard etc. So why Mustang is desktopstein and not desktop? Or rather: what desktop should have?

For me to consider mainboard as desktop class it needs to have few things:

• microATX or miniITX format
• USB ports (more than two, USB 3.0 speed should be available)
• on-board sound with 3.5mm output (do not care USB/PCIe/SPI - has to work)
• x16 PCI Express slot for graphics (can have 8 lines)
• SATA controller working with optical drives (good-bye X-Gene 1)
• at least four SATA ports
• standard DIMM slots (without ECC requirement)

When it comes to AArch64 there is no such ones so far. Something can be arranged from few mainboards by adding extra cards or usb devices. And so far no new rumours about new hardware ;(

UPDATE: why 4 drives? I have system hdd, scratch hdd (to test installers) and want to have optical drive (also for installers).

Multimedia support

I use Fedora on my machines. And as lot of people know this distribution has strange rules when it comes to multimedia support. Forget about MP3, H264 and few other things.

Normally (on x86(-64) architecture) people go for RPMFusion repositories. But they do not support aarch64… So I did local rebuilds of everything needed to get MPlayer, VLC and H264 support for GStreamer (so Firefox is able to play YouTube videos).

There is no Adobe Flash support so some of YT videos does not work. Too bad that it includes live streaming as I am unable to watch Linaro Connect keynotes on desktop (good that it works on Android tablet).

Three years ago I was working on getting AArch64 toolchain built. Then lot of things happened.

Nowadays if you are lucky you can even have AArch64 hardware. The problem is that there is no desktop class one still. Mustang and Seattle are server boards, Juno is development platform, Hikey is out of stock, Dragonboard 410c has 1GB of memory (same as Hikey) and rest of “publicly available” AArch64 hardware is in Android or iOS devices.

I played with getting X11 running for quite long time so decided that it is time to make use of it. Moved Mustang mainboard into microatx tower case, added PCI Express riser with Radeon HD5450 graphics card in it. Then two hard drives (one for systems, second for testing installers) and fun started. Two monitors connected, speakers got some waves from cheap (1.5€) USB sound card, mouse and keyboard also migrated from my x86-64 desktop. I called resulting machine “desktopstein”.

Pressed power and machine booted. Nothing on screen for about 1.5-2 minutes because UEFI firmware does not initialize graphics and then some messages appeared and I was able to login to text console. Enabled lightdm and logged into KDE session.

Konsole works, Firefox works (copied whole profile from x86-64), Thunderbird fetched mails (also copied profile). No Chrome nor Chromium is fine. Had to switch to other music provider than Deezer as their web player requires Adobe Flash. But there are still lot of MP3 streaming services so VLC (from own rpmfusion rebuild) got something to play.

So far works good. During week I plan to do my normal work and try some random things.

During last years lot of work went into design of SIMD instructions for different cpus. X86-64 has some, Power has and so does AArch64.

But why I am writing about it? Simple — build failures. Or rather test failures. Especially in scientific software like HepMC or alglib. They build fine but that’s all.

The difference was small — something like e-15 or smaller but it was enough to make tests fail. And what to blame? SIMD of course.

AArch64 has FMA (fused multiply-add) instructions which speed up calculations but result is more precise than traditional way when all operations are done one by one. This is enough for tests to fail :-(

But there is solution for it. To degrade you need to add “-ffp-context=off” to compiler flags. This disables use of FMA so results of tests are same as on x86-64 (on pre-Haswell/Bulldozer cores). As a bonus it works on powerpc64(le) and s390(x) too.

Thanks goes to David Abdurachmanov and Andrew Pinski for finding out which exactly flag needs to be used (instead of -fno-expensive-optimisations I used before).

During fixing software to make it build and run on AArch64 sooner or later you can meet magical constant named PAGE_SIZE. And in most situations it will be used in a wrong way.

What it does is simple — tell how big page size is. But it does not work that way on AArch64 architecture as we have different values possible: 4K, 16K (may not be supported in all cpus) and 64K with latter being used in Fedora and other distributions. There were some packages which we built at time of 4K kernel being used and then wondered why things fail under 64K kernel…

But how to handle it as a userspace software developer? Simplest solution is to use sysconf(_SC_PAGESIZE) function call (same as getpagesize()). But remember to not hardcode anything based on what you get as a result. Otherwise your application can misbehave when run on kernel with other PAGE_SIZE size.

The good part is that if someone uses PAGE_SIZE constant in code then it will just do not compile on AArch64 as it is not present in system headers. From what I checked sys/user.h header has it defined on some platforms and does not on other so it can not be assumed as available.

UPDATE: added 16K page size which may not be supported in some cpus.