I may go back to Linaro

Few days ago my manager asked me if I would like to go back to Linaro. This time not as ‘Linaro-but-Canonical engineer’ but as ‘Red Hat assigned engineer’. That made me thinking…

Those three years at Linaro were good time. Learnt a lot about cross toolchains, got possibility to work on bootstrapping AArch64 support in OpenEmbedded, Debian/Ubuntu and then Fedora/RHEL. Met many skilled people from around the world, travelled into places which I would probably not visit on my own.

Going back sounds good. From my discussions with few people from Linaro there is more and more AArch64 related work there (and I have some hardware at home) so my rusty arm32 skills can rust in peace. Have to take a closer look at what exactly is on a plate there to take and find some place.

So if you work for Linaro and will be at FOSDEM (or devconf.cz) then I would love to talk.

Running 32-bit ARM virtual machine on AArch64 hardware

It was a matter of days and finally all pieces are done. Running 32-bit ARM virtual machines on 64-bit AArch64 hardware is possible and quite easy.


  • AArch64 hardware (I used APM Mustang as usual)
  • ARM rootfs (fetched Fedora 22 image with “virt-builder” tool)
  • ARM kernel and initramfs (I used Fedora 24 one)
  • Virt Manager (can be done from shell too)


Start “virt-manager” and add new machine:

Add new machine

Select rootfs, kernel, initramfs (dtb will be provided internally by qemu) and tell kernel where rootfs is:

Storage/boot options

Then set amount of memory and cores. I did 10GB of RAM and 8 cores. Save machine.

Let’s run

Open created machine and press Play button. It should boot:

Booted system

I upgraded F22 to F24 to have latest development system.

Is it fast?

If I would just boot and write about it then there will be questions about performance. I did build of gcc 5.3.1-3 using mock (standard Fedora way). On arm32 Fedora builder it took 19 hours, on AArch64 builder 4.5h only. On my machine AArch64 build took 9.5 hour and in this vm it took 12.5h (slow hdd used). So builder with memory and some fast storage will boost arm32 builds a lot.

Numbers from “openssl speed” shows performance similar to host cpu:

The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2               1787.41k     3677.19k     5039.02k     5555.88k     5728.94k
mdc2                 0.00         0.00         0.00         0.00         0.00 
md4              24846.05k    81594.07k   226791.59k   418185.22k   554344.45k
md5              18881.79k    60907.46k   163927.55k   281694.58k   357168.47k
hmac(md5)        21345.25k    69033.83k   177675.52k   291996.33k   357250.39k
sha1             20776.17k    65099.46k   167091.03k   275240.62k   338582.71k
rmd160           15867.02k    42659.95k    88652.54k   123879.77k   140571.99k
rc4             167878.11k   186243.61k   191468.46k   192576.51k   193112.75k
des cbc          35418.48k    37327.19k    37803.69k    37954.56k    37991.77k
des ede3         13415.40k    13605.87k    13641.90k    13654.36k    13628.76k
idea cbc         36377.06k    38284.93k    38665.05k    38864.71k    39032.15k
seed cbc         42533.48k    43863.15k    44276.22k    44376.75k    44397.91k
rc2 cbc          29523.86k    30563.20k    30763.09k    30940.50k    30857.44k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00 
blowfish cbc     60512.96k    66274.07k    67889.66k    68273.15k    68302.17k
cast cbc         56795.77k    61845.42k    63236.86k    63251.11k    63445.82k
aes-128 cbc      61479.48k    65319.32k    67327.49k    67773.78k    66590.04k
aes-192 cbc      53337.95k    55916.74k    56583.34k    56957.61k    57024.51k
aes-256 cbc      46888.06k    48538.97k    49300.82k    49725.44k    50402.65k
camellia-128 cbc    59413.00k    62610.45k    63400.53k    63593.13k    63660.03k
camellia-192 cbc    47212.40k    49549.89k    50590.21k    50843.99k    50012.16k
camellia-256 cbc    47581.19k    49388.89k    50519.13k    49991.68k    50978.82k
sha256           27232.09k    64660.84k   119572.57k   151862.27k   164874.92k
sha512            9376.71k    37571.93k    54401.88k    74966.36k    84322.99k
whirlpool         3358.92k     6907.67k    11214.42k    13301.08k    14065.66k
aes-128 ige      60127.48k    65397.14k    67277.65k    67428.35k    67584.00k
aes-192 ige      52340.73k    56249.81k    57313.54k    57559.38k    57191.08k
aes-256 ige      46090.63k    48848.96k    49684.82k    49861.32k    49892.01k
ghash           150893.11k   171448.55k   177457.92k   179003.39k   179595.95k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.000322s 0.000026s   3101.3  39214.9
rsa 1024 bits 0.001446s 0.000073s    691.7  13714.6
rsa 2048 bits 0.008511s 0.000251s    117.5   3987.5
rsa 4096 bits 0.058092s 0.000945s     17.2   1058.4
                  sign    verify    sign/s verify/s
dsa  512 bits 0.000272s 0.000297s   3680.6   3363.6
dsa 1024 bits 0.000739s 0.000897s   1353.1   1115.2
dsa 2048 bits 0.002762s 0.002903s    362.1    344.5
                              sign    verify    sign/s verify/s
 256 bit ecdsa (nistp256)   0.0005s   0.0019s   1977.8    538.3
 384 bit ecdsa (nistp384)   0.0015s   0.0057s    663.0    174.6
 521 bit ecdsa (nistp521)   0.0035s   0.0136s    286.8     73.4
                              op      op/s
 256 bit ecdh (nistp256)   0.0016s    616.0
 384 bit ecdh (nistp384)   0.0049s    204.8
 521 bit ecdh (nistp521)   0.0115s     87.2

Red Hat Enterprise Linux Server for ARM 7.2 Development Preview released!

When I started working for Red Hat I got a list of packages in RHEL 7.0 which did not built for AArch64. Some time later I worked on merging those fixes in Fedora and upstream. Red Hat Enterprise Linux 7.0 got released. Then 7.1 followed. Then CentOS developers added AArch64 target based on work we did in RHEL.

Yesterday Red Hat Enterprise Linux 7.2 got released. What makes this version special is one paragraph:

Red Hat is also making available Red Hat Enterprise Linux Server for ARM 7.2 Development Preview, which was first made available to partners and their customers in June 2015. This Development Preview enables new partner hardware and additional features for the ARM architecture.

Which means AArch64 port. Working out of box on SBSA/SBBR compliant hardware.

Unbricking APM Mustang

Firmware update usually ends well. Previous (1.15.19) firmware failed to boot on some of Mustangs at Red Hat but worked fine on one under my desk. Yesterday I got 1.15.22 plus slimpro update and managed to get machine into non-bootable state (firmware works fine on other machines).

So how to get APM Mustang back into working state?

  • Get a SD card and connect it to an PC Linux machine with reader support.
  • Download Mustang software from MyAPM (1.5.19 was latest available there).
  • Unpack “mustang_sq_1.15.19.tar.xz” and then “mustang_binaries_1.15.19.tar.xz” tarballs.
  • Write the boot loader firmware to the SD card: “dd if=tianocore_media.img of=/dev/SDCARD“.
  • Take FAT formatted USB drive and put there some files from “mustang_binaries_1.15.19.tar.xz” archive (all into root directory):
    • potenza/apm_upgrade_tianocore.cmd
    • potenza/tianocore_media.img
    • potenza/UpgradeFirmware.efi
  • Power off your Mustang
  • Configure the Mustang to boot from SD card via these jumpers change:
    • Find HDR9 (close to HDR8, which is next to PCIe port)
    • Locate pin 11-12 and 17-18.
    • Connect 11-12 and 17-18 with jumpers
  • Insert SD card to Mustang SD port
  • Connect serial cable to Mustang and your PC.
  • Run minicom/picocom/screen/other-preferred-serial-terminal and connect to Mustang serial port
  • Power up Mustang and you should boot with SD UEFI firmware:
X-Gene Mustang Board
Boot firmware (version 1.1.0 built at 12:25:21 on Jun 22 2015)
TianoCore 1.1.0 UEFI 2.4.0 Jun 22 2015 12:24:25
CPU: APM ARM 64-bit Potenza Rev A3 1400MHz PCP 1400MHz
     SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Board: X-Gene Mustang Board
The default boot selection will start in   2 second 
  • Press any key to get into UEFI menu.
  • Select “Shell” option and you will be greeted with a list of recognized block devices and filesystems. Check which is USB (“FS6” in my case).
Shell> fs6:
FS6:\> ls  
Directory of: FS6:\
08/04/2015  00:28              39,328  UpgradeFirmware.efi
08/27/2015  19:20                  56  apm_upgrade_tianocore.cmd
08/27/2015  19:20           2,098,176  tianocore_media.img
  • Flash firmware using “UpgradeFirmware.efi apm_upgrade_tianocore.cmd” command.
  • Power off
  • Change jumpers back to normal (11-12 and 17-18 to be open).
  • Eject SD card from Mustang
  • Power on

And your Mustang should be working again. You can also try to write other versions of firmware of course or grab files from internal hdd.

Fedora 23 and unsupported ARM/AArch64 devices

Week ago Fedora 23 got released. Also for ARM and AArch64 architectures. But it does not mean that it supports all possible hardware.


There is the installation guide which lists two supported hardware platforms (besides QEMU):

  • Applied Micro Mustang
  • AMD Seattle

And then we got email from Clive Messer with question why we do not support 96Boards, ie. HiKey and Dragonboard as they are cheap and available.

I am not surprised with such question. It would be great to have support for both boards but their current state makes it quite hard. There is no support for them in mainline kernel, Dragonboard needs some firmware files which license forbids packaging it (note bolded part):

Distribution of the Redistributable Binary Code is subject to the following restrictions: (i) Redistributable Binary Code may only be distributed in binary format and may not be distributed in source code format: (ii) Redistributable Binary Code may not be distributed on a stand alone basis but may be redistributed in conjunction with and as a part of a software application created by you; (iii) the Redistributable Binary Code may only operate in conjunction with platforms incorporating Qualcomm Technologies, Inc. chipsets; (iv) redistribution of the Redistributable Binary Code must include the .txt file setting forth the terms and condition of this Agreement; (v) you may not use Qualcomm Technologies’ or its affiliates or subsidiaries name, logo or trademarks; and (vi) copyright, trademark, patent and any other notices that appear on the Materials may not be removed or obscured.

So even if we get mainline kernel working on it some things will not work without non-free files.

Then we have bootloaders. Hikey can be flashed with UEFI but (according to bootloader install documentation) you need to keep partitions in some magic way. Where is “there has to be one ef00 type partition formatted with FAT” as it is with other UEFI powered machines? Dragonboard 410c uses fastboot ;(


On ARM side there is common question about “most readily available and used board, with the most units sold and the biggest community” one. I think that developers from that community do not want their board supported in main distributions like Debian or Fedora.

Heresy? Do not think so. What needs to get board supported was told many times. Mainline kernel support, firmware blobs with redistribution license, drivers for graphics and sane bootloader (UEFI, U-Boot, maybe some other too).


So if you want to have your board supported then spend some time on mainlining your changes/drivers. And then come to us.

AArch64 desktop: last day

Each year you can hear “this is a year of Linux desktop” phrase. After few days with AArch64 desktop I know one thing: it is not a year of ARMv8 Linux desktop.

Web browsing

OK, I can be spoiled by speed of my i7-2600k desktop but situation when Firefox with less than 20 tabs open is unable to display characters I type into textarea fast enough shows that something is wrong (16GB ram machine). And tell me that this is not typical desktop use of web browser…

YouTube. Main source of any kind of videos. Sometimes it works, but most of time I lack patience to wait until it will start (VP9 and h264 codecs support present). And no way to watch “live hangouts”.

And say bye to music streaming services like Deezer or Spotify.


I am not a game player. Installed Quake3 (which I never played before) and it worked, SuperTuxKart worked as well. But that does not prove anything as both those games have low requirements.

It probably never will be gaming platform on Linux desktop.


For my style of development it is fine. But all I need is terminal and gVim ;D

Other hardware?

I think that results may be affected by a fact that all I have here is Applied Micro Mustang based on X-Gene 1 cpu. It is one of first ARMv8 processors in Linux world and it is optimized for server use rather than desktop.

One thing is sure: in next year I will try this experiment with other AArch64 hardware. Just hope it will be sooner than in a year from now (which is my feeling after lack of new aarch64 hardware announcements from Linaro members during this week Linaro Connect).