1. Unbricking APM Mustang

    Firmware update usually ends well. Previous (1.15.19) firmware failed to boot on some of Mustangs at Red Hat but worked fine on one under my desk. Yesterday I got 1.15.22 plus slimpro update and managed to get machine into non-bootable state (firmware works fine on other machines).

    So how to get APM Mustang back into working state?

    • Get a SD card and connect it to an PC Linux machine with reader support.
    • Download Mustang software from MyAPM (1.5.19 was latest available there).
    • Unpack “mustang_sq_1.15.19.tar.xz” and then “mustang_binaries_1.15.19.tar.xz” tarballs.
    • Write the boot loader firmware to the SD card: “dd if=tianocore_media.img of=/dev/SDCARD.
    • Take FAT formatted USB drive and put there some files from “mustang_binaries_1.15.19.tar.xz” archive (all into root directory):
      • potenza/apm_upgrade_tianocore.cmd
      • potenza/tianocore_media.img
      • potenza/UpgradeFirmware.efi
    • Power off your Mustang
    • Configure the Mustang to boot from SD card via these jumpers change:
      • Find HDR9 (close to HDR8, which is next to PCIe port)
      • Locate pin 11-12 and 17-18.
      • Connect 11-12 and 17-18 with jumpers
    • Insert SD card to Mustang SD port
    • Connect serial cable to Mustang and your PC.
    • Run minicom/picocom/screen/other-preferred-serial-terminal and connect to Mustang serial port
    • Power up Mustang and you should boot with SD UEFI firmware:
    X-Gene Mustang Board
    Boot firmware (version 1.1.0 built at 12:25:21 on Jun 22 2015)
    PROGRESS CODE: V3020003 I0
    PROGRESS CODE: V3020002 I0
    PROGRESS CODE: V3020003 I0
    PROGRESS CODE: V3020002 I0
    PROGRESS CODE: V3020003 I0
    PROGRESS CODE: V3020002 I0
    PROGRESS CODE: V3020003 I0
    PROGRESS CODE: V3021001 I0
    TianoCore 1.1.0 UEFI 2.4.0 Jun 22 2015 12:24:25
    CPU: APM ARM 64-bit Potenza Rev A3 1400MHz PCP 1400MHz
         32 KB ICACHE, 32 KB DCACHE
         SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
    Board: X-Gene Mustang Board
    The default boot selection will start in   2 second
    • Press any key to get into UEFI menu.
    • Select “Shell” option and you will be greeted with a list of recognized block devices and filesystems. Check which is USB (“FS6” in my case).
    Shell> fs6:
    FS6:> ls
    Directory of: FS6:
    08/04/2015  00:28              39,328  UpgradeFirmware.efi
    08/27/2015  19:20                  56  apm_upgrade_tianocore.cmd
    08/27/2015  19:20           2,098,176  tianocore_media.img
    • Flash firmware using “UpgradeFirmware.efi apm_upgrade_tianocore.cmd” command.
    • Power off
    • Change jumpers back to normal (11-12 and 17-18 to be open).
    • Eject SD card from Mustang
    • Power on

    And your Mustang should be working again. You can also try to write other versions of firmware of course or grab files from internal hdd.

    Written by Marcin Juszkiewicz on
  2. Fedora 23 and unsupported ARM/AArch64 devices

    Week ago Fedora 23 got released. Also for ARM and AArch64 architectures. But it does not mean that it supports all possible hardware.


    There is the installation guide which lists two supported hardware platforms (besides QEMU):

    • Applied Micro Mustang
    • AMD Seattle

    And then we got email from Clive Messer with question why we do not support 96Boards, ie. HiKey and Dragonboard as they are cheap and available.

    I am not surprised with such question. It would be great to have support for both boards but their current state makes it quite hard. There is no support for them in mainline kernel, Dragonboard needs some firmware files which license forbids packaging it (note bolded part):

    Distribution of the Redistributable Binary Code is subject to the following restrictions: (i) Redistributable Binary Code may only be distributed in binary format and may not be distributed in source code format: (ii) Redistributable Binary Code may not be distributed on a stand alone basis but may be redistributed in conjunction with and as a part of a software application created by you; (iii) the Redistributable Binary Code may only operate in conjunction with platforms incorporating Qualcomm Technologies, Inc. chipsets; (iv) redistribution of the Redistributable Binary Code must include the .txt file setting forth the terms and condition of this Agreement; (v) you may not use Qualcomm Technologies’ or its affiliates or subsidiaries name, logo or trademarks; and (vi) copyright, trademark, patent and any other notices that appear on the Materials may not be removed or obscured.

    So even if we get mainline kernel working on it some things will not work without non-free files.

    Then we have bootloaders. Hikey can be flashed with UEFI but (according to bootloader install documentation) you need to keep partitions in some magic way. Where is “there has to be one ef00 type partition formatted with FAT” as it is with other UEFI powered machines? Dragonboard 410c uses fastboot ;(


    On ARM side there is common question about “most readily available and used board, with the most units sold and the biggest community” one. I think that developers from that community do not want their board supported in main distributions like Debian or Fedora.

    Heresy? Do not think so. What needs to get board supported was told many times. Mainline kernel support, firmware blobs with redistribution license, drivers for graphics and sane bootloader (UEFI, U-Boot, maybe some other too).


    So if you want to have your board supported then spend some time on mainlining your changes/drivers. And then come to us.

    Written by Marcin Juszkiewicz on
  3. AArch64 desktop: last day

    Each year you can hear “this is a year of Linux desktop” phrase. After few days with AArch64 desktop I know one thing: it is not a year of ARMv8 Linux desktop.

    Web browsing

    OK, I can be spoiled by speed of my i7-2600k desktop but situation when Firefox with less than 20 tabs open is unable to display characters I type into textarea fast enough shows that something is wrong (16GB ram machine). And tell me that this is not typical desktop use of web browser…

    YouTube. Main source of any kind of videos. Sometimes it works, but most of time I lack patience to wait until it will start (VP9 and h264 codecs support present). And no way to watch “live hangouts”.

    And say bye to music streaming services like Deezer or Spotify.


    I am not a game player. Installed Quake3 (which I never played before) and it worked, SuperTuxKart worked as well. But that does not prove anything as both those games have low requirements.

    It probably never will be gaming platform on Linux desktop.


    For my style of development it is fine. But all I need is terminal and gVim ;D

    Other hardware?

    I think that results may be affected by a fact that all I have here is Applied Micro Mustang based on X-Gene 1 cpu. It is one of first ARMv8 processors in Linux world and it is optimized for server use rather than desktop.

    One thing is sure: in next year I will try this experiment with other AArch64 hardware. Just hope it will be sooner than in a year from now (which is my feeling after lack of new aarch64 hardware announcements from Linaro members during this week Linaro Connect).

    Written by Marcin Juszkiewicz on
  4. AArch64 desktop: day two

    First day behind me. Had to disconnect DVD drive as X-Gene SATA controller still does not like it. And to resolve some issues with multimedia support. Got some questions too.

    What is desktop class?

    There were questions about desktop class hardware, links to Gigabyte mainboard etc. So why Mustang is desktopstein and not desktop? Or rather: what desktop should have?

    For me to consider mainboard as desktop class it needs to have few things:

    • microATX or miniITX format
    • USB ports (more than two, USB 3.0 speed should be available)
    • on-board sound with 3.5mm output (do not care USB/PCIe/SPI - has to work)
    • x16 PCI Express slot for graphics (can have 8 lines)
    • SATA controller working with optical drives (good-bye X-Gene 1)
    • at least four SATA ports
    • standard DIMM slots (without ECC requirement)

    When it comes to AArch64 there is no such ones so far. Something can be arranged from few mainboards by adding extra cards or usb devices. And so far no new rumours about new hardware ;(

    UPDATE: why 4 drives? I have system hdd, scratch hdd (to test installers) and want to have optical drive (also for installers).

    Multimedia support

    I use Fedora on my machines. And as lot of people know this distribution has strange rules when it comes to multimedia support. Forget about MP3, H264 and few other things.

    Normally (on x86(-64) architecture) people go for RPMFusion repositories. But they do not support aarch64… So I did local rebuilds of everything needed to get MPlayer, VLC and H264 support for GStreamer (so Firefox is able to play YouTube videos).

    There is no Adobe Flash support so some of YT videos does not work. Too bad that it includes live streaming as I am unable to watch Linaro Connect keynotes on desktop (good that it works on Android tablet).

    Written by Marcin Juszkiewicz on
  5. AArch64 desktop: day one

    Three years ago I was working on getting AArch64 toolchain built. Then lot of things happened.

    Nowadays if you are lucky you can even have AArch64 hardware. The problem is that there is no desktop class one still. Mustang and Seattle are server boards, Juno is development platform, Hikey is out of stock, Dragonboard 410c has 1GB of memory (same as Hikey) and rest of “publicly available” AArch64 hardware is in Android or iOS devices.

    I played with getting X11 running for quite long time so decided that it is time to make use of it. Moved Mustang mainboard into microatx tower case, added PCI Express riser with Radeon HD5450 graphics card in it. Then two hard drives (one for systems, second for testing installers) and fun started. Two monitors connected, speakers got some waves from cheap (1.5€) USB sound card, mouse and keyboard also migrated from my x86-64 desktop. I called resulting machine “desktopstein”.

    Pressed power and machine booted. Nothing on screen for about 1.5-2 minutes because UEFI firmware does not initialize graphics and then some messages appeared and I was able to login to text console. Enabled lightdm and logged into KDE session.

    Konsole works, Firefox works (copied whole profile from x86-64), Thunderbird fetched mails (also copied profile). No Chrome nor Chromium is fine. Had to switch to other music provider than Deezer as their web player requires Adobe Flash. But there are still lot of MP3 streaming services so VLC (from own rpmfusion rebuild) got something to play.

    So far works good. During week I plan to do my normal work and try some random things.

    Written by Marcin Juszkiewicz on
  6. From a diary of AArch64 porter — vfp precision

    During last years lot of work went into design of SIMD instructions for different cpus. X86-64 has some, Power has and so does AArch64.

    But why I am writing about it? Simple — build failures. Or rather test failures. Especially in scientific software like HepMC or alglib. They build fine but that’s all.

    The difference was small — something like e-15 or smaller but it was enough to make tests fail. And what to blame? SIMD of course.

    AArch64 has FMA (fused multiply-add) instructions which speed up calculations but result is more precise than traditional way when all operations are done one by one. This is enough for tests to fail :-(

    But there is solution for it. To degrade you need to add “-ffp-context=off” to compiler flags. This disables use of FMA so results of tests are same as on x86-64 (on pre-Haswell/Bulldozer cores). As a bonus it works on powerpc64(le) and s390(x) too.

    Thanks goes to David Abdurachmanov and Andrew Pinski for finding out which exactly flag needs to be used (instead of -fno-expensive-optimisations I used before).

    Written by Marcin Juszkiewicz on
  7. From a diary of AArch64 porter — PAGE_SIZE

    During fixing software to make it build and run on AArch64 sooner or later you can meet magical constant named PAGE_SIZE. And in most situations it will be used in a wrong way.

    What it does is simple — tell how big page size is. But it does not work that way on AArch64 architecture as we have different values possible: 4K, 16K (may not be supported in all cpus) and 64K with latter being used in Fedora and other distributions. There were some packages which we built at time of 4K kernel being used and then wondered why things fail under 64K kernel…

    But how to handle it as a userspace software developer? Simplest solution is to use sysconf(_SC_PAGESIZE) function call (same as getpagesize()). But remember to not hardcode anything based on what you get as a result. Otherwise your application can misbehave when run on kernel with other PAGE_SIZE size.

    The good part is that if someone uses PAGE_SIZE constant in code then it will just do not compile on AArch64 as it is not present in system headers. From what I checked sys/user.h header has it defined on some platforms and does not on other so it can not be assumed as available.

    UPDATE: added 16K page size which may not be supported in some cpus.

    Written by Marcin Juszkiewicz on
  8. How to get Xserver running out of box on AArch64

    As I want to have AArch64 desktop running with as small amount of changes needed as possible I decided that it is time to get Xserver to just run on APM Mustang.

    Current setup

    To get X11 running I need to have xorg.conf file. And this feels strange as on x86(-64) I got rid of it years ago.

    Config snippet is small and probably could be even smaller:

    Section "Device"
            Identifier " radeon"
            Driver  "radeon"
            BusID "PCI:1:0:0"
    Section "Screen"
            Identifier      "Screen"
            Device          "radeon"
    Section "DRI"
            Mode 0666

    Without it Xserver failed to find graphics card.

    Searching for solution

    I cloned Xserver git repository and started hacking. During several hours (split into few days) I added countless LogMessage() calls to source code, generated few patches and sent them to x-devel ML. And finally found out why it did not work.

    Turns out that I was wrong — Xserver was able to find graphics card. But then it went to platform_find_pci_info() function, called pci_device_is_boot_vga() and rejected it.

    Why? Because firmware from Applied Micro did not initialized card so kernel decided not to mark it as boot gfx one. I do not know is it possible to get UEFI to properly initialize pcie card on AArch64 architecture but there are two other ways to get it working.

    hack Xserver

    We can hack Xserver to not check for pci_device_is_boot_vga() or to use first available card if it returns false:

    diff a/hw/xfree86/common/xf86platformBus.c b/hw/xfree86/common/xf86platformBus.c
    index f1e9423..d88c58e 100644
    --- a/hw/xfree86/common/xf86platformBus.c
    +++ b/hw/xfree86/common/xf86platformBus.c
    @@ -136,7 +136,8 @@ platform_find_pci_info()
         if (info) {
             pd->pdev = info;
    -        if (pci_device_is_boot_vga(info)) {
    +        if (pci_device_is_boot_vga(info) || xf86_num_platform_devices == 1)
    +        {
                 primaryBus.type = BUS_PLATFORM;
                 primaryBus.id.plat = pd;

    This may not work on multi-gpu systems. In that case try removing “== 1” part.

    hack Linux kernel

    If firmware does not give us boot gfx card then maybe we can mark first one as such and everything will work? This is how PowerPC has it solved. So let’s take their code:

    diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
    index b3d098b..eea39ba 100644
    --- a/arch/arm64/kernel/pci.c
    +++ b/arch/arm64/kernel/pci.c
    @@ -18,6 +18,7 @@
     #include <linux/of_pci.h>
     #include <linux/of_platform.h>
     #include <linux/slab.h>
    +#include <linux/vgaarb.h>
    @@ -84,3 +85,15 @@ struct pci_bus *pci_acpi_scan_root()
            return NULL;
    +static void fixup_vga(struct pci_dev *pdev)
    +       u16 cmd;
    +       pci_read_config_word(pdev, PCI_COMMAND, &cmd);
    +       if ((cmd & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) || !vga_default_device())
    +               vga_set_default_device(pdev);
    +                             PCI_CLASS_DISPLAY_VGA, 8, fixup_vga);


    Both hacks work. I can just run Xserver and get X11 working. But which one will get into upstream and then to Fedora and other Linux distributions? Time will show.

    There are some issues with those solutions. If there are multiple graphics cards in a system then which one is primary one? Can their order change after firmware or kernel update?

    Thanks goes to Dave Airlie for help with Xserver, Mark Salter for pointing me to PowerPC solution and Matthew Garrett for discussion about issues with kernel solution.

    Written by Marcin Juszkiewicz on
Page 22 / 105