Tag Archives: development

From a diary of AArch porter –- testsuites

More and more software come with testsuites. But not every distribution runs them for each package (nevermind is it Debian, Fedora or Ubuntu). Why it matters? Let me give example from yesterday: HDF 4.2.10.

There is a bug reported against libhdf with information that it built fine for Ubuntu. As I had issues with hdf in Fedora I decided to look and found even simpler patch than one I wrote. Tried it and got package built. But that’s all…

Running testsuite is easy: “make check”. But result was awesome:

!!! 31294 Error(s) were detected !!!

It does not look good, right? So yesterday I spent some time yesterday on searching for architecture related check and found main reason for so big amount of errors — unknown systems are treated as big endian… Simple switch there and from 31294 it dropped to just 278 ones.

Took me a while to find all 27 places where miscellaneous variations of “#if defined(__aarch64__)” were needed and finally got to point where “make check” simply worked as it should.

So if you port software do not assume it is fine once it builds. Run testsuite to be sure that it runs properly.

10 years ago I got write access to OpenEmbedded

It was 8th May of 2004 when I did first push to OpenEmbedded repository. It was BitKeeper at that time but if someone wants to look then commit can be seen in git.

I will not write about my OE history as there are several posts about it on my blog already:

It was nice to be there through all those years to see how it grows. From a tool used by bunch of open source lovers who wanted to build stuff for own toys/devices, to a tool used by more and more companies. First ones like OpenedHand, Vernier. Then SoC vendors started to appear: Atmel, Texas Instruments and more. New architectures were added. New rewrites, updates (tons of those).

Speaking of updates… According to statistics from Ohloh.net I am still in top 5 contributors in OpenEmbedded and Yocto project ;)

There were commercial devices on a market with OpenEmbedded derived distributions running on them. I wonder how many Palm Pre users knew that they can build extra packages with OE. And that work was not lost — LG Electronics uses WebOS on their current TV sets and switched whole development team to use OpenEmbedded.

Since 2006 we got annual meetings and this year we have two of them: European as usual and North America one for first time (there was one few years ago during ELC but I do not remember was it official).

There is OpenEmbedded e.V. which is non-profit organization to take care of OE finances and infrastructure. I was one step from being one of its founders but birth of my daughter was more important ;)

And of course there is the Yocto project. Born from OpenedHand’s Poky helped to bring order into OpenEmbedded. Layers (which were discussed since 2006 at least) were created and enforced so recipes are better organized than it was before. It also helped with visibility. Note that when I write OpenEmbedded I mean OpenEmbedded and Yocto project as they are connected.

I remember days when Montavista was seen as kind of competitor (“kind of” because they were big and expensive while we were just a bunch of guys). Then they moved to OpenEmbedded and dropped own tools. Other company with such switch was Denx. 3 years ago they released ELDK 5.0 which was OE based and made several releases since then.

What future will bring? No idea but it will be bright. And I will still be somewhere nearby.

AArch64 can build OpenEmbedded

In 2012 I was porting OpenEmbedded to target AArch64 so I can say that I did first OE builds for that architecture.

But today I did kind of reverse thing:

Build Configuration:
BB_VERSION        = "1.21.1"
BUILD_SYS         = "aarch64-linux"
NATIVELSBSTRING   = "Fedora-21"
TARGET_SYS        = "arm-oe-linux-gnueabi"
MACHINE           = "genericarmv7a"
DISTRO            = "nodistro"
DISTRO_VERSION    = "nodistro.0"
TUNE_FEATURES     = "armv7a vfp thumb neon callconvention-hard"
TARGET_FPU        = "vfp-neon"

Yes — I did build on AArch64 machine targeting ARMv7a system. Had to edit one patch (pseudo-native was set to use very old glibc symbols which are not available on 64-bit ARM) but after that build was running just fine.

I did not tested resulting binaries.

%patch should DIE

I am not maintaining packages in Fedora. But my work is related to building them for AArch64 architecture. And this means editing patches and/or adding new ones…

When I was doing AArch64 bootstrap in OpenEmbedded it was easy. Altering Debian/Ubuntu packages was also simple task. Why? Because in both situations patches are applied in developer friendly way so it is visible which ones are applied in which order and (as quilt is used) they are also easy to refresh and edit.

But not in Fedora. Here we have %setup and %patch macros which use workflow from 80′s. Yes, patch <pX file.patch is not modern way of handling patches which may need work. Especially when you first fetch and install all build dependencies just to get build failed cause nth patch does not fully apply. Or when your work requires to alter patches 3rd, 7th and 13th.

Sure, there is %autosetup macro which should solve a problem. But it does not. There are packages where some patches are reverse applied or with other patch level than 1. I saw also ones where there is extra directory change during applying diffs.

So dear package maintainers — please migrate to %autosetup (with quilt or git) to make life of other developers easier. I think that there will be more people satisfied when this will happen than just me.

It is 10 years of Linux on ARM for me

It was somewhere between 7th and 11th February 2004 when I got package with my first Linux/ARM device. It was Sharp Zaurus SL-5500 (also named “collie”) and all started…

At that time I had Palm M105 (still own) and Sony CLIE SJ30 (both running PalmOS/m68k) but wanted hackable device. But I did not have idea what this device will do with my life.

Took me about three years to get to the point where I could abandon my daily work as PHP programmer and move to a bit risky business of embedded Linux consulting. But it was worth it. Not only from financial perspective (I paid more tax in first year then earned in previous) but also from my development. I met a lot of great hackers, people with knowledge which I did not have and I worked hard to be a part of that group.

I was a developer in multiple distributions: OpenZaurus, Poky Linux, Ångström, Debian, Maemo, Ubuntu. My patches landed also in many other embedded and “normal” ones. I patched uncountable amount of software packages to get them built and working. Sure, not all of those changes were sent upstream, some were just ugly hacks but this started to change one day.

Worked as distribution leader in OpenZaurus. My duties (still in free time only) were user support, maintaining repositories and images. I organized testing of pre-release images with over one hundred users — we had all supported devices covered. There was “updates” repository where we provided security fixes, kernel updates and other improvements. I also officially ended development of this distribution when we merged into Ångström.

I worked as one of main developers of Poky Linux which later became Yocto Linux. Learnt about build automation, QA control, build-after-commit workflow and many other things. During my work with OpenedHand I also spent some time on learning differences between British and American versions of English.

Worked with some companies based in USA. This allowed me to learn how to organize teamwork with people from quite far timezones (Vernier was based in Portland so 9 hours difference). It was useful then and still is as most of Red Hat ARM team is US based.

I remember moments when I had to explain what I am doing at work to some people (including my mom). For last 1.5 year I used to say “building software for computers which do not exist” but this is slowly changing as AArch64 hardware exists but is not on a mass market yet.

Now I got to a point when I am recognized at conferences by some random people when at FOSDEM 2007 I knew just few guys from OpenEmbedded (but connected many faces with names/nicknames there).

Played with more hardware then wanted. I still have some devices which I never booted (FRI2 for example). There are boards/devices which I would like to get rid of but most of them is so outdated that may go to electronic trash only.

But if I would have an option to move back that 10 years and think again about buying Sharp Zaurus SL-5500 I would not change it as it was one of the best things I did.

Xulrunner/AArch64 on a way to upstream

I finally sent Xulrunner support for AArch64 upstream. From 76KB patch I went to 14 patches with 132KB in total due to other diff options like more context lines.

Will not bother with whole history of a patch. It involved three external projects:

  • libffi (merged upstream long time ago, xulrunner needs to update their copy)
  • double-conversion (see Qt/AArch64 post for details) also update needed
  • libevent got fix for deprecated syscalls which needs to be merged into xulrunner’s copy

Splitting patch went quite easy thanks to help from Xulrunner developers: mstange froydnj Tomcat Ms2ger glandium which told me how to submit my patch, which already existing bugs to update and who assign to code reviews.

And for patches please go to dependency tree of bug #962534.

And one note: AArch64 big endian was not fully covered. Endianness info was provided in most places for both little and big options.

The story of Qt/AArch64 patching

In over a year of my work as AArch64 porter I saw a lot of patches. But Qt one has the most interesting history.

Around year ago when I was building whatever possible during my Linaro work we got to the point when Qt jumped into a queue. Build failed but fixing was quite easy — all I had to do was to take “webkitgtk” patch written by Riku Voipio and apply it to Qt 4. Resulting file landed in “meta-aarch64″ layer of OpenEmbedded and is still there.

Time passed. More common distributions like Debian, Fedora, OpenSUSE, Ubuntu (alphabetical order) started working on their AArch64 ports. And one day Fedora and Ubuntu started working on building Qt 4. I do not know who wrote QAtomic stuff but I saw few versions/iterations of it and it took me quite long time (first on model, then on real hardware) to get it fully working in Fedora — used Ubuntu version of patches.

Up to this moment it was over 9 months and still no upstreaming was done. So one day I decided to go for it and opened QTBUG #35442. Then reopened issue #33 in “double-conversion” project (which is used in few places in Qt) as they got good fix and merged wrong one (but it is fixed now). For that one I opened a request to update to newer version of “double-conversion” as QTBUG #35528.

But story did not end there. As Qt 4 development is more or less ended I was asked to provide fixes for Qt 5. Took me a while. Had to create a graph of build time dependencies between Qt 5 components (had to break few in meantime) and slowly module after module I got most of it built.

There were 3 components which required patching:

First one is solved upstream and waits for Qt guys. I was told that 5.3 will get it updated. Second one is already reviewed and merged. Last one left and here is a problem as it looks like the only person who does QtWebKit code reviews is Allan Sandfeld Jensen but he can not review code he sent. I am not able to do that due to Qt Contributor License Agreement which needs to be signed and (due to legal stuff) I can not do that.

So how it looks now? I would say that quite good. One 3rd party project needs update (in two places of Qt 5) and one patch needs to get through code review. I still need to send package updates to Fedora bug tracker. Ubuntu will need to merge patches when they move to 5.2 version.