Nine years of embedded Linux

Nine years ago I bought Sharp Zaurus SL-5500 as my first Linux PDA. And due to this I am where I am.

I could say that it started two years earlier when I saw PalmOS devices at local geek meetings. But it took me over year before Palm m105… Then was Sony Clie SJ30 — gorgeous device. High resolution, memory card, 16bit colour. Too bad that applications did not make use of it.

So I went for Linux. There were two options: Zaurus or iPaq. Went for former one as it had keyboard. It was good choice.

Quickly started development of packages and joined OpenEmbedded team. Then became one of OpenZaurus developers. After year or something took over release maintenance and released few last versions. 3.5.4(.1) were the best tested releases of OZ ever — I had over hundred testers for each RC image and they provided installation reports, bug reports and fixes. And it had unified installer for whole range of devices (took me several months to get it polished and few guys added own tweaks). When Ångström distribution started I was the one who officially ended OpenZaurus development.

And all that was in free time. But in mean time I created my consulting company. CELF was my first customer ;)

One nice evening I got question on irc and due to that I left dark side of IT and went from PHP programming to embedded Linux full-time. OpenedHand had interesting projects and clients with many devices. Imagine operating system + kernel + Python + GStreamer in 16 megabytes of flash… And I managed to get it done. While working for them I used proper developer boards (not only customer devices) and there were funny moments…

When we worked with ST Microelectronics on NDK-15 (later replaced by NHK-15 from ST Ericsson) I had to merge two kernel trees from two separate teams. Took me 2 days of mangling 20-30MB diffs but got it done. There are people at ST-E which reminded me this during one of Linaro Connects ;D

Also on GUADEC 2007 when we presented new interface for Openmoko phones NDK-15 had to wait for me as no one at stand was able to get it running (U-Boot config needed changes).

But then Intel acquired OpenedHand… The craziest trip of my life was return from London to my parents place. For three months I even had email but never used it due to problems with Intel corporate network and Linux (do not ask).

Next was Bug Labs and their BUG device. I cleaned their Poky trees, migrated to latest version and later to use OpenEmbedded directly. Less challenges but I also had few other customers at that time to keep me busy. Some of them were OH customers before and went to me for help.

Time passed, 2010 came. One day Canonical made another attempt to seduce me and this time I decided that it looks like good opportunity so I accepted. Sent BUG 2.0 prototype back to NYC and few weeks later I made crazy train trip to small nowhere near Brussels to meet my new coworkers from NewCore. 1-2 weeks later we got our current name: Linaro.

Total change… From embedded devices to ‘Yes, it is ARM. So what?’ kind as we support(ed) devices powerful enough to run normal desktop software. Many changes for me — from OpenEmbedded where you can (cross) build everything in few hours to Ubuntu packaging where sending package for inclusion into archive meant few hours of buildd queue and then few of build. But I learnt a lot here and met another set of hackers including grey beards ones ;)

And all that because I bought Sharp Zaurus SL-5500 nine years ago…

What interest me in ARM world

When I published my last post about ARM boards there were many questions and suggestions with interesting devices. Thank You all for it.

But there were also suggestions about ARM9 or ARM11 based devices. So I decided that it is good time to write what interest me now in ARM world.

But first some inventory. I had/used/have several devices with ARM cpu:

  • StrongARM (armv4) one:

    • Sharp Zaurus SL-5500 (which took me to ARM world)
  • ARM920 (armv4t) ones:

    • Openmoko GTA01 bv3, bv4 (s3c2410)
    • EDB9301 (EP9301 cpu)
    • Sim-One (EP9307)
  • ARM926 (armv5te) ones:

    • Sharp Zaurus sl-5600 (pxa250)
    • Sharp Zaurus c760/sl-6000 (pxa255)
    • Sharp Zaurus sl-c3000 (pxa272)
    • Sheevaplug (kirkwood)
    • Atmel devboards (at91sam9263, at91sam9m10)
    • ST-Microelectronics/ST-Ericsson NDK-15, NHK-15 (st88n15)
    • Nokia 770 (omap1710)
    • Linksys NSLU2 (ixp425 iirc)
  • ARM1136 (armv6) ones:

    • Nokia N810 (omap2430)
    • Bug r1.0, r1.2 (i.mx31)
  • Cortex-A8 (armv7a) ones:

    • Beagleboard B7, B7, C3 (omap3430)
    • Nokia N900 (omap3430)
    • Nexus S (exynos3)
    • Genesi Efika MX Smartbook (i.mx51)
    • Freescale Quickstart (i.mx53)
  • Cortex-A9 (armv7a) ones:

    • Pandaboard EA1, A1 (omap4430)
    • Archos G9 80 (omap4430)

All of that during last 8 years. Most of my ARM live so far was around ARM926 based devices (some of them still can not be listed here) and I do not want to go there again. Kirkwood core was fastest one with 1.2GHz clock and 512MB of RAM it was really fast machine. I only missed Serial ATA in my Sheevaplug (rev 1.0) but even with hard drive on USB it was nice improvement.

Then I played a bit with ARM11 processors. Ok, they were faster than most of ARM9 cpus but I already had experience with Sheevaplug. And after few months first Cortex-a8 board landed on my desk — I got Beagleboard B7 from Bug labs as test platform for their new device. This was improvement!

I still remember my reaction when connected it to normal LCD monitor and saw it used at 720p resolution (1680×1050 was a bit hard for omap3). Moved to Nokia N900 few months later and found that fast cpu means nothing when paired with slow storage and not enough memory for system.

So today I prefer to not look below Cortex-A9 (or comparable cores like ones from Qualcomm or Marvell). Hope to play one day with Cortex-A5 (which should replace ARM926 one day) just to see how low-end armv7a cpu behave.

And wait for ARMv8 to hit market.

PandaBoard: my story

It was 24th March 2010 when one friend asked me do I want to be added to beta testers list for new omap hardware. One of questions was “what would you like to have on board” so I replied:

  • hdmi out (does not care much about vga/svideo/composite out)
  • 2xSD slots (SD or microsd type)
  • ethernet (but rather not on usb)
  • serial on db9/icd10 + serial/jtag by miniusb (think sheevaplug)
  • OTG is not needed but can be present
  • BT would be nice but not required as I have 5 micro dongles here
  • few usb ports — if possible (not omap3530) on more then one hub
  • few leds (multicolor?) would be nice (bug 2.0 has 2xblue + 2xmulticolor)
  • few buttons including power/reset ones
  • and 5V 2.1/2.5mm power jack. I do not need power-on-otg because it require 500mA ports
  • onboard lcd+ts is not needed for me
  • ah… and mounting holes like in beagleboard so board can be mounted anywhere
  • connector with i2c/spi/gpio/etc/etc
  • I missed audio in/out
  • battery for rtc

And suggested to place most of connectors on 2 edges as it helps to organize desk. Atmel’s at91sam9m10 was given as example cause it has all connectors on top and left edge.

And time passed… At UDS-M TI people said that there will be cheap OMAP4 based board named PandaBoard. During dinner (later same day) I got added second time to early adopters list. I wonder how Rob Clark reacted when he saw me on a list already :D

And again time passed… Ubuntu/ARM people were playing with prototypes of PandaBoard (ES1.0, ES2.0 6-layer etc) and I had occasion to play with boards during Ubuntu/Linaro platform sprint in Prague. It looked nice (if you did not looked at ES1.0 one) and was more or less working fine.

And finally at 15th September I was told that at the end of month there will be production run from which several boards will be shipped to early adopters and few selected projects. Board travelled half of the world, then got back to US and at the end of UDS-N I got it.

Arrived home, powered BeagleBoard C3 off and started to assemble new board. Panda got several accessories connected:

  • +5V 3.5A power supply
  • powered USB hub
  • small USB keyboard
  • wireless USB mouse
  • 20″ LCD monitor with 1680x1050px resolution (this is also connected to my desktop)
  • 320GB Serial-ATA hard drive in SATA->USB enclosure

Also connected Ethernet, serial (by usb-serial dongle + 2 usb extenders) and used one of floating SD cards to have place for bootloaders and kernel. Config is much nicer then it was when I used BeagleBoard.

As operating system I am using Ubuntu 11.04 ‘natty’ as this is current development version and I have some things to check under it. Anyway I plan to move backwards and install 10.10 ‘maverick’ as primary system cause this will allow me to test omap4 hardware acceleration of graphics and audio/video decoding.

What I am using it for? Package building and testing. So far rebuilt whole KDE4 but it was segfaulting all the time on EfikaMX Smartbook so I am waiting for official ones (as there are some things to fix there first).

PandaBoard: Beagleboard XM killer?

It was known since previous UDS that there will be OMAP4 based PandaBoard available for developers. And some time ago was started (for now with temporary website). Boards are still not available at distributors but there are some of them in different projects (like Ubuntu/ARM), some are on a way to new users (mine for example).

When final price was announced many people said that PandaBoard is BeagleBoard XM killer due to same (179USD) price. But is it? Let have a look.

First group of users for such boards are software developers. If they do not work for hardware companies then usually want to get more power for same price. So they will choose PandaBoard.

Second group would be companies which want to produce own hardware based on OMAP3/4. Here it depends on how soon OMAP4 chips will be available in small orders. As OMAP3 can be bought now and BBXM is available to buy many will choose it as this allow to get own hardware ready to market in less then year with having working platform for own developers so final device will start with ready software. One of such is BUG 2.0 which I used at prototype phase. It was designed after using BeagleBoards with BUGBoard extension as base for hardware development.

And Beagleboard XM is available to buy today — with fast CPU, 512MB ram, Ethernet, few USB ports it is big update to previous versions. I never used it — BB C3 is still my primary ARM development system. But in 2-3 weeks situation will change and BB will meet another C3 and one B7 versions in a box due to arrival of PandaBoard.

What makes a good developer board?

During FOSDEM 2010 I had discussion with Ulf Samuelsson from Atmel and few other guys about developer boards. What is required on them and what should be avoided. Some time later I had a talk with one person about new OMAP3 based board and what I would like to see on it. So I decided to write something in public.

So far I used mostly ARM developer boards from ST Microelectronics/ST Ericsson, Atmel, Cirrus Logic, Intel, Simple machines, Bug Labs, Texas Instruments. Some were better then others etc. But what ideal developer board should have? Let me try to create a list:

  • 2 serial ports (one can be null modem, second should have RTS/CTS/DTR lines)
  • working Ethernet not placed on USB bus (so it works when USB does not)
  • powered USB host port (more then one would be great)
  • USB device port
  • JTAG connector
  • one power input — +5V or +12V — other should be forbidden as those ones can be taken from PC PSU which can power multiple devices at same time
  • SD/MMC slot — even if it is over slow SPI (like on Sim.One — 250KB/s max)
  • GPIO pins
  • I²C bus
  • SPI bus
  • keypad with Up/Down/Left/Right + Enter at least
  • easily reachable reset button (pins to short are acceptable as micro switch can be put on them)
  • few LEDs controlled by system
  • all connectors put on one or two edges of board — top one + one of side ones are ok (Atmel at91sam9m10-ekes for example)
  • mounting holes (so board can be mounted on A5 sheet holder for presentation at stand)
  • backup battery for RTC
  • U-Boot
  • quite fresh kernel (not NHK-15 due to 2.6.20 kernel which is 3 years old now)

What to avoid:

  • female serial port connectors (Atmel NGW100) — most devs will find 3 null modem cables before straight one
  • flat cables which connect “debug boards” with main board (Openmoko phones, NHK-15 from ST Ericsson)
  • RJ45 connectors for serial console (Sim.One) — DB9 or properly done USB->RS232 adapter on-board are best
  • placing connectors on all edges (BeagleBoard — but it had to be cheap)
  • non standard bootloaders (U-Boot is what I prefer)
  • Ethernet on USB (Bug 2.0) — it is hard to use when you have problems with USB Host
  • jumpers (Atmel boards)
  • non standard connectors (Bug r1.2 and it’s Handylink crap — next versions use iPod connector which is easier to use)

I am trying to not cover should developer board contain display with touchscreen or not as it depends on type of board. But if screen is present then more then QVGA (320×240) would be nice (WVGA anyone?). Some kind of video out connector can also be used but would be nice to have one of VGA/DVI/HDMI so normal PC monitor can be used — Composite video and S-Video require searching for some kind of TV…

Which boards are my favorites? There are few:

  • FriendlyARM with WVGA screen — cheap product which gives access to everything needed to start with embedded Linux
  • Atmel AT91SAM9263-EK — my first own developer board
  • BeagleBoard Cx — has own problems but I like the power of it

Another job change

Since OpenedHand was acquired by Intel I worked with few customers. The biggest one was Bug Labs Inc. with which I spend lot of time on hacking Poky Linux and OpenEmbedded to make their BUG devices prosper in hacking community.

Thanks to developers there Java land is not so strange for me (not that I started to like it) and I know which projects exist in that area. Many of changes done for BUG landed in OpenEmbedded metadata and helped other projects. Last release of Poky ‘pinky’ (stable branch which we used with R1.4) was done due to out improvements and bug fixes (we got credit on them). It was great time and I really enjoyed it.

The open source companies have this nice feeling — developers work on code to make it better and better (as other people look at code) and are friendly to own employees and contractors.

What next? My first job where my experience from OpenEmbedded will be used in a project which does not derive from it. Yes — no OpenEmbedded, no Poky Linux. But it will be GNU/Linux still and still ARM architecture.

In 3 weeks from now I will work for Canonical as Foundations OS Engineer. The goal is to make Ubuntu/ARM fly on supported devices (armv7a only). This will be full time job but I hope that will have possibility to do some OE related things from time to time.