How to speed up mock

Fedora and related distributions use “mock” to build packages. It creates chroot from “root cache” tarball, updates it, installs build dependencies and runs build. Similar to “pbuilder” under Debian. Whole build time can be long but there are some ways to get it faster.

kill fsync()

Everytime something calls “fsync()” storage slow downs because all writes need to be done. Build process goes in a chroot which will be removed at the end so why bother?

Run “dnf install nosync” and then enable it in “/etc/mock/site-defaults.cfg” file:

config_opts['nosync'] = True

local cache for packages

I do lot of builds. Often few builds of same package. And each build requires fetching of RPM packages from external repositories. So why not cache it?

In LAN I have one machine working as NAS. One of services running there is “www cache” with 10GB space which I use only for fetching packages — both in mock and system and I use it on all machines. This way I can recreate build chroot without waiting for external repositories.

config_opts['http_proxy'] = 'http://nas.lan:3128'

Note that this also requires editing mock distribution config files to not use “mirrorlist=” but “baseurl=” instead so same server will be used each time. There is a script to convert mock configuration files if you go that way.

decompress root cache

Mock keeps tarball of base chroot contents which gets unpacked at start of build. By default it is gzip compressed and unpacking takes time. On my systems I switched off compression to gain a bit at cost of storage:

config_opts['plugin_conf']['root_cache_opts']['compress_program'] = ""
config_opts['plugin_conf']['root_cache_opts']['extension'] = ""

tmpfs

If memory is not an issue then tmpfs can be used for builds. Mock has own plugin for it but I do not use it. Instead I decide on my own about mounting “/var/lib/mock” as tmpfs or not. Why? I only have 16GB ram in pinkiepie so 8-12GB can be spent on tmpfs while there are packages which would not fit during build.

parallel decompression

Sources are compressed. Nowadays CPU has more than one core. So why not using it with multithreaded gzip/bzip2 depackers? This time distro file (like “/etc/mock/default.cfg” one) needs to be edited:

config_opts['chroot_setup_cmd'] = 'install @buildsys-build /usr/bin/pigz /usr/bin/lbzip2'
config_opts['macros']['%__gzip'] = '/usr/bin/pigz'
config_opts['macros']['%__bzip2'] = '/usr/bin/lbzip2'

extra mockchain tip

For those who use “mockchain” a lot this shell snippet may help finding which packages failed:

for dir in *
do 
    if [ -e $dir/fail ];then
        rm $dir/fail
        mv $dir _fail-$dir
    fi
done

summary

With this set of changes I can do mock builds faster than before. Hope that it helps someone else too.

2 thoughts on “How to speed up mock”

    1. If only Debian/Ubuntu let you do the parallel decompression part. By default the apt library has the compression libraries built-in so it uses those over any apt.conf settings to redirect it to lbzip2, pigz, pxz.

      All of the parallel compression utilities set up threads for read/write as well as the ones for actual decompression so you tend to get a slight performance boost (probably down to the IO scheduler and not blocking memory-bound decompression thread) even on a single core build system.

Comments are closed.