We have designed and developed a C++ application that uses Qt for the user interface. We need to create a executable that anyone can use to run the application. For Windows and Apple systems, there is little variation in operating system structure so simply building the application on that platform is sufficient. This is not the case for Linux with its many distributions and variations.

Single Distribution

The first step is to get an executable for the developer machine. This is quite simple - it can be done in the Qt Creator IDE or using the terminal with qmake and make. qmake generates a Makefile based on the contents of a project file.

The specifics of the project file aren’t important; this file is created automatically by QtCreator and defines a few configurations (compiler versions and required Qt modules) as well as files to include and compile. Here is a minimal example of a project file Application.pro:

QT       += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

CONFIG += c++17

SOURCES += \
    application-src/main.cpp \
    application-src/mainwindow.cpp

HEADERS += \
    application-headers/mainwindow.h

FORMS += \
    application-forms/mainwindow.ui

INCLUDEPATH += application-headers/
INCLUDEPATH += application-src/

RESOURCES += \
    application-styling/application-styling.qrc

qmake reads this file and generates a Makefile:

#############################################################################
# Generated by qmake (3.1) (Qt 6.2.4)
#############################################################################

####### Compiler, tools and options

CC            = gcc
CXX           = g++
MKDIR         = mkdir -p
COPY          = cp -f

####### Output directory

OBJECTS_DIR   = ./

####### Files

SOURCES       = ../application-src/main.cpp \
		../application-src/mainwindow.cpp

OBJECTS       = main.o \
		mainwindow.o 

DIST          = /usr/lib/x86_64-linux-gnu/qt6/mkspecs/features/spec_pre.prf \
		/usr/lib/x86_64-linux-gnu/qt6/mkspecs/common/unix.conf \
		/usr/lib/x86_64-linux-gnu/qt6/mkspecs/common/linux.conf \
		../application-src/main.cpp \
		../application-src/mainwindow.cpp

####### Build rules

Application: ui_mainwindow.h $(OBJECTS)  
	$(LINK) $(LFLAGS) -o $(TARGET) $(OBJECTS) $(OBJCOMP) $(LIBS)

####### Sub-libraries

mocclean: compiler_moc_header_clean compiler_moc_objc_header_clean compiler_moc_source_clean

####### Compile

main.o: ../application-src/main.cpp ../application-headers/mainwindow.h
	$(CXX) -c $(CXXFLAGS) $(INCPATH) -o main.o ../application-src/main.cpp

####### Install

install_target: first FORCE
	@test -d $(INSTALL_ROOT)/opt/Application/bin || mkdir -p $(INSTALL_ROOT)/opt/Application/bin
	$(QINSTALL_PROGRAM) $(QMAKE_TARGET) $(INSTALL_ROOT)/opt/Application/bin/$(QMAKE_TARGET)
	-$(STRIP) $(INSTALL_ROOT)/opt/Application/bin/$(QMAKE_TARGET)

Running make uses the GNU Compiler Collection to generate an elf binary that can be executed.

Multiple Distributions

A critical limitation of the executable generated above is that it depends on certain libraries in a way that makes it distribution dependent.

Discovering the Issue

The most likely way this issue would be discovered is by running the application on different distributions - typically a given with even with a small user base. However, for this particular project, the issue was intrestingly identified in the CI/CD pipeline.

You may wish to automate the building and publishing of the application when the source code is pushed to remote git repository. Git hosting services often include an “Actions” feature that can use docker to achieve this. A minimal image that is often chosen is Alpine Linux. Required packages are installed with the Alpine package manager apk and the process for building the application outlined above is followed.

To test the generated binary, we run it on our development machine. This happens to be Arch Linux - an operating system that is organised differently to Alpine Linux. Running the binary results in an error: Error: file not found.

To understand the cause of this error, we need to better understand how Make and, more specifically, the GNU compiler works.

Compilation and Linking Process

The compiler’s actions can be divided into 4 stages handled by different components.

Examining the output of each stage gives a better insight into the process.

The component of importance here is the linker.

Dynamic Linking

Dynamic link object files are identified by the .so for “shared object” extension on Linux and .dll for “dynamically linked library” on Windows.

When linking, the symbols of the object file is added instead of the entire file itself.

The following command lists all the symbols in the libcurl library:

objdump -T /usr/lib/libcurl.so

During runtime, missing symbols are resolved by the system. This makes the application more resource efficient and means changes to shared objects can be easily propagated. The comes at the cost of comptatibility since more assumptions are made about the system - such as having the shared object in an accessible path.

Static Linking

All object files are stored directly in the resulting executable. Statically linked libraries are identified by an .a for “archive” extension.

Static linking results in better compatibility since it minimizes external dependencies. However, is increases resource consumption and makes it more difficult to update.

Static linking Qt is not trivial. The official installation only includes dynamic libraries and static linking may potentially face licensing restrictions. Static linking requires building Qt from source and additional configuration.

Packaging with AppImage

AppImages takes a different approach. The executable binary is generated using the same process outlined before, but this is then packaged with most required files to run with minimal external dependencies.

It should be noted that AppImages do still have some dependencies, but these are likely fulfilled by default by your distribution. There are some exceptions; Filesystem in Userspace (FUSE), which allows non-root users to mount filesystems and is required for the AppImage to work, is not bundled with older AppImages and may need to be installed explicitly.

If this is your first time packaging an application with AppImage, getting it to work may take time. After much trial and error, it seems that packaging with AppImage has two requirements: a compliant AppDir and a specific version of glibc.

AppDir Structure

Creating the AppDir is quite simple and can be done manually; this is how it is organized:

.
└── AppDir
    └── usr
        ├── bin
        │   └── app-name.elf
        └── share
            ├── applications
            │   └── app-name.desktop
            └── icons
                └── hicolor
                    └── 256x256
                        └── apps
                            └── app-name.svg 

app-name.svg is an image file that will be displayed as the application’s icon. app-name.elf is the binary that was generated.

app-name.desktop is a desktop entry file:

Type=Application
Exec=app-name
Name=app-name
Icon=app-name
Categories=Utility;

Note that the file extension is excluded in the entries for Exec and Icon.

glibc

glibc is a standard C library that provides many common and important functions. The glibc contraint is a little more nuanced.

To produce binaries that are compatible with many target systems, build on the oldest still-supported build system.

linuxdeployqt - the chosen packaging tool - recommends Ubuntu and enforces this system constraint quite strictly to encourage developers to support the most platforms.

The initial attempt on Alpine Linux failed because the glibc version was too new. From the documentation, it wasn’t entirely clear if “still-supported” referred to the “end of standard support” or “end of life”. Ubuntu 14.04 was the oldest system that had not reached end of life. Not only did the system have network issues (there was a bug with libcurl3 that made TLS connections unusable), but the application didn’t even compile. Sections of the source code needed to be modified because the glibc version was too old and didn’t recognise some of the newer syntax.

At this stage, the packaging tool was abandoned and alternatives were tested - this time on Arch Linux, a rolling distribution with the latest versions of all packages. After much frustration, the issue was narrowed down to the glibc version again (this was not as obvious as it is in hindsight). Arch Linux does not support rolling back to a specific version, so the correct version was manually downloaded from https://archive.archlinux.org/packages/g/glibc/ and forcefully installed. This promptly broke the system, causing a blue screen of death on shutdown (a first time on Linux). Luckily this was inside a virtual machine. Before shutdown, new terminals could not be opened, but some commands in an already open terminal worked and was used to copy some important data to a thumbdrive. The thumbdrive later failed due to an unrelated issue.

Ubuntu 22.04 - the oldest system that still had standard support - was used next. Everything went smoothly once the format for the AppDir was established and application was successfully packaged and run on different distributions.

A example of a command that might be used to create an AppImage on Ubuntu 22.04:

./linuxdeploy.AppImage -appimage \
    AppDir/usr/share/applications/app-name.desktop \
    -qmake=/usr/bin/qmake6 qmldir=application-styling/ \
    -qmldir-importscanner=/usr/lib/qt6/libexec/qmlimportscanner

The qmldir and importscanner are required to include styling and image assets defined as Qt Resources.

It is intresting to see the lifespan of the software we build, and the chain of dependencies it requires.