# RPMs ## What you'll learn in this chapter * Modifying a Clear Linux OS kernel RPM * Creating a new RPM from an upstream release * Adding the custom RPM to the update content * Deploying the change to the target system ## Concepts Creating RPM files is a relatively simple task where source code and build instructions are used to generate a binary archive with some metadata. The RPM format has been in use for a very long time and is an efficient method to convey both content and metadata in a single file format. Clear Linux OS uses the RPM file format for intermediate storage. This allows developers to solve partial problems and not have to rebuild an entire OS altogether. This increases speed while still providing all the tools and data needed to prevent breaking dependencies. RPM files are created as the output of `rpmbuild`, which takes a `spec` file as instructions, together with the source code, patches, and miscellaneous files that may be needed during the build. For this purpose, the developer doesn't really work with RPM files, instead they work with `spec` files and miscellaneous files that are the input to the `rpmbuild` phase. This is what we call a **package** in Clear Linux OS terminology. These packages all live in their own individual git tree. They are published for users to see, review, and reuse at the following github URL: ``` [https://github.com/clearlinux-pkgs] ``` Working with `spec` files is tedious work. A Linux distribution typically contains thousands of them and they mostly repeat metadata that can automatically be generated or discovered, so we really don't want to spend too much time on the metadata aspect. The Clear Linux OS team also has created tooling to make maintaining and creating these `spec` files easier and largely automated, so that developers can focus on the content instead. We use the `common` tooling to bypass some of the hurdles that make packaging more difficult and get to borrow most of the upstream content for free. This allows us to modify targeted components for the purpose of our mix quickly. The RPM files are built using `mock`. This program creates a buildroot where the source code that needs to be compiled is in a separated environment. This prevents the compiler from accidentally pulling in system libraries that it wasn't meant to pull in. The buildroot will only contain libraries that are explicitly required through dependencies or as part of the buildroot toolchain. The buildroot dependencies are obtained from both local and upstream Clear Linux OS `yum` repositories. Dependencies are resolved using `dnf` or `yum` such that the buildroot is complete, before rpmbuild commences with the build process. ## `common` A repository at github specifically exists to deal with the creation and maintainance of `spec` files. It needs to be set up once and we have a handy shell script to do this. You can find it either on the `https://github.com/clearlinux/common/` repository but it's also in the `files` folder in the training repository. ``` ~ $ how-to-clear/files/clr/user-setup.sh ``` This command creates a folder called `clearlinux`. Note we are executing it in the `~` folder, but you can set it up and move it in its entirety to a new location without issues. Note we assume you cloned the training git project at this location, too. Inside the `clearlinux` workspace you'll find a `Makefile` and the `projects` and `packages` folders. For now, you won't need to deal with the `projects` folder other than knowing that this is where `autospec` is stored and the `common` project lives. These two items are the heart of the tooling that deals with making packages for Clear Linux OS. ``` ~ $ cd clearlinux ``` ### `packages` Under the `packages` folder we will store the folders that contain the `spec` and other files. From there we will create RPM files as needed for our training and testing. The common tooling allows us to reuse existing Clear Linux OS packages from upstream, and if we do, these will end up in here. If we end up making our own custom RPM files, we don't necessarily need to put them here at all, but we'll set up the workspace so that `mixer` can use the generated RPM files easily. It is easiest to do this all in a consistent way. In the end, the `mixer` tooling doesn't care where the RPM files come from and how they are generated. The RPM files can even be copied in if you want to bypass building them. We can quickly borrow some of the upstream packages if we desire. This makes for an excellent starting point and shows off the tooling that we have. ``` ~/clearlinux $ make clone_dmidecode ~/clearlinux $ cd packages/dmidecode ``` Since `dmidecode` is already present in Clear Linux OS, we can just start using that package and make modifications to it. This allows us to bypass most of the hurdles of packaging, and gives us a starting point that is as close to what Clear Linux OS uses as possible. We can make a quick change to this package, and change the revision to a new number that's higher than the Clear Linux OS version, and rebuild it. This way we can include it in our mix and know for sure that it's our version and not the version from the Clear Linux OS RPM repository instead. Add the following lines to the `excludes` file: ``` /usr/bin/biosdecode /usr/bin/ownership /usr/bin/vpddecode /usr/share/man/man8/biosdecode.8 /usr/share/man/man8/ownership.8 /usr/share/man/man8/vpddecode.8 ``` These files are not needed by dmidecode. Due to security restrictions, they are useless on Clear Linux OS because they require `/dev/mem` to be available, which is not the case on Clear Linux OS. We can therefore just remove them from the RPM files without penalty. ``` ~/clearlinux/packages/dmidecode $ make autospec ``` We end up with several new RPM files under `results/`. This brings us to the next phase: Adding `dmidecode` into our mix content and pushing it to our target device. ## Adding `dmidecode` to our mix We need to maintain an RPM repository. An RPM repository is a combination of a few RPM files and some metadata that allows programs like `yum` and `dnf` to follow and include dependencies when we're asking it to use specific RPM files. In the mixer folder, we've already created a location for RPM files when we used the `mixer init --local-rpms` command in an earlier chapter. We will use this location to put our newly generated RPM files and thereby convey them to mixer so it can include them as needed. The `make autospec` command creates RPM files under each package. We could copy these files manually over to the `local-rpms` folder in the `mix` folder structure. In the future, we plan to have tools available to do this more efficiently. ``` ~/clearlinux/packages/dmidecode $ cp results/*x86_64*rpm ~/mix/local-rpms/ ``` Next, we can include `dmidecode` in several ways to our update content. We can create a new local bundle, we can modify an existing upstream bundle, or we can include an upstream bundle that already has `dmidecode` present. For simplicity, we'll make a new local bundle: ``` ~/mix $ mixer bundle edit dmidecode ``` Edit the file and insert the `dmidecode` package name, without quotes or leading `#` characters. Save the file, and add it to the bundle list: ``` ~/mix $ mixer bundle add dmidecode ``` Note that the order here isn't a mistake - `edit` allows you to register a new bundle, and `add` inserts it to the list of bundles that will be included in the build. Add the `dmidecode` RPM file name to the bundle, and you're ready to deploy the change: ``` ~/mix $ mixer build all ``` After this, we can go to our target device and install the new bundle after we update, and use it: ``` ~ # swupd update ~ # swupd bundle-add dmidecode ~ # dmidecode ``` We can verify that our `biosdecode` program is missing in the same way, as expected. ## Linux kernel RPM One of the more common use cases that people ask us about is how they can modify the Linux kernel on the device. This is very similar to the above demonstration, but we'll go into the kernel specific aspects in this section. First, we will modify the `linux-kvm` kernel that is already used on the system so that we don't need to switch kernels on the device. For this reason we want to make sure that we've already started with the proper kernel package for the target device. In this training, we've settled on the KVM kernel image, but for your purposes, you may need to choose a different kernel or perhaps even make a significantly new kernel. ``` ~/clearlinux $ make clone_linux-kvm ~/clearlinux $ cd packages/linux-kvm ``` We can do a ton of changes to this kernel. For this example, we'll disable an option that is easily verifiable later on. On the target, we can see that the `btrfs` filesystem is supported by the current kernel by doing: ``` ~ # modprobe btrfs ~ # grep btrfs /proc/filesystems ``` To disable this, edit the `config` file in the `linux-kvm` package folder and change the line below from: ``` CONFIG_BTRFS_FS=m ``` to: ``` # CONFIG_BTRFS_FS is not set ``` We need to make sure to increment the package release number to make sure that our new RPM version is used instead of the version on the remote RPM repository: ``` ~/clearlinux/packages/linux-kvm $ make bumpnogit ``` Now we can build the RPM files: ``` ~/clearlinux/packages/linux-kvm $ make build ``` This will take a little bit of time, as the kernel isn't small. Once the process finishes, you should have 2 binary RPM files under the `results` folder that we can give back to mixer again: ``` ~/clearlinux/packages/linux-kvm $ cp results/*x86_64.rpm ~/mix/local-rpms/ ``` Switch back to the mixer, as we can now mix in our changed kernel. ``` ~/clearlinux/packages/linux-kvm $ cd ~/mix ``` ``` ~/mix $ mixer add-rpms ~/mix $ mixer build all ``` Then we can switch to the target device and verify the changes again: ``` ~ # swupd update ~ # reboot ~ # modprobe btrfs modprobe: FATAL: Module btrfs not found ... ``` Success! ## Exercises * Use `make shell` in a package folder after building, or when a build error occurs. * Use `make autospecnew NAME= URL=` on something that isn't in Clear Linux OS yet. * Use `make repoadd` in packages that need new dependencies and build one custom package against another new custom package. * Use `make clone` or `make pull` in the top-level folder. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://clr.gitbook.io/clearlinux-docs/06-rpms.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.