.NET in Gentoo in 2023

:: dotnet, gentoo, linux, powershell

By: Maciej Barć

.NET ecosystem in Gentoo in year 2023

The Gentoo Dotnet project introduced better support for building .NET-based software using the nuget, dotnet-pkg-base and dotnet-pkg eclasses. This opened new opportunities of bringing new packages depending on .NET ecosystem to the official Gentoo ebuild repository and helping developers that use dotnet-sdk on Gentoo.

New software requiring .NET is constantly being added to the main Gentoo tree, among others that is:

  • PowerShell for Linux,
  • Denaro — finance application,
  • Ryujinx — NS emulator,
  • OpenRA — RTS engine for Command & Conquer, Red Alert and Dune2k,
  • Pinta — graphics program,
  • Pablodraw — Ansi, Ascii and RIPscrip art editor,
  • Dafny — verification-aware programming language
  • many packages aimed straight at developing .NET projects.

Dotnet project is also looking for new maintainers and users who are willing to help out here and there. Current state of .NET in Gentoo is very good but we can still do a lot better.

Special thanks to people who helped out

Portage Continuous Delivery

:: gentoo, linux

By: Maciej Barć

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

Set up your repo with eselect-repository

Install the my-org repository:

1
eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1
emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1
echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1
emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1
0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1
0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1
0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8
#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1
0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3
pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

Example Gentoo overlays

Firefox is still the best browser. Deal with it Google!

:: browser

By: Maciej Barć

Firefox began as the first open source browser to live through the browser wars, overcoming Microsoft’s Internet Explorer and continues to deliver competition-smashing technology to this day.

Chromium code

The only advantages of Chromium are that it was adopted by Electron and spread partially because of a more liberal license and Google’s own efforts.

Google will never be able to cope with the worst imaginable code base of Chromium.

Chromium is near-impossible to compile

On a 4cores/8threads Ryzen CPU Chromium compiles in ~12h and requires at least 20GB of disk space for build. At the same time Firefox compiles in ~1.5h and requires ~8GB for disk space.

Programming language adoption

Additionally Firefox team was able to rewrite a very large portion of Firefox codebase in Rust which improved the browser’s safety. There were attempts to add rust to Chromium but they all are in more of a addon-like fashion.

Porting to UNIXes

Because Chromium is extremely large it’s very hard to port and maintain for Linux and BSD based systems. There were numerous bugs with Chromium’s UI on Linux that cause crashes on pressing random controls. I believe Google has no Linux testers beside the “free software freeloaders” (wink, wink, IBM :P).

Anti-competition

This days Google has to result to dirty tactics where certain Google-owned websites would either refuse to work on Firefox or give a fake performance hit that is entirely caused by malicious JavaScript code.

Several popular FOSS-related sources have covered this news recently, check them out on the WWW.

Common Project Layout, version 0

:: dev

By: Maciej Barć

This is a tongue-in-cheek “draft” for Common Project Layout, version 0. It will probably never become any sort of adopted standard but I think it is good to share some ideas that I had while working on this.

Definition

Common Project Layout (CPL) is a set of good practices for structuring medium-to-large monorepo-like software repositories.

Benefits

CPL helps with code organization. It can be a good “framework” (in a very loose meaning of this word) to modularize product components.

It can make large repositories easier to work with and understand.

Upfront limitations

CPL is strictly designed for “hosting” software and all the non-code assets are left up to the engineers to decide their location.

For example branding assets could be put into the Branding top-level directory, but on the other hand are we sure they will stay the same with major version?

Since we can agree that we consider documentation “producers” (not the produced artifacts) to be code we could also acknowledge that some assets could have their own versioned subproject.

Requirements

Versioning

CPL requires that the software is versioned inside directories whose names include the version. Recommended pattern is to name directories vMAJOR where MAJOR is either the current tagged major version or one that will be if no tags exist. It is also recommended to group the vMAJOR directories under one common directory, for example Source.

Subprojects

The vMAJOR could theoretically contain all the source code mixed together but it should be grouped and organized by their purpose.

Subproject is defined as a directory inside a versioned (vMAJOR) directory. “Versioned subproject” and “subproject” are synonymous to CPL.

To mark the purpose of a subproject, whether it is to be used as a helper or as a “container” for source that is actually exposed (or binaries created from it), it should be adequately named.

For helpers name does not matter but for source subproject it should be prefixed by project name.

For example we could have this layout:

1
2
3
4
5
6
7
8
Source/
└── v1/
    ├── Makefile
    ├── VERSION
    ├── admin/
    ├── make/
    ├── my-project-app/
    └── my-project-util/

In the above example my-project-app and my-project-lib are the source subproject and admin and make are subproject that are there only to help in building, managing and deploying the actual source subprojects.

At the and it is up to the engineer to choose if something is considered a source subproject. For example: If we have a helper subproject that all it does is hold Docker / Podman files for creating a development container what should we name it? As of now I had named them PROJECT-dev-container.

Recommendations

Make and admin

I think it is a good practice for each vMAJOR to have a Makefile, or equivalent in other build system, that will call scripts inside vMAJOR/admin directory that each take care of some small / specific task.

For example the vMAJOR/Makefile recipe for build can call admin/build_my_project_app.py and admin/build_my_project_lib.py. Each those scripts would call the “real” build system specific to the subproject they act upon.

VERSION file

It is nice to have a VERSION file in the vMAJOR directory. It can be reused by build tools and also to show what was the last version worked upon inside vMAJOR, the latest git tag can either be put on different major version or simply not be there yet.

References

See those repositories for referencing the CPL layout:

Using gzexe for shipping Racket executables

:: dev, lisp, tutorial

By: Maciej Barć

Racket executables made by raco exe are known to be quite large. One of tools that can be used to help reduce the size of produced binaries is the gzexe program.

gzexe is a tool that can compress a executable binary. It can be acquired by installing gzip on most Linux distributions (included in the app-arch/gzip package on Gentoo).

Creating a hello-world executable with Racket

Write following contents to hello-world.rkt file:

1
2
3
4
5
6
7
#lang racket/base

(define (main)
  (displayln "It REPLs. Ship it!"))

(module+ main
  (main))

To make a binary run:

1
raco exe --orig-exe -v -o hello-world hello-world.rkt

The file hello-world will be produced.

This is what file hello-world says about it:

1
2
3
hello-world: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
for GNU/Linux 3.2.0, stripped

This “small” executable weights 46 MB!

In comparison busybox weights around 2 MB.

Compressing with gzexe

Keep in mind that gzexe will overwrite the compressed file and create a backup with appended "~".

1
gzexe hello-world

And this gives us only 8,5 MB. Nice!

In comparison bazel, which is a single-binary build system written in JAVA, executable takes 33 MB on my Gentoo machine. I tried compressing it with gzexe and it reduces it only by 10%, to around 29 MB.

gzexeis not a silver bullet but with Racket exes it works very nicely.

Genkernel in 2023

:: gentoo, linux, tutorial

By: Maciej Barć

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

  1. Copy old config

    1
    2
    cd /usr/src
    cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/
    
  2. Remove old kernel build directories

    1
    rm -r linux-6.1.31-gentoo
    
  3. Run initial preparation

    1
    ( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )
    
  4. Call genkernel

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    genkernel                                                       \
        --no-menuconfig                                             \
        --no-clean                                                  \
        --no-clear-cachedir                                         \
        --no-cleanup                                                \
        --no-mrproper                                               \
        --lvm                                                       \
        --luks                                                      \
        --mdadm                                                     \
        --nfs                                                       \
        --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')"    \
        all
    
  5. Rebuild the modules

    If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

    1
    emerge -1 @module-rebuild
    

Shell script setup

:: dev

By: Maciej Barć

Good practices

Use sh

If you do not need bash features, then use sh, it is installed on every UNIX-like system.

1
#!/bin/sh

Exit on failure

Shell scripts continue even if a command returns error. To fail right away use:

1
set -e

Trap C-c

Catch Control-c and exit.

1
trap "exit 130" INT

Use script directory

Assume we are executing a script from directory /Admin, where / is the root of a given project directory.

1
2
script_path="${0}"
script_root="$(dirname "${script_path}")"

We can use ${script_root} to call other scripts form the Admin directory, but we can also use it to relate to the /.

1
2
3
project_root="$(realpath "${script_root}/../")"
echo "[INFO] Entering directory: ${project_root}"
cd "${project_root}"

So with above we can run commands form / (repository root). Like for example make and other:

1
2
make
python3 ./Admin/serve_1.py

Even better

Use Python for repository maintenance scripts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
from os import chdir
from os import path

from subprocess import run
from sys import argv

script_path = path.realpath(__file__)
script_root = path.dirname(script_path)
project_root = path.realpath(path.join(script_root, ".."))

print(f" * Entering directory: {project_root}")
chdir(project_root)

leftover_args = argv[1::]
command_arguments = ["make"] + leftover_args

cmd_string = " ".join(command_arguments)
print(f" * Executing command: {cmd_string}")
run(command_arguments, check=True)

ELisp ebuilds good practices

:: emacs, gentoo, lisp, packaging

By: Maciej Barć

Check load path

Some Elisp package compilation failures are caused by not setting the loadpath correctly. It mostly happens when you compile source from a directory that is not the current working directory. For example:

1
elisp-compile elisp/*.el

In most cases you can cd or override the S variable to set it to location where ELisp source resides.

But in other cases you can append to load path the directory with source, see:

1
BYTECOMPFLAGS="${BYTECOMPFLAGS} -L elisp" elisp-compile elisp/*.el

Do not rename auto-generated autoload file

elisp-make-autoload-file allows to name the generated autoload file. For sake of easier debugging and writing Gentoo SITEFILEs, please do not rename the generated file.

The name of that file should always be ${PN}-autoloads.el.

Use new elisp-enable-tests function

elisp-enable-tests allows to set up IUSE, RESTRICT, BDEPEND and the test runner function for running tests with the specified test runner.

The 1st (test-runner) argument must be one of:

  • buttercup — for buttercup provided via app-emacs/buttercup,
  • ert-runner — for ert-runner provided via app-emacs/ert-runner,
  • ert — for ERT, the built-in GNU Emacs test utility.

The 2nd argument is the directory where test are located, the leftover arguments are passed to the selected test runner.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
EAPI=8

inherit elisp

# Other package settings ...

SITEFILE="50${PN}-gentoo.el"
DOCS=( README.md )

elisp-enable-tests buttercup test

Remove empty SITEFILEs

Recently a feature was added to elisp.eclass that will cause build process to generate the required SITEFILE with boilerplate code if it does not exist.

So if your SITEFILE looked like this:

1
(add-to-list 'load-path "@SITELISP@")

… then, you can just remove that file.

But remember to keep the SITEFILE variable inside your ebuild:

1
SITEFILE="50${PN}-gentoo.el"

Remove pkg.el files

The *-pkg.el files are useless to Gentoo distribution model of Emacs Lisp packages and should be removed. It is as simple as adding this line to a ebuild:

1
ELISP_REMOVE="${PN}-pkg.el"

Beware that some packages will try to find their ${PN}-pkg.el file, but in most cases this will show up in failing package tests.

Use official repository

It is tedious to repackage Elpa tarballs, so use the official upstream even if you have to snapshot a specific commit.

To snapshot GitHub repos you would generally use this code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# First check if we have the correct version to prevent
# autobumping package version without changing the commit.
[[ ${PV} == *_p20220325 ]] && COMMIT=65c496d3d1d1298345beb9845840067bffb2ffd8

# Use correct URL that supports snapshots.
SRC_URI="https://github.com/domtronn/${PN}/archive/${COMMIT}.tar.gz
    -> ${P}.tar.gz"

# Override the temporary build directory variable.
S="${WORKDIR}"/${PN}-${COMMIT}

Include live version support

We do not want to be worse than the Melpa unstable :D

So, why not allow the given package to be used live?

Even if you do not push the live package to the overlay, please include support for it.

1
2
3
4
5
6
7
8
if [[ ${PV} == *9999* ]] ; then
    inherit git-r3
    EGIT_REPO_URI="https://github.com/example/${PN}.git"
else
    SRC_URI="https://github.com/example/${PN}/archive/${PV}.tar.gz
        -> ${P}.tar.gz"
    KEYWORDS="~amd64 ~x86"
fi

Ask for tags

Git is good, git tags are good. In case if upstream does not tag their package or just forgets to, kindly ask them to create a git tag when bumping Emacs package versions.

Debugging Frog blog with syntax macros

:: blog, lisp, dev, tutorial

By: Maciej Barć

Constructing debugging syntax

I wanted to echo parameter values when I set them in my blog’s frog.rkt config file.

Nothing simpler in Racket!

First I create this macro for echoing a single parameter value when it is set:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
(define-syntax-rule (verbose-set-parameter parameter-id parameter-value)
  (begin
    ;; Set the parameter.
    (parameter-id parameter-value)

    ;; Then call the parameter and print it's value.
    ;; The "'parameter-id" is special syntax
    ;; for turning a "parameter-id" identifier to a symbol.
    ;; We can also write it like:
    ;; > (quote parameter-id)
    ;; to be less confusing.
    (printf "[DEBUG] (~a ~v)\n" 'parameter-id (parameter-id))))

then, I create a wrapper for above macro that can take multiple parameter pairs:

1
2
3
4
5
(define-syntax-rule (verbose-set-parameters (parameter-id parameter-value) ...)
  (begin
    ;; Unpack a chain of "(parameter-id parameter-value)" pairs
    ;; using the "..." syntax.
    (verbose-set-parameter parameter-id parameter-value) ...))

Using the macro

Afterwards we can call it like so:

1
2
3
(verbose-set-parameters
 (current-title "XGQT's blog")
 (current-author "Maciej Barć"))

Notice that even the form of setting a parameter, that is (parameter-procedure "value"), remains the same, but in reality it is just similar to how the syntax macro pattern-matches on it.

Inspecting macro expansion

In racket-mode inside GNU Emacs we can inspect the macro expansion with racket-expand-region. Stepping through the expansion provided this result:

1
2
3
4
5
6
7
(begin
  (begin
    (current-title "XGQT's blog")
    (printf "[DEBUG] (~a ~v)\n" 'current-title (current-title)))
  (begin
    (current-author "Maciej Barć")
    (printf "[DEBUG] (~a ~v)\n" 'current-author (current-author))))

Time hackery in VirtualBox

:: vm, windows

By: Maciej Barć

Unwanted (VDI) present

Recently I was given a task to run a outdated piece of commercial software on a very outdated (in)famous operating system. Unlucky for me on the given VDI (VirtualBox disk image) the so-called “Guests Additions” that enabled “better” time synchronization were already installed.

Fighting VBox time

So, this is what I had to do to keep the old clock:

  • turn off networking card in the virtual machine configuration, because we do not want the system to use NTP or other newt service to get the “new” time,
  • disable getting the host’s clock time: VBoxManage setextradata "WinXP" "VBoxInternal/Devices/VMMDev/0/Config/GetHostTimeDisabled" 1,
  • and… this magic: VBoxManage modifyvm "WinXP" --biossystemtimeoffset "-341597644449", this thing does the very weird thing of telling the virtual machine BIOS how much to set it back in time, the offset here is total milliseconds between some old date and current date.

Getting the old date

With the Python script snippet below you will get the milliseconds to set back the VirtualBox clock to 01.01.2012 01:00.

1
2
3
4
from datetime import datetime

round((datetime.strptime("01.01.2012 01:00", "%d.%m.%Y %H:%M")
      - datetime.now()).total_seconds() * 1000)

Comparing objects in Racket

:: dev, lisp, tutorial

By: Maciej Barć

Equality methods

By implementing a method for equality equal-to? and two extraction methods equal-hash-code-of and equal-secondary-hash-code-of we can define our own object comparison rules.

For more info see Object Equality and Hashing.

Consider the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
(define integer%
  (class* object% (equal<%>)
    (super-new)

    (init-field [value 0])

    (define/public (equal-to? other-object recur)
      (= value (get-field value other-object)))

    (define/public (equal-hash-code-of hash-code)
      (hash-code value))

    (define/public (equal-secondary-hash-code-of hash-code)
      (hash-code value))))

If we create a new integer% object we can notice that it is not transparent (we can not inspect values of any of it’s fields).

1
2
(new integer%)
;;  => (object:integer% ...)

But if we compare two fresh integer% objects they will be equal.

1
2
(equal? (new integer%) (new integer%))
;;  => #true

Transparent class

A transparent cvlass is a class with the inspect expression valuye se to #false.

From Racket documentation Creating Classes:

Just as for structure types, an inspector controls access to the class’s fields, including private fields, and also affects comparisons using equal?.

Consider the following example:

1
2
3
4
5
6
7
8
(define integer%
  (class object%

    (super-new)

    (inspect #false)

    (init-field [value 0])))

If we create a new integer% object we can see it’s field values.

1
2
(new integer%)
;;  => (object:integer% 0)

And if we compare two fresh integer% objects they will be equal.

1
2
(equal? (new integer%) (new integer%))
;;  => #true