Blog

• 1: Learning
• 1.1: My Minimal Rust Operating System
• 1.1.1: Introduction
• 1.1.2: Part 1 - A Freestanding Rust Binary
• 1.1.3: Part 2 - A Minimal Rust Kernel
• 2: My Projects
• 2.1: Homieclips

1 - Learning

The documents here are for projects that I have taken up in order to learn something I am interested in. They typically follow a guide created by someone else that I follow along.

1.1 - My Minimal Rust Operating System

1.1.1 - Introduction

What am I doing?

I am going through the Writing an OS in Rust guides created by Phil Oppermann.

I am hosting the source code in this repo. For each part of the original guide I will create an article on what I did and any notable changes or thoughts in regards to what was originally done, I will also create a git branch to save what I had done for each part of the guide.

Why am I doing this?

I want to learn lower level systems programming and I want to learn it through Rust as I am an interested in learning the capapilities of the language and if I want to switch to C/C++ it will be easier learning the fundamentals through Rust first.

What has been setup before starting the tutorial.

• Created the initial Rust barebones project.
• Since I use the Nix package manager I have created a Flake.nix that I will use with nix-direnv to control my developer environments
• I have created the repository on Github here and committed all the current changes to it.

Next

Now I can start with the guide and will leave any updates on further pages.

1.1.2 - Part 1 - A Freestanding Rust Binary

Rust Analyzer and no_std

Starting this guide everything appears to be the working as intended with the guide. So far the only thing worth noting is that the Rust Analyzer is throwing errors regarding the use of the custom panic implementation as it still refers to the standard library despite the use of #![no_std].

Looking online there does appear to be some possible fixes and I may apply them if none of what is applied in future tutorials fixes it.

Nix issues with a different build targets

For my developer environment I use nix-direnv with a flake.nix which will install and setup any special packages as well as the Rust version I want.

Previously I was using the nightly version of Rust in order to use no_std however I couldn’t install extra targets though the flake.nix directly. Luckily the flake was already setup to install Rust according to a supplied rust-toolchain.toml file.

So I created the following rust-toolchain.toml file and committed it to git so Nix could properly detect it.

[toolchain]
channel = "nightly-2024-12-20"
components = ["rust-src", "rustfmt", "clippy"]
targets = ["thumbv7em-none-eabihf"]
profile = "default"  

When that was complete direnv reloaded and I was able to cross compile the existing example code.

1.1.3 - Part 2 - A Minimal Rust Kernel

Changes made before starting part 2 of the tutorial

Before starting part two I noticed that there were two issues I had to look at.

Unsafe no_mangle

When attempting to build with the thumbv7em-none-eabihf target it was getting an error saying that #[no_mangle] needed to be wrapped in an unsafe block. The original guide made no mention of this, however after a quick google search I was able to find the relevant RFC to consider some attributes unsafe here. I looked at the linked PR and found that the change was made to nightly as recently as Oct 31 2024 and since I was on a nightly version from the end of December 2024 it means it affected me.

I then made the change to the code like so

// main.rs
#![no_std]
#![no_main]
use core::panic::PanicInfo;

#[unsafe(no_mangle)]
pub extern "C" fn _start() -> ! {
    loop {}
}

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

Rust Analyzer still referencing the std library

Rust analyzer was giving false-positive errors as it still references the std library despite the use of #![no_std]. In order to fix this I would need to tell rust analyzer to not check on all targets. Since I use the Helix modal editor for dev work I can create a local languages.toml file with any changes that are relevant for the projects.

After creating .helix/languages.toml in the base of my repo and adding the following contents rust analyzer no longer reported errors based on using the std library.

[language-server.rust-analyzer.config.check]
allTargets = false
targets = "aarch64-apple-darwin"

Actually working on Part 2

Thankfully while working on part two there were no issues involving my Nix setup or using a newer version. I only had to add the llvm-tools-preview to my rust-toolchain.toml file so that the feature was added as a part of my dev shell and then add cargo-bootimage to the list of Nix packages that should be available as well.

Thoughts

• I had a look at the later versions of the bootloader library in case trying to use it would be worth it but as of right now it seems too large a task. Will possibly look at it later when I have more experience with OS dev.

2 - My Projects

2.1 - Homieclips

Goals

Fundamental

• Allow my friends to upload short clips that they have created
• Be able to stream the videos from any device quickly and easily
• A pleasing user experience
• Hosting clips in an object storage solution like S3
• Discord authentication and sign in
• Automatic transcoding of uploaded files to HLS including multiple video qualities
• When uploading games naming the game they belong to
• Only those I choose can sign in via Discord and access the site

Future

• Use the IGDB to provide the list of games to select as well as use the cover art for categorising
• Include playlists for viewing
• Have the ability to download the clips

Services

Main Backend & Frontend: Homie Clips

Repository Link: https://github.com/Cal-lifornia/homieclips

This is both the main frontend and backend of the Homie Clips app which is being built in Golang.

The backend is an Api built in Golang with the Gin library. The main purpose of the backend is to handle the CRUD actions between the frontend and the main Mongo database. These include:

• Creating, deleting, updating and reading of the clips data from the database
• Authentication via Auth0
• Login and logout logic

The frontend of the app is also served via Gin and the frontend is built with vanilla javascript and Tailwind CSS.

To stream videos the Video.js library is being used as it can natively stream HLS and has lots of customisable options. For that to work the data for the main home page is fetched and the link for the video is fetched by adding the unique object id onto the S3 url for the video.

The frontend still requires a fair amount of work, with the ability to upload clips being the main missing feature as I am aiming to have the ability to use multipart uploads directly from the client to S3 so that the main server isn’t being used for that. This is because the main server will be hosted at home.

HSL Transcoder

Github Link: https://github.com/Cal-lifornia/homieclips-hsl-transcoder

The HSL Transcoder service is a background service that utilises FFMPEG, AWS S3 and SQS libraries to automatically transcode any files that have been uploaded to the S3 Homie Clips bucket to HSL fragments in multiple resolutions. Those fragments are then uploaded to the S3 bucket and marked as complete in the MongoDB, the original uploaded file is then copied to another location in the bucket so that it is archived.

Currently the service works as intended however two main features are missing:

• Transcoding will only really work for 4K and QHD, this is because a function to detect the resolution of the original clip and then use the correct FFMPEG command needs to be added.
• Automatic thumbnail generation and upload using FFMPEGs thumbnail filter

Database

The database for Homie Clips uses MongoDB, originally Mongo was only being used for testing as I hadn’t used it before, but I found that it was very useful and since I don’t need it to be lighting quick it is suitable for the project.

The key part of the database is the Clips collection, which has each clips object_id, this id is used to name the clip that is uploaded to the S3 Homie Clips bucket, so it allows the clip to be easily found and used in the frontend without having to use a separate S3 library. In the future this will also allow changes to happen in the database and as long the object_id remains, base functionality will stay.