View .fit Files in the Terminal With f2i
&& [ code, rust, astronmy ] && 0 comments
Ever wanted to preview astronomical .fit files directly in your terminal without the need for DS9? Neither have I, but no matter - now you can.
It’s also just a crazy fast thumbnailer. Python is great but Rust beats it out if you can get it to work for you. In this case, ndarray and cfitsio were up to the task.
Messing with AI Bots for Fun with django-llm-poison
&& [ python, code, django ] && 4 comments
The internet is filling up with AI slop and you and I dear reader, are unwilling accomplices to this rapid decline. Big tech may have created the models but the models are trained on our words. Social media posts, forum rants and of course blogs like this one, all hoovered up into the LLM.
At least in some cases, we have a choice of whether to continue to feed the machine.
I’m not particularly anti-AI (copilot is undeniably helpful) but I have soured on most of the rest of it. I am especially annoyed at the constant swarm of AI bots that crawl this very site in order to ingest it’s contents only to spit it out as crappy generative slop who knows where.
My answer is a middle-finger of sorts to these bots. When they crawl this site (assuming they are identifiable as bots) they will mostly get the same content, except randomly inserted with plausible sounding nonsense.
To demonstrate view this very post with bot-mode enabled.
I’ve packaged it up as a reusable Django app django-llm-poison so that others running Django powered websites can use it as well.
It works by generating Markov chains from the content on the site. When a bot requests content the response is the same but with every few sentences replaced by Markov nonsense (I am aware of the irony of using very primitive generative “AI” to combat current AI). In this way, the quality of training data provided by this site is greatly diminished while remaining the same for human visitors.
Of course this site is an insignificant blip in the vast sea of information, but nonetheless, I derive a certain satisfaction for this small act of techno-disobedience.
The Arch Linux Subreddit is Horrifying
&& [ code, linux, arch ] && 1 comments
This is not a rant about elitest arch-using neckbeards (I am one!) being mean on the internet. No, the Arch Linux Subreddit is horrifying because it provides a glimpse into just how absolutely busted some of it’s user’s machines are.
I’ve been using Arch for the better part of 10 years. It’s been rock-solid the entire time. But I keep hearing about it’s supposed instability. Well, now I understand why. This is a lesson in why no matter how good your software is PEBCAK nullifies all.
As a perfect example consider this thread: How Often Do You Run sudo pacman syu?
This is subtle, but potentially the most offending post in the entire thread. This user is constantly performing what is what is called a partial upgrade. Esentially they are installing software that might depend on updated libraries that are not present on the system because they are not doing full upgrades. Very bad, and the arch wiki warns explicitly against doing this this exact thing. If I had to try to cause my system to become unstable, I couldn’t think of a better way than this.
It gets better:
This is just abuse! Not only of the machine but of Arch’s mirrors. There is so much redundancy in here it hurts. Absolutely unnecessary.
Also in this thread a lot of people saying “never” “every 3 months” or “when something breaks”. I’m not saying you need up update impulsively, but a rolling release distro works best when it’s actually kept rolling. Massive updates only a few times a year can be problematic.
It’s not my intention to make fun of less experienced people here, or the slice of them that happen to hang out on Reddit. But when you hear someone complain that Arch is unstable, keep in mind the myriad of ways it might not be the distribution’s fault.
A Repository of Themes for the Cosmic Desktop
&& [ code, linux, cosmic ] && 2 comments
System76 recently released the first alpha of their new desktop environment, COSMIC. It has some neat theming capabilities, something I’ve missed since who knows how long ago when GNOME started removing the ability to theme GTK.
The COSMIC settings app already provides the ability to import and export themes. So I thought it would be cool if there was a place where people could share thier creations. Like the gnome-look.org of old, before it was completely gimped.
Since no such site existed, I made it myself!
I didn’t want to have to rely on screenshots. They are boring, inconsistent, not always demonstrative of the actual product. So I decided that extracting the actual theme parameters and applying them to a fake “window” using CSS would be a fun approach. As a bonus, these preview windows are interactive! For example to download a theme, you press a “download theme” button which is styled in the theme itself.
In order to extract the CSS, I had to write a small rust program to parse the theme .ron (Rusty Object Notation) files and spit out CSS variables. In the future I’d like to extend this program to do other stuff (like output editor color schemes).
I manage to build and deploy the site in about 2 days. So far people seem to like it and a few have even uploaded their own themes.
They range from the scary:
To the sublime:
But it’s nice to see options other than the standard Catppuccin, Gruvbox, etc.
This was the most fun I’ve had with a weekend project for some time now. I’m looking forward to seeing how it evolves.
Landed a PR for COSMIC and I'm Irrationaly Excited About It
&& [ code, linux, cosmic ] && 0 comments
My parents gave me my first computer when I was 14 years old. I played a lot of Unreal Tournament at the time. At some point I heard about this thing called Linux that actually played UT better than Windows. So my dad drove me to Circuit City and I bought a box full of CDs that was Redhat Linux 8. What followed was weeks of running a nearly inoperable computer as I formatted and re-formatted the disk every time I ran into some issue I couldn’t fix - which was a lot.
Eventually I figured it out and I’ve been using Linux and my daily driver ever since. I did get into programming, but mostly for the web. I even made a website about Unreal Tournament which is a hilarious time capsule.
That all said, I always looked up to and admired the (in my mind) godlike developers that wrote the software I used every day. Gnome, KDE, the kernel, Tiling WMs, Firefox… these were the realms of the programmers I looked up to the most. I’ve always had a latent desire to contribute myself, but never really got comfortable with the tech to do so.
Until today! I managed to land a small PR in cosmic-files - what I would consider to be “major” open source software. Or soon to be.
Anyway, I hope this is just the beginning.
Bender DD 2: Running Gnome Apps Outside of Builder
&& [ code, linux, bender, gnome ] && 0 comments
Builder is pretty neat as an IDE for GNOME. For bootstrapping a new GNOME app, I found it to be downright amazing. Once I started getting down to writing actual code however, Builder’s limitations started to become apparent. Christian Hergert is doing an amazing job writing an IDE but wow, it must be an uphill battle trying to turn a GTK text widget into a full blown text editor. Honestly if it used embedded Neovim it could be perfect; Builder does have VIM emulation which was almost good enough, but the undo stack is buggy and caused me to lose 30+ minutes of work at one point. The second time it happened, I started to look for ways to work on Bender outside of Builder.
It’s not entirely clear what commands Builder uses to run an application. But as of this writing the bootstrapped application is a Flatpak app utilizing Meson as the build system. Armed with that knowledge, we can use the following commands to build and launch the app from it’s working directory:
flatpak-builder flatpak-build-dir com.my.App.json --force-clean --user --install
flatpak run com.my.App//master
Replace com.my.App with your application’s ID.
The first time you run these commands, you’ll run into errors.
First, check the modules section of the com.my.App.json file. You’ll find
something that looks like this:
"type": "git",
"url": "file:///home/mario/Documents"
It needs to be edited to either point to a real git repo, or changed to point to the actual project directory:
"type": "dir",
"path": "/home/mario/Documents/Bender"
If you see a validation error about a missing metadata, edit the
data/com.my.App.metainfo.xml.in file to include a summary:
<component type="desktop">
...
<summary>If you ain't crashin, you ain't smashin</summary>
</component>
There may be other issues, but the errors should make them obvious.
Now you can throw those initial commands in a Makefile or some script and you’re writing GNOME apps in your editor of choice!
Here is Bender launched by the ctrl+b shortcut in Sublime Text:
Bender Development Diary 1: A Webdev Goes Native
&& [ code, linux, bender, gnome, vala ] && 0 comments
I’ve been writing code using Linux as my main OS for over a decade now. Despite this long and fruitful relationship I have yet to do any real native development for this beloved platform.
I tell you what, I can whip up some damn good JSON APIs. But it’s time to try something new. I’ll be creating an application for Linux. Specifically for GNOME. Using GTK.
Picking a Language
From what I can tell there are a few options for developing on the GNOME platform: C, Rust, Python Javascript and Vala. The reason for this is GObject. When you are developing for GNOME, what you are working with are the C libraries that all start with G: GObject, GIO, Gee, etc. So the language you choose either needs to have bindings or be able to call into the C libraries directly.
I’m only considering languages with native bindings. I’m not elite enough to use FFI (although I would really love to program in Zig).
C: I’m not smart enough 🙃
Rust: Honestly long-term this is probably the way to go. But after looking at the GObject bindings and some GTK apps written in Rust I’ve decided I don’t really want to fight both GTK and Rust at the same time.
Python: I already write a ton of Python. So naturally, this is where I started. The GObject bindings look decent. Anything Async looks not great (network calls being the big one). Python has always been really bad at this. We have Asyncio now, but GObject does not support it. The concurrency model is based around callbacks. GNOME has it’s own network library, Libsoup. You could use Requests or HTTPX but then I think you are relegated to managing threads 🤢 (could be wrong on this). So as I see it, by choosing Python you don’t get to use any of the things that make writing Pyhon great like Asyncio or the excellent third party libraries. But you are stuck with Python’s bad parts: a runtime dependency, weak typing, etc.
Javascript: No. 💨
So that really only leaves…
Vala is a C#-esque language developed under the GNOME umbrella, designed specifically for creating GNOME apps. It compiles down to C and provides GObject bindings by default. The niche application (developing GNOME apps) is both it’s strength and weakness. The integration with GNOME technologies looks great. But Vala’s Stack Overflow presence is a barren wasteland.
This means there better integration, but less prior art to pull from. I might actually have to think and write code using documentation as a reference. It’s scary, but an opportunity for personal growth. I welcome the challenge (for now).
Is it a dead language? There is a lot of hand-wringing by people online that think so and one very infamous blog post that I could find. Maybe it is. But I don’t really care. There are some pretty awesome Vala apps under development right now and that’s good enough for me.
The creat [sic] Unix System Call
&& [ code, linux, c ] && 0 comments
The start of section 8.3 of the venerable The C Programming Language by Brain Kernighan and Dennis Ritchie reads:
Other than the default standard input, output and error, you must explicitly open files in order to read or write them. There are two system calls for this,
openandcreat[sic].
It is very rare to see [sic] in a text about software because typos in software can be fixed. So why here?
Many UNIX commands are 6 characters or less
If you’ve mucked around in the Linux command line at all, you’ve probably run into this. Why is ‘umount’ not spelled ‘unmount’? is a great SO question that goes into this. The TL;DR is that back in the day, there were real technical limitations on the number of characters that could be used in, for example, file names. In fact, the pdp-11 on which Ken Thomson wrote the original Unix used a character encoding called Radix 50 that could store a maximum of 6 characters in a single machine word. Whether this limitation was real when these system calls were written is unclear, but the practice of using abbreviated words probably persisted.
But wait, creat is only 5 characters. So why drop the ‘e’?
It might actually be a typo
In the 1984 book The UNIX Programming Environment by Brian Kernighan & Rob Pike page 204 the following footnote appears:
Ken Thompson was once asked what he would do differently if he were redesigning the UNIX system. His reply: “I’d spell creat with an e.”
My pure conjecture? Ken Thompson was probably used to thinking up short
names for commands. creat was easy - just drop the ‘e’, and he may
have not even realized that the full word create would have been only
6 characters.
Redemption?
In 2009 Ken Thompson made this commit to the Go programming language:
spell it with an “e”
All is well that ends well ☺️
Writing a HTTP Server in Zig
&& [ code, zig ] && 2 comments
I continue my Zig adventure by following up an echo server with a HTTP server.
I’ve been doing web development the majority of my career. Yet I never really thought too much about HTTP servers, much less what it would take to implement one. So it made perfect sense for me when I started to learn Zig to build one. The problem space is a nice mix of socket programming and string handling.
The source is available on Github.
To start the server:
zig run http.zig
I’m not super confident that any of the following code is “good Zig” but here it is anyways.
Constants: Errors and Mime-Types
I defined program-wide custom errors at the top where they are easy to reference. As well as an anonymous struct of structs that maps file extensions to mime-type strings. This is important later.
const std = @import("std");
const net = std.net;
const fs = std.fs;
const mem = std.mem;
const expect = std.testing.expect;
pub const ServeFileError = error{
HeaderMalformed,
MethodNotSupported,
ProtoNotSupported,
UnknownMimeType,
};
const mimeTypes = .{
.{ ".html", "text/html" },
.{ ".css", "text/css" },
.{ ".png", "image/png" },
.{ ".jpg", "image/jpeg" },
.{ ".gif", "image/gif" },
};
I really do like how Zig does error handling. The error-tuples reminds me of Golang, but without the annoying need to handle them explicitly every time they are returned.
The Main Loop
This is the main loop off the program. I probably could have factored it more.
It starts off with some pretty standard socket programming: listening on an address and then setting up a loop to listen for data on that address. The Zig standard library seems well designed here.
I also encountered my first browser behavioral peculiarity. It would seem that (at least Firefox) attempts to open a connect to the remote server of the target of an anchor tag when the user hovers over it with their mouse. Presumably this is to optimize load speed in anticipation of a click. However no data is actually sent until the user clicks, and will time out after about 10 seconds. This required a special case in this code.
The majority of the rest of the code is string parsing/formatting, followed by sending the result down the socket to the browser.
pub fn main() !void {
std.debug.print("Starting server\n", .{});
const self_addr = try net.Address.resolveIp("0.0.0.0", 4206);
var listener = try self_addr.listen(.{ .reuse_address = true });
std.debug.print("Listening on {}\n", .{self_addr});
while (listener.accept()) |conn| {
std.debug.print("Accepted connection from: {}\n", .{conn.address});
var recv_buf: [4096]u8 = undefined;
var recv_total: usize = 0;
while (conn.stream.read(recv_buf[recv_total..])) |recv_len| {
if (recv_len == 0) break;
recv_total += recv_len;
if (mem.containsAtLeast(u8, recv_buf[0..recv_total], 1, "\r\n\r\n")) {
break;
}
} else |read_err| {
return read_err;
}
const recv_data = recv_buf[0..recv_total];
if (recv_data.len == 0) {
// Browsers (or firefox?) attempt to optimize for speed
// by opening a connection to the server once a user highlights
// a link, but doesn't start sending the request until it's
// clicked. The request eventually times out so we just
// go agane.
std.debug.print("Got connection but no header!\n", .{});
continue;
}
const header = try parseHeader(recv_data);
const path = try parsePath(header.requestLine);
const mime = mimeForPath(path);
const buf = openLocalFile(path) catch |err| {
if (err == error.FileNotFound) {
_ = try conn.stream.writer().write(http404());
continue;
} else {
return err;
}
};
std.debug.print("SENDING----\n", .{});
const httpHead =
"HTTP/1.1 200 OK \r\n" ++
"Connection: close\r\n" ++
"Content-Type: {s}\r\n" ++
"Content-Length: {}\r\n" ++
"\r\n";
_ = try conn.stream.writer().print(httpHead, .{ mime, buf.len });
_ = try conn.stream.writer().write(buf);
} else |err| {
std.debug.print("error in accept: {}\n", .{err});
}
}
Parsing the header
This is straight string parsing. While the std library has some nice inclusions, coming from Python this still seems verbose and difficult. But perhaps that’s not a fair comparison.
I used structs here to give some structure to the return type of the parseHeader function.
const HeaderNames = enum {
Host,
@"User-Agent",
};
const HTTPHeader = struct {
requestLine: []const u8,
host: []const u8,
userAgent: []const u8,
pub fn print(self: HTTPHeader) void {
std.debug.print("{s} - {s}\n", .{
self.requestLine,
self.host,
});
}
};
pub fn parseHeader(header: []const u8) !HTTPHeader {
var headerStruct = HTTPHeader{
.requestLine = undefined,
.host = undefined,
.userAgent = undefined,
};
var headerIter = mem.tokenizeSequence(u8, header, "\r\n");
headerStruct.requestLine = headerIter.next() orelse return ServeFileError.HeaderMalformed;
while (headerIter.next()) |line| {
const nameSlice = mem.sliceTo(line, ':');
if (nameSlice.len == line.len) return ServeFileError.HeaderMalformed;
const headerName = std.meta.stringToEnum(HeaderNames, nameSlice) orelse continue;
const headerValue = mem.trimLeft(u8, line[nameSlice.len + 1 ..], " ");
switch (headerName) {
.Host => headerStruct.host = headerValue,
.@"User-Agent" => headerStruct.userAgent = headerValue,
}
}
return headerStruct;
}
At least we have slices!
Parsing the Request Path
Again, this is normal string parsing. We do ensure that the browser is only performing a GET over HTTP/1.1
pub fn parsePath(requestLine: []const u8) ![]const u8 {
var requestLineIter = mem.tokenizeScalar(u8, requestLine, ' ');
const method = requestLineIter.next().?;
if (!mem.eql(u8, method, "GET")) return ServeFileError.MethodNotSupported;
const path = requestLineIter.next().?;
if (path.len <= 0) return error.NoPath;
const proto = requestLineIter.next().?;
if (!mem.eql(u8, proto, "HTTP/1.1")) return ServeFileError.ProtoNotSupported;
if (mem.eql(u8, path, "/")) {
return "/index.html";
}
return path;
}
Reading the Local File
The File API seems to be well thought out in Zig. Here we translate the requested
path into a local file - or else return an error.FileNotFound which we can easily
translate into a 404 status higher up the call stack.
pub fn openLocalFile(path: []const u8) ![]u8 {
const localPath = path[1..];
const file = fs.cwd().openFile(localPath, .{}) catch |err| switch (err) {
error.FileNotFound => {
std.debug.print("File not found: {s}\n", .{localPath});
return error.FileNotFound;
},
else => return err,
};
defer file.close();
std.debug.print("file: {}\n", .{file});
const memory = std.heap.page_allocator;
const maxSize = std.math.maxInt(usize);
return try file.readToEndAlloc(memory, maxSize);
}
Speaking of 404s, this is what that looks like:
pub fn http404() []const u8 {
return "HTTP/1.1 404 NOT FOUND \r\n" ++
"Connection: close\r\n" ++
"Content-Type: text/html; charset=utf8\r\n" ++
"Content-Length: 9\r\n" ++
"\r\n" ++
"NOT FOUND";
}
Detecting the mime-type.
Nothing too interesting here, but necessary:
pub fn mimeForPath(path: []const u8) []const u8 {
const extension = std.fs.path.extension(path);
inline for (mimeTypes) |kv| {
if (mem.eql(u8, extension, kv[0])) {
return kv[1];
}
}
return "application/octet-stream";
}
Testing
Originally I was writing tests inline adjacent to the functions they were testing. I think
I might like doing that, for smaller files with a focused purpose. But for this project I moved
the files out to test_http.zig for clarity. They can be run with zig test test_http.zig.
Final Thoughts
This was an extremely fun exercise to lean more Zig. The cool thing about an HTTP server is that there is so much to implement but a lot of it isn’t very complex. However, I think I’m going to leave this one here.
A Simple Echo Server in Zig
&& [ code, zig ] && 1 comments
Recently I’ve been trying to hone my low-level programming skills. Zig is cool and it’s less painful than Rust.
Eventually I’d like to implement a HTTP server. We’ll see if I get there. As a baby step, here is a simple echo server written in Zig:
const std = @import("std");
pub fn main() !void {
std.debug.print("Starting server\n", .{});
const self_addr = try std.net.Address.resolveIp("127.0.0.1", 42069);
var listener = try self_addr.listen(.{ .reuse_address = true });
std.debug.print("Listening on {}\n", .{self_addr});
while (listener.accept()) |conn| {
std.debug.print("Accepted connection from: {}\n", .{conn.address});
_ = try conn.stream.write("Welcome to my server.\n");
var buffer: [4096]u8 = undefined;
while (true) {
const bytes_recv = try conn.stream.read(&buffer);
const chunk = buffer[0..bytes_recv];
if (chunk.len == 0) break;
std.debug.print("message: {s}", .{chunk});
_ = try conn.stream.writer().print("No u {s}", .{chunk});
}
} else |err| {
std.debug.print("error in accept: {}\n", .{err});
}
}
Here’s what happens when you telnet 127.0.0.1 42069
and enter “Learn Zig”:
❯❯❯ telnet 127.0.0.1 42069
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Welcome to my server.
Learn Zig!
No u Learn Zig!
Note this compiles and runs on Zig version 0.12-dev. It doesn’t even run on 0.11. So YMMV if you are here from the future.