Claude’s blog: everything but the cafeine

Claude's blog: everything but the cafeine

Reinoud Elhorst (nick: Claude)

20+ years professional development and management experience in startups; always working on hobby projects. Happy cyclist, SUPper, swimmer.

Coding, crazy projects, modelling, leisure, the occasional travel info or product review. It’s all inclusive here; just bring your own coffee.

You can read more about me here.

Recent posts

How to structure a python project with multiple entry points

In a python project, you might have multiple command-line-tools, each in their own subdirectory, that all need access to some shared code. Python will not allow one to use relative imports to directories higher than the entry-point. So starting your entry-point with import ..lib will directly fail with

ImportError: attempted relative import with no known parent package`. In this post I explore what I think is the most pythonic solution, and then some other solutions.

Multi language Hugo Site

In this post I describe a way to make a single-language hugo site, dual- (or multi-)language. There are many approaches to this, and which approach is best, depends on kind of multi-language site you want. So before you just implement these steps, please read below what you’re going to get, and then decide if this is (or is not) the best way forward. I mostly wrote this down since it’s my second time doing it, and someone (including future-me) might find these comments helpful.

Best practices for making a PyPI package (including automatic building, 2023)

I created a bunch of commandline programs in Python to help me with machine learning tasks. Last week I got so annoyed that every time I changed or extended a program, I had to also update the commandline interface for it, that I decided to do something about it. And so clargs was born, a package that takes a function, inspects it’s signature, and creates the correct commandline interface through Python’s argparse.

This post is not about this particular package, but about the lessons I learned how to create a package. It’s the latest and greatest and best practices in July 2023. This includes setting up testing (with tox and unittest), setting up GitHub actions (both so that every commit gets tested, and that a new version gets built and pushed to PyPI whenever I tag a new release). It even solves the problem that I want to have one and only one place where I specify the version number!

Rust's nom as a streaming parser

The go-to parser in Rust seems to be nom[^go-to-parser], and it has streaming built-in. However the good news stops there, since it’s not completely clear what this does and how it’s supposed to work. There is an (unanswered) github issue asking for a small example, another issue explaining that there are some corner cases that cannot be solved with the streaming parser, a stackoverflow question on the subject.

It took me some time before I understood how to think about streaming parsing, so I decided to write down my thoughts, both for my own future reference, as others.

Docker on M1/M2 macbook performance

Just a quick note to describe what some others had already found before. I was running a compiler (the Nordic Semi Chip Compiler) under docker. The docker image is available only for x86_64/amd64 architecture, which was fine while I was giving it all a first spin. However when I started to do more and more compilations, I started to get frustrated by the compilation times, and I was wondering how much could be saved by using an arm64 docker image.

Hugo: Recurring events with iCalendar support

A recurring event In the Hacker Space Trójmiasto, which I a visit when I’m in town, there are regular events. We were looking for a way to put these events on the (Hugo) website. This is easy enough for simple events that happen occasionally (just hard-code the dates), but we had some extra requirements (two for the end-user, and two technical requirements):

An easy way to define recurring meetings (like the monthly picnic, or the bi-weekly board-meeting). Ideally the members could download the event-data in iCalendar (.ics) format. Of course you don’t want to involve a programmer every time that an event gets changed (e.g. an organiser may want to cancel an event last-minute), so whatever format we use, needs to be readable (and editable) by humans :) Ideally everything will work “in-Hugo”, so without an extra pre-compiler step, or something else. Just to make clear what I’m talking about here. There is a section “events”, which contains pages (in Hugo terms). A page is for a named “event-type” (like “picnic”), not a single event (like “picnic on 5 December 2022”). So a single page is for multiple dates. On this page we want to show all the dates for the event, and allow someone to download these dates as an iCalendar (.ics) file. To give you an idea of the structure, visit the events-list here. Note that, depending on when you’re reading this, the system as described in this post may not be live yet.

How to export from Blender to X3D with textures

This document is to describe the 3D Scanner App -> Blender -> X3D conversion. 3D Scanner App is a great app for making 3d scans. It can scan large areas, however on my iPhone 13 Pro it crashes when a single scan is more than 4GB large (note: this is not a hard limit; sometimes it crashes with 3.5GB, sometimes I have seen it succeed with 4.5GB scans). In the best case the scan is retained, however in my experience you should consider the scan lost in this case. It seems to hit the 4GB mark after 15 minutes of scanning. In order to be on the safe side, and have a bit of leeway, I like to plan the scans so that they take 10 minutes each. In theory you could make two scans with only 10cm overlap, and still align them later, however it depends on how distinct and changing the scene is; you will need to find the same object in both scans in order to align. I was scanning a rocky field, and I certainly needed 5 meter overlap in order to match different scans. After doing all this, the scans need to be stiched back together, this is what we use Blender for (and obviously, while we’re here, we might do any number of things; simplifying meshes, adding things, etc). Finally I want to export the scan to a X3D file, so I may use it on the web. It took me a good amount of time to get the textures correctly included in the final result, so I’m writing myself this little note on how to do this :).

How I wrote the Show on Map Safari Extension (for MacOS)

A couple of weeks ago I had to look for a company to rent some Nerf guns from, for a long weekend in the Ardennes with friends. There are several companies in The Netherlands and Belgium doing just that, but I wanted to find one that was on the route. I found myself copy-pasting city and village names into Google Maps to find where all these places were located.

Was this unworkable? No. But was it slightly annoying? Yes. So as I good nerd, I decided to create a solution.

Create a static HTTPS site on your custom domain using Cloud Formation

Every now and then I find myself with the need to quickly create a small website, for instance as a static backend or a help page to one of my apps. I have a couple of requirements for this:

I want a custom (sub)domain. For now I host everything on claude-apps.com, but this could change. I want to serve my website on HTTPS (obviously). I want the website to be securely setup, in a minimal number of clicks. I want to be able to access the log files. I don’t want anything I didn’t ask for (in the form of trackers, or advertisements, or pay walls, etc).

WorkOutDoors: The workout app I would have written for myself

Ever since I got my first Apple Watch, I looked for apps to track workouts. In the beginning, any app would do. The most important thing was to give me feedback on how my workout was going; all the actual work was being done on the iPhone. Over time however two things changed.

First, I started to get more serious about my workouts; I would run the occasional 10k, or half marathon, and wanted to do training for this. This meant more complex workouts: intervals training, runs in hilly terrain, etc.

Performance of a Go library (fzf-lib) in the browser

In past posts in this series, I looked at how to convert a Go library in order for it to work in the browser. We used both the standard go compiler and TinyGo to compile Go code to WebAssembly, and we used GopherJS to compile Go code to JavaScript. If you did not read that post, I very much advice you to do so now; this post builds on that one, and readers are assumed to know the information in that post

Converting SVG into PNG

In one of my previous posts I mentioned that I was still looking for the best way to convert SVG into PNG. If you just Google “Convert SVG to PNG” you will get hundreds of hits, from Inkscape, to online tools, to photo editors that can import SVGs (I know Pixelmator Pro claims to support SVG imports). For simple SVGs these options may work, but I’m tend to explore the boundaries of what is possible in SVGs. Also, if you offer to convert an SVG to an PNG, you better (at least) offer me a way to specify the resolution that I want my PNG to be; you cannot just assume that the viewBox of the SVG says anything about the resolution it’s meant to be shown at.

Using a Go library (fzf-lib) in the browser

In this post I will describe how to compile a Go library for use in the browser. It assumed that you’re familiar with the previous post in this series, as well have at least a basic understanding of Go, JavaScript and TypeScript.

I’m going to expose fzf-lib, so that we can make calls to it from JavaScript. fzf-lib is a library-port I made earlier from Junegunn Choi’s amazing command line program fzf.

The following items are discussed in this post:

Making speech bubbles in SVG

Earlier today I wanted to make some speech bubbles for an SVG image I was working on. They’re not hard to make, but getting them right takes a bit of fiddling. So I wrote down some examples, for later reference.

UPDATE: I made a small SVG speech bubble generator, based on the method described on this post. Have fun with it! Go can talk to Browsers and Node. The (svg) Go Gopher above is licensed under the Creative Commons 3.0 Attribution License, and was drawn in SVG by Renee French. Here I show how to create speech bubbles.

Interface between Go 1.16 (compiled to WebAssembly) and JavaScript (syscall/js)

Last month I posted a story about creating a stand-alone library from Junegunn Choi’s fzf. This was the first step in an effort to produce a version of fzf that runs in the browser. A second step would be to compile fzf-lib to run in the browser. Doing exactly that will be the content of a next post (after which there will be a couple more, looking at performance, looking at whether it’s the smart thing to do, packaging the whole thing into an npm package). In today’s post I will focus on how to make an interface between Go and JavaScript code. There is some documentation out there, but it’s few are far between (and some outdated); hopefully the explanations and examples here will help others (among which future me).

Creating Fzf into a library: fzf-lib

Over the past coupe of weeks I converted fzf into a go library, to be used in other projects.

See fzf-lib on  Github Introduction Ever since I discovered Junegunn Choi’s fzf I’ve been a huge fan. It has all the properties of a great product: extremely low learning curve, intuitive usage (just start typing, and pick your result), intuitive result order (the thing that you most likely wanted is on top), and extremely fast, even when searching through millions of lines of text. Plus, once you get the hang of things, it does have the power to use smartly chosen meta-characters that make your search so much easier. It does all this through a highly configurable terminal interface, with helpful previews, and a vim plugin. I think I use fzf hundreds of times a day, both during work and free time!

Using KaTeX on github pages

Every now and then I write a blog post with some math in it. Math is hard enough as it is, without the added frustration of having to do a lot of effort (as blog writer) to make it look nice, or a lot of effort (as reader) to decode some math formula written in text, such as Newton’s law of gravity: F = G * (m_1*m_2)/ r^2. Luckily there are tools such as {\KaTeX} that make math look good: F = G \times \frac{m_1 \times m_2}{r^2} This post explains how to get it working on your github pages (or any html page) through the use of javascript and client-side rendering.

How to embed SVG images on Medium

As you may know, I am a great fan of SVG. I don’t think that SVG gets enough credit on the web, as a small file size, infinite resolution, declarative, next gen image format. A reason why SVG is not so popular, may be that may places that allow image upload, don’t allow SVG; such is the case here at Medium as well. There may be good reasons for this (cross browser compatibility, security, SVGs acting different based on how you embed them). However, in the end, I would love to embed some SVGs in Medium.

Ways to use SVG in your html page

Lately I’ve rekindled my interest in SVG. Using SVG I can declaratively create icons / diagrams / etc, in such a way that both I and the computer can read the sourcecode, and that version control systems, such as github, can easily show me the differences between files. The coverimage for this post was created as a 20 line SVG image (and then converted to PNG since Medium doesn’t support SVG images) — it also shows you that great technology such as SVG is no match for poor design skills. If you have never played with SVG, I can definitely advise you to start!

Better Lambdas with pip packages in CloudFormation

After writing about how to create AWS Lambda Layers based on pip packages, I set out to make writing lambda functions in CloudFormation a bit easier. In the official CloudFormation AWS::Lambda::Function you either must upload your functions as zipfile to S3 and reference them from there, or, if you want them inline, you run into 3 large issues

You’re limited to 4096 bytes of code (meaning in my experience that as the code gets more complex, you start saving on comments, function names, error checking, etc, making the whole thing even harder to read and maintain). You’re limited to a single file. Maybe not so bad if you only have 4096 bytes, but still not good for readability You are limited to the builtin packages cfnresponse and boto3; installing packages from pip is not that easy. This limitation is mitigated by the pip layers mentioned before. Now, let me lead of by saying that for more complex environments (production environments), I definitely recommend having uploading packages through the S3 way. At this time you should also have multiple stacks (or even multiple accounts), and your own tooling for doing stack stuff. However for someone who just experiments a bit on their own account with a small project now and then, it’s nice to have self-contained stacks, that do the job from start to end.

AWS Custom Resources security "trap" — or why it’s bad to give lambda execute rights to non-admin

AWS CloudFormation allows the use of Custom Resources. These are great if one has to do some things outside AWS (say you create a stack and you want as part of the stack also create a github repository with a bunch of access rights). In practice however I have mostly used them to do things inside AWS that either do not have a CloudFormation interface (yet), or where the CloudFormation interface does not do what I need (for example, build a lambda layer based on pip packages). It takes quite some hassle to set it all up, but once it works, it works like a breeze!

AWS Lambda (python) with packages through pip (in CloudFormation)

I absolutely love CloudFormation as a tool for creating small and large items on AWS. Having code-based infrastructure, of easily maintaining your system in git, seeing differences, etc is pure joy. There are however (many) times when CloudFormation (or AWS in general) seems to miss some things. In such cases, blogs like this one should help you :).

This document is here mostly for historical reference. There is a new and better method, that doesn’t have some of the security implications present in this post. TL;DR scroll down to find a CloudFormation custom resource that builds Lambda Layers based on a list of pip packages.

MacPy3D — 3D Model As Code for Python

I’ve always been the kind of person who cannot resist a good challenge, especially in programming. In addition, I’m one who likes to know what is going on, not just one to accept magic, accept bugs, accept that the computer knows better than me.

People who grew up with WordPerfect on DOS remember the days that in a word processor, you could look “under water” and see exactly what was going on. I remember “programming” logos and icons in javascript + canvas, rather than using photoshop — these days I try to exclusively use SVG. All so that my products are human-readable code, that I can check in, see diffs between versions, etc.

Timing Morton code on Python on Apple Silicon

Last week I got my new Apple Silicon Macbook Pro M1. I was very excited to do some very simple tests to see how fast python could calculate the Morton Code for a 3D case. I need this for a small project I’m working on, and I found out in a previous iteration that this is taking quite some CPU time. My previous MacBook is a 13" with 2.7GHz Quad Core i7 — the top of the 13" line in 2019; the promises out there suggest that the M1 should be faster. Let’s see.

Cycling in Wrocław, Poland

Being Dutch, I was born on a bicycle. As a youngster, your bike is your only legal faster-than-foot way; later. I started to appreciate the fact that I never had problems parking, and even later I realised that cycling kept me in shape, and gave me lots of environmental karma points. Most of all however, I love the wind in my face, the feeling of freedom, the rest it gives me, whether on a 5 minute shopping trip, or a 12 hours-a-day roadtrip to Berlin or Paris. So whenever I go somewhere, especially for a longer trip, I like to know what my cycling options are, what the rules are (officially, and, more importantly, practically), whether car-drivers are going to be respectful, or will try to kill you. And, obviously, how to find the best roads, those roads that are quiet, but don’t require you to cycle through mud, end up in a field with only option to turn around, etc. In order for others to make this easier, I’m sharing my experiences.

Setup Neovim as Python IDE with virtualenvs -- Explanation of the steps necessary to install Neovim as a Python IDE (end of 2020)

Every now and then you feel overconfident and decide that a full reinstall of your whole system is in order. It always takes way more time than you anticipated, but in the end you’re left with something better (in the computer), and you understand the world a little bit better yourself.

A large challenge every time is to get (neo)vim setup in just the right way to work as a full fledged Python IDE (or, it should be noted that vim was never designed to be an IDE; however we want to at least set it up as a Python development system). One of the major things I struggle with is how to separate all the (python-)pieces. It’s very tempting to just install everything in global scope somewhere, but since in that case you’re using one scope for your tools and the code that you’re writing, this is asking for problems (in addition, it’s generally a bad idea to put everything together in a global scope).