BYOC | howto

Instructions on how to do something (step by step)

How to structure a python project with multiple entry points

In a python project, you might have multiple command-line-tools, each in their own subdirectory, that all need access to some shared code. Python will not allow one to use relative imports to directories higher than the entry-point. So starting your entry-point with import ..lib will directly fail with

ImportError: attempted relative import with no known parent package`. In this post I explore what I think is the most pythonic solution, and then some other solutions.

Multi language Hugo Site

In this post I describe a way to make a single-language hugo site, dual- (or multi-)language. There are many approaches to this, and which approach is best, depends on kind of multi-language site you want. So before you just implement these steps, please read below what you’re going to get, and then decide if this is (or is not) the best way forward. I mostly wrote this down since it’s my second time doing it, and someone (including future-me) might find these comments helpful.

Best practices for making a PyPI package (including automatic building, 2023)

I created a bunch of commandline programs in Python to help me with machine learning tasks. Last week I got so annoyed that every time I changed or extended a program, I had to also update the commandline interface for it, that I decided to do something about it. And so clargs was born, a package that takes a function, inspects it’s signature, and creates the correct commandline interface through Python’s argparse.

This post is not about this particular package, but about the lessons I learned how to create a package. It’s the latest and greatest and best practices in July 2023. This includes setting up testing (with tox and unittest), setting up GitHub actions (both so that every commit gets tested, and that a new version gets built and pushed to PyPI whenever I tag a new release). It even solves the problem that I want to have one and only one place where I specify the version number!

Hugo: Recurring events with iCalendar support

A recurring event In the Hacker Space Trójmiasto, which I a visit when I’m in town, there are regular events. We were looking for a way to put these events on the (Hugo) website. This is easy enough for simple events that happen occasionally (just hard-code the dates), but we had some extra requirements (two for the end-user, and two technical requirements):

An easy way to define recurring meetings (like the monthly picnic, or the bi-weekly board-meeting). Ideally the members could download the event-data in iCalendar (.ics) format. Of course you don’t want to involve a programmer every time that an event gets changed (e.g. an organiser may want to cancel an event last-minute), so whatever format we use, needs to be readable (and editable) by humans :) Ideally everything will work “in-Hugo”, so without an extra pre-compiler step, or something else. Just to make clear what I’m talking about here. There is a section “events”, which contains pages (in Hugo terms). A page is for a named “event-type” (like “picnic”), not a single event (like “picnic on 5 December 2022”). So a single page is for multiple dates. On this page we want to show all the dates for the event, and allow someone to download these dates as an iCalendar (.ics) file. To give you an idea of the structure, visit the events-list here. Note that, depending on when you’re reading this, the system as described in this post may not be live yet.

How to export from Blender to X3D with textures

This document is to describe the 3D Scanner App -> Blender -> X3D conversion. 3D Scanner App is a great app for making 3d scans. It can scan large areas, however on my iPhone 13 Pro it crashes when a single scan is more than 4GB large (note: this is not a hard limit; sometimes it crashes with 3.5GB, sometimes I have seen it succeed with 4.5GB scans). In the best case the scan is retained, however in my experience you should consider the scan lost in this case. It seems to hit the 4GB mark after 15 minutes of scanning. In order to be on the safe side, and have a bit of leeway, I like to plan the scans so that they take 10 minutes each. In theory you could make two scans with only 10cm overlap, and still align them later, however it depends on how distinct and changing the scene is; you will need to find the same object in both scans in order to align. I was scanning a rocky field, and I certainly needed 5 meter overlap in order to match different scans. After doing all this, the scans need to be stiched back together, this is what we use Blender for (and obviously, while we’re here, we might do any number of things; simplifying meshes, adding things, etc). Finally I want to export the scan to a X3D file, so I may use it on the web. It took me a good amount of time to get the textures correctly included in the final result, so I’m writing myself this little note on how to do this :).

How I wrote the Show on Map Safari Extension (for MacOS)

tech howto safari

A couple of weeks ago I had to look for a company to rent some Nerf guns from, for a long weekend in the Ardennes with friends. There are several companies in The Netherlands and Belgium doing just that, but I wanted to find one that was on the route. I found myself copy-pasting city and village names into Google Maps to find where all these places were located.

Was this unworkable? No. But was it slightly annoying? Yes. So as I good nerd, I decided to create a solution.

Create a static HTTPS site on your custom domain using Cloud Formation

Every now and then I find myself with the need to quickly create a small website, for instance as a static backend or a help page to one of my apps. I have a couple of requirements for this:

I want a custom (sub)domain. For now I host everything on claude-apps.com, but this could change. I want to serve my website on HTTPS (obviously). I want the website to be securely setup, in a minimal number of clicks. I want to be able to access the log files. I don’t want anything I didn’t ask for (in the form of trackers, or advertisements, or pay walls, etc).

Interface between Go 1.16 (compiled to WebAssembly) and JavaScript (syscall/js)

tech howto fzf go wasm javascript

Last month I posted a story about creating a stand-alone library from Junegunn Choi’s fzf. This was the first step in an effort to produce a version of fzf that runs in the browser. A second step would be to compile fzf-lib to run in the browser. Doing exactly that will be the content of a next post (after which there will be a couple more, looking at performance, looking at whether it’s the smart thing to do, packaging the whole thing into an npm package). In today’s post I will focus on how to make an interface between Go and JavaScript code. There is some documentation out there, but it’s few are far between (and some outdated); hopefully the explanations and examples here will help others (among which future me).

Logging HomeKit data to Google Sheets for free: walkthrough

My home is smart! My home is smart enough to switch off the lights when I’m not there (and then switch them on for a couple of hours each night to avoid break-ins). My home is smart enough to warn me if the front door is left open, or when I should open the window because the level of CO2 is getting too high. However, my home also is very forgetful…

How to embed SVG images on Medium

As you may know, I am a great fan of SVG. I don’t think that SVG gets enough credit on the web, as a small file size, infinite resolution, declarative, next gen image format. A reason why SVG is not so popular, may be that may places that allow image upload, don’t allow SVG; such is the case here at Medium as well. There may be good reasons for this (cross browser compatibility, security, SVGs acting different based on how you embed them). However, in the end, I would love to embed some SVGs in Medium.

Better Lambdas with pip packages in CloudFormation

After writing about how to create AWS Lambda Layers based on pip packages, I set out to make writing lambda functions in CloudFormation a bit easier. In the official CloudFormation AWS::Lambda::Function you either must upload your functions as zipfile to S3 and reference them from there, or, if you want them inline, you run into 3 large issues

You’re limited to 4096 bytes of code (meaning in my experience that as the code gets more complex, you start saving on comments, function names, error checking, etc, making the whole thing even harder to read and maintain). You’re limited to a single file. Maybe not so bad if you only have 4096 bytes, but still not good for readability You are limited to the builtin packages cfnresponse and boto3; installing packages from pip is not that easy. This limitation is mitigated by the pip layers mentioned before. Now, let me lead of by saying that for more complex environments (production environments), I definitely recommend having uploading packages through the S3 way. At this time you should also have multiple stacks (or even multiple accounts), and your own tooling for doing stack stuff. However for someone who just experiments a bit on their own account with a small project now and then, it’s nice to have self-contained stacks, that do the job from start to end.

AWS Lambda (python) with packages through pip (in CloudFormation)

I absolutely love CloudFormation as a tool for creating small and large items on AWS. Having code-based infrastructure, of easily maintaining your system in git, seeing differences, etc is pure joy. There are however (many) times when CloudFormation (or AWS in general) seems to miss some things. In such cases, blogs like this one should help you :).

This document is here mostly for historical reference. There is a new and better method, that doesn’t have some of the security implications present in this post. TL;DR scroll down to find a CloudFormation custom resource that builds Lambda Layers based on a list of pip packages.

Setup Neovim as Python IDE with virtualenvs -- Explanation of the steps necessary to install Neovim as a Python IDE (end of 2020)

tech howto neovim

Every now and then you feel overconfident and decide that a full reinstall of your whole system is in order. It always takes way more time than you anticipated, but in the end you’re left with something better (in the computer), and you understand the world a little bit better yourself.

A large challenge every time is to get (neo)vim setup in just the right way to work as a full fledged Python IDE (or, it should be noted that vim was never designed to be an IDE; however we want to at least set it up as a Python development system). One of the major things I struggle with is how to separate all the (python-)pieces. It’s very tempting to just install everything in global scope somewhere, but since in that case you’re using one scope for your tools and the code that you’re writing, this is asking for problems (in addition, it’s generally a bad idea to put everything together in a global scope).