Tags tech

In a python project, you might have multiple command-line-tools, each in their own subdirectory, that all need access to some shared code. Python will not allow one to use relative imports to directories higher than the entry-point. So starting your entry-point with import ..lib will directly fail with

ImportError: attempted relative import with no known parent package`. In this post I explore what I think is the most pythonic solution, and then some other solutions.

I created a bunch of commandline programs in Python to help me with machine learning tasks. Last week I got so annoyed that every time I changed or extended a program, I had to also update the commandline interface for it, that I decided to do something about it. And so clargs was born, a package that takes a function, inspects it’s signature, and creates the correct commandline interface through Python’s argparse.

This post is not about this particular package, but about the lessons I learned how to create a package. It’s the latest and greatest and best practices in July 2023. This includes setting up testing (with tox and unittest), setting up GitHub actions (both so that every commit gets tested, and that a new version gets built and pushed to PyPI whenever I tag a new release). It even solves the problem that I want to have one and only one place where I specify the version number!

The go-to parser in Rust seems to be nom[^go-to-parser], and it has streaming built-in. However the good news stops there, since it’s not completely clear what this does and how it’s supposed to work. There is an (unanswered) github issue asking for a small example, another issue explaining that there are some corner cases that cannot be solved with the streaming parser, a stackoverflow question on the subject.

It took me some time before I understood how to think about streaming parsing, so I decided to write down my thoughts, both for my own future reference, as others.

Just a quick note to describe what some others had already found before. I was running a compiler (the Nordic Semi Chip Compiler) under docker. The docker image is available only for x86_64/amd64 architecture, which was fine while I was giving it all a first spin. However when I started to do more and more compilations, I started to get frustrated by the compilation times, and I was wondering how much could be saved by using an arm64 docker image.

A couple of weeks ago I had to look for a company to rent some Nerf guns from, for a long weekend in the Ardennes with friends. There are several companies in The Netherlands and Belgium doing just that, but I wanted to find one that was on the route. I found myself copy-pasting city and village names into Google Maps to find where all these places were located.

Was this unworkable? No. But was it slightly annoying? Yes. So as I good nerd, I decided to create a solution.

In past posts in this series, I looked at how to convert a Go library in order for it to work in the browser. We used both the standard go compiler and TinyGo to compile Go code to WebAssembly, and we used GopherJS to compile Go code to JavaScript. If you did not read that post, I very much advice you to do so now; this post builds on that one, and readers are assumed to know the information in that post

In this post I will describe how to compile a Go library for use in the browser. It assumed that you’re familiar with the previous post in this series, as well have at least a basic understanding of Go, JavaScript and TypeScript.

I’m going to expose fzf-lib, so that we can make calls to it from JavaScript. fzf-lib is a library-port I made earlier from Junegunn Choi’s amazing command line program fzf.

The following items are discussed in this post:

Earlier today I wanted to make some speech bubbles for an SVG image I was working on. They’re not hard to make, but getting them right takes a bit of fiddling. So I wrote down some examples, for later reference.

UPDATE: I made a small SVG speech bubble generator, based on the method described on this post. Have fun with it! Go can talk to Browsers and Node. The (svg) Go Gopher above is licensed under the Creative Commons 3.0 Attribution License, and was drawn in SVG by Renee French. Here I show how to create speech bubbles.

Last month I posted a story about creating a stand-alone library from Junegunn Choi’s fzf. This was the first step in an effort to produce a version of fzf that runs in the browser. A second step would be to compile fzf-lib to run in the browser. Doing exactly that will be the content of a next post (after which there will be a couple more, looking at performance, looking at whether it’s the smart thing to do, packaging the whole thing into an npm package). In today’s post I will focus on how to make an interface between Go and JavaScript code. There is some documentation out there, but it’s few are far between (and some outdated); hopefully the explanations and examples here will help others (among which future me).

My home is smart! My home is smart enough to switch off the lights when I’m not there (and then switch them on for a couple of hours each night to avoid break-ins). My home is smart enough to warn me if the front door is left open, or when I should open the window because the level of CO2 is getting too high. However, my home also is very forgetful…

Every now and then I write a blog post with some math in it. Math is hard enough as it is, without the added frustration of having to do a lot of effort (as blog writer) to make it look nice, or a lot of effort (as reader) to decode some math formula written in text, such as Newton’s law of gravity: F = G * (m_1*m_2)/ r^2. Luckily there are tools such as {\KaTeX} that make math look good: F = G \times \frac{m_1 \times m_2}{r^2} This post explains how to get it working on your github pages (or any html page) through the use of javascript and client-side rendering.

As you may know, I am a great fan of SVG. I don’t think that SVG gets enough credit on the web, as a small file size, infinite resolution, declarative, next gen image format. A reason why SVG is not so popular, may be that may places that allow image upload, don’t allow SVG; such is the case here at Medium as well. There may be good reasons for this (cross browser compatibility, security, SVGs acting different based on how you embed them). However, in the end, I would love to embed some SVGs in Medium.

Lately I’ve rekindled my interest in SVG. Using SVG I can declaratively create icons / diagrams / etc, in such a way that both I and the computer can read the sourcecode, and that version control systems, such as github, can easily show me the differences between files. The coverimage for this post was created as a 20 line SVG image (and then converted to PNG since Medium doesn’t support SVG images) — it also shows you that great technology such as SVG is no match for poor design skills. If you have never played with SVG, I can definitely advise you to start!

AWS CloudFormation allows the use of Custom Resources. These are great if one has to do some things outside AWS (say you create a stack and you want as part of the stack also create a github repository with a bunch of access rights). In practice however I have mostly used them to do things inside AWS that either do not have a CloudFormation interface (yet), or where the CloudFormation interface does not do what I need (for example, build a lambda layer based on pip packages). It takes quite some hassle to set it all up, but once it works, it works like a breeze!

I absolutely love CloudFormation as a tool for creating small and large items on AWS. Having code-based infrastructure, of easily maintaining your system in git, seeing differences, etc is pure joy. There are however (many) times when CloudFormation (or AWS in general) seems to miss some things. In such cases, blogs like this one should help you :).

This document is here mostly for historical reference. There is a new and better method, that doesn’t have some of the security implications present in this post. TL;DR scroll down to find a CloudFormation custom resource that builds Lambda Layers based on a list of pip packages.

Generally when making things in 3D, we like to talk about objects. For instance a cube, or a pyramid. When making a 3D modelling library, it therefore pays to have a very good idea of what an object actually is, and what different kinds of objects one can have. Also, it seems that other programs use the term manifold object; I have done my best to explain what that means and what MacPy’s thoughts about this are.

I’ve always been the kind of person who cannot resist a good challenge, especially in programming. In addition, I’m one who likes to know what is going on, not just one to accept magic, accept bugs, accept that the computer knows better than me.

People who grew up with WordPerfect on DOS remember the days that in a word processor, you could look “under water” and see exactly what was going on. I remember “programming” logos and icons in javascript + canvas, rather than using photoshop — these days I try to exclusively use SVG. All so that my products are human-readable code, that I can check in, see diffs between versions, etc.

Last week I got my new Apple Silicon Macbook Pro M1. I was very excited to do some very simple tests to see how fast python could calculate the Morton Code for a 3D case. I need this for a small project I’m working on, and I found out in a previous iteration that this is taking quite some CPU time. My previous MacBook is a 13" with 2.7GHz Quad Core i7 — the top of the 13" line in 2019; the promises out there suggest that the M1 should be faster. Let’s see.

Every now and then you feel overconfident and decide that a full reinstall of your whole system is in order. It always takes way more time than you anticipated, but in the end you’re left with something better (in the computer), and you understand the world a little bit better yourself.

A large challenge every time is to get (neo)vim setup in just the right way to work as a full fledged Python IDE (or, it should be noted that vim was never designed to be an IDE; however we want to at least set it up as a Python development system). One of the major things I struggle with is how to separate all the (python-)pieces. It’s very tempting to just install everything in global scope somewhere, but since in that case you’re using one scope for your tools and the code that you’re writing, this is asking for problems (in addition, it’s generally a bad idea to put everything together in a global scope).