Category theoretic ideas in data engineering systems design

Posted on Sun 22 September 2024 in Software • Tagged with data, math, software-engineering

I have been reading – at a very shallow level – about category theory (wiki), and thinking about its applicability to software and system design for data engineering. This post is a loose collection of thoughts on the subject, and a manifesto for employing a more rigorous language when building such systems …


Continue reading

On regulation in software engineering

Posted on Sun 21 July 2024 in Software • Tagged with software-engineering, politics

Emergency services, GPs, hospitals, transportation systems and the like are funded, at least in part, by public money. When their computer systems go down, the public is badly affected and people can get hurt, and lives can be put at risk. So what regulations and quality controls are there on …


Continue reading

Logging in Python

Posted on Sun 07 July 2024 in Python • Tagged with python, backend

Having reliable, detailed but searchable logs is an essential part of any application, especially of long-running services that need to be debugged and monitored for uptime and stability. As one of the oldest modules in the Python standard library, the logging module provides a very powerful set of tools for …


Continue reading

Object creation patterns in Python: Static factory methods

Posted on Tue 21 May 2024 in Python • Tagged with python, object-oriented-programming, design-patterns

The basic way to create and initialize an object in Python is using the default constructor, defined as the __init__() method:

class MyClass:
    def __init__(self, x: float) -> None:
        self.x: float = x
        print(f"created an object with x = {x}")


obj = MyClass(5)

Usually, initial values of instance variables …


Continue reading

In Praise of Visualization

Posted on Sat 11 May 2024 in Software • Tagged with simulation, physics, project-management

A little investment into building a GUI for live visualizations and control provides huge improvements in iteration speed.

I recently came across the excellent video Coding Adventure: Simulating Fluids wonderfully explained by Sebastian Lague. The video walks through the steps to build a simple smoothed particle hydrodynamics simulation from nothing …


Continue reading

Communicating with the past and the future

Posted on Fri 10 May 2024 in Software • Tagged with documentation, project-management

Originally written May 2022.

Introduction

In his dialogue Phaedrus, Plato has unflattering things to say about the development of writing, arguing among other things that it weakens the memory:

And so it is that you by reason of your tender regard for the writing that is your offspring have declared …


Continue reading

Are vegans vegetarians? and related questions

Posted on Sun 28 April 2024 in Software • Tagged with logic, type-theory

I recently asked the following questions:

Assuming all other requirements are the same:

  1. Is a vegan a vegetarian?
  2. Is a vegetarian a vegan?

To my (somewhat) surprise, this drew quite a lot of interest from colleagues, friends, and even people on Facebook I hadn't talked to for years, with lots …


Continue reading

Self-hosting a minimal website with nginx and HTTPS

Posted on Sat 27 April 2024 in Software • Tagged with backend, nginx, https

This website is now self-hosted on a Raspberry Pi server at home. I had struggled with nginx and HTTPS in the past, so was rather pleasantly surprised to see how straightforward it was.

I couldn't find anything on the web that walked one through the whole process, so here are …


Continue reading

for loops in Python are a bit weird

Posted on Sun 11 February 2024 in Python • Tagged with python, javascript

A quiz:

What does the following JavaScript code print and why?

for (let i = 0; i < 3; i++) {
  console.log(i);
  i -= 3
}

What does the following Python code print and why?

for i in range(3):
    print(i)
    i -= 3


The uncertainty principle and spectrograms

Posted on Tue 09 January 2024 in Physics • Tagged with physics, music, teaching

In the previous post I discussed how, when measuring the frequency (tempo) of a signal that itself changes in time, there is necessarily a tradeoff between the precision in the measured frequency and precision in the time at which the measurement is taken: A more precise measurement of frequency requires …


Continue reading