Posts

An Introduction To The dbt Plugin API

In dbt-core 1.6.0, Michelle Ark made an exciting addition: a plugin system! This system lets third-party code seamlessly integrate models and artifacts into dbt Core’s compilation process, offering new possibilities for dbt users. Initially proposed by Kshitij Aranke in March 2023 as part of a discussion around creating a python sdk for dbt, the plugin system first came to fruition to support dbt Labs’ proprietary multi-project collaboration product.

Despite its quiet release, the dbt Core plugin system brings great benefits to the OSS dbt community, allowing native cross-project references without importing projects as packages, crafting synthetic model nodes, and creating custom artifacts. Though undocumented at present, this post aims to unravel the plugin interface, guiding you on creating your own plugins and tapping into the dbt plugin system’s potential.

To Hell And Back

This summer I learned a new card game called “To Hell and Back”. Similar to “Oh Hell” and “Rats!”, “To Hell and Back” is a trick-taking card game where you bid the number of tricks you intend to take, and you must take exactly that number of tricks per hand in order to win. Bid correctly, and you earn your bid and 10 extra points. Lose your bid and you get zilch. Unlike its other variations, “To Hell and Back” starts with all players being dealt one card, and each hand the number of cards dealt per hand increases until we hit our maximum for all players. In the case of one-card hands, the differences between success and failure are luck of the draw and careful bidding.

Visual Guide to DBSCAN Clustering

Clustering has become an everyday process for grouping together observations based on similar factors. This is particularly true when working with spatial data. For some of my ongoing research into applying spatial Statistics to fluorescence microscopy, I’ve been applying DBSCAN dbscan to binary images of fluorescence-tagged chromosomes to localize chromosomes. The Scikit Learn Pnython library provides a blisteringly fast DBSCAN implementation that can cluster 78 million observations in 6 seconds.

Creating Cellular Automata: Life-like Cellular Automata

In a continuation of understanding models of life, one of the most interesting cellular automatons is a two dimensional “life-like” automaton. The first life-like automaton was created by John Conway in 1970, and was published in the October 1970 release of Scientific American. The intrigue that surrounds the automaton comes from the emergence and self-organization of highly complex patterns as the simulation evolves. As a result, these automata have attracted the interest of computer scientists, mathematicians, biologists, and physicists.

Creating Cellular Automata: Elementary Cellular Automata

Sometimes the best way to learn about something is to create it. If I want to learn about an interesting subset of mathematics, eigenvalues perhaps, the explanation of the math can only go so far. I must let my pencil do the talking as I learn through construction. Programming is no different. So, when I became interested in cellular automata, I decided to make some examples of different types. Today, we can go over elementary automata.