Technology

A Table of Contents shortcode for WordPress articles

A Table of Contents helps users navigate (long) blog posts. I use them on both posts and post. The desktop version always shows the table on the right side in the sidebar (using a text-widget with a shortcode). On mobile, I’ll only show it on long articles, using a shortcode under the first paragraph.

Submenu navigation for category pages

In WordPress you have two main taxonomies: categories and tags. I use categories as a taxonomy tree. That is why I want to show the submenu on the category page. It does not come out of the box, so I created something that renders the submenu items for me.

Shortcodes for simple post lists in WordPress

Recently, I worked on my theme for KeesTalksTech. To gain performance, I need to rely less on plugins, that’s why I needed a simple way to show small lists of posts in my sidebar.

I’ve created 2 short codes: one that shows recent posts, used in the new section and one that shows specific posts, used in the highlights section.

Reading 400k+ key/values from Redis fast

At Wehkamp we use Redis a lot. It is fast, available and implemented as a managed AWS service called ElastiCache. Sometimes we need to extract data from Redis, and usually I use the redis-cli to interact from the command-line. But what if you need to get the values of 400k+ keys? What would you do? Is there an effective way to query multiple key/values from Redis?

Connect to AWS ElastiCache (Redis) with redis-cli

This week I needed to query an ElastiCache instance on AWS – which is Amazons version of Redis. I could not find a decent free client to query this remote dictionary, so I ended up using redis-cli on Ubuntu. Turns out: Redis is a wonderful and powerful system to work with.

Spark: replace array with IDs with values; or: how to join objects?

This week we’ve been looking at joining two huge tables in Spark into a single table. It turns out that it is not a straightforward exercise to join data based on an array of IDs. In this blog I’ll show one way of doing this.

Streaming a Kafka Topic to a Delta Table on S3 with Spark Structured Streaming

Our data strategy specifies that we should store data on S3 for further processing. Raw S3 data is not the best way of dealing with data on Spark, though. In this blog I’ll show how you can use Spark Structured Streaming to write JSON records of a Kafka topic into a Delta table.

Easy Spark optimization for max record: aggregate instead of join?

There is a lot of code that needs to make a selection based on a maximum value. One example are Kafka reads: we only want the latest offset for each key, because that’s the latest record. What is the fastest way of doing this?

Add more color to the Python code of your Databricks notebook [retired]

Tired of the dull Python syntax highlighting in Databricks? Just copy this code into your Magic CSS editor, change it (to your own style), pin it & enjoy!

Kafka, Spark and schema inference

At Wehkamp we use Apache Kafka in our event driven service architecture. It handles high loads of messages really well. We use Apache Spark to run analysis. From time to time, I need to read a Kafka topic into my Databricks notebook. In this article, I’ll show what I use to read from a Kafka topic that has no schema attached to it. We’ll also dive into how we can render the JSON schema in a human-readable format.