Let’s run Jupyter notebooks in a Visual Studio Code development container, so we keep our host system clean and our development setup replicable. We’re building a scraper, so let’s add support for Puppeteer (pyppeteer) as well!
Read the article Jupyter Notebooks + VSCode Dev Container with Puppeteer support
To make a setup more resilient we could allow for actions to be retried when they fail. We should not “hammer” our underlying systems, so it is wise to wait a bit before retrying (exponential backoff). Let’s see how something like this could be done in Python. Note: this only works if actions are idempotent and you can afford to wait.
Read the article Python utility function: retry with exponential backoff
Use Python to determine Dutch plural and singular forms from a CSV, with dutch-pluralizer, inflect, pandas, and custom overrides.
Read the article Pluralize and Singularize a List of Dutch Words with Python
At Wehkamp we use Redis a lot. It is fast, available and implemented as a managed AWS service called ElastiCache. Sometimes we need to extract data from Redis, and usually I use the redis-cli to interact from the command-line. But what if you need to get the values of 400k+ keys? What would you do? Is there an effective way to query multiple key/values from Redis?
Read the article Reading 400k+ key/values from Redis fast
Let’s see if we can use PIL to crop model images and resize them to a 2:3 ratio using Python Image Library (PIL). When all images on an overview are the same ratio, the overview looks way nicer. And… let’s try to make the model on the image, the center of the image.
Read the article Cropping model images using PIL
When working with images in a Python notebook I like to visualize them on a grid. Just calling display is not enough as it renders the images underneath each other. Let’s use Matplotlib to generate a single image with an image grid on it.
Read the article Plotting a grid of PIL images in Jupyter
I imagine your first thought is: why? Well, at Wehkamp we do a lot of cross platform development, but sometimes we end up with shell scripts that do stuff with Docker and Python. Usually that’s not a problem for Mac, but for Windows it’s a different thing. I have a MacBook Pro, but I’m a .NET developer, that’s why I prefer Windows, so I run Bootcamp. This article will show how to do Python development in the Windows Subsystem for Linux (WSL) using Visual Studio Code and Docker.
Read the article Visual Studio Code + Docker + Python using WSL in Windows on Boot Camp
Last week I was working on a Databricks script that needed to produce a Slack message as its final outcome. I lifted some code that used a Slack client that was PIP-installed. Unfortunately, I could not use the package on my cluster. Fortunately, the Slack API is so simple, that you don’t really need a package to post a simple message to a channel. In this article I’ll show you the simplest way of producing awesome messages in Slack.
Read the article Simple Python code to send messages to a Slack channel (without packages)
As an engineer, I love to parametrise my applications. That’s why I love the widget-feature of Databricks notebooks, which allows me to do this with a nice UI. In this blog I’ll explore how to build a True/False widget and a list widget. I also show how to validate the values of required fields.
Read the article Adding True/False and list value widgets to your Databricks notebook
At Wehkamp we use AWS Lambda to classify images on S3. The Lambda is triggered when a new image is uploaded to the S3 bucket. Currently we have over 6.400.000 images in the bucket. Now we would like to run the Lambda for all images of the bucket. In this blog I’ll show how we did this with a Python 3.6 script.
Read the article Trigger Lambda for large S3 Bucket with SQS