Automation Blog Posts- Page 4 of 7

Task: Save Article to Markdown

Let’s use a small snippet to convert any article to Markdown. This might come in handy if you want to repost your articles to blogs that support Markdown like DEV.

Read article

Amazon S3, Automation, Node.js

Using the S3P API to copy 1.3M of 5M of AWS S3 keys

This week we had to exfil some data out of a bucket with 5M+ of keys. After doing some calculations and testing with a Bash script that used AWS cli, we decided to go a more performant route and use s3p. They claim to be 5-50 times faster than AWS cli 😊.

Read article

bash

Named Arguments in a Bash Script

Let’s face it, using positional arguments like $1 and $2 for your arguments is not very descriptive and not very flexible. There is a better way to supply arguments: with this simple trick you’ll get named arguments in your-script, which is way better 🤓.

Read article

.NET Core / C#, Automation

.NET Core & the Google Analytics Real Time Data API

At Wehkamp, we use Google Analytics extensively to measure how many users are active. It is hard to correlate active users with your other statistics that “live” in Prometheus. In this blog I’ll show how you can use the Google Real Time API to produce statistics for active users, page views and screen views.

Read article

bash, Projects, Python, Redis

Reading 400k+ key/values from Redis fast

At Wehkamp we use Redis a lot. It is fast, available and implemented as a managed AWS service called ElastiCache. Sometimes we need to extract data from Redis, and usually I use the redis-cli to interact from the command-line. But what if you need to get the values of 400k+ keys? What would you do? Is there an effective way to query multiple key/values from Redis?

Read article

bash, Redis

Connect to AWS ElastiCache (Redis) with redis-cli

This week I needed to query an ElastiCache instance on AWS – which is Amazons version of Redis. I could not find a decent free client to query this remote dictionary, so I ended up using redis-cli on Ubuntu. Turns out: Redis is a wonderful and powerful system to work with.

Read article

PowerShell, Windows

Resetting the terminal colors on Windows

I have no idea how I came to this point, but the yellow colors in my terminal (both cmd and PowerShell) are not bright yellow anymore. So I want to reset my colors back to the old values! Turns out that getting them back is not as straightforward as I had hoped…

Read article

bash

Poll JSON endpoint until value changes with bash & curl

Today I had some data coming into our event driven landscape, so I needed to know when my data was processed. As it constituted the processing of 400.000+ records (and I had more things to do), I needed a small script that kept an eye on my JSON API endpoint to see if the values are changed.

Read article

Automation, Boot Camp, Docker, Python, WSL

Visual Studio Code + Docker + Python using WSL in Windows on Boot Camp

I imagine your first thought is: why? Well, at Wehkamp we do a lot of cross platform development, but sometimes we end up with shell scripts that do stuff with Docker and Python. Usually that’s not a problem for Mac, but for Windows it’s a different thing. I have a MacBook Pro, but I’m a .NET developer, that’s why I prefer Windows, so I run Bootcamp. This article will show how to do Python development in the Windows Subsystem for Linux (WSL) using Visual Studio Code and Docker.

Read article

Automation, CI/CD, Docker

Docker on Synology: from git to running container; the easy way

My Synology disk crashed and so did my Docker set up. Basically, the CI/CD pipeline for my programs no longer existed. The wonderful thing of an awful crash like this, is that I could rethink my setup. The result is what I would call “a poor man’s CI/CD”. It’s just Git, Docker, Docker Compose and Cron. It is easy to set up and it might be all you need.

Read article