Skip to main content

About

Hi there! I'm Remy, a Linux/UNIX sysadmin, my primary focus is on building high available cloud environments in OpenStack, Amazon, Microsoft Azure or on premise with Ansible and Terraform (or any other configuration management tool). My keywords are OpenStack, cloud, virtualization, high availability, scaling and clustering, security, (Open)SSL, Python, Powershell, Bash and general linux/UNIX system administration. Besides that I also develop software in Python.

I currently work for VolkerWessels, the largest Dutch civil engineering construction company as an (Azure) cloud specialist.

I've worked as a sysadmin (in a devops role) for the Erasmus University Medical Center (a large hospital and medical university in Rotterdam), Digidentity (a Dutch certificate authority, the company that develops DigiD and other Ruby on Rails applications), CloudVPS (an OpenStack Cloud provider, ISP, managed service provider and datacenter) and a few other smaller companies. I like to design, build, document and manage large, complex and high available systems. I'm a team player that loves to work with configuration management tools and version control systems. If I do something more than three times I automate it. Planning, documentation, monitoring and backups are things I do first, not when the time allows it later. Last but not least I have an interest in legacy systems like the PDP-11, PDP-8 and operating systems like OpenVMS, HP-UX and old UNIX systems.

To contact me, see my resume, get my GPG or S/MIME key see the about page.

This is my personal website, please do note that these articles do not reflect or are based on work, opinions or policies of any of my (previous) employers. Any resemblance to reality is pure coincidence.

This site started in 2006 as my form of (public) documentation. It has grown to include software, tutorials, snippets and articles on linux/UNIX, system administration and everything related with over 10.000 unique visitors a day as of 2018-06. The URL is the phonetic way you say my name (Ray-Mii), since non-Dutch speakers always have trouble with the correct pronunciation.


Latest Items

Line total (up+down sum) in PHP Network Weathermap

13-11-2018 | Remy van Elst

With PHP Network Weathermap you can create a birds-eye view of network components from your monitoring system (like LibreNMS, Cacti or anything else with an RRD database). It can display simple maps with components and links between, showing up and down traffic, but also complex systems with custom components, like Nagios status, temperature or other information. For network and system administrators seeing the seperate in and out traffic of a link is fine, we can sum up two numbers. A co worker filling the role of service manager asked me if it was possible to sum up in and out and show that, including the scale (different colours depending on link usage). This co worker is not interested in the seperate up/down link speed but wants to know how much traffic a location is using in total. Using a clever workaround, you can display a line's total usage, including the scale. This article also gives some more tips on weathermap, colouring and scale.

Read more...

Three new NitroKeys! Nitrokey Pro 2, Storage 2 and a FIDO-U2F Nitrokey

08-11-2018 | Remy van Elst

Last week I received several newsletters from Nitrokey. As you might know, I'm a fan of their (mostly open source) hardware security devices. Their newsletters introduced two new keys, the Nitrokey Pro 2 and the Nitrokey FIDO-U2F key. On their website I also saw the Nitrokey Storage Pro 2. This article is a summary of the newsletters and goes over the new features in the new hardware. It boils down to a new OpenPGP smartcard version (3.3, it was 2.1) in the Nitrokey Pro 2 and Storage 2. The FIDO-U2F device is an entirely new Nitrokey (with a button).

Read more...

Use Ubuntu behind a Microsoft ForeFront TMG proxy with cntlm

27-10-2018 | Remy van Elst

Recently I had to deploy a few machines in a network where outgoing network access was forced through a Microsoft Forefront TMG proxy. For all the Windows clients this went automatically due to domain policies, for Linux this has to be set up manually. Defining the proxy in /etc/environment was not enough since NTML authentication is required, which is not supported by default. I found cntlm, a piece of software which acts as a local proxy, translating all requests to authenticated NTLM requests to your upstream proxy. This guide covers the (offline) installation, setup, getting the correct password hash and system-wide configuration. It should work on a desktop as well, but I did not test that.

Read more...

Encrypt and decrypt files to public keys via the OpenSSL Command Line

25-10-2018 | Remy van Elst

This small tutorial will show you how to use the openssl command line to encrypt and decrypt a file using a public key. We will first generate a random key, encrypt that random key against the public key of the other person and use that random key to encrypt the actual file with using symmetric encryption.

Read more...

Find files in tar archives and extract specific files from tar archives

17-10-2018 | Remy van Elst

This is a small tip, to find specific files in tar archives and how to extract those specific files from said archive.

Read more...

Reddit Gold for Caldera Openlinux 1.2

26-09-2018 | Remy van Elst

Someone liked my Reddit post regarding a few old CD's I found of Caldera Openlinux 1.2, including source code and floppies so much they gilded it. I got some special internet points today.

Read more...

Service checks in LibreNMS (http, all other Nagios plugins)

10-09-2018 | Remy van Elst

LibreNMS is becoming one of my favorite monitoring tools. Setup and getting started is easy and it has enough advanced options and tunables. I recently discovered that LibreNMS is able to check services as well. Services, in this context, means, executing Nagios plugins (like check_http, check_ping, etc). This allows you to check services that SNMP does not cover by default, like HTTP(s) health checks, certificate expiry, tcp port checks (e.g. rdp) and anything for which you can write a Nagios plugin yourself. The performance data, if available, is graphed automatically. Alerting is done with the regular LibreNMS alerts. This guide covers the setup of services (it's not enabled by default) and a few basic checks, like an http health check, certificate expiry and SSH monitoring.

Read more...

tping - ping with a timestamp

03-09-2018 | Remy van Elst

tping is a bash alias I once got from an old co-worker. It's ping, but with a timestamp. Instead of looking at the increased icmp_seq number you now have a timestamp.

Read more...

Linux on Microsoft Azure? Disable this built-in root-access backdoor (wa-linux-agent)

22-08-2018 | Remy van Elst

Are you running Linux on Microsoft Azure? Then by default anyone with access to your Azure portal can run commands as root in your VM, reset SSH keys, user passwords and SSH configuration. This article explains what the backdoor (wa-linux-agent) is, what it is meant to do, how it can be disabled and removed and what the implications are. OpenStack/QEMU also have an agent/backdoor which is covered in this article as well.

Read more...

Python script to talk to LibreNMS API and get alerts and hosts

08-08-2018 | Remy van Elst

This script talks to the LibreNMS API to receive a list of down devices and alerts. The LibreNMS dashboard provides widgets for alerts and host statusses, but there is no easy way to access that output via the API. Using Python I was able to get certain information and output it as HTML or text using PrettyTable. It can be included in other systems or be used in a chain of monitoring customizations. z

Read more...

nginx 1.15.2, ssl_preread_protocol, multiplex HTTPS and SSH on the same port

06-08-2018 | Remy van Elst

The NGINX blog recently had a nice article on a new feature of NGINX 1.15.2, $ssl_preread_protocol. This allows you to multiplex HTTPS and other SSL protocols on the same port, or as their blog states, 'to distinguish between SSL/TLS and other protocols when forwarding traffic using a TCP (stream) proxy'. This can be used to run SSH and HTTPS on the same port (or any other SSL protocol next to HTTPS). By running SSH and HTTPS on the same port, one can circumvent certain firewall restrictions. If the session looks like HTTPS, nginx will handle it, if it looks like something else, it will forward it to the configured other program. I used to use SSHL to get this functionality, but now it's built into the nginx webserver. This small guide will cover the installation of the latest version of nginx on Ubuntu (16.04) and configuring this multiplex feature.

Read more...

Site updates for accessibility, text only pages and skip to main content

01-08-2018 | Remy van Elst

I've made some new improvements to this website. Raymii.org is generated using my self-written static site generator named ingsoc, the new features are focussed on accessibility. If you are using a screen reader or command-line browser this will benefit you. Or if you like to archive stuff offline. The two main improvements are a text-only version of every content page (article/tutorial etc) and a 'Skip to main content' link.

Read more...

Send email with multiple inline images via bash with a loop

23-07-2018 | Remy van Elst

Recently I had a request from a user that whished to receive a scheduled email with two screenshots. The screenshots were automated via AutoIt on a network share, the user manually logged in every evening to check the pictures. With bash and postfix/sendmail we can automate this process, the user now doesn't have to login but can just check their email. There are a lot of snippets and guides to attach emails via the shell, but displaying multiple images inline as an HTML mail was something I had to figure out. You cannot embed the image in base64 HTML because Outlook doesn't show that, you must use the Content-ID style embed. Like UUENCODE, but more complicated. (The next step in this process with the user is to automate the reason why they have to check those screenshots every night, that is something for another article)

Read more...

log_vcs - Ansible callback plugin that creates VCS (git) branches for every Ansible run

10-07-2018 | Remy van Elst

This Ansible callback plugin creates a VCS branch every time you run Ansible. If you ever need to go back to a certain state or environment, check out that branch and be sure nothing has changed. This is useful when you have multiple environments or multiple people deploying and continually develop your Ansible. When you often deploy to test / acceptance and less often to production, you can checkout the last branch that deployed to production if a hotfix or other maintenance is required, without having to search back in your commits and logs. I would recommend to develop infrastructure features in feature branches and have the master branch always deployable to production. However, reality learns that that is not always the case and this is a nice automatic way to have a fallback.

Read more...

Windows 7 installer on a KVM Linux VPS (Windows on Digital Ocean)

01-07-2018 | Remy van Elst

For fun I wanted to install Windows 7 on a KVM Linux VPS (on [Digital Ocean) but it should work for any KVM or XEN-HVM VPS with console access). I was experimenting with Grub2 and ISO booting, since grub2 can natively boot a linux ISO. For Windows this is not possible, the installer needs to be extracted on a FAT32 partition from which you boot. On a normal system I would repartition the disk using a live CD, but on a VPS where an ISO cannot be booted this is troublesome. If I could boot from an ISO I would use that to install Windows, but where's the fun in that? I had to figure out how to shrink an EXT4 filesystem from a running Ubuntu VPS, which is possible, however very risky, with pivot_root. Next the partiton table can be converted to MBR, the partition can be resized, a FAT32 partiton and filesystem can be created, the Windows Installer files copied onto that, some Grub config and a reboot later, you're in the Windows 7 Installer.

Read more...

Syslog configuration for remote logservers for syslog-ng and rsyslog, both client and server

21-06-2018 | Remy van Elst

Syslog is the protocol, format (and software) linux and most networking devices use to log messages. All kinds of messages, system, authentication, login and applications. There are multiple implementations of syslog, like syslog-ng and rsyslog. Syslog has the option to log to a remote server and to act as a remote logserver (that receives logs). With a remote logging server you can archive your logs and keep them secure (when a machine gets hacked, if root is compromised the logs on the machine are no longer trustworthy). This tutorial shows how to set up a syslog server with rsyslog and syslog-ng and shows how to setup servers as a syslog client (that log to a remote server) with syslog-ng and rsyslog.

Read more...

snap install mosaic, the first graphical webbrowser on Ubuntu

14-06-2018 | Remy van Elst

On one of my favorite podcasts from Jupiter Broadcasting, either Linux Action News or Linux unplugged (252 I think, not sure), Allan Pope was talking about Snap packages and how there are now WinePacks, a snap with Wine and a single (Windows) application packaged. During the discussion he dropped that Mosaic, the first graphical web browser, is available as a snap package, for modern distributions. I installed it, after huge download (1.5 MB), playing around with it is quite fun. In this post I'll discuss how to install it, what works and what doens't in the modern age on Ubuntu 18.04

Read more...

Chrome 68 is depcrecating HPKP (HTTP Public Key Pinning)

12-06-2018 | Remy van Elst

In 2014 I published an article on HPKP, http public key pinning. It allows a site operator to send a public key in an http header, forcing the browser to only connect when that header is found. It was ment to redice the risk of a compromised certificate authority (since any CA can create a certificate for any website). Quite secure, but it was often wrongly configured, forgotten until certificates expired and there were some security issues like a false pin. Late 2017 Google announced that HPKP would be removed in Chrome 68 and that version is released now, so HPKP is no longer supported. This post goes into the reasoning behind the removal, the possible replacement (Expect-CT) and how to remove HPKP from your site.

Read more...

That time when one of my HP-UX servers lost half of it's RAM (and how to connect to an HP iLO 2 with modern OpenSSH (7.6+))

06-06-2018 | Remy van Elst

One of my favorite sayings is: 'Hardware is stupid, move everything to the cloud!'. The cloud is just someone elses computer, but at least I'm not responsible for the hardware anymore, since hardware breaks. When a VM breaks, because you use configuration management and version control, just roll out a new one. We all know that's not true, but still, the thought of it is nice. Last week one of the HP-UX machines had a failing disk and this week it's back with a whole new issue. After it was rebooted (due to issues with the services running on it), the Event Monitoring Service (EMS) sent an email regarding RAM issues and after manual checking it seems the machine lost half of it's RAM. It should have 16 GB and now it only has 8 GB. You might imagine my suprise. This post goes into my troubleshooting, since I was not able to go to the machine, shut it down and check if the RAM was still there. I'll cover the use of cstm (Support Tool Manager), how to connect to the HP iLO (out of band access) with modern OpenSSH (7.2) and the steps I took to gather information on what might have happened.

Read more...

GPG noninteractive batch sign, trust and send gnupg keys

01-06-2018 | Remy van Elst

Recently a team I consult for started using a shared password manager, pass. It uses GPG keys and presents itself as the standard unix password manager, but in essence it's nothing more than a wrapper around GPG encrypted files. We all had to generate new keys since the team is new and we were not allowed to use existing keys. Using a new, empty keyring, I generated my key and imported their keys. I wanted to trust, sign and publish all keys to a keyserver, this article shows how to do that noninteractively in batch form. Saves me doing the same thing four times, since now it's just four people, but next time it might be a hundred.

Read more...

HP-UX 11.31 System information and find out part number of a failed disk with sasmgr

18-05-2018 | Remy van Elst

On one of my regular scheduled datacenter visits one of the older HP-UX Itanium machines had an orange light on the front. These systems are not (yet) monitored, but still in use so the disk had to be replaced. Not knowing anything about this system or which parts were used, I managed to find the exact part number and device type so we could order a spare. This small guide uses sasmgr to get the data on HP-UX 11.31.

Read more...

All Items