• Installing GitLab in an internal Docker Container behind a public Nginx

    There are several options available to install GitLab on your server. The recommended way by GitLab is to use GitLab Omnibus, but this will bundle with an included Nginx server. If you already have your own instance of Nginx you probably do not want to install another one - it cannot listen on the default ports anyway. Thus, you have to look for another method to install GitLab. Another possibility would be to install GitLab from source. In this case however, you will have to install PostgreSQL, Redis and Nodejs.

  • Health Monitoring and yet another Health Status Monitoring Service

    When you’re running websites or any other service you want to know its state at any time and be informed when something goes wrong. For example, I started molescrape almost one year ago and during long-term website crawling you need some form of application monitoring. During the first few days, you might check the status of the crawling manually, but some weeks later you won’t. And at some point in time your application will fail. And you will need to notice, or you will go days or even weeks without noticing. In this article I will give a short overview over monitoring solutions (especially simple ones for hobby projects) and introduce my own monitoring solution.

  • Failure Detection for Molescrape: Anomaly Detection on Daily Signals

    At my scraping platform molescrape.com, I have setup a constant monitoring of the number of items collected per spider to detect when a spider fails (e.g. because of system problems or because the website changed). Currently, the threshold has to be set manually for each newly added spider. As this is increased effort for the user, I have been working on a system to automatically detect a useful threshold.

  • Compiling xmr-stak from scratch

    The most commonly used mining program for mining Monero at the moment is xmr-stak. Some time ago it was split into separate projects for xmr-stak-cpu and xmr-stak-gpu, but now they are both combined into xmr-stak.

  • Messaging Systems: An Overview over RabbitMQ, Kafka, ZeroMQ and Mosquitto - Part 1

    When dealing with messaging systems there are a lot of options available from classical message brokers to simple libraries that handle the messaging logic without a central server. Almost all of them have some differences and each of them has a reason to exist. In this article I will compare a few popular ones and very different ones, namely the message broker RabbitMQ, the distributed streaming platform Kafka, the socket and concurrency library ZeroMQ and the lightweight MQTT broker Mosquitto. You will see that each of them has their own advantages and differences from the others and you should choose one according to your needs.