Projects

Monitoring and Deployment - Part 4

Introduction

In previous parts [1] [2] [3] , I described the development process for our microservices. Initially, I intentionally omitted monitoring capabilities.

Monitoring ensures system reliability, aids in debugging, and provides operational insights, making it crucial as our system scales.

Now, I’ll address that by integrating monitoring into our system.

Monitoring Flask Applications

Prometheus is a powerful tool for collecting, querying, and visualizing metrics, making it an excellent choice for microservices monitoring.

How to use scraped data - Part 3

Introduction

This is the third article in a series about an automated monitoring and publishing project. If you missed the earlier parts:

• In the first article, I described the data collection process using web scraping.
• In the second article, I explained how to organize data exchange between microservices using a data bus.

Today, I will focus on creating a Telegram bot that publishes the collected data in a channel, providing a convenient interface for interaction.

Databus for Scrapers – Part 2

Introduction

In the first part, we discussed a project that automates data collection from listing websites using modular web scrapers in Flask. Now, we’ll add a centralized data exchange mechanism — the databus.

Instead of having scrapers interact directly with end consumers (such as a Telegram bot or a dashboard), we use RabbitMQ as a message broker and Redis to prevent duplicate data. This approach ensures scalability, fault tolerance, and ease of extension.

Web Scrapers on Flask - Part 1

Introduction

Recently, I faced a challenge to help a friend who is searching for an apartment in Belgrade. The requirements were strict: a limited budget, a specific neighborhood, and minimal time for searching. Manually monitoring dozens of aggregator websites was clearly inefficient. The solution became obvious: automation.

I decided to create a project that gathers all relevant listings from multiple sources and presents them in a convenient format. This would allow my friend to focus on choosing an apartment rather than endlessly searching for one.