HPC Monitoring

Designed and implemented a professional, multi-level monitoring system for production-grade infrastructure within the international H2020 LEXIS project at IT4Innovations. The goal was to consolidate metrics from disparate sources into a single pane of glass.

The system operates across four monitoring levels. At the hardware layer, it tracks physical servers (Intel Cascade Lake), CEPH storage (120 TB HDD/32 TB SSD), and 100 Gbit/s Mellanox networking. The virtualization layer covers VMware vSphere and OpenStack cloud environments. Infrastructure services monitoring handles pfSense (firewall/DHCP), Active Directory, VPN, and HAProxy. At the application level, it tracks performance for LEXIS platform instances.

The core engine is Prometheus for time-series data with Grafana for unified visualization and alerting. Data collection relies on Node Exporter (Linux), WMI Exporter (Windows), Blackbox Exporter, and SNMP Exporter, feeding into Prometheus DB, InfluxDB, MSSQL, and Elasticsearch. Log management runs through Graylog with pipelines, Cerebro for Elasticsearch, and Syslog integration. Ansible handles IaC deployment of agents and configurations, and Active Directory integration via LDAP provides Grafana RBAC.

The result was unified visibility merging SQL, NoSQL, and time-series data sources, with proactive alerting via Mattermost and Telegram channels. A multi-level backup strategy based on the 3-2-1 rule with CEPH RBD snapshots rounded out the infrastructure, significantly reducing Mean Time to Detection (MTTD) for system anomalies.