Fast access to critical weather data with Proxmox VE

When meteorologists analyze an evolving weather situation, access to current data needs to be fast and reliable. At MeteoSwiss, NinJo brings together large volumes of meteorological information and turns them into visualizations that support forecasts and warnings.

As data volumes and performance requirements increased, the existing Solaris/SPARC platform reached its limits. MeteoSwiss therefore needed a new infrastructure that could combine high performance with flexibility and cost efficiency.

Weather data at the heart of daily operations

The Federal Office of Meteorology and Climatology MeteoSwiss is Switzerland's national weather and climate service. Around 390 employees collect, manage, and analyze weather and climate data, operate national measurement networks, prepare forecasts, and issue warnings.

NinJo is one of MeteoSwiss’ key applications. The visualization system combines data from numerical weather models, ground stations, satellites, radar systems, lightning detection, radiosondes, and other sources. It also serves as a lead system for severe weather warnings in Switzerland. The workload is substantial: NinJo generates around 30,000 graphical products every day, while each of the 8 production server instances imports and processes approximately 1.8 TB of data per day. This places high demands on CPU resources, memory, storage performance, and I/O latency.

Storage latency became the bottleneck

The former NinJo environment ran on Sun Solaris/SPARC systems with storage connected through redundant Fibre Channel links. Over time, the platform could no longer provide the response times required by the application. In some cases, meteorologists had to wait up to 30 seconds for the system to display a sounding. The infrastructure team reached a point where further optimization of the existing setup could not deliver a significant improvement.

Storage latency therefore became one of the decisive technical requirements for the replacement platform. At the same time, budget constraints meant that MeteoSwiss’ standard ESXi hypervisor was not a viable option for this project. The team needed an alternative that could combine performance, flexibility, standard server hardware, and a favorable price-performance ratio.

Proxmox Virtual Environment (Proxmox VE) supported the PCIe NVMe hardware that MeteoSwiss wanted to evaluate, making it a promising candidate for the new platform.

Finding the right storage architecture

Roger Widmer, the system engineer responsible for NinJo's technical applications in 2016, started the proof of concept with a single server and then expanded the environment to evaluate cluster functionality and VM behavior. The team also tested different storage options. Tests with the existing enterprise SAN did not provide the required latency, while local PCIe NVMe storage delivered the response times NinJo needed for complex queries.

The team combined several NVMe devices in RAID10 ZFS pools and placed the VM ZVOLs on this local storage. The move from Solaris/SPARC to x86 also required MeteoSwiss to rebuild the surrounding application environment. During the roughly 2-year implementation project, the old and new platforms ran in parallel to ensure a reliable transition for the meteorologists using NinJo.

Initially, MeteoSwiss used pve-zsync to replicate important VMs to another Proxmox VE host. Today, the relevant services are designed with redundancy at the application level, while the team mainly uses pve-zsync for VM migrations between different Proxmox VE clusters. A hardware lifecycle update in 2025 brought the environment to its current configuration.

A platform built for demanding workloads

Today, MeteoSwiss operates one Proxmox VE cluster at each of 2 locations. The current infrastructure includes:

• 2 Proxmox VE clusters, one at each location
• 4 physical servers per cluster, for a total of 8 Proxmox VE hosts
• 32 VMs across Production, Deployment/Integration, and Development
• 2 AMD EPYC processors with 32 cores each per server
• 1 TB RAM per server
• Approximately 35 TB of local NVMe storage per server

The cluster architecture gives the infrastructure team flexibility in day-to-day operations. MeteoSwiss can move VMs between hosts when required and perform maintenance without interrupting services.

Operating ZFS at this scale has also provided useful lessons. The team initially underestimated the memory requirements of the ZFS Adaptive Replacement Cache. MeteoSwiss has since optimized the configuration step by step using recommendations from the Proxmox documentation.

Stable operations and a pragmatic recovery model

The Proxmox VE platform has proven stable and high-performing in long-term production use. Local NVMe storage provides the I/O performance NinJo requires, while virtualization gives the infrastructure team the flexibility to operate and maintain the environment efficiently. MeteoSwiss describes the solution as cost-efficient, stable, and straightforward to maintain.

The recovery concept follows a similarly pragmatic approach. MeteoSwiss does not use traditional VM backups for the NinJo instances. If a VM needs to be rebuilt, the team reinstalls it and restores its configuration with Puppet. Input data remains stored outside the virtualization platform and can be made available again when required. MeteoSwiss also values the favorable price-performance ratio of the Proxmox subscription. In relation to the overall investment in the NinJo infrastructure, subscription costs represent a relatively modest expense.

Expanding beyond NinJo

Meteorological data volumes continue to grow, and MeteoSwiss expects the requirements for fast, reliable processing to increase further. The organization therefore continues to focus on performance and scalability while integrating additional data sources and workloads. Proxmox VE already supports applications beyond NinJo. MeteoSwiss uses the platform for smaller local installations in its regional centers and for pollen detection systems. In the pollen environment, powerful GPUs support data processing, and Proxmox VE gives the team the flexibility to assign GPU resources to individual VMs according to workload requirements.

About MeteoSwiss
The Federal Office of Meteorology and Climatology MeteoSwiss is Switzerland’s national weather and climate service. Around 390 employees work at locations including Zurich, Geneva, Payerne, and Locarno. MeteoSwiss operates national observation networks, collects and analyzes weather and climate data, provides forecasts and warnings, and contributes to meteorological and climatological research. More: https://www.meteoschweiz.admin.ch/