Reliable File-Based JSON Storage in Flask Apps

In a world where databases are ubiquitous, it might seem unusual to find a system relying on file-based JSON storage. However, for certain workloads, this approach offers an effective and reliable solution. This article takes a closer look at such a system, running on Alpine Linux under Gunicorn, managing over 60 retail stores, and surviving power outages and operator mistakes.

Why JSON Files in 2025?

The system in question manages the configuration data for more than 60 retail stores, with around 50 SKUs per location. Operators tweak delivery schedules, safety stock logic, and product exclusions several times a day through a Flask UI. The data is config-like, not an event stream, and writes happen at human cadence. The downside is that naive JSON writes can corrupt the main file on crashes, leading to app failure. To address this, the storage layer is built around atomic writes and recoverable reads.

Atomic Writes: Making Crashes Boring

To ensure data integrity, the system uses an atomic write pattern. This involves writing to a temporary file, flushing it, synchronizing with the operating system, and atomically swapping the temporary file with the main file using os.replace(). This approach ensures that the main file is either the previous valid file or the new valid file, never a half-file.

Recoverable Reads: Forensic-Safe Defaults

When it comes to reads, the system handles failures gracefully. If the main file is corrupted and there is no backup, it returns the default configuration without writing over the evidence. This forensic-safe approach allows for post-mortem analysis of what went wrong.

Production Integration: SettingsManager

The storage layer is integrated into the Flask managers that operators interact with daily. The settings manager loads on app startup and reloads on demand. Operators change delivery days, safety stock thresholds, and product exclusions through the web UI, which triggers the atomic write.

The Incident: When .bak Saved the System

A power outage during an operator-driven edit led to a real-world test of the system's resilience. The recovery logic behaved as designed, loading from the backup file, restoring the main file, and preserving the broken file for investigation. Without the rolling backup and forensic-safe defaults, the system would have started with in-memory defaults, potentially causing issues for the store in question.

Where This Stops Working

While this approach works well for the current scale, there are clear triggers where a database becomes the simpler option. These include high-frequency writes, concurrent writers, large files, and complex query patterns. When these conditions are met, SQLite is an obvious next step.

In conclusion, the system's file-based storage layer offers reliable configuration management for certain workloads. It's a reminder that matching storage to workload is crucial. JSON files with careful atomicity work for config-heavy apps with infrequent writes. When multi-writer coordination, partial updates, or indexed queries are needed, a database becomes the simpler option. As the system expands to manage more stores, these triggers may be reached, and a move to SQLite will be considered.