2 pointsby nova_engine_dev13 hours ago1 comment
  • nova_engine_dev10 hours ago
    Author here. NovaEngine v4.0 is optimized specifically for cloud log datasets where redundancy patterns repeat across services and time windows. Instead of generic compression, it uses a lightweight pattern recognition + deduplication pipeline tuned for log entropy. Current benchmarks show ~40 MB/s single-thread throughput on commodity hardware, with up to ~90% space savings on highly redundant logs (Kubernetes, app logs, infra telemetry). I’m especially interested in feedback on the architecture choices and ideas for scaling this further (parallelization, streaming ingestion, or edge use cases). Happy to answer any technical questions.