r/datastructures • u/tejaromalius_ • 4h ago
Cuckoo filter w/ xxHash3 beats SQLite in performance by ~120x times!
We needed a way to reduce the barcode data stored on handheld terminals while keeping scans fast and reliable compared with SQLite. We tested a Cuckoo filter with 16-bit fingerprints, ran a standard benchmark, and the result?
Cuckoo + SQLite cuts 50/50 lookup time by about 44–48% (~1.8× faster); unknown lookups are 22–30× faster with BLAKE3 or 65–127× with xxh3; known lookups are 2–6% slower. The standalone filter is 2–8 MiB versus SQLite’s 17.11–38.99 MiB, but adding it alongside SQLite increases total storage.
Read more: https://github.com/tejaromalius-labs/CuckooFiltersBarcodes