Notice: epoch aggregate values issue in db-sync 13.7.0.x

3 min

We are notifying db-sync operators of a bounded issue affecting two derived aggregate fields in the epoch table across db-sync versions 13.7.0.0 through 13.7.0.4.

The Cardano chain itself is unaffected, canonical source data is intact, and full recovery of the impacted aggregates is straightforward in every affected deployment.

What's happening

After upgrading from 13.6.0.5 (Persistent) to the 13.7.x Hasql release line, some operators have observed inflated values being written to epoch.out_sum and epoch.fees at the first epoch boundary following a restart, rollback, cache miss, or upgrade.

What's affected, and what's not

The issue is strictly limited to two aggregated values in the epoch table: epoch.out_sum and epoch.fees.

Unaffected:

  • The Cardano blockchain and its consensus operation
  • Canonical chain data and source-of-truth tables (tx, block, and related)
  • Ledger state, balances, and on-chain history
  • All other db-sync tables and fields

Because the impacted fields are derived from canonical source data, they can always be recomputed. No information is lost, and no permanent inconsistency is introduced. Recomputed values match the canonical chain in every case.

Operators relying on these aggregates for reporting, analytics, accounting, or dashboards may see incorrect totals following a restart or upgrade event.

Workarounds available now

Any one of the following will mitigate the issue immediately:

  1. Run the validation/fix scripts that are shipped with db-sync:
    • scripts/validate-epoch-table.sql
    • scripts/fix-epoch-table.sql
    • scripts/check-and-fix-epoch-table.sh
  2. Run db-sync with the --disable-epoch flag.
  3. Ignore epoch.out_sum and epoch.fees and recompute directly from tx and block as needed.

Resolution of this issue was supported through collaboration and coordinated response efforts between Intersect, Input Output | Group, Ensurable Systems, the Cardano Foundation, and DripDropz.

Fix and longer-term direction

A corrected db-sync snapshot is expected within the next few hours. The permanent fix will ship with db-sync 13.7.1.0.

Longer term, the epoch table may be removed or no longer materialised. Because every value it holds is fully derivable from canonical chain data, removing the redundant layer reduces surface area without losing any information.

Cardano's design deliberately separates canonical, immutable chain data from the derived views built on top of it. That separation is what makes incidents like this bounded, recoverable, and contained to the indexing layer.

We’ll update again when db-sync 13.7.1.0 is available.