GEYSER — Solana × AccountsDb

One Solana Slot, Multiple Banks: Fork-Aware Geyser Indexing

10 min read
SolanaGeyserRustIndexing

Your Geyser plugin buffers account updates by slot and writes a slot to the analytics tables when the slot is rooted. For as long as a slot meant one bank, that was correct. In October, Agave’s accounts database finished a migration whose stated purpose is to let “multiple banks with the same slot” exist side by side.

A bank is one replay of one block. When a node replays a slot twice, both replays stream their account writes and transactions to every Geyser plugin as they execute, and only one of them is ever finalized. The interface has been naming the bank behind each update since version 4.3.0. An indexer that keys by slot alone merges the two, and the abandoned bank’s data reaches the finalized table.

Here is where the second bank comes from, what a plugin is and is not told, what a slot-keyed indexer commits wrongly, and the bank-keyed buffer that fixes it, with tests that show both.


A slot is not a bank

A slot is a position in time: the window one leader has to produce a block. A bank is a node’s execution of a specific block at that position, with its own account changes, its own transaction results and its own ID. The Solana SDK says it plainly: a BankId “is unique even if the slot number of two different slots is the same”.

Agave’s accounts database used to assume the opposite inside its write cache, the in-memory layer that holds a bank’s account writes until its slot is rooted and flushed. Between late September and the end of October, one Agave engineer moved it from slot keys to bank keys:

BankId becomes its own type (#15620)2 Oct
Ancestors carry (slot, bank ID) (#15857)8 Oct
Write cache rekeyed by bank ID (#16102)20 Oct
Roots added by bank (#16262)23 Oct
Purge by bank ID (#16360)27 Oct
Flush and freeze by bank ID (#16447, #16459)28 Oct
Slot-to-bank map removed (#16497)28 Oct
Purge deferred to bank drop (#16508)31 Oct

The two highlighted changes say what the series is for. #16360 switched purging to bank IDs “so multiple banks with the same slot can exist in the write cache”. #16497 removed the last lookup by slot, after which the cache “no longer asserts one bank per slot: banks at the same slot each keep their own cache, and removing a bank removes only its own cache.” Before the series, the old bank’s cache had to be purged inline before the slot could be created again. #16508 moved that purge to the background and measured it falling from about 2 milliseconds to about 100 microseconds.


Where the second bank comes from

Three paths in Agave’s replay stage clear a bank and let the same slot be replayed into a new one.

  • Switching to the cluster’s version of a block. A leader can produce more than one version of a block for its slot. Under Alpenglow, live on mainnet since 21 October, votor tells replay when the version the cluster certified differs from the one the node replayed. Replay repairs the cluster’s version, then clears its own bank for that slot and every descendant and replays again. The same slots come back with new bank IDs.
  • Fast leader handover. The first slot of a leader’s four-slot window may start on an optimistic parent, before the previous leader’s last block is settled. If the leader then switches parents, the block carries an UpdateParent marker, and replay clears the in-progress bank and recreates it from the marker’s position in the block.
  • The leader’s own optimistic bank. On the producing side, the same handover clears the bank the leader started building and replaces it.

The second and third paths are fast leader handover, and replay’s restart path checks the FastLeaderHandover11111111111111111111111111 feature, which has not been activated on mainnet. The first needs only a leader that produced two versions of a block, which is rare, but the path is live today. The write-cache series is what lets these replays happen without the old bank’s cache being torn down first.


What a Geyser plugin is told, and what it is not

Since agave-geyser-plugin-interface 4.3.0, the callbacks that carry data name the bank they came from. In version 6.0.0, the current one on 31 October:

update_account_for_bankslot + bank ID, every write
notify_transaction_for_bankslot + bank ID, every transaction
update_bank_status · Rootedbank ID + parent, once per rooted slot
update_slot_status · Deadslot only, no bank ID
notify_entry_update_parentcleared bank ID, entry plugins only
bank cleared to switch block versionsno callback at all

Three things in that table decide how an indexer has to be built.

Discards are mostly silent. The only notice of a cleared bank is the UpdateParent entry notification, and the plugin manager sends it only to plugins that enable entry notifications, which most account indexers do not. A bank cleared to switch block versions produces nothing at all. By the time either happens, the bank’s writes and transactions have already been delivered.

Finality is per bank, and ordered. When votor roots a block, the RPC tracker sorts the newly rooted chain by slot and reports each slot once, with its bank ID and with the previous root as its parent. The chain arrives contiguous: a slot between a rooted slot and its reported parent was never rooted. And a bank’s Rooted arrives after its data. Account writes are delivered synchronously while the bank replays, before it can freeze. Transactions travel through a separate service, but the root notification waits on that service’s declared work before it is processed.

Old plugins are blind. The slot-only callbacks were deprecated in 4.3.0, and each new bank-aware method’s default implementation forwards to its old counterpart, dropping the bank ID on the way. A plugin that still implements only update_account keeps compiling and keeps receiving data, with no way to tell two banks of one slot apart.


What a slot-keyed indexer commits

To see the failure without a validator, we fed the event sequence these paths produce to two small indexers in Rust. One buffers by slot and commits a slot when it is rooted. The other buffers by slot and bank ID. Slot 104 is replayed twice: bank 5 executes transactions a and b, writing account X to 70 and account Y to 1, and is then cleared. Bank 6 executes transaction c, writing X to 90, and is rooted.

rustthe same events, two indexers
// bank 5: the version that is abandoned
ix.transaction(104, 5, sig(0xa));
ix.account(104, 5, w(X, 70, 2));
ix.transaction(104, 5, sig(0xb));
ix.account(104, 5, w(Y, 1, 3));
// bank 6: the version the cluster finalizes
ix.transaction(104, 6, sig(0xc));
ix.account(104, 6, w(X, 90, 4));
ix.rooted(104, 103, 6);

// slot-keyed: Y = 1 is "finalized", slot 104 has 3 transactions
// bank-keyed: no Y, slot 104 has 1 transaction (c), X = 90

The slot-keyed indexer commits a value for Y that never existed on the finalized chain, and counts three transactions in a block that has one. Volume, fee and activity tables built on it are inflated by exactly the abandoned bank’s work.

It does get X right, and only by luck. Both banks wrote X, and the higher write_version happened to belong to bank 6, because the clearing paths remove the old bank before the new one replays. Swap the arrival order and a third test shows the slot-keyed indexer committing the abandoned value. Nothing in the interface promises that order, and the write-cache series exists to make same-slot banks cheaper to keep around.


Key by bank, commit on the rooted bank

The fix is a different buffer key and one rule at root time:

rustfork-aware buffer (condensed)
pub struct BankKeyed {
    pending: BTreeMap<(Slot, BankId), Pending>,
    pub finalized: Finalized,
}

impl BankKeyed {
    // update_account_for_bank / notify_transaction_for_bank
    pub fn account(&mut self, slot: Slot, bank_id: BankId, w: AccountWrite) {
        self.pending.entry((slot, bank_id)).or_default().write(w);
    }

    // update_bank_status(slot, parent, Rooted, bank_id): one call per newly
    // rooted slot, in slot order, parent = the previous root. Every bank
    // buffered in (parent, slot] is decided now: keep the rooted one.
    pub fn rooted(&mut self, slot: Slot, parent: Slot, bank_id: BankId) {
        let decided: Vec<_> = self
            .pending
            .range((parent + 1, BankId::MIN)..=(slot, BankId::MAX))
            .map(|(key, _)| *key)
            .collect();
        for key in decided {
            let p = self.pending.remove(&key).expect("key was just listed");
            if key == (slot, bank_id) {
                p.commit(slot, &mut self.finalized);
            }
        }
    }
}

The range does two jobs. It drops other banks at the rooted slot, the same-slot case, and it drops banks at slots the chain skipped, an ordinary abandoned fork, because a slot between a root and its reported parent will never be rooted. Banks above the rooted slot stay buffered until their own turn. Handling notify_entry_update_parent is optional: it lets a cleared bank’s buffer go early, but correctness rests on the rooted bank ID, the one signal every plugin receives.

Six tests pin both behaviours down. Two show the slot-keyed indexer committing the abandoned bank’s data, one of them through arrival order alone. The others show the bank-keyed buffer handling the same-slot case, an abandoned fork at a skipped slot, descendants of the root that must stay buffered, and the early discard. Inside a bank, the highest write_version per account is its final state, and that rule is still correct. It just has to be applied per bank, not per slot.


What this does not show

The tests replay event sequences derived from Agave’s source. They are not a captured Geyser stream from a validator, and we have not measured how often a mainnet node clears a bank today. With fast leader handover inactive, only the block-version switch can do it on mainnet, and that needs a leader to have produced two versions of a block.

The ordering guarantees in section 03 come from reading the notification paths, not from a fault-injection test. And a Dead status still names only a slot. A plugin that must react to a dead bank, rather than simply never committing it, cannot tell from that call which of a slot’s banks died.


Same-Slot Banks: Questions People Actually Ask

Can two Solana banks have the same slot?

Yes. A bank is one replay of one block, and Agave gives every bank a unique bank ID even when two banks share a slot. Under Alpenglow a node replaces its bank for a slot when the cluster settles on a different version of that block, and fast leader handover adds a second way once its feature activates.

Does Geyser tell a plugin which bank an update came from?

Since agave-geyser-plugin-interface 4.3.0, yes: update_account_for_bank, notify_transaction_for_bank and update_bank_status all carry a bank_id. The older slot-only callbacks are deprecated, and the default bank-aware methods forward to them without the bank ID.

How does an indexer know which bank was finalized?

Agave reports every newly rooted slot through update_bank_status with SlotStatus::Rooted and the rooted bank’s ID, in slot order, with the previous root as the parent. Commit only that bank, and discard every other bank buffered between the parent and the rooted slot.

Is buffering by slot until the slot is rooted enough?

No. A slot-keyed buffer merges every bank that ever replayed the slot, so accounts written only by an abandoned bank and transactions executed only in it reach the finalized table. Key the buffer by slot and bank ID.

Does Agave notify a plugin when it discards a bank?

Not in general. A bank cleared by fast leader handover produces an UpdateParent entry notification, but only to plugins that enable entry notifications. A bank cleared to switch to the cluster’s version of a block produces no notification at all. The rooted bank ID is the only signal every plugin gets.


The one-line version

A slot can have more than one bank, Geyser streams all of them, and only the bank named in the Rooted status is the finalized one. Key the buffer by slot and bank ID, and implement the _for_bank callbacks.

The pipeline this buffer belongs in is the subject of ingesting the Solana firehose with Rust, and the certificates that decide which bank is rooted are in Alpenglow finality in numbers. What a block says about its own producer is in which client built each block.

Sources: Agave master at commit 5817ebe009, the merge of #16508: replay_stage.rs, update_parent.rs, optimistically_confirmed_bank_tracker.rs, entry_notifier.rs and transaction_status_service.rs; pull requests #16360, #16497 and #16508; the GeyserPlugin trait in agave-geyser-plugin-interface 6.0.0; and solana_clock::BankId. The feature account FastLeaderHandover11111111111111111111111111 does not exist on mainnet.

xroot.devOn-chain research, published in full
𝕏 @xrootdev
← Back to all notes