Storage Power Consensus

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus

The Storage Power Consensus (SPC) subsystem is the main interface which enables Filecoin nodes to agree on the state of the system. Storage Power Consensus accounts for individual storage miners’ effective power over consensus in given chains in its Power Table. It also runs Expected Consensus (the underlying consensus algorithm in use by Filecoin), enabling storage miners to run leader election and generate new blocks updating the state of the Filecoin system.

Succinctly, the SPC subsystem offers the following services:

• Access to the Power Table for every subchain, accounting for individual storage miner power and total power on-chain.

• Access to Expected Consensus for individual storage miners, enabling:

  • Access to verifiable randomness Tickets as provided by drand for the rest of the protocol.
  • Running Leader Election to produce new blocks.
  • Running Chain Selection across subchains using EC’s weighting function.
  • Identification of the most recently finalized tipset, for use by all protocol participants.

Distinguishing between storage miners and block miners

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.distinguishing-between-storage-miners-and-block-miners

There are two ways to earn Filecoin tokens in the Filecoin network:

• By participating in the Storage Market as a storage provider and being paid by clients for file storage deals.
• By mining new blocks, extending the blockchain, securing the Filecoin consensus mechanism, and running smart contracts to perform state updates as a Storage Miner.

There are two types of “miners” (storage and block miners) to be distinguished. Leader Election in Filecoin is predicated on a miner’s storage power. Thus, while all block miners will be storage miners, the reverse is not necessarily true.

However, given Filecoin’s “useful Proof-of-Work” is achieved through file storage ( PoRep and PoSt), there is little overhead cost for storage miners to participate in leader election. Such a Storage Miner Actor need only register with the Storage Power Actor in order to participate in Expected Consensus and mine blocks.

On Power

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.on-power

Quality-adjusted power is assigned to every sector as a static function of its Sector Quality which includes: i) the Sector Spacetime, which is the product of the sector size and the promised storage duration, ii) the Deal Weight that converts spacetime occupied by deals into consensus power, iii) the Deal Quality Multiplier that depends on the type of deal done over the sector (i.e., CC, Regular Deal or Verified Client Deal), and finally, iv) the Sector Quality Multiplier, which is an average of deal quality multipliers weighted by the amount of spacetime each type of deal occupies in the sector.

The Sector Quality is a measure that maps size, duration and the type of active deals in a sector during its lifetime to its impact on power and reward distribution.

The quality of a sector depends on the deals made over the data inside the sector. There are generally three types of deals: the Committed Capacity (CC), where there is effectively no deal and the miner is storing arbitrary data inside the sector, the Regular Deals, where a miner and a client agree on a price in the market and the Verified Client deals, which give more power to the sector. We refer the reader to the Sector and Sector Quality sections for details on Sector Types and Sector Quality, the Verified Clients section for more details on what a verified client is, and the CryptoEconomics section for specific parameter values on the Deal Weights and Quality Multipliers.

Quality-Adjusted Power is the number of votes a miner has in the Secret Leader Election and has been defined to increase linearly with the useful storage that a miner has committed to the network.

More precisely, we have the following definitions:

• Raw-byte power: the size of a sector in bytes.
• Quality-adjusted power: the consensus power of stored data on the network, equal to Raw-byte power multiplied by the Sector Quality Multiplier.

Beacon Entries

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.beacon-entries

The Filecoin protocol uses randomness produced by a drand beacon to seed unbiasable randomness seeds for use in the chain (see randomness).

In turn these random seeds are used by:

• The sector_sealer as SealSeeds to bind sector commitments to a given subchain.
• The post_generator as PoStChallenges to prove sectors remain committed as of a given block.
• The Storage Power subsystem as randomness in Secret Leader Election to determine how often a miner is chosen to mine a new block.

This randomness may be drawn from various Filecoin chain epochs by the respective protocols that use them according to their security requirements.

It is important to note that a given Filecoin network and a given drand network need not have the same round time, i.e. blocks may be generated faster or slower by Filecoin than randomness is generated by drand. For instance, if the drand beacon is producing randomness twice as fast as Filecoin produces blocks, we might expect two random values to be produced in a Filecoin epoch, conversely if the Filecoin network is twice as fast as drand, we might expect a random value every other Filecoin epoch. Accordingly, depending on both networks' configurations, certain Filecoin blocks could contain multiple or no drand entries. Furthermore, it must be that any call to the drand network for a new randomness entry during an outage should be blocking, as noted with the drand.Public() calls below. In all cases, Filecoin blocks must include all drand beacon outputs generated since the last epoch in the BeaconEntries field of the block header. Any use of randomness from a given Filecoin epoch should use the last valid drand entry included in a Filecoin block. This is shown below.

Get drand randomness for VM

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.get-drand-randomness-for-vm

For operations such as PoRep creation, proof validations, or anything that requires randomness for the Filecoin VM, there should be a method that extracts the drand entry from the chain correctly. Note that the round may span multiple filecoin epochs if drand is slower; the lowest epoch number block will contain the requested beacon entry. Similarly, if there has been null rounds where the beacon should have been inserted, we need to iterate on the chain to find where the entry is inserted. Specifically, the next non-null block must contain the drand entry requested by definition.

Fetch randomness from drand network

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.fetch-randomness-from-drand-network

When mining, a miner can fetch entries from the drand network to include them in the new block.

type DrandBeacon struct {
	client dclient.Client

	pubkey kyber.Point

	// seconds
	interval time.Duration

	drandGenTime uint64
	filGenTime   uint64
	filRoundTime uint64

	localCache *lru.Cache[uint64, *types.BeaconEntry]
}

func BeaconEntriesForBlock(ctx context.Context, bSchedule Schedule, nv network.Version, epoch abi.ChainEpoch, parentEpoch abi.ChainEpoch, prev types.BeaconEntry) ([]types.BeaconEntry, error) {
	{
		parentBeacon := bSchedule.BeaconForEpoch(parentEpoch)
		currBeacon := bSchedule.BeaconForEpoch(epoch)
		if parentBeacon != currBeacon {
			// Fork logic
			round := currBeacon.MaxBeaconRoundForEpoch(nv, epoch)
			out := make([]types.BeaconEntry, 2)
			rch := currBeacon.Entry(ctx, round-1)
			res := <-rch
			if res.Err != nil {
				return nil, xerrors.Errorf("getting entry %d returned error: %w", round-1, res.Err)
			}
			out[0] = res.Entry
			rch = currBeacon.Entry(ctx, round)
			res = <-rch
			if res.Err != nil {
				return nil, xerrors.Errorf("getting entry %d returned error: %w", round, res.Err)
			}
			out[1] = res.Entry
			return out, nil
		}
	}

	beacon := bSchedule.BeaconForEpoch(epoch)

	start := build.Clock.Now()

	maxRound := beacon.MaxBeaconRoundForEpoch(nv, epoch)
	if maxRound == prev.Round {
		return nil, nil
	}

	// TODO: this is a sketchy way to handle the genesis block not having a beacon entry
	if prev.Round == 0 {
		prev.Round = maxRound - 1
	}

	cur := maxRound
	var out []types.BeaconEntry
	for cur > prev.Round {
		rch := beacon.Entry(ctx, cur)
		select {
		case resp := <-rch:
			if resp.Err != nil {
				return nil, xerrors.Errorf("beacon entry request returned error: %w", resp.Err)
			}

			out = append(out, resp.Entry)
			cur = resp.Entry.Round - 1
		case <-ctx.Done():
			return nil, xerrors.Errorf("context timed out waiting on beacon entry to come back for epoch %d: %w", epoch, ctx.Err())
		}
	}

	log.Debugw("fetching beacon entries", "took", build.Clock.Since(start), "numEntries", len(out))
	reverse(out)
	return out, nil
}

func (db *DrandBeacon) MaxBeaconRoundForEpoch(nv network.Version, filEpoch abi.ChainEpoch) uint64 {
	// TODO: sometimes the genesis time for filecoin is zero and this goes negative
	latestTs := ((uint64(filEpoch) * db.filRoundTime) + db.filGenTime) - db.filRoundTime

	if nv <= network.Version15 {
		return db.maxBeaconRoundV1(latestTs)
	}

	return db.maxBeaconRoundV2(latestTs)
}

Validating Beacon Entries on block reception

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.validating-beacon-entries-on-block-reception

A Filecoin chain will contain the entirety of the beacon’s output from the Filecoin genesis to the current block.

Given their role in leader election and other critical protocols in Filecoin, a block’s beacon entries must be validated for every block. See drand for details. This can be done by ensuring every beacon entry is a valid signature over the prior one in the chain, using drand’s Verify endpoint as follows:

func ValidateBlockValues(bSchedule Schedule, nv network.Version, h *types.BlockHeader, parentEpoch abi.ChainEpoch,
	prevEntry types.BeaconEntry) error {
	{
		parentBeacon := bSchedule.BeaconForEpoch(parentEpoch)
		currBeacon := bSchedule.BeaconForEpoch(h.Height)
		if parentBeacon != currBeacon {
			if len(h.BeaconEntries) != 2 {
				return xerrors.Errorf("expected two beacon entries at beacon fork, got %d", len(h.BeaconEntries))
			}
			err := currBeacon.VerifyEntry(h.BeaconEntries[1], h.BeaconEntries[0])
			if err != nil {
				return xerrors.Errorf("beacon at fork point invalid: (%v, %v): %w",
					h.BeaconEntries[1], h.BeaconEntries[0], err)
			}
			return nil
		}
	}

	// TODO: fork logic
	b := bSchedule.BeaconForEpoch(h.Height)
	maxRound := b.MaxBeaconRoundForEpoch(nv, h.Height)
	if maxRound == prevEntry.Round {
		if len(h.BeaconEntries) != 0 {
			return xerrors.Errorf("expected not to have any beacon entries in this block, got %d", len(h.BeaconEntries))
		}
		return nil
	}

	if len(h.BeaconEntries) == 0 {
		return xerrors.Errorf("expected to have beacon entries in this block, but didn't find any")
	}

	last := h.BeaconEntries[len(h.BeaconEntries)-1]
	if last.Round != maxRound {
		return xerrors.Errorf("expected final beacon entry in block to be at round %d, got %d", maxRound, last.Round)
	}

	for i, e := range h.BeaconEntries {
		if err := b.VerifyEntry(e, prevEntry); err != nil {
			return xerrors.Errorf("beacon entry %d (%d - %x (%d)) was invalid: %w", i, e.Round, e.Data, len(e.Data), err)
		}
		prevEntry = e
	}

	return nil
}

Tickets

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.tickets

Filecoin block headers also contain a single “ticket” generated from its epoch’s beacon entry. Tickets are used to break ties in the Fork Choice Rule, for forks of equal weight.

Whenever comparing tickets in Filecoin, the comparison is that of the ticket’s VRF Digest’s bytes.

Randomness Ticket generation

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.randomness-ticket-generation

At a Filecoin epoch n, a new ticket is generated using the appropriate beacon entry for epoch n.

The miner runs the beacon entry through a Verifiable Random Function (VRF) to get a new unique ticket. The beacon entry is prepended with the ticket domain separation tag and concatenated with the miner actor address (to ensure miners using the same worker keys get different tickets).

To generate a ticket for a given epoch n:

randSeed = GetRandomnessFromBeacon(n)
newTicketRandomness = VRF_miner(H(TicketProdDST || index || Serialization(randSeed, minerActorAddress)))

Verifiable Random Functions are used for ticket generation.

Ticket Validation

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.ticket-validation

Each Ticket should be generated from the prior one in the VRF-chain and verified accordingly.

Minimum Miner Size

• State reliable
• Theory Audit wip
• Edit this section
• section-systems.filecoin_blockchain.storage_power_consensus.minimum-miner-size

In order to secure Storage Power Consensus, the system defines a minimum miner size required to participate in consensus.

Specifically, miners must have either at least MIN_MINER_SIZE_STOR of power (i.e. storage power currently used in storage deals) in order to participate in leader election. If no miner has MIN_MINER_SIZE_STOR or more power, miners with at least as much power as the smallest miner in the top MIN_MINER_SIZE_TARG of miners (sorted by storage power) will be able to participate in leader election. In plain english, take MIN_MINER_SIZE_TARG = 3 for instance, this means that miners with at least as much power as the 3rd largest miner will be eligible to participate in consensus.

Miners smaller than this cannot mine blocks and earn block rewards in the network. Their power will still be counted in the total network (raw or claimed) storage power, even though their power will not be counted as votes for leader election. However, it is important to note that such miners can still have their power faulted and be penalized accordingly.

Accordingly, to bootstrap the network, the genesis block must include miners, potentially just CommittedCapacity sectors, to initiate the network.

The MIN_MINER_SIZE_TARG condition will not be used in a network in which any miner has more than MIN_MINER_SIZE_STOR power. It is nonetheless defined to ensure liveness in small networks (e.g. close to genesis or after large power drops).