How does Ethereum obtain randomness

Obtaining true randomness on the blockchain is very challenging. For Ethereum, true randomness is required; otherwise, the distribution of validators on the chain could be predictable and susceptible to malicious attacks. The key to generating randomness is to ensure the unpredictability of the random source. However, on-chain data such as txId, blockHash, and nonce are all predictable and cannot be used as sources of randomness.

Ethereum addresses this issue using randao. The origin of the randao concept is no longer traceable, but it references this project: https://github.com/randao/randao.

The specific approach of randao is as follows: A DAO (Decentralized Autonomous Organization) allows anyone to participate, and these participants collaborate to generate random numbers. A RANDAO smart contract must be created on the blockchain, defining the rules for generating random numbers in three steps:

First Step: Collecting valid sha3(s) - Participants in the random number generation must first send m ETH as collateral to the smart contract C within a specified time interval (for example, a range of 6 blocks, approximately 72 seconds) and simultaneously send a sha3(s) value to smart contract C, where s is a number known only to the participant.
Second Step: Collecting valid s - After the first step, those who submitted sha3(s) must send s to smart contract C within a specified time interval. Smart contract C checks whether sha3(s) matches the previously submitted value. Identical s values are saved in a seed set for generating random numbers.
Third Step: Calculating the random number - After all the secret numbers s have been successfully collected, smart contract C uses a function f(s1,s2,…,sn) to calculate the random number. The result of the random number is written into the storage of the smart contract and is sent to all other smart contracts that previously requested the random number. Smart contract C returns the collateral to the participants, and then the reward is divided equally among all participants, with the reward coming from other smart contracts requesting the random value.

Of course, the randao algorithm in Ethereum does not follow this approach exactly; instead, it cleverly utilizes BLS aggregate signatures to achieve this. BLS aggregate signatures require all validators to generate them together, which is equivalent to all validators participating in the generation of randomness.

In the randao algorithm, randomness is not regenerated from scratch each time but is accumulated. We can think of randao as a deck of cards; as each participant takes turns shuffling, randomness accumulates over time.

In the consensus layer, there are two key fields for generating randomness:

RandaoMixes: Represents the current randomness, which evolves with the creation of each new block. Over time, RandaoMixes accumulates the randomness generated by all blocks.
RandaoReveal: Stores the BLS aggregate signature from all Validators.

In each block, a BLS signature is aggregated, resulting in a new RandaoReveal value. This newly generated RandaoReveal mixes with the RandaoMixes value to produce a new RandaoMixes value, which becomes the new source of randomness.

Even if the randomness of each block is weak, the accumulated randomness is high. The randomness generated in the Nth epoch is used to calculate the distribution of validators in the N+2nd epoch.

The calculation of randao is completed in the consensus layer. In the BeaconState, there is a variable maintained called randaoMixes, which will be different in each slot:

type BeaconState struct {
  //...
	slot                                primitives.Slot
	// ...
	randaoMixes                         customtypes.RandaoMixes
	// ...
}

Every time a new block is produced, the randaoReveal variable within the block is used to update the value of randaoMixes：

func ProcessRandao(
	ctx context.Context,
	beaconState state.BeaconState,
	b interfaces.ReadOnlySignedBeaconBlock,
) (state.BeaconState, error) {
	// ...
	// get RandaoReveal value
	randaoReveal := body.RandaoReveal()
	if err := verifySignature(buf, proposerPub, randaoReveal[:], domain); err != nil {
		return nil, errors.Wrap(err, "could not verify block randao")
	}
  // update randaoMixes
	beaconState, err = ProcessRandaoNoVerify(beaconState, randaoReveal[:])
	if err != nil {
		return nil, errors.Wrap(err, "could not process randao")
	}
	return beaconState, nil
}

func ProcessRandaoNoVerify(
	beaconState state.BeaconState,
	randaoReveal []byte,
) (state.BeaconState, error) {
	// ...
	// update randaoMixes
	if err := beaconState.UpdateRandaoMixesAtIndex(uint64(currentEpoch%latestMixesLength), [32]byte(latestMixSlice)); err != nil {
		return nil, err
	}
	return beaconState, nil
}

When validators need to be redistributed, a new random number seed is generated using the randaoMixes：

func Seed(state state.ReadOnlyBeaconState, epoch primitives.Epoch, domain [bls.DomainByteLength]byte) ([32]byte, error) {
	lookAheadEpoch := epoch + params.BeaconConfig().EpochsPerHistoricalVector -
		params.BeaconConfig().MinSeedLookahead - 1

  // read randaoMix
	randaoMix, err := RandaoMix(state, lookAheadEpoch)
	if err != nil {
		return [32]byte{}, err
	}
	seed := append(domain[:], bytesutil.Bytes8(uint64(epoch))...)
	seed = append(seed, randaoMix...)

	seed32 := hash.Hash(seed)

	return seed32, nil
}

// read RandaoMix
func RandaoMix(state state.ReadOnlyBeaconState, epoch primitives.Epoch) ([]byte, error) {
	return state.RandaoMixAtIndex(uint64(epoch % params.BeaconConfig().EpochsPerHistoricalVector))
}

This implementation is so elegant, using BLS signatures to solve both the problem of signature aggregation and the issue of the randomness source.

Additionally, after the completion of the Merge upgrade, the block.difficulty in the execution layer no longer serves any other purpose and is instead used to return the latest randao value. In Solidity versions after 0.8.18, a new field block.prevrandao was added to represent the latest randao value. Both variables return the same value, and you can choose which one to use depending on the version of Solidity.