a straggler slows everything down

1. Overhead for parallelization
2. Synchronization is required
3. Load imbalances
4. Amdahl’s Law

1. Popular Content
2. Poor hash functions

Better than linear scalability (per server >1 user)

Per resource added, exactly one extra user can additionally use the service

Per resource added, less than one user can additionally use the service

Too expensive to make everything fully redundant

1. Replication
2. Reconsumption

Replicate the data and service and have the job run again.

Remember the data lineage for a job and re-run task. Easier for stateless.

May introduce data consistency problems (e.g. data consumed, new data pushed)

• choosing the location of the DC
  • e.g. Iceland
• raise temperature of aisles
• reduce conversion of energy (e.g. directly work at 12V)
• reuse dissapated heat

1.15-1.18

• Power
• Cooling
• Shelter
• Security

Inverse of data center efficiency

High numbers of read only operations

1. Get consistent snapshot (read: back-up snapshot) of leader’s database (already saved somewhere)
2. Copy snapshot to follower
3. Since leader likely has update in this time, get the leader’s replication log
4. Apply log. Once processed, follower should be caught-up

Catch Up Recovery

Failover

synchronous or asynchronous

• Single-leader replication
• Multi-leader replication
• Leaderless replication

• Scalability
• Less contention
• Improve performance
• Optimize storage costs
• Improve Security

If a partition is unfair, that is, one part has more data or queries.

Disproportionally high load

• horizontal
• vertical
• functional

• key range
• hash-based

• Network Behavior
• Node Behavior
• Timing Behavior

1. Vector
2. Lamport

• Cannot update state immediately, must wait for delivery
• Need fault tolerant total order broadcast

Empty AppendEntries RPCs

100-500ms

In consensus 1 or more nodes propose values, only one of which is accepted. Crashes are tolerated in this system.

As for Atomic Commits, every node votes whether or not it wishes to commit or abort. Commits require all nodes, whilst aborts require 1+ nodes to vote abort. Crashes cause a direct abort.

Linearizability only handles a single transaction, whilst Serializability groups transactions together, that can be executed in parallel.

Yes, known as Strict Serializability

Operation Based CRDT

• Commutative: s1 U s2 = s2 U s1
• Associative: (s1 U s2) U s3 = s1 U (s2 U s3)
• Idempotent: s1 U s1 = s1