Independent parallel processing of tasks / sub-requests ==> Can add additional servers for further concurrency
Must be able to handle and recover from both software and hardware failures. Services and data must be replicated for redundancy.
Service must have 100% 24/7 uptime
Data stored/produced by the services must be consistent
Consistency Availability Partition Tolerance
Cost of additional latency
Better performance, applications are harder to write
Predictable low latency processing with high throughput
The last 0.X% of the request latency distribution time
In a distributed system, failures happen all the time. Design the application to be self-healing
Build redundancy into your application to avoid having single points of failure.
Minimize coordination between application services to achieve better scalability
Design your application so that it can scale horizontally, adding or removing new instances on demand.
Use partitioning to work around database, network and compute limits.
Scaling without having a state is trivial
Latency is king. Caching helps to significantly reduce the job’s latency
Pick the storage technology that is the best fit for your data and how it will be used.
Partition/Aggregate compute pattern is one that scales pretty well
An evolutionary design is key for continuous innovation
Service Level Agreement
High cost powerful CPUs, with more core and more memory
Add more lower cost servers
Power Usage Effectiveness is the ratio of total energy used at DC vs that which is delivered to computing equipment
All IT Systems (servers, networking etc.) + Other Equipment (cooling, UPS, generators, lights, fans)
Resources owned and operated by one org, usable by any paying customer
Resources owned, operated and used by a single org.
Note: Can be on-prem and/or hosted
Combine Public and Private cloud. Better control over sensitive data, whilst taking advantage of scale of public cloud provider.
e.g. patient database in private cloud, but API in public cloud
Use multiple clouds for an application / service. Redundant, but introduces complexity (API differences, migration)
Resources located locally (or in a data center operated by company)
Resources hosted and managed by third-party
Rent IT infrastructure (servers and VMs), as well as corresponding networking and security measures.
Get a on demand environment for development/deployment
Build app, without managing servers, as cloud vendors provide that. Using Lambdas, so better scaling?
Cloud hosted software, cloud provider handles all software and infrastructure.
Keeping a copy of the same data on multiple machines that are connected via network
Leaders are the replicas, which can be written to
Followers are the nodes, who apply the replication log, they receive from leaders
Leader waits until all replicas confirm changes, before reporting success to user. Followers are guaranteed to have up to date versions of data. If follower does not answer, no writes can be done.
Leader sends update message to all replicas, however responds success when it has made changes
When running the same query on leader and follower results in different results. There is no limit to how far a replica can fall behind
Avoid time skips (e.g. comment appearing, and then gone, due to not being in other replica yet), by reading same data always from the same replica.
Client detects stale response and writes the new response
Background process checks for inconsistencies and fixes them (unlike replication log, order is not preserved)
A copy of the dataset
the ability to view changes (read data) right after making those changes
For large datasets, or very high throughput, we need to break the data up into partitions. A piece of data, belongs to one exact partition
Splitting a table into multiple segments, whereby each segment has a different set of data.
Split a table into multiple tables, whereby some columns are in the one table, and some in the other. Ideally, the table holds data that is linked together.
When it is possible to functionally partition data, e.g. a read only table and read-write table, one should partition these.
Message latency no greater than known upper bound Executed at a known speed
System is asynchronous for a finite time, synchronous otherwise
Messages may be delayed arbitrarily, could be paused, no timing guarantee
Crash at any time, stops executing
Crash at any time, may resume and some point in time
Deviates from algorithm, could crash, could be malicious
System as a whole is not working
some part of the system is not working
crash (crash-stop/crash-recovery), deviating from algorithm (Byzantine)
dropping or significantly delaying messages
System as a whole continues working, despite faults (some maximum number of faults assumed)
Algorithm that detects whether another node is faulty
labels a node as faulty if and only if it has crashed
Mean Time to Recovery
Mean Time Between Failures
Service Level Objective Percentage of requests that need to return a correct response within a specific time
e.g. 99.9% of Requests in a a day get a response in 200ms
Service Level Agreement Contract specifying a SLO, typically with penalties for violation
Count number of seconds elapsed
Difference between two clocks
Time since a fixed date (e.g. Unix Epoch)
Time since arbitrary point (e.g. boot)
Count Events (messages sent)
Parts per Million Used to measure Quartz drift
When several nodes come to an agreement about an single value
no two nodes decide differently
no node decides twice
if a node decides value v, then v was proposed by some node.
every node that does not crash, eventually decides some value.
Time period, whereby there is at most 1 leader (sometimes none)
Atomicity, Consistency, Isolation, Durability
Combine Serializability and Linearizability
The outcome of a series of parallel executed transactions is the same if it were executed serially (without overlapping time)
If there are no more updates, replicas will eventually go towards the same state (may take a long time)
If two replicas communicate with one another, they will converge towards the same state, even if updates occur. Consists of…
Replicas process operations based off of local state only
Any two replicas that have processed the same set of updates are in the same state (even if processing order differs)
Last Writer Wins
Order of delivery does not matter