High-level design components : The building blocks of your HLD!

on

    In this post, I will discuss significant design components used in a typically high-level design doc. These are the building blocks of any HLD and one must be very confident about their fundamentals before one can use them and come up with an HLD. This list is NOT exhaustive and we have a lot of other components also existing. But if we know these, learning the others won't be a problem. These are the ones most frequently used. Also note, I will not be talking about the "database" or "persistent" component as they are a huge topic in themselves and have been discussed or will be discussed in my other post!

1. Load balancers
Load balancing refers to efficiently distributing incoming network traffic across a group of backend servers, also known as a server farm or server pool. Load balancers act as a point which distributes incoming requests to various servers. This is critical because:
  1. We want optimum utilization of resources. We do not want one server to starve while other servers are overwhelmed. [Efficiency]
  2. We do not want our service to go down just because one of the servers is down. We would ideally want the traffic to be redirected to other servers. [Availibility]
  3. We want to have the flexibility to add/remove servers based on requirements without having to change major infrastructures. [Scalability]
  4. We can cache frequent requests and their responses in the load balancer. In these cases, it will simply return the response instead of redirecting the query. [Speed] (More about caching below)
  5. Load balancers act as a reverse proxy for the servers (The requester will not know which server from what IP address responded to the request and it will only see the load balancers IP). [Security]
The load balancing can be done using various algorithms which include:
  1. Round robin: Distributing to all servers in a circular manner.
  2. Least connections: Routing requests to servers with minimal load/connections at a given point.
  3. Least response time: The server which is closest and has the least response time.
  4. Hashing: Based on the hash of the request, route the request to one of the n server pools.
Load balancers usually support dynamic configuration of servers for many cases where based on usage, new servers are automatically set up and if unused, released (for eg. Amazon EC2 servers). The load balancer automatically registers new servers and de-registers released servers. A load balancer can be hardware-based or software-based.
In terms of networking, the load balancers are usually employed at the level  (Transport layer) or at layer 7 (Application layer). For more on OSI network layers, refer to this.
2. Cache
Cache Memory is a special very high-speed memory. It is used to speed up and synchronize with a high-speed CPU. Cache memory is costlier than main memory or disk memory but more economical than CPU registers. Cache memory is a high-speed memory type that acts as a buffer between RAM and the CPU. It holds frequently requested data and instructions so that they are immediately available to the CPU when needed. Cache memory is used to reduce the average time to access data from the Main memory. The cache is a smaller and faster memory that stores copies of the data from frequently used main memory locations. There are various different independent caches in a CPU, which store instructions and data.
Levels of memory:
  • Level 1 or Register –It is a type of memory in which data is stored and accepted that are immediately stored in the CPU. The most commonly used register is accumulator, Program counter, address register etc.
  • Level 2 or Cache memory –It is the fastest memory which has faster access time where data is temporarily stored for faster access.
  • Level 3 or Main Memory –It is the memory on which the computer works currently. It is small in size and once power is off data no longer stays in this memory.
  • Level 4 or Secondary Memory –It is external memory which is not as fast as the main memory but data stays permanently in this memory.
Cache Performance: When the processor needs to read or write a location in the main memory, it first checks for a corresponding entry in the cache.
  • If the processor finds the memory location in the cache, a cache hit has occurred and data is read from the cache.
  • If the processor does not find the memory location in the cache, a cache miss has occurred. For a cache miss, the cache allocates a new entry and copies in data from the main memory, and then the request is fulfilled from the contents of the cache.
The performance of cache memory is frequently measured in a quantity called the Hit ratio = hit / (hit + miss) = no. of hits / total accesses.

Application of Cache Memory:

  1. Usually, the cache memory can store a reasonable number of blocks at any given time, but this number is small compared to the total number of blocks in the main memory.
  2. The correspondence between the main memory blocks and those in the cache is specified by a mapping function.
  3. Primary Cache – A primary cache is always located on the processor chip. This cache is small and its access time is comparable to that of processor registers.
  4. Secondary Cache – Secondary cache is placed between the primary cache and the rest of the memory. It is referred to as the level 2 (L2) cache. Often, the Level 2 cache is also housed on the processor chip.
The cache layer is usually deployed at the load-balancer layer or other places. This is very important for fast access and less traffic to back-end calls.
TTL (Time to leave) - Time for which an entry is persisted in the cache. Post that, it is replaced/released
Some examples of caches are Redis and Elastic cache.
3. Queues
A queue is an intermediate stage where "information" is stored before it is polled (picked) and processed one at a time or in batches. The queue can maintain data in either a FIFO (first in first out) or random order. The queue is an intermediary stage which has multiple applications. Some of which are:
  1. Keep track of multiple requests information when they come together so that they can be processed one after the other if the processing speed is not at par.
  2. Keeping the failed requests in a separate place so that they may be retried at a later time or may be used for analytics purposes. This type of queue is called a Dead letter queue (DLQ) generally.
  3. It organizes the requests such that they can be processed in a particular order.
Some examples of queues are Kafka and Amazon SQS.
4. Proxies
A proxy is an intermediate server which accepts requests from one source and redirects them to some other destination. The reason to use the proxy is that the sender or receiver of the request does not usually know where did this request/response originate from. Proxies are usually of two types:
  1. Forward proxy: It is a server which receives the requests and sends them to the internet or servers making sure the identity of the requester is hidden.
  2. Backward proxy: It is a server which receives the responses from servers or the internet and redirects them to the private device making sure to hide the details of the responder.
Proxies are very important pieces to ensure anonymity and ensure security. Load balancers are a good example of a forward proxy as well as a backward proxy. You can read more about them here.
5. CDN (Content delivery network)
A content delivery network (CDN) refers to a geographically distributed group of servers which work together to provide fast delivery of Internet content. A CDN allows for the quick transfer of assets needed for loading Internet content including HTML pages, javascript files, stylesheets, images, and videos. The popularity of CDN services continues to grow, and today the majority of web traffic is served through CDNs, including traffic from major sites like Facebook, Netflix, and Amazon.
While a CDN does not host content and can’t replace the need for proper web hosting, it does help cache content at the network edge, which improves website performance. Many websites struggle to have their performance needs to be met by traditional hosting services, which is why they opt for CDNs. By utilizing caching to reduce hosting bandwidth, helping to prevent interruptions in service, and improving security, CDNs are a popular choice to relieve some of the major pain points that come with traditional web hosting.

6. DNS
The Domain Name System (DNS) is the phonebook of the Internet. Humans access information online through domain names, such as nytimes.com or espn.com. Web browsers interact through Internet Protocol (IP) addresses. DNS translates domain names to IP addresses so browsers can load Internet resources.
Each device connected to the Internet has a unique IP address which other machines use to find the device. DNS servers eliminate the need for humans to memorize IP addresses such as 192.168.1.1 (in IPv4), or more complex newer alphanumeric IP addresses such as 2400:cb00:2048:1::c629:d7a2 (in IPv6).

The 8 steps in a DNS lookup:

  1. A user types ‘example.com’ into a web browser and the query travels into the Internet and is received by a DNS recursive resolver.
  2. The resolver then queries a DNS root nameserver (.).
  3. The root server then responds to the resolver with the address of a Top Level Domain (TLD) DNS server (such as .com or .net), which stores the information for its domains. When searching for example.com, our request is pointed toward the .com TLD.
  4. The resolver then makes a request to the .com TLD.
  5. The TLD server then responds with the IP address of the domain’s nameserver, example.com.
  6. Lastly, the recursive resolver sends a query to the domain’s nameserver.
  7. The IP address for example.com is then returned to the resolver from the nameserver.
  8. The DNS resolver then responds to the web browser with the IP address of the domain requested initially.
7. Service
A service is software that performs automated tasks, responds to hardware events, or listens for data requests from other software. In a user's operating system, these services are often loaded automatically at startup, and run in the background, without user interaction.

    With this knowledge about what all components are typically used in an HLD or system design, readers could now embark on a better journey towards the HLD and design.

Amrit Raj

Comments

Post a Comment