Skip to content
Home » Design a URL shortening system like bit.ly

Design a URL shortening system like bit.ly

Requirements

  • Given a URL, the system should generate a unique shortened URL.
  • When the user accesses the shortened URL, the system should redirect them to the original URL.
  • The system should be able to handle high traffic and be scalable.
  • It should have a low latency, as users expect quick redirection.
  • The system should have high availability.

Database Schema

The core table you would need is a table to store the URL mappings:

-- Create UrlTable
CREATE TABLE UrlTable (
    id INT PRIMARY KEY AUTO_INCREMENT,
    short_url VARCHAR(255),
    original_url VARCHAR(255),
    created_at TIMESTAMP,
    expires_at TIMESTAMP,
    user_id INT,
    FOREIGN KEY (user_id) REFERENCES UserTable(user_id)
);

-- Create UserTable
CREATE TABLE UserTable (
    user_id INT PRIMARY KEY AUTO_INCREMENT,
    username VARCHAR(255),
    password VARCHAR(255)
);
SQL

This schema is simplified for the demonstration. For a production system, additional data such as URL access statistics and user roles might be necessary.

RESTful API Design

Shorten a URL

POST /api/urls/

Request Body

{
	"original_url": "https://www.example.com",
	"user_id": 1234,
	"expires_at": "2023-06-01T00:00:00Z"
}
JSON

Response

{
	"short_url": "https://bit.ly/abc123"
}
JSON

Access a shortened URL

GET /api/urls/{short_url}

Response

Redirect to the original URL or return an error if the URL is expired or doesn’t exist.

Get URL details

GET /api/urls/{short_url}/details

Response 

{
	"original_url": "https://www.example.com",
	"created_at": "2023-05-01T00:00:00Z",
	"expires_at": "2023-06-01T00:00:00Z"
}
JSON

Delete a shortened URL

DELETE /api/urls/{short_url}

Response 

{
	"status": "success"
}
JSON

High-level System Design

URL Shortening

When the user posts a request to shorten a URL, the system will create a new record in the UrlTable, generate a unique id, convert this id to a base62 string, and store it as short_url.

URL Redirection

When a GET request is made to access a short_url, the system will look up the short_url in the UrlTable, check if it’s expired, and if not, redirect to the original_url.

Scalability and High Availability

  • To ensure the system can handle high traffic, use load balancers to distribute incoming requests across multiple servers. This design allows for horizontal scaling by adding more servers as needed.
  • The database can be sharded based on the id to distribute the data and load across multiple database nodes.

Caching

To reduce database load and improve response time, a caching layer can be added. Popular or recently accessed URLs can be cached. Use an expiration policy like LRU (Least Recently Used) to manage the cache.

Cleanup

A separate service can periodically scan the UrlTable for expired URLs and delete them.

Security

  • Ensure that all data is transmitted over secure connections (HTTPS).
  • Store only hashed versions of user passwords.
  • Implement rate limiting to prevent abuse.

Components

Web Server

Handles incoming HTTP requests and sends HTTP responses. It can be implemented with Node.js, Django, Flask, etc.

Application Server

Handles the logic of URL shortening, URL redirection, etc. This can be implemented in any language like Python, Java, Go, etc.

Database

Stores the URL mappings and other metadata. We can use a key-value store like Redis or a NoSQL database like Cassandra.

Cache

Stores recently or frequently accessed URLs. Can use Memcached or Redis.

Load Balancer

Distributes incoming network traffic across multiple servers to ensure no single server is overwhelmed.

This is a basic outline of a URL shortening system. Many other aspects, such as security, rate limiting, analytics, etc., would need to be considered for a fully-fledged, production-ready system.

Tags: