How I Cut Campus Server Latency by 73%

10th Grade · Shanghai First Prize (11th Shanghai Maker Star Competition)

The Problem

Our school had a website. When too many students tried to access it at the same time — like during course registration — the server would slow down. Most requests finished quickly, but some took a very long time.

This is called tail latency. When 99% of requests finish in 10ms but 1% take 500ms, the average looks fine. But those 1% of users had a terrible experience.

Network architecture diagram showing campus infrastructure

My Approach

I built a simulated server environment on a Raspberry Pi to test different load balancing strategies. The idea was to see which strategy reduced tail latency the most.

Campus infrastructure with core switches and servers

I tested three strategies:

Round Robin: Each request goes to the next server in order. Simple, but it does not consider how busy each server is.

Least Connections: Each request goes to the server with the fewest active connections. This sounds smart, but it requires checking every server before each decision.

Weighted Round Robin: Servers get requests based on their capacity. Faster servers get more requests.

Docker container architecture for load balancing

The R-MLO Mechanism

I designed a Resource-Monitor Load Offloading (R-MLO) mechanism. It uses lightweight containers as offloading nodes, monitoring hardware usage in real-time. When traffic spikes for too long, it distributes excess requests to idle backup nodes.

What I Found

The results surprised me. Least Connections reduced average latency the most, but it did not help tail latency as much as I expected. The reason was that checking every server took time itself — the overhead of making a smarter decision sometimes outweighed the benefit.

Weighted Round Robin did better for tail latency because it spread the load more evenly without the overhead of checking connection counts.

Latency comparison results showing P99 tail latency reduction

Latency distribution chart showing improvement

The biggest insight was that the best strategy depends on the workload. If requests come in bursts, Least Connections works better. If the load is steady, Weighted Round Robin is simpler and often good enough.

Implementation

I implemented the system using Docker containers and nginx for load balancing. The deployment involved configuring upstream servers and writing a shell script to handle the routing decisions.

The Campus Website

I also built a simple campus website to measure the improvement in a real-world setting. Students could use it to check their grades, view class schedules, and access course materials.

The results showed that the optimized strategy reduced P99 latency by 73%. This meant that even during peak hours, the website stayed responsive.

What I Learned

This project changed how I think about performance. Before, I thought faster hardware was always the answer. Now I know that the way you distribute work matters just as much as how fast each piece runs.

It also taught me that there is no single best solution. Every strategy has tradeoffs. The art is in choosing the right one for your specific situation.

← Back to Home