You can't fix what you can't measure. I like this paper's detailed measurement of RPCs in production. A recurring theme in this paper is variation. At hyperscale, RPCs exhibit significant diversity in processing time, CPU/memory usage, and request sizes. dl.acm.org/doi/pdf/10.1... Thoughts in 🧵 - ThreadSky

About ThreadSky

qianli.dev • 9 days ago

You can't fix what you can't measure. I like this paper's detailed measurement of RPCs in production. A recurring theme in this paper is variation. At hyperscale, RPCs exhibit significant diversity in processing time, CPU/memory usage, and request sizes.

https://dl.acm.org/doi/pdf/10.1145/3600006.3613156
Thoughts in 🧵

Comments

qianli.dev•9 days ago

Google's internal services are widely distributed and replicated for high availability. Interestingly, most rely on a service-oriented architecture rather than microservices. Many of these services are compute- and data-intensive, so decomposing into microservices would add substantial overhead.

qianli.dev•9 days ago

Hyperscale RPC latencies are significantly higher (milliseconds) and noisier than previous research assumptions (microseconds). There's major variation with latencies ranging from hundreds of microseconds to several seconds. Additionally, service usage is highly skewed -- popular RPCs dominate.

qianli.dev•9 days ago

While application processing time usually dominates the latency, RPC overhead (e.g., queuing, (de)serialization) is non-negligible. At the 90th percentile, the RPC latency tax spikes to 96%. I think this overhead is also a good factor to consider while decomposing services into microservices.

qianli.dev•9 days ago

Looking forward to follow-up research on load balancing, scheduling, and software-hardware co-design in datacenters!

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply