Optimizing Geolocation API performance for our IP lookup tool?

We've been running our 'IP Lookup Tool - Geo-locate Any IP Address & Get Details' for a while now, and it's gaining traction. However, as traffic scales, we're encountering significant performance bottlenecks, particularly around the external Geolocation API calls we rely on.

While most IP lookups are snappy, we're seeing unpredictable and often unacceptable spikes in response times for specific IP ranges. These aren't necessarily obscure IPs, but they seem to hit a slower path in the API provider's infrastructure, leading to timeouts and a degraded user experience. This also puts undue strain on our own server resources as connections hang.

# Standard, fast lookup
curl -s -w "Total time: %{time_total}\n" "https://api.geolocation.example.com/v1/ip/8.8.8.8"
{"ip":"8.8.8.8","country":"US","city":"Mountain View"}Total time: 0.150

# Problematic lookup, high latency/timeout
curl -s -w "Total time: %{time_total}\n" "https://api.geolocation.example.com/v1/ip/172.217.160.0"
{"error":"Timeout or service unavailable"}Total time: 4.870

We've implemented basic caching, but it doesn't fully address the real-time latency for uncached lookups. How are other developers and founders managing Geolocation API performance in high-volume IP tools? Are there advanced caching strategies, intelligent load balancing across multiple API providers, or specific circuit breaker patterns that have proven effective? We're looking for practical, battle-tested advice to ensure consistent, low-latency responses.

Help a brother out please...