Laravel SEO sitemap issues!

We launched our 'Dynamic XML Sitemap for Laravel & All Websites' product, and it's been a nightmare lately. We're now facing critical, frustrating issues with its core functionality under real-world load, and I'm completely stuck and desperate for some help. Our sitemap generation logic is failing unpredictably, leading to either missing new URLs or including old, deleted ones. This directly impacts our Laravel SEO efforts and Google indexing, which is a massive problem for our users and us. We've tried so much already to fix this. We've implemented various caching layers like Redis and Memcached, hoping to speed up URL fetching and reduce the load. We've adjusted cron job frequencies for sitemap regeneration, literally trying everything from every 5 minutes to hourly, thinking maybe the timing was off. We've also experimented with different database query optimization techniques for fetching content URLs, spending hours trying to make those queries as efficient as possible. We've thoroughly reviewed server logs for memory limits, execution timeouts, or any other resource constraints that might be popping up during sitemap generation. And of course, we've confirmed all our robots.txt and sitemap.xml paths are correctly configured and accessible, ruling out the obvious stuff. But despite all these efforts, we're still seeing specific, frustrating failures. New pages are sometimes taking hours to appear in the sitemap, even after manual regeneration attempts, which is just unacceptable for dynamic content. Conversely, pages that have been deleted or unpublished are occasionally lingering in the sitemap for extended periods, causing Google to try and index dead links. We're also seeing intermittent 500 errors during peak content update periods, which seem to coincide directly with sitemap regeneration processes, indicating a deeper performance bottleneck we can't pinpoint. To make matters worse, Google Search Console is reporting 'Couldn't fetch' errors for our sitemap URL sporadically, which is a huge red flag and means our SEO is taking a hit. We are completely stuck and need immediate advice. What advanced strategies, debugging methodologies, or architectural patterns are we overlooking to ensure a truly robust, auto-updating, and future-proof dynamic XML sitemap for high-traffic Laravel applications? How do others manage this at scale without constant manual intervention or significant performance degradation