Laravel Sitemap & SEO Optimization

Hey everyone,

I'm currently elbow-deep in developing a robust 'Dynamic XML Sitemap' solution for Laravel, which is absolutely crucial for effective Laravel SEO optimization. The goal is to make it auto-updating and future-proof for various client sites.

However, I'm hitting a significant wall with performance bottlenecks and memory exhaustion when trying to generate sitemaps for very large datasets—think 500,000+ URLs, sometimes pushing into the millions. This issue crops up consistently during scheduled generation tasks and, predictably, escalates under concurrent access scenarios. I'm finding that the process often grinds to a halt with memory limits being breached, despite increasing PHP's memory limit considerably.

Here’s a typical error I'm seeing:

[2023-10-27 10:30:05] local.ERROR: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/app/Services/SitemapGenerator.php on line 125

I'm seeking advice on best practices, architectural patterns, or specific Laravel packages/techniques to efficiently handle large-scale dynamic sitemap generation without consistently exceeding memory limits or causing server instability. How are others in the community managing sitemap generation for sites with millions of URLs without resorting to static file generation or massive server upgrades? I've tried chunking but it still accumulates memory over time.

Help a brother out please...