Experiencing critical memory exhaustion during large-scale dynamic XML sitemap generation for Laravel SEO optimization

hey everyone, so i'm pretty deep into developing a dynamic XML sitemap generator for our main Laravel application. it's a SaaS platform, and we're looking at potentially millions of records across various models that need to be included. the whole goal here is robust Laravel SEO optimization, making sure Google and other search engines can properly index everything we've got.

the core problem i'm consistently hitting is 'allowed memory size exhausted' errors. it's happening when i try to generate comprehensive sitemaps for sites with extensive data. i've already bumped up the php memory_limit significantly, but it just seems to delay the inevitable for really large datasets. we're using a custom package for this, not some off-the-shelf solution, because we need a lot of specific control.

my current approach involves iterating through records using chunking, which helps a lot initially. but for extremely large datasets, the aggregate memory usage for all the model objects, or even just the raw string concatenation for the XML elements, becomes totally prohibitive. i'm thinking beyond basic chunking or direct stream processing at this point. i'm really seeking strategies that might involve more advanced queueing, background processing, or maybe even a completely different architectural approach for generating these very large XML files without running into these crippling memory issues during our laravel development cycle.

i'm hoping some of you have tackled similar challenges in high-scale Laravel development environments. i'm really waiting for some expert advice on scalable, memory-efficient methods for generating dynamic XML sitemaps for Laravel, especially for applications that demand frequent updates and handle massive amounts of data. any insights would be super helpful!