Persistent Laravel dynamic sitemap implementation cache invalidation issue with large datasets causing 500 errors

i'm running into a persistent cache invalidation problem with my laravel dynamic sitemap implementation, specifically for very large sites (1M+ URLs).

the issue often manifests as 500 errors during sitemap regeneration, even with explicit cache clearing. it seems like a race condition or incomplete invalidation causing resource limits to be hit.

here's a representative error from the logs:

[2023-10-27 14:35:01] production.ERROR: Maximum execution time of 300 seconds exceeded {\"exception\":\"[object] (Symfony\\Component\\ErrorHandler\\Error: Maximum execution time of 300 seconds exceeded in /var/www/html/vendor/spatie/laravel-sitemap/src/SitemapGenerator.php:123)\"}

looking for robust strategies or alternative approaches for managing large-scale dynamic sitemap generation and cache integrity. thanks in advance!