XML sitemap generation for huge sites?

Hi everyone, I'm new here and just saw the thread about sitemap generator crashes. We're a small team, and while we're not crashing yet, I'm trying to learn best practices before we hit that wall as we scale.

Our SaaS has grown quite a bit, and we're now pushing close to 50,000 unique content pages. We've been using a basic online XML sitemap generator and sometimes a WordPress plugin for our blog, but I'm getting worried about scalability.

The Problem We're Anticipating

I've noticed that when we try to generate a full sitemap for our main app, it either times out, or the online tools hit their page limits. I'm imagining what happens when we reach 100k, 500k, or even a million pages. I'm trying to understand how large enterprises or even medium-sized SaaS companies manage their XML sitemap generation efficiently and reliably without bogging down their primary servers or incurring huge costs.

My Questions for the Community:

• What are the common strategies for truly large sites (think hundreds of thousands to millions of URLs)?
• Are there specific hosting setups or server configurations that are better suited for this intensive task?
• Should we consider a dedicated microservice just for sitemap generation, perhaps running on a separate server or serverless function?
• Any recommendations for robust open-source libraries or paid services that excel at this scale?
• How do you handle real-time or near real-time updates for content that changes frequently, ensuring the sitemap is always fresh without constant full regeneration?