struggling with website performance optimization for dynamic content

hey folks,

• Context & Problem Setup:

  • just rolled out a major update introducing much more dynamic, user-specific content across our SaaS platform.
  • we're seeing a noticeable dip in website performance optimization metrics, especially TTFB and LCP, for logged-in users. it's a real headache.
  • our current stack is Next.js frontend, Node.js API, and PostgreSQL on AWS (ECS/RDS). we use CloudFront for static assets and some basic API caching via Redis.

• Observed Bottlenecks:

  • api calls for personalized dashboards are often slow, even with optimized queries. these personalized data fetches are our main culprit.
  • server-side rendering (SSR) for these dynamic pages is hitting CPU limits on our ECS instances during peak times. we're just burning through resources.
  • redis cache hit ratio for highly personalized data is lower than expected because each user's data is so unique; traditional caching isn't cutting it.

• Strategies Attempted (and their limitations):

  • scaled up ECS instances, but it's getting expensive and not linearly improving performance for complex queries. diminishing returns are kicking in.
  • optimized database indexes and some complex SQL queries, which helped a bit, but not enough for the new load.
  • implemented more aggressive client-side caching where possible, but core dynamic content still requires fresh server fetches, which brings us back to square one.

• Seeking Advanced Solutions:

  • what are some advanced website performance optimization strategies for highly dynamic, user-specific content that goes beyond traditional caching and horizontal scaling? i'm looking for out-of-the-box ideas.
  • specifically looking for insights into edge computing for dynamic data, distributed caching for personalized content, or perhaps alternative data fetching/hydration patterns for Next.js that handle unique payloads better. any thoughts on these areas would be super helpful.

thanks in advance!