Seeking open-source SERP tracking software recommendations for SaaS

Our SaaS product has reached a critical juncture where our reliance on commercial SEO tools for comprehensive SERP tracking and analysis is becoming unsustainable. While platforms like Ahrefs and SEMrush have served their purpose for initial growth, the escalating costs associated with enterprise-level keyword volumes and the inherent vendor lock-in are beginning to significantly impact our operational expenditure as we scale. We are actively seeking to transition to a more flexible, cost-effective, and ultimately, a more controllable solution that provides us with full ownership and granular insight into our SEO data pipeline.

From a technical standpoint, our primary requirement is a solution that offers robust, unthrottled API access for seamless integration with our internal data analytics platform. The current off-the-shelf commercial SERP trackers simply do not provide the level of granular control we need over data collection parameters, indexing methodologies, and custom reporting logic. Specifically, we need the capability to track highly dynamic, long-tail, and niche keywords across an extensive array of geographic regions and device types, which necessitates custom proxy management, sophisticated CAPTCHA handling, and advanced headless browser integration. The black-box nature of commercial offerings prevents us from fine-tuning these critical aspects.

We have extensively evaluated and utilized several prominent commercial SERP tracking software solutions, including Ahrefs, SEMrush, Moz, and even some niche players. Consistently, they fall short in critical areas. The prohibitive costs for tracking hundreds of thousands of keywords across multiple locales are simply not viable long-term. Furthermore, the lack of deep customization options for data extraction logic, the severely limited API call quotas, and the complete inability to self-host or directly own the raw, unfiltered SERP data are major impediments. We are looking to move away from relying on proprietary, opaque ranking algorithms and instead build our data intelligence based on raw, verifiable search engine results.

Therefore, I am reaching out to this community for recommendations on mature, open-source SERP tracking software or frameworks. Ideally, these solutions would be self-hostable, provide extensive and unconstrained API capabilities, support integration with custom proxy networks, and allow for the implementation of our own custom ranking algorithms or advanced data validation routines. While we have a slight preference for Python or Node.js-based solutions due to our existing tech stack, we are open to any robust platform that facilitates deep technical customization and offers the necessary flexibility for a high-volume, enterprise-grade SEO data operation. What open-source SERP tracking software or methodologies are fellow analytical experts using to overcome the limitations of commercial tools for scalable, highly customized SEO data collection?