ASN lookup accuracy problems?

hey everyone, i just launched a new web tool called 'What is My ISP?' – pretty straightforward, it's supposed to find your internet service provider. but i'm running into some really frustrating inconsistencies, especially when trying to identify the *actual* last-mile ISP. the core issue is that so many IPs, particularly for mobile networks, residential users behind CGNAT, or even folks on corporate networks that route all their traffic through central hubs, frequently resolve to transit providers like cogent or level 3, or even cloud services like AWS or google cloud. it's not giving me the end-user's direct internet provider, you know, the comcasts and at&ts of the world. this is a big block to accurate isp identification. i've thrown a lot at this. we've leveraged multiple commercial IP geolocation APIs – maxmind, ipinfo.io, you name it – scrutinizing their ASN, organization, and ISP fields. i've also gone old school, doing direct whois queries on IP ranges and ASNs, and cross-referenced that with public BGP routing tables and RIR data. tried reverse DNS lookups too, but let's be honest, for most residential IPs, that's a dead end. my main focus has been on ASN lookup data as the primary identifier, assuming that's the most reliable path. here’s where i'm really stuck: despite all this extensive ASN data, often the 'ISP' field in these databases, or even what i derive from whois, still reflects the transit provider (like cogent, level 3) or the network owner (like AWS, google cloud) rather than the true residential/business ISP. how do services like speedtest.net achieve better accuracy in identifying the *local* ISP? its like they have some secret sauce. is there a specific technique to differentiate transit ASNs from last-mile access ASNs more reliably? perhaps by analyzing peering relationships, or maybe there's more granular RIR allocations i'm missing? i'm looking for methods to improve isp identification beyond just the top-level ASN. so, what advanced techniques or data sources can be used to accurately determine the *end-user's direct internet service provider* when standard IP-to-ASN/ORG mapping falls short? any insights on BGP path analysis, or specific community-driven IP blacklists/whitelists for ISPs would be super helpful. i'm open to anything that improves this granular isp identification. help a brother out please...