There are also other methods, like using zmap/zgrab to probe for servers that respond to VPN software handshakes, which can in theory be run against the entire IP space. (this also highlights non-commercial VPNs which are not generally the target of our detection, so we use this sparingly)
It will never cover every VPN or proxy in existence, but it gets pretty close.
Assuming your VPN identification service operates commercially, I trust that you are in full compliance with all contractual agreements and Terms of Service for the services you utilize. Many of these agreements specifically prohibit commercial use, which could encompass the harvesting of exit node IP addresses and the subsequent sale of such information.
Illinois law makes it a misdemeanor to violate web site ToS, though. And felony for the second time IIRC. Other states probably also.
Why? It's not like there's any real moral (or, likely, legal) reason to care beyond avoiding the service's ban hammer.
https://www.ilga.gov/Documents/legislation/ilcs/documents/07...
... "the owner authorizes patrons, customers, or guests to access the computer network and the person accessing the computer network is an authorized patron, customer, or guest and complies with all terms or conditions for use of the computer network that are imposed by the owner;"
Unless you're the one-in-a-million unlucky user who gets prosecuted under the CFAA's very generic "unauthorized access to a protected computer" clause, like Aaron Swartz. It seems the general consensus is this doesn't apply to breaking a website ToS, and Aaron was only in so much trouble because he broke into a network closet, as well as for copyright violation. But consult a lawyer if unsure. (That's another difference: A business will ask a lawyer if it wants to do something shady, while an individual will simply avoid doing it)
How does the buyer even know what the precision and recall rates might be?
The legitimate end-user will then no longer be able to use e.g. SoundCloud.
The more concise word for this is “botnet”. Computers participating in one should be quarantined until they stop.
Often times random shovelware apps will have these proxy SDKs embedded in them, and the only mention of it being part of the software is buried in some long ToS that nobody reads.
But the more sites that require a residential VPN for normal use, the less legitimate that argument becomes.
You just track and block /24 or /16 as necessary.
Even with IPv6 it's not a huge problem. With a few samples we can know that a provider is operating in a given /64 or /48 or even /32 space, and can assign a confidence level that the range is used for VPNs.
https://ip-ranges.amazonaws.com/ip-ranges.json
https://digitalocean.com/geo/google.csv
(And even if they don't publish them, you can just look up the ranges owned by any autonomous network with the appropriate registry.)
Its not perfect ofc, but its not meant to be. Its usually just used as a safety blanket for geoblocked intellectual property, like netflix.
Maybe they mean commercial VPN providers that run on the cloud?
You're a great HN user and commenter and your contributions are much appreciated! I don't want to come across like a bag of bricks but if you would use this feedback to fine-tune a bit, that would be appreciated.
(You may be right that other commenters were breaking the guidelines worse, but we just don't come close to seeing everything, and a lot of what we do see happens by random access.)
Example 1. I run Blockada on my Android phone, so I can block every ad even in apps and I can more or less firewall them (the outside calls). Blockada runs as a local VPN and unfortunately Android allows only one active VPN. So it's either Blockada or Wireguard. I'm with Blockada but I might occasionally want to disable it and enable Wireguard. I never did it yet because:
Example 2. WireGuard does not run everywhere. My little home ARM based server has a Linux kernel with some special driver to manage its hardware (it's pretty common on non-Raspberry ARM devices) and WireGuard does not run on it. It requires a newer kernel that I still cannot upgrade to and maybe I will never be able to. So I don't have anything to VPN to.
I might eventually put online a Raspberry, even an old model 3, as a bastion host on the home end of the VPN, but then it would be something else to care about and to power. It's not worth the mind share and the wattage so far.
https://news.ycombinator.com/item?id=45926849
Besides the political implications, I think we should try to find an objective taxonomy, it's clear that privacy VPNs and network security VPNs are different products semantically, commercially and legally, even if the same core tech is used.
Possibly the configuration and network topology is different even, making it a technically different product, similar to how a DNS might be either an authorative server for a TLD, an ISP proxy for an end user, a consumer blacklist like pihole, or an industrial blacklist like spamhaus. It would be a non trivial mistake to conflate any pair of those and bring one up in an argument that refers to the other.
It's not that he doesn't know the difference. He's making the argument that since there's no _technical_ difference there can be no legal difference.
In most places the law is exercised pragmatically, interpreted by presumed intention. That's why legal precedent is important. You likely won't convince any judge being anal about the wording (maybe if the law gets applied for the first time). You can derail anything semantically. Furthermore, despite apparent belief, laws are frequently formulated in such a way that a particular wider term is extended to help interpretation. Eg. "It is prohibited to use a VPN in a way capable and intended to obscure one's physical internet access point identification". (Not a lawyer, not a native speaker, don't get anal with this wording, either.) I very much doubt any legally binding document would even use the term 'VPN' primarily to describe the technical means for anonymization, but rather describe it functionally.
It doesn't really matter that a single person has found a loophole because many, many other people don't have such a luxury, and that's what the lawmakers are aiming for.
It's going to be interesting when the majority of the UK accesses the internet via VPN because of the increasingly ridiculous hoops that the UK makes them go through, and the government tries to stop them while also allowing VPNs to be used by the tech sector.
I agree, these are two separate legal processes powered by the same technology. But the internet doesn't have any awareness of legality (thankfully) so we're stuck with only the technical meaning.
I doubt that.
The tech is the same, though. That's the point.
It’s not taking about IPsec tunnels between networkers, or a connection back to your home. It’s talking about surfshark
The point, again, is that the tech is the same, and there's no method for determining what purpose the VPN is being used for.
I do actually give VPN access to my mother that is not technically competent but I have full access to her computer and locked her down as much as possible
But no, Tailscale did not pay me for this comment. I do happen to know someone that works there though.
Entirely missing the point that setting up a VPN exit node on your own or someone else's connection is a crazily esoteric super weird nerdy thing outside of communities like HN, and Tailscale on an Apple TV box will not only work but automatically update itself with no intervention on your part, and that the person whose house it is in needs extremely minimal technical skill to do what you tell them to over the phone.
Thanks again, devilbunny
You're very welcome.
Bit of a non sequitur, you would have to outline your entire usage pattern to even submit that as N=1.
GEOIP providers dont sit on your home network. They do accept data from third parties, and are themselves (likely) subscribed to other IP addressing lists. Mostly they are a data aggregator, and its garbage in > garbage out.
If someone, say netflix, but other services participate, flag you as having an inconsistent location, they may forward those details on and you can get added to one of these lists. You might see ip bans at various content providers.
But the implementation is so slapshod that you can just as likely, poison a single ip in a CGNAT pool, and have it take over a month for anyone to act on it, where some other users on your same ISP might experience the issue.
These things can also be weighted by usage, larger amounts of traffic are more interesting because it can represent a pool of more users, or more IP infringement per user.
You can also get hit from poor IP reputation, hosting a webserver with a proxy or php reverse shell, or a hundred other things.
(Also, larger ISPs might deal with a GEOIP provider selling lists of VPN users that include their IP address space, legally, rather than just going through the process of getting the list updated normally. This means the GEOIP providers can get skittish around some ISPs and might just not include them in lists)
or in my case, have a VM on same subnet as other poor actors and thus get bad rep from others.
I just tried it with a well known commercial VPN and I had no problems accessing the site and its music content.
But those data packages are expensive and not available with each wanted origin country. Also you need hardware on your side. But it is an option, just saying.
If using a VPN for access is forbidden by the ToS, you only need to detect a VPN connection once to prove violation.
The IPv4 address space to consider is limited and it is technically absolutely feasible to exhaustively scrape and block the majority of VPN endpoints. Realistically any VPN provider will have some rather small IPv4 subnets make do, shit's expensive. More so, for the trivial case, VPN anonymization works best, when many people share one IP endpoint, naturally the spread is limited. There are VPN providers, some may even be trustworthy, which have the mission of "flying under the radar" with residential IPs and all, but they are way, waaaay more expensive. For most people that's no option.
IPv6 is a different matter, but with the very increase in tracking and access control discussed here, that may be even more of a reason, IPv6 is not going to be a thing any time soon....
Thinking about it, maybe this AI monetization FOMO and monopoly protectionism, will incidentally lead to a technological split of the web. IPv4 will become the "corpo net" and IPv6 will be the "alt net". I think there may be a chance to make IPv6 the cool internet of the people, right now!
But an IP address is not a person (legally in the US at least), and many IPv4 addresses get re-used fairly often. My home 5G internet changes IP every single day, and it's a constant struggle because other users often get my IP blocked for things I didn't do. I cannot even visit etsy.com for example. Just for fun I even checked 4chan and the IP was banned for CP, months before I ever had this particular IP (because I'm paranoid and track all that stuff).
That's a completely different matter (and still probably reasonable suspicion for a search, anyway). If an account/service ID evidently uses a service through a VPN there is no uncertainty of ToS violation. Of course someone could have hacked your account and used a VPN, it doesn't ultimately prove you did it, but nevertheless the account can be flagged/blocked correctly for VPN usage.
> many IPv4 addresses get re-used fairly often
The VPN's servers won't be using changing, "random" IPs. That's something ISPs do when assigning residential IPs. VPNs with residential IPs are not common. (I am not sure those VPNs are even really legal offerings.)
If your ISP uses NAT for its subnet space, you could argue it's technically similar to a VPN. However, same as with VPN exit scraping/discovery, those IP spaces can be determined and processed accordingly. I am also sure those ISP subnets for residential IPs are actually publicly defined and known. Eg. the Vodafon IP may get temporarily flagged for acute suspicious behavior, but won't get your account flagged for VPN violation, or even blocked permanently, since it's known to be the subnet of a mobile ISP, which uses NAT.
Additionally, I presume e.g. SoundCloud prohibits anonymizing VPNs, not everything that's technically a VPN or similar.
They probably assume some amount of collateral damage, a small number of VPN users still flying under the radar, the bulk of VPN users being properly targeted, and the vast majority of users not noticing anything.
yeah sure, if you ignore the existence of literally every mobile isp.
A lot of shady shit under that term. Used by all the harmful services - scammers, AI crawlers... :)
As a bonus you may even get discount codes for your VPN!
For real tho, fuck all those rent-seeking control freaks. Piracy was almost dead, we had a good deal. But no, it's never enough, so here we are.
Also, some piracy boards are actually pretty steady, nice and cool communities, and listing to local files feels way more intentional.
Tradeoff is that it seems to be a browser only thing. Some tools like the default macOS curl seem to be integrated with it.
If 20% of people are using VPNs, blocking them is going to be a double-digit hit.
Traffic is sent through two independent relays: Apple sees your IP but not the destination, while a 3rd party egress partner sees the destination but not your IP, with encryption preventing either side from correlating both ends. It's some of the benefits of Tor. But of course you still need to put a lot of faith in Apple's implementation, which is the hardest part.
Quite frankly, it's a bit silly to paint Apple as some privacy fortress, who wouldn't have to comply with law enforcement/intelligence to unmask/tap traffic. I mean, for a lot of people VPN choice is done considering legal jurisdictions somewhere far away. Apple could/would never possibly offer this level of protection.
There were also plenty of corp-ware in existence that had Flash as *absolutely mandatory*.
It's a very limited VPN as it only works for Safari/Mail and only anonymizes you to your region/country.
This shit has been going on for maybe 5 years but no one seems to know or care.
Instead of blocking or limiting features to whitelist users with approved behavioral patterns and limit / block those that don't -- such as loading a page and immediately commenting or doing things that normal humans don't do, they block IP addresses and ASNs.
I just close the browser tab and remind myself not to waste my time caring, there'll be other platforms.
My router is setup for WireGaurd and it'll never be disabled.
Shame on SoundCloud
As someone who has both spent quite a bit of time writing scrapers and later lots of headache on blocking malicious bots from accessing websites, I can tell you this has become futile. Bot makers aren't stupid. If you put in a check for how fast actions are performed, they will put in a sleep timer in their script. If you start blocking residential IPs because many people use it, you are probably just blocking a school or dormitory, while the real bots will quickly move to another IP once they smell something is off. Today with modern multimodal LLMs, you can bypass almost every "human-check" imaginable. And if they can't pass something, most of your users sure as hell won't either. Not because it is too hard, but because it will take too long to solve. The sweet 3-15s actionable human intelligence threshold has been passed by now. The cats and dogs type captchas were already solved more than 12 years ago by simple CV machine learning. The tech has progressed an insane amount since then. In the end I always ended up basically doing what SoundCloud did here if my service was sensitive: Block entire countries, all tor exit nodes and all known VPN ASNs. That will get it down by like 90%. Bear in mind that anyone who wants to put in some effort will still easily bypass this, but at least the low-effort guys from third world countries will take a while before they catch on. So you can go back to doing some actual work in the meantime.
This is the main issue here, the web has become actively hostile to normal people in the quest to monetize every second of online activity.
"Completely indifferent" and "Corporations are completely amoral" are more accurate.
It's the difference between someone trying to drown you, versus someone trying to fish while you drown just off the bank. Same end, of course.
That's not to say corporations don't come awfully close to the comic book concept of evil. By definition, a corporation's prime purpose is an uncaring commitment to making money, and if you've gone public, making all the money. That's awfully close to being the opposite of the "good" ideals of generosity and kindness.
Snowden showed the NSA has taps upstream, so in my book: that's over. I'm fairly convinced if your company reaches a size where it could potentially be a national security threat, the government comes knocking (Facebook, Apple, Twitter, etc.), so that seems like it's over. You have the AI companies scraping god knows what. And, I imagine most countries have corollaries.
Really, all the bad actors I'd encounter in my daily travels would be ones who want to steal money from me. That's a simple ideology. I can handle that. My identity gets stolen, my bank account...there's multiple levels of billion dollar companies with vested interest in me not losing faith in "the system," so I'm not worried about it really.
If a company wants to associate my phone number to glean all my purchases forever in order to target tailored ads to me, fine. Again, it's in the spirit of taking my money, which is a simple ideology.
If the neighbors want to snoop on my traffic, hats off to them for having the capacity to live two lives: both theirs, and mine after they figure out my day-to-day dealings. Doubt they have time to do much about it. Hard enough to live one life in 24 hours.
If the government wants to try and keep tabs on everything to see who's making ICBMs and who isn't, or whatever else they want to do, that's their prerogative but it seems like a complex goal that doesn't affect me.
Did you travel to get an abortion? Someone might be interested in charging you with a felony. Did you associate too closely with non-citizens? Maybe you're one too. Did you reserve a hotel room? Probably willing to pay more for flights there. Do you frequent hacker news? Might not be so in favor of the current political establishment.
But beyond that I disagree with your sentiment.
These things need to be stopped as they come. Withholding data and living a life of fearful "what ifs" cannot preemptively stop atrocity. Of course I'll never know what past information can be used against me in the future; weaponized in ways I cannot fathom. It's a possibility. Hindsight is 20/20, but "you can't predict the future," so how would I know? I have to live my life. I gotta do SOMETHING.
The crux of all of those "what ifs" is beholden to if the person correlating that data has social agency to act upon it. If that's the case, anyone could be my next predator. Anyone could be the next Hitler waiting to exterminate me based on my non-citizen camaraderie or political leanings.
Data is just a predictor, it is not the truth. If my life provided a data point for a yet-to-be-born hostile dictator to perjure me, I will deal with that when it comes, but I can't live my life out of fear.
I compare it to ecology. You're saying you will deal with the sea when it has risen to your doorstep rather than reduce emissions, or even build a levy. You've chosen to not worry about the sea, either because you don't think you can stop it, or it is not convenient for the moment to try. People who believe the sea is rising can't help but fear it because they are rational. People building privacy levies are not living in fear, they are reacting rationally to a hazard.
It's as simple as that: No thanks, then I slide the slider on WireGuard and then I have an encrypted tunnel that all of my devices can communicate with each other, use a DNS through the tunnel with domains blocked and I can control what phones home and what doesn't. I'm not concerned with foreign governments, snoopy neighbors, war driving, or anything.
I can't solve all the problems but there are no what ifs on my end, *What if" -> No.
I'm not a number in some algorithm or malicious because I route my data securely, I'm a human being.
For a longer argument, see The Eternal Value of Privacy, by Bruce Schneier in 2006: <https://web.archive.org/web/20241203195026/https://www.wired...>.
... or were you simply using a VPN and that's the most likely culprit for a general failure of the service ?
Genuinely curious ...
They also block data center IP's
Signing up for Reddit through a VPN has been difficult to impossible for a long time.
The amount of abuse that comes through VPN-sourced IP addresses is much higher than normal. It's common to block it on any social media site.
Could also be spam/abuse prevention. Credential stuffing often goes through VPNs, signup over VPN is a strong signal for future abuse or issues in various ways.
I'd like to like that won't come back, but voting rights for women are back on the table, apparently, and SoundCloud is apparently worth age-gating, so I guess not.
VPNs often use providers with excellent peering and networking - the same providers that scrapers would want to use.
edit: Ah, it's based on login status
Soundcloud these days is nothing but a spambot filled website that have ripped countless users’ tracks and scam to earn fake followers, which the platform doesn’t block these bot but instead shallow banning proper users. The support is also nonexistent and my support ticket hasn’t being been responded for more than an year.
I ended up trashed my account because I got shadow banned for no reason while they keep on stripping off basic features. Some of the users in my community also faced the similar stories.
Unless there is an irreplaceable feature in SoundCloud you rely on, I see no reason to use it.
Patreon also banned VPN
YouTube, Reddit - locked out, requiring to log into account, on pretense of security and care concerns, yeah to identify and track VPN users.
I don’t have an SC account myself. But get me a file and I will gladly torrent it for others. Some of my torrents have been active for well over a decade, and have hit ratios in the tens of thousands.
>However, the company's response included a configuration change that disrupted VPN connectivity to the site. SoundCloud has not provided a timeline for when VPN access will be fully restored.
https://www.bleepingcomputer.com/news/security/soundcloud-co...
There is pretty much zero regulatory oversight. The ownership structure of VPN companies is opaque, often owned by holding companies. For example Kape Technologies owns ExpressVPN, PIA, CyberGhost, etc.
Many VPN providers may be shady, but none as shady as the governments and corporations that drive people to using them.
there are more elegant solutions similar to yt-dlp. scdl is very user friendly and automatically embeds metadata.
with soundcloud, i just got a generic 403 from cloudfront
combine that with country-level internet filter, the internet is getting harder and harder to use :(
I used that once and got in trouble with the client since the ruleset was over blocking.
aren't Cloudflare exit nodes also content edge servers so impossible to block?
I worked around it but it was a pain
rights holders keep demanding geo fences and identity checks... service providers comply because they don't want to get sued.
BUT... the blunt tool is to block whole swaths of IPs... then we all scramble.
i think the conversation around Apple or any single company saving us is missing the point.
ALSO... even if a big platform rolled out an anonymizing proxy... regulators would still push for carve outs... copyright exemptions... law enforcement taps.
the root is the business model... ad targeting... licensing... fraud detection... all of which depend on tying a real person to a real IP.
HOWEVER... if enough of us treat VPN use as normal... the calculus changes.
blocking a few percent of weirdos is easy... blocking half your paying users is not.
i don't know the answer... but i suspect it's going to get more fragmented before it gets better.
Lidarr relies on people ripping this music, and also adding the metadata to Musicbrainz, which just simply isn't going to happen for most SC uploads.