Tor users are actively discriminated against by website operators

Computer scientists have documented how a large and growing number of websites discriminate against people who browse them using Tor.

Tor is an anonymity service that is maintained with assistance from the US State Department and designed in part to allows victims of censorship in countries like China and Iran to surf the web. New research show how corporations are discriminating against Tor users, in some cases partly because it’s harder to classify anonymous users for the purpose of pushing ads at them.

Many websites block access from the Tor network, either deliberately or because they are reacting to malicious traffic originating from the Tor network. One particular problem is that content distribution networks (CDNs) like CloudFlare are used by many popular websites and these very often block Tor users, occasionally to the surprise of website operators who enabled CloudFlare.

A (heated) discussion thread on the Tor website, involving the Cloudflare staff, can be found here.. El Reg’s interview with CloudFlare boss Matthew Prince on the controversy can be found here.

But the issue extends far beyond CloudFlare and affects surfers visiting popular websites using anonymisation software much more generally.

You're not getting in with jeans

Tor users face various annoyances in their web browsing experience in general, ranging from pages saying “Access denied” to having to solve CAPTCHAs before continuing. These hurdles disappear if the same website is accessed without Tor.

The growing trend of websites extending this kind of “differential treatment” to anonymous users undermines Tor’s overall utility, and adds to the traditional threats to Tor, such as attacks on user privacy, or governments blocking access to Tor, etc.

Computer scientists tried to quantify these problems and answer related questions such as how prevalent anti-Tor discrimination might be and whether there is any pattern in where these Tor-unfriendly websites are hosted (or located).

To answer these questions, researchers conducted comprehensive network and application layer measurements in order to log websites that block Tor. The researchers scanned the entire IPv4 address space on port 80 from Tor exit nodes before fetching the homepage from the most popular 1,000 websites from all Tor exit nodes. Measurements from this exercise were compared with a baseline from non-Tor control measurements.

The experiment uncovered what the boffins describe as “significant evidence of Tor blocking”. At least 1.3 million IP addresses that would otherwise allow a TCP handshake on port 80 block the handshake if it originates from a Tor exit node. The researchers also found at least 3.67 per cent (or more than one in 30) of the most popular 1,000 websites block Tor users at the application layer.

A paper (pdf), Do You See What I See? Differential Treatment of Anonymous Users, was presented this week at the Network and Distributed System Security Symposium (NDSS) conference in San Diego, USA. Computer scientists from the University of Cambridge, University College London, University of California, Berkeley and International Computer Science Institute (Berkeley) collaborated in putting together the study.

University of Cambridge doctoral candidate Sheharbano Khattak summarises the researchers findings in a post on the University of Cambridge Computer Laboratory Security Group's Light Blue Touchpaper blog here.

Khattak explains that the researchers identified CloudFlare, Amazon Web Services and Akamai as dominant Tor blockers, “highlighting the amplified blocking effect such centralised web services may create when their Tor-unfriendly policy trickles down to thousands of their client websites.”

“We think that some of this blocking is caused by blacklists that include Tor exit nodes, yet other instances likely arise when abuse generated from Tor exit nodes trigger automated blocking mechanisms on websites,” she writes.

Researchers view the process of quantifying problems faced by Tor users and identifying websites that treat traffic from the Tor network differently as the first step in driving change. Engaging with major players on the web such as CloudFlare in order to brainstorm possible solutions ought to be the next stage in the process, according to researchers.

ISPs as well as content delivery networks are also part of the problem but making progress on that front may be difficult, Khattak warns.

"There is not much we can do in the case of entities such as ISPs and countries that preemptively block all Tor exit nodes as a matter of policy, beyond some alleviation in the form of awareness campaigns to highlight the problem (such as, Tor’s “Don’t Block Me” initiative)," she writes. "With abuse-based blocking, we need solutions to enable precise filtering beyond IP address blocking of Tor exit nodes, so that benign Tor users don’t have to suffer from the abusive actions of other Tor users sharing the same exit node." ®