Feeds

Insecure indexing risk dissected

How did THAT get out?

  • alert
  • submit to reddit

Intelligent flash storage arrays

It's embarrassing when future PR items, upcoming security advisories or boilerplates for obituaries that are not meant to be visible to external users drift into the public domain. These documents might get accidentally uploaded to the wrong part of a website but mischievous attacks can also play a role.

Web application security researcher Amit Klein this week published a paper explaining how "insecure indexing" allows attackers to expose hidden files on web servers. Some site-installed search engines index files that search engines are programmed to ignore. Typically search engines look in a root domain for a special file called "robots.txt" which tells the robot (spider) which files it may download.

If an attacker can get to internal search engines he can get around files denied to him by the Robots Exclusion Standard. Klein explains that these attacks are "fundamentally different from exploiting external (remote) search engines".

Klein explains various attack techniques, ranging all the way from guessing a file name from names that already exist to targeted search strings and far more complicated traffic-intensive attacks, and concludes with methods for detecting insecure indexing and suggested defences. "Crawling style indexing should be preferred over direct file indexing. If file-level indexing cannot be avoided, more consideration should be made when deploying a search engine that facilitates it. In particular those search engines should be systematically limited to the visible resources (or at the very least, to accessible resources)," he writes.

The paper - Insecure Indexing Vulnerability: Attacks Against Local Search Engines - can be found on the Web Application Security Consortium's site here. ®

Related stories

Botnets strangle Google Adwords campaigns
Phishers suspected of eBay Germany domain hijack
Interview with a link spammer
Google's No-Google tag blesses the Balkanized web
Google exposes web surveillance cams
Major flaw found in Google Desktop

Beginner's guide to SSL certificates

More from The Register

next story
You really need to do some tech support for Aunty Agnes
Free anti-virus software, expires, stops updating and p0wns the world
USB coding anarchy: Consider all sticks licked
Thumb drive design ruled by almighty buck
Attack reveals 81 percent of Tor users but admins call for calm
Cisco Netflow a handy tool for cheapskate attackers
Privacy bods offer GOV SPY VICTIMS a FREE SPYWARE SNIFFER
Looks for gov malware that evades most antivirus
Patch NOW! Microsoft slings emergency bug fix at Windows admins
Vulnerability promotes lusers to domain overlords ... oops
Oi, Europe! Tell US feds to GTFO of our servers, say Microsoft and pals
By writing a really angry letter about how it's harming our cloud business, ta
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Reducing the cost and complexity of web vulnerability management
How using vulnerability assessments to identify exploitable weaknesses and take corrective action can reduce the risk of hackers finding your site and attacking it.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.