Feeds

Insecure indexing risk dissected

How did THAT get out?

  • alert
  • submit to reddit

Internet Security Threat Report 2014

It's embarrassing when future PR items, upcoming security advisories or boilerplates for obituaries that are not meant to be visible to external users drift into the public domain. These documents might get accidentally uploaded to the wrong part of a website but mischievous attacks can also play a role.

Web application security researcher Amit Klein this week published a paper explaining how "insecure indexing" allows attackers to expose hidden files on web servers. Some site-installed search engines index files that search engines are programmed to ignore. Typically search engines look in a root domain for a special file called "robots.txt" which tells the robot (spider) which files it may download.

If an attacker can get to internal search engines he can get around files denied to him by the Robots Exclusion Standard. Klein explains that these attacks are "fundamentally different from exploiting external (remote) search engines".

Klein explains various attack techniques, ranging all the way from guessing a file name from names that already exist to targeted search strings and far more complicated traffic-intensive attacks, and concludes with methods for detecting insecure indexing and suggested defences. "Crawling style indexing should be preferred over direct file indexing. If file-level indexing cannot be avoided, more consideration should be made when deploying a search engine that facilitates it. In particular those search engines should be systematically limited to the visible resources (or at the very least, to accessible resources)," he writes.

The paper - Insecure Indexing Vulnerability: Attacks Against Local Search Engines - can be found on the Web Application Security Consortium's site here. ®

Related stories

Botnets strangle Google Adwords campaigns
Phishers suspected of eBay Germany domain hijack
Interview with a link spammer
Google's No-Google tag blesses the Balkanized web
Google exposes web surveillance cams
Major flaw found in Google Desktop

Internet Security Threat Report 2014

More from The Register

next story
George Clooney, WikiLeaks' lawyer wife hand out burner phones to wedding guests
Day 4: 'News'-papers STILL rammed with Clooney nuptials
Shellshock: 'Larger scale attack' on its way, warn securo-bods
Not just web servers under threat - though TENS of THOUSANDS have been hit
Apple's new iPhone 6 vulnerable to last year's TouchID fingerprint hack
But unsophisticated thieves need not attempt this trick
PEAK IPV4? Global IPv6 traffic is growing, DDoS dying, says Akamai
First time the cache network has seen drop in use of 32-bit-wide IP addresses
Oracle SHELLSHOCKER - data titan lists unpatchables
Database kingpin lists 32 products that can't be patched (yet) as GNU fixes second vuln
Researchers tell black hats: 'YOU'RE SOOO PREDICTABLE'
Want to register that domain? We're way ahead of you.
Stunned by Shellshock Bash bug? Patch all you can – or be punished
UK data watchdog rolls up its sleeves, polishes truncheon
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.