Pot. Kettle. Black.

Buy something, sell it. I know stupid passengers are being ripped off not buying from source but my friendly local corner shop sells milk at a quid a pint and I know that going to Sainsbury's I can get it at 60p. But why should it be illegal? StupidJet

Moneysupermarket.com for example uses something in the region of two million IP addresses.

How does that work then? I doubt they own 2 million IP addresses as the IP4 address space is running low.

Anyone have the technical details?

I can understand sites which resell flights with their own mark-up, like Expedia. However, sites which depend on agent's fees or advertising revenue, and still send you to Easyjet's site for the final booking, like Skyscanner, I hope would be exempt. Mostly because Skyscanner is a great aggregator, and I would probably just disregard any cheapo that opted out from having their flights appear on it.

While I am sure this is very effective at stopping people blocking MoneySupermarket it is a little wasteful of address space. Also, shouldn't the vendors have the right to choose if they are exploited by screen scraping bots such as this?

What if expedia and the like are screen scraping another service (paying for the EasyJet data) and this third-party doesn't prohibit screen-scraping?

Sounds like a bit of a nightmare proving where the data has come from originally (although I'm sure Expedia would play nice and fess up)

(No doubt there's a backlog of similar comments wending their way through the moderation system but here goes...)

2 million IP addresses?! I would have thought even Google doesn't have that number of machines, let alone that number with individual *public* IP addresses.

Me thinks the PR monkey needs more training...

Moneysupermarket.com uses 2 million ip addresses? Surely that's not right. Now we know where all those ipv4 addresses have gone!

"...argue the defence that it was unaware of easyJet's terms and conditions, which prohibit screen scraping."

If a company publishes their prices online, then how can they stop someone else from using them? How do some silly T's&c's restrict that?

Price comparison websites provide a simple service. They just automate an activity that you or I would have to do manually - i.e. visit each individual site and compare the costs (including delivery charges, time to deliver, delivery options etc).

I work in Holland, so when I need to fly over, I simply visit the sites of KLM, BA and BMI; manually compare the flight costs and options (times). I don't see the problem in Expedia (or anyone else) automating that for me.

They also provide advertising for companies, too. I.e. Easyjet should pay Expedia for showing off their cheap prices to people that might not otherwise have gone to their site or known that they service the route.

...

I just don't get this... Is it a simple case of easyJet's Lawyers having too much time on their hands, or a CEO on a stupid power trip?

So that's why we need IP6! Seriously? 2 million, when ISPs won't even issue Class C ranges to most companies? What do they need them for apart from dubious anti-blocking practices?

Hi,

Has Moneysupermarket created the biggest bot army ever? How can it have 2 million IP addresses? And of course all in several blocks all around as I can block all the 16 millions address of one class A range in 2 seconds.....

Sounds like some people are still thrying to BS the Daily Mail journalist (oops sorry.... I know they don't recruit any!).

"No, we're not going to give free advertising to these websites"

"But [price comparision website] Moneysupermarket.com for example uses something in the region of two million IP addresses making the website very hard to block."

And I just checked out their website, and will probably use it to book my flight next month, as it offers a great comparison service. Interestingly, on exactly the same flight that I was going to book, expedia charges around 16 pounds more than ebookers.

Im guessing ebookers are also the recipient of easyjets emails...

If they are targeting price comparison sites too, then they need to be very careful about this - the ONLY reason I travelled to Cyprus on Easyjet a few weeks ago was because I saw their flights slightly cheaper than others on Skyscanner. I didn't even know Easyjet were flying to Cyprus now, so with little advertising on Easyjet's part and without that site, they wouldn't have had my bum on that plane.

Why don't sleazyJet just use a Captcha, like everyone else does?

inetnum: 193.195.186.8 - 193.195.186.11

netname: MONEYSUPERMARKET

erm...

quantity 3 public IP numbers

Not quite 2 million, then ;-)

What exactly is EasyJet's problem?

Expedia are only making use of information which is already available to the general public, and EasyJet are still getting paid for the flights.

The only people who can be said to be losing out are the customers, because Exdedia must be adding a markup. But presumably they are happy to pay this markup if it means they do not have to check prices right, left and centre.

So, that will be quantity FOUR public IP addresses, then

It's Friday...

I guess easyjet won't be competing on price any more then - that's the only reason I could find for not wanting to appear on comparison sites any longer - instead they'll be trading on reputation. Bwahahaha.

Actually they'll be trading on their reputation as a cheap'n'cheerless low-cost carrier that (used to) run the cheapest flights - and now wants more money (in order to stay in business with oil at $XXX per barrel) and would prefer everyone not to be able to compare them with their competition.

Yup, a deluded owner with pretensions towards grandeur and a hypocritical denial of their under-cutting start in the industry.

PH because she's easy and also owns a jet and is also orange coloured.

One-and-a-bit public IP addresses, even:

193.195.186.8 is the network address.

193.195.186.9 is by convention reserved for the router (but that machine can be running its own daemons).

193.195.186.10 is a usable public IP address.

193.195.186.11 is the broadcast address.

that they've missed an opportunity to gouge the customer. They'll be charging for clean air next.

So let's get this straight. EasyJet want sites who are advertising their brand for free, and in the process bringing them customers without paying commission rates, to stop or face legal action? Stelios really needs to start thinking about harnessing the opportunities from working with the likes of Expedia etc, not trying to make them sworn enemies.

Considering the onslaught of RyanAir this is not exactly the brightest commercial decision ever made. But then, EasyJet have a history of dumbass commercial decisions.

Thought I'd check out the site with the poor TV adverts, just to see what all the fuss is about, comparing prices of flights I have already booked direct with BA for later this year (at the time easyjet were more expensive!). The site asks me how many travellers, so I respond 3 since there's 2 adults and an infant who'll sit on our laps- clicking through to easyjet there's no where to actually specify ages! I doubt I'll go back there...

I've flown Easyjet in the past, but the prices are just not low enough to put up with all the long queues, extra charges to check in baggage, seating (non assigned & squashed in). They should welcome any other site that generates them some traffic!

indeedy, these conventions apply

point being: a tiny public address presence

real point being: irrelevant & ignorant hype by spokesperson in the actual debate about corporation X getting all litigious about competitor Y 'screen scraping' (alliterative euphemism for 'thieving?)

I flew from Zurich to Frankfurt on Easyjet in January. The only reason I knew that route even existed was because I saw it on Expedia. I booked directly on the Easyjet website - but it was because of Expedia that Easyjet got my business.

It's not really that hard to protect your website against most scrapers with a little bit of work.

For one thing, you display your prices on screen as an image, and never have the price shown within the page as a text entry. That'll slow the buggers down.

Also, having lots of variability within your code, different form element names, different page structure, etc, all randomised, will make Mr Scraper's job very difficult.

Jolyon

They have about 60 IP addresses for the travel side. The 2 million IP addresses are those of the customers deeplinking into the providers site, by banning travelsupermarket/moneysupermarket they would be turning away that many customers.

Sounds more like an excuse from the lazy admins at easyJet

193.195.186.8 - 193.195.186.11

and also

194.159.99.32 - 194.159.99.63

So, more than 4 but less than 2 million.

If Paris's millions are only really 32 then I'm going to have to stop loving her (money)

So basically on top of the allready very slow sites they typically run, you want to add multiple website errors and some big images? Heh.

I suspect the reason easyJet are doing this is because they depend heavily on a pricing model that relies on customers behaving in a certain way.

The models will depend on only a small minority of passengers buying the cheapest tickets and the vast majority paying full price. This is how they acheive seemingly much cheaper flights while maintaining a profit margin. The no frills aspect doesnt really represent a whole lot of price difference.

Advertising cheaper tickets more widely will break the pricing model and hence they take the action.

Ah, but if you display key information as images, you're making your site inaccessible to blind people who use screenreaders, therefore effectively breaking the law under the Disability Discrimination Act!

>So basically on top of the allready very slow sites they typically run,

>you want to add multiple website errors and some big images? Heh.

Not really! A price in an image as a PNG or GIF is going to be bugger-all in size (yes much > than plain text, but not a significant hit overall).

Plus, who says randomized page content and form field names, etc would have to generate "website errors". Do it properly and it can still all generate perfectly compliant and valid HTML. It's just it'll be slightly different HTML every time you look at the source, but the result would render the same.

Now.... I'm not suggesting that it is right or wrong for companies to be scraping EasyJet's content, I find the comparison sites very useful - but not always cheaper - trying to book a flight to Texas recently found that the air fair on continental.com was far cheaper than any of the fares - even from continental - listed on the main comparison sites. Same with airport parking - almost always cheaper to go direct to the parking company websites and join their 'club'!

So, I'm the comparison site's worst customer. I use them as an index and then go and check up/book directly on the vendor's website. So I'd love these people to keep scraping the EasyJet site for my benefit!

Jolyon

...because they would have to have a name, so the price comparison site could just pick up on the name and use that. More of a pain, but workable.

Unless of course you had a cipher that renamed the images at the server prior to each refresh... or some other crap like that!

"It's not really that hard to protect your website against most scrapers with a little bit of work.

For one thing, you display your prices on screen as an image, and never have the price shown within the page as a text entry. That'll slow the buggers down"

Doing that would breach the Disabilities Discrimination Act (Many companies breach it by doing this sort of thing, Ingram Micro etc). Those people with poor or no sight can have web pages 'read' to them, putting text into a graphic stops this being possible.

You can have Text as Graphics if you also include the text e.g. alt="my data"

Tony - it would be easy to write a script that generated images with a unique URL. Not every URL is a file - many (most?) are caught and the content is generated automatically by a script. This can be done with images too.

Chris - it isn't illegal to use text only as graphics. The law says the site operator must make 'reasonable adjustments' to make it suitable for disabled users.

If there is evidence that the site is being 'scraped' - it might not be reasonable to make the prices available as text (unless there was a better way to stop the site being scraped).

Even then, websites could allow alternative methods - e.g. a button that will play a sound file of the price.

As EasyJet.com is a very well known website, EasyJet would have to be careful to make sure any changes they make complied with the law, but - as ever - nothing is black and white.

You have raised a very good point.

Damn you both!

Jolyon

I guess EJ want to sell travel insurance, hotels, car hire, etc etc on top of the wafer thin flight margin to make some more dosh. (not used expeidia, but I guess they also sell these prodcuts too)

...and ends up with a revenue cut of 40% of sales which previously came via those sites and now goes to their competitors instead. Great success!

if easyjet dont want access to experions customers and marketing in this cooling economic climate, then more fool them.

Whereas your allocations would most likely be correct in a LAN environment this is not a LAN environment.

It is unlikely that an address range of 8-11 will use 8 as subnet and 11 as broadcast as this would (a) make half the addresses unusable and (b) be pointless.

You can have a subnet with all zeros or ones, since about IOS version 12.0 it has been default on Cisco kit to allow this. (previously it was allowed, it was just not allowed by default).

As an example look at a typical home user type allocation of say:

1.1.1.1/32

In this instance, 1.1.1.1 would be the network, 1.1.1.1 would be the broadcast and 1.1.1.1 would be reserved for the router leaving -2 addresses available for your computer.

In an internet facing scenario, every IP address will belong to a NAT router, it is extremely rare (and dangerous) to attach non-firewalls directly to the interweb in a corporate environment.

Look at

http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080093f18.shtml

for some explanations.

"Chris - it isn't illegal to use text only as graphics. The law says the site operator must make 'reasonable adjustments' to make it suitable for disabled users."

You are correct in that the law doesn't say that expicitly, but how else do you "make it suitable for disabled users." having a button that played a sound file to you is the only other way I can see of complying, and that wouldn't realy be sensible. Can you think of another way?

I don't have a sight problem but do like to save information out and search it. e.g. I get sent many many details of special offers and may want to find them some time later, if it comes to me as text then that can easily be filed and searched, as graphics no chance.

I've asked companies like Ingrams a number of times how do they think they comply, but they never reply.

A lot more than 4, but still < 2M:

inetnum: 91.102.184.0 - 91.102.191.255

netname: UK-MSFG-20061106

descr: Moneysupermarket.com Limited

country: GB

org: ORG-ML58-RIPE

and would not always have the pain tolerance to look at an individual airline's website. Plus Easyjet should be grateful for any business they can get, especially with the price of oil rising daily (and don't they have many aircraft on order?) I don't think you need to be a chartered accountant to work that out.

Nothing has to be passed to the browser that would give it any clue what the images are about. Here, for information only, is how it works:

1. The server sends a cookie to the browser when a login session is established. That cookie is a filename (or a key within a database) on the server. The file (or database record) contains the actual price to be disguised.

2. When your browser asks for the images showing the prices, it sends back the same cookie along with each request..

3. The server looks up the price in the appropriate file or database, renders the text as an image and returns the image to the browser.

Not that I condone such practices for a minute. I actually think Expedia are doing EasyJet a favour by sending customers their way ..... but no doubt EasyJet will have to mention out loud in court whatever it is that they think they are being deprived of.

How does this work?

How does Expedia sell Easyjet tickets? Surely your credit card transaction has to go via Easyjet or else Easyjet won't get paid. If that happens then how does Expedia get its cut? If Expedia take your money and then buy the ticket on your behalf using their own account Easyjet could simply refuse the transaction.

I'm not denying they can do it, I'd just like to know how.

Also why is this not a problem for Ryanair? I've just tried Stansted-Krakow 8th July, return 15th July. On Expedia all I get is LOT at over GBP500 (change in Munich) but at Ryanair I can get it for GBP120.

Actually I tried Luton-Krakow for same dates and Expedia finds nothing at all but Easyjet has seats at GBP137. So has Expedia already caved in? Or is Expedia, as I've always suspected, only interested in selling expensive tickets and gets a commission from the airlines like any other travel agent.

IT? because there are only two icons with a question mark and I don't wan't to go to Paris.

According to yesterdays article, Easyjet flights are available through the GDS. So why would Expedia need to screenscrape - Easyjet are putting the info in the GDS!

GDS queries may cost more than web scraping, but I'm pretty sure that Expedia doesn't webscrape every other airlines website for prices - getting a comparison would be next to impossible if they had to wait for slow airline websites to respond.

Expedia can also use the GDS to book the flight on for _their_ customer. If Easyjet doesn't want companies to use the GDS to book Easyjet flights, they shouldn't be listing their flights there.

When I go web shopping to certain web sites I often use two or more computers. The reason for this is variable pricing where the price offered to you reflects the level of interest that you have in the product. The specific products offered will vary as well. So depending on what I have in mind I often come in with a 'clean' computer, pull the URL and then hit the site from the computer I do the purchasing from. This seems to bypass the logic.

I suspect Easyjet is also using this type of mechanism. They are locked in a battle with consumers -- they want to sell the product for more than the consumer wants to pay so there's a delicate confidence trick being worked. If EJ knows you're a regular customer who's not that discriminating (doesn't comparason shop) then they'll drop a bit more onto the price.

Its a fun activity, a bit like watching the price tags in stores go up and down throughout the week. Its also a waste of my time -- I usually haven't got the time or the interest for these games.

So a third party website (acting like my 'clean' computer) messes the game up which is why EJ doesn't like it. They don't know who's buying the ticket so they don't know how much they can charge.

BTW --- Amazon was the first to experiment with this.

Seriously, how do these website purchase tickets? I suppose they could have a system that just registers with the Easyjet website and uses that (easy enough with a little ingenuity).

However, this would also be expensive, as they would buy the tickets at the same cost as the consumers, then have to add something to make a profit.

Still, having seen the prices at Expedia and Lastminute, they may well be doing this.

AVG seems to have a lock on having multiple computers on the Internet running their product. Since we now know that AVG will auto-follow links and pre-load and pre-screen content, it isn't that far of a stretch to sell this (screen scraping) service to Easyjet competitors. This alone could cover the developments costs of AVG and allow it to become commercial freeware.

The counter-measure to this would be unthinkable as Easyjet would have to identify and block nearly every AVG user.

RyanAir regularly send out similar letters. In fact Expedia even help RyanAir to identify sites that are selling their flights!

@Anonymous Coward (GDS): I am a developer for a site similar to expedia and I've never heard of 'GDS' as an API to buy flights from. We can buy flights through different systems, ranging from screen-scraping ViewData (an old teletext-style booking interface), screen-scraping websites, scheduled flat-file imports (in CSV, EDI or AVALIBLE format) or ideally XML services. The majority of low cost airlines we get through screen-scraping. We send search requests to all the airlines simultaneously and if and airline doesn't respond in time then they won't make it into our flight results.

@Stuart Castle: Yes - most travel sites selling EasyJet/RyanAir tickets will be booking the tickets on the website's debit card, then adding their own markup and debit / credit card fee on top. Without the low cost airlines offering account based systems they don't have much choice.

(Posted as anonymous because I don't want this to get me or my employer in trouble)

"However, this would also be expensive, as they would buy the tickets at the same cost as the consumers, then have to add something to make a profit.

Still, having seen the prices at Expedia and Lastminute, they may well be doing this."

According to these articles, that's exactly what Easyjet are alleging the screen scrapers do. And indeed that is exactly how a screen-scraped interaction works. Dead easy with Python, Perl, PHP or similar.