Happy Friday: Busted Barracuda update borks corporate firewalls
IT admins left with dead boxes blocking all traffic
Updated A firmware update pushed to Barracuda firewalls has knocked out boxes in large firms and crippled networks, we're told.
The change was sent out to customer gear by Barracuda on Friday morning Pacific Time, according to a source familiar with the matter. The update is automatically installed, and promptly caused the devices to fall over.
It is believed bugged geolocation data in the update causes the hardware's processors to overload and crash systems. Rebooting boxes does not clear the problem: you're still left with a dead firewall. The geolocation database is typically used to filter traffic from particular countries.
Barracuda's support team admitted this is a "known issue," and the biz is working on a fix: some customers have already received corrected data. However, applying the fix involves manually changing files so it you're a European Barracuda user, your Friday night is not going to be much fun.
"A support rep had to connect in to our network and run some commands on every firewall to get it going," a source at one Barracuda customer told El Reg.
"This was a little tricky as at least one of our sites was COMPLETELY down. If you rebooted your firewall (as we did in one site) to try and clear the problem, then you were completely down after it booted. Support told me after I called in not to reboot but it was too late.
"I was originally told it was something related to the geolocation reference data (which is updated periodically during the day). I would expect any data that gets pushed to EVERY customer device would go through some rigorous QA. This isn’t the first time their QA group have failed us."
Our reader is right: last November Barracuda’s Email Security Service took a dive, for example. The fault was thought to have been caused by a denial-of-service attack against Barracuda's servers, although the company declined to confirm that.
"Good news: no one was ever exposed from a security perspective," a spokesperson for Barracuda told us. "Bad news: [this is] not what the comms person wants to hear about on Friday afternoon." ®
Updated to add
A Barracuda spokesperson has been in touch to add: "We were able to proactively identify an error in the application definition file, and confirmed that there was no security exposure to our customers. The problem was quickly resolved and we are working with impacted customers to ensure all firewalls are updated with the correction."