Feeds

MS struggles to contain the Slammer worm

: The worm that turned

  • alert
  • submit to reddit

Intelligent flash storage arrays

MemoWatch An insight into the problems faced by Microsoft indealing with the Slammer (aka Sapphire) worm is revealed in internal company memos leaked to El Reg.

The email memos from Microsoft security team (18 in all of which we publish only the first [most illuminating] four) reveal a giant corporation struggling to contain the effects of a virulent worm, which knocked many of its Internet services offline.

Those who blame system admins for the spread of the worm might do well to ponder on the fact even Microsoft had not applied its fix which, it's now clear, was far from easy to apply.



From: Mike Carlson (ITG)


Sent: Saturday, January 25, 2003 8:04 AM


To: EIS Data Center Clients; uDRM Operations Team; Gregory Wood's Direct


Reports; IR/ID


Cc: HUSK:Corporate Data Center Release Management; ITG DR Command Team; ITG DR Functional Command Team; Randy Coggan; Jim DuBois Direct Reports; Steven Rees

< Microsoft Confidential - Do Not Forward! >

Event Description:

At approx 10:00 pm traffic on the corporate network jumped dramatically, eventually bringing nearly all services to a crawl. The root cause appears at this time to be a virus attacking SQL servers on UDP port 1434. The problem is not confined to MS only, as there are reports of widespread impact at other companies and across the Internet.

Start Date/Time:

Approximately 10:00 PM PST, Friday 1/24/2003.

Impact:

All apps and services are potentially affected and performance is sporadic as best. The network is essentially flooded with traffic, making it difficult to gather details concerning the impact.

Status Update:

Confirmed SQL SP3 does protect the system from the virus, but believe we may have a hot fix that could be remotely applied to SP2 as well. All systems owners should make plans to update their systems asap.

ETA: None.

Action Required:

Owners should be planning to upgrade their systems asap.

Next Update: 0900



From: Mike DeGooyer


Sent: Saturday, January 25, 2003 9:15 AM


To: EIS Data Center Clients; Gregory Wood's Direct Reports; IR/ID


Cc: Broadband Networking All; Corporate Data Center Release Management; ITG


DR Command Team; ITG DR Functional Command Team; Randy Coggan; Jim DuBois Direct Reports; Steven Rees



Microsoft Confidential

Do Not Forward

Corporate Network Outage: 1-24-03

Update: Network traffic is still affected and inhibiting traffic. Effected resources have been identified and operations is isolating network traffic to apply the fix upon completion.

Next Update: 1000 PST or when implementation steps are ready.

Action Required:

None at this time. Effected resources owners will be contacted.

Description:

At approximately 10:00pm PST traffic on the corporate network increased dramatically due to a virus attack directed at SQL Server. The interruption was not directed at Microsoft Resources.

Event Summary:

8:00am Specific file versions and server impact is being assessed.

7:00am Engineering testing application of hot fix that could be remotely applied to SP2.

6:00am Engineering confirmed SQL SP3 protects systems. Preparing to engage effected server owners.

5:00am Identifying that patch that will fully protect the systems.

4:00am Virus behavior was identified. Working on resolution process.

3:00am SQL Dev and engineering are engaged isolating the issue.

2:00am Engineering engaged to isolate the issue and behavior patterns.

1:16am Notification sent clients.

____________

Mike DeGooyer
EIS Release Management- BGIT

[contact details deleted]



From: Mike DeGooyer


Sent: Saturday, January 25, 2003 11:23 AM


To: Mike DeGooyer; EIS Data Center Clients; Gregory Wood's Direct Reports;


IR/ID


Cc: Broadband Networking All; Corporate Data Center Release Management; ITG


DR Command Team; ITG DR Functional Command Team; Randy Coggan; Jim DuBois Direct Reports; Steven Rees; uDRM Operations (MSE); Corporate Data Center Release Management



Update: HELP NEEDED: If you have servers that are nonessential, please shut down the MSSQLSERVER service as well as SQL Agent (so SQL doesn't restart) so that we can eliminate nonessential noise/traffic on the network. Your urgent assistance with this will be very helpful.

SQL Development and Engineering are engaged. Network traffic is still affected and inhibiting traffic. Operations are isolating network traffic to apply the fix upon completion.

Next Update: 1200 PST or when implementation steps are ready.

____________

Mike DeGooyer

EIS Release Management- BGIT

[contact details deleted]



From: Chad Lewis


Sent: Saturday, January 25, 2003 12:35 PM


To: EIS Data Center Clients; Gregory Wood's Direct Reports; IR/ID; ITG DR


Command Team; ITG DR Functional Command Team


Cc: Broadband Networking All; Corporate Data Center Release Management;


Randy Coggan; Jim DuBois Direct Reports; Steven Rees; uDRM Operations


(MSE); Phil Nguyen (ITG); Mike DeGooyer; Building 11 MCSS Members; TK MCSS


Staff; Jim Pauley; Global Networks Operations Center; Steven Rees; Paul


Olson; Peter Tutak; MSN IA Core



**Microsoft Confidential - Do Not Forward!**

Status Update:

If you have SQL servers that are nonessential, please shut down the MSSQLSERVER service as well as SQL Agent (so SQL does not restart) so that we can eliminate nonessential noise/traffic on the network. Your urgent assistance is required.

Within the next 60 minutes, the 039 SQL patch will be scripted for deployment to all SQL servers outside the data center to reduce traffic volume on the network. For details on 039, visit
www.microsoft.com/technet/treeview/default.asp?url=/technet/security/bulletin/MS02-039.asp.
Impact will be a cycling of the SQL service.

We are investigating the best method of addressing the data center space and will communicate during next update.

Next update 1:30pm, or as needed.

Chad Lewis

EIS Release Management




By Monday morning (the time of the last memo sent to us) Microsoft had more or less got on top of the problem but reported it continued to "monitor client traffic volumes and corporate network impacts" caused by the worm.

All this disruption, which was also felt by banks, ISPs and other organisation, was preventable with a six-month old patch. But if Microsoft doesn't apply its own fixes what chance do the rest of us have? ®

Related Stories

ATMs, ISPs hit by Slammer worm spread
SQL worm slams the Net

External Links

SQL Slammer worm advisory by security tools firm ISS
More on the SQL Server 2000 vulnerability Slammer exploits, alert by Next Generation Security Software

Top 5 reasons to deploy VMware with Tegile

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.