At times G-theory may experience system outages or conduct server maintenance. This could be due to any number reasons. Notices will be posted here whenever an event has occurred explaining the reason for it. If you have any questions about any of the notices you see here please contact us.

[Mar 11, 2005, 11:21 AM] Level3 Connectivity Issue
We are seeing intermittent connectivity issues with Level3 this morning. The explanation we've been given is they have several circuits down, and the BGP session on the router terminating those circuits has failed to timeout. Thus, Level3 is accepting traffic for routes it does not have available, and subsequently blackholes the traffic. We have removed Level3 from our BGP route announcements; however, customers whose local ISPs use Level3 for transit may still experience intermittent problems until the issue is resolved.

[Feb 26, 2005, 09:57 AM] Scheduled Maintenance
Beginning around 11:30 a.m. Eastern time this morning, all www and mail servers will be taken down briefly for firmware upgrades. Downtime per server should be approximately 10 minutes.

[Feb 08, 2005, 08:55 PM] Scheduled Maintenance
Around 9:00 p.m. Eastern time, we will begin maintenance on all user servers to apply security updates. Downtime per server should be under 10 minutes.

[Dec 08, 2004, 08:34 PM] Maintenance Complete
The scheduled network maintenance is complete. Downtime was under 15 minutes.

[Dec 08, 2004, 04:50 PM] Planned Maintenance
Around 7:30 p.m. this evening, we will begin performing scheduled maintenance on various elements of our network. This will result in a brief service outage. Another notice will be posted when the maintenance is complete.

[Dec 02, 2004, 05:37 AM] Scheduled Maintenance
Beginning around 6:00 a.m., all servers will receive system maintenance followed by a reboot. Downtime per server should be less than 5 minutes.

[Oct 12, 2004, 01:29 PM] Network Outage
Shortly after 1:00 p.m. today, a BGP routing issue caused parts of our network to become temporarily inaccessible. The problem was corrected after a duration of approximately 15 minutes. We apologize for any inconvenience.

[Aug 05, 2004, 03:34 PM] Network Downtime
At approximately 2:55 p.m. Eastern time today, a massive utility power feed spike critically damaged the primary UPS cluster which powers the datacenter in which our equipment resides. Power was promptly restored from secondary UPS clusters and backup maintenance power. Due to the severity of damage, the primary UPS cluster has been taken out of service and will be replaced within the next few hours. Total downtime was approximately 12 minutes.

[Jul 30, 2004, 09:50 PM] Scheduled Maintenance
Tonight after 10:00 p.m. Eastern time, all servers will undergo maintenance followed by a reboot. Downtime per server should be less than 5 minutes.

[Jul 13, 2004, 07:04 PM] Scheduled Maintenance
Tonight after 8:00 p.m. Eastern time, all servers will be taken offline briefly for maintenance. Downtime per machine should be less than 5 minutes.

[Jul 13, 2004, 02:14 PM] Temporary Outage
Shortly before 2:00 p.m. Eastern time, we experienced a fiber issue connecting a primary distribution switch to the network core. This caused a brief but widespread blackout to many parts of our network. Downtime was less than 15 minutes.

[Jun 25, 2004, 02:05 PM] Server Downtime
www2 and our primary mail server were taken offline briefly for unscheduled maintenance. Downtime was less than 10 minutes per server.

[May 16, 2004, 03:30 PM] Email Virus Filtering Implemented
As you know, the level of virus infected email traffic worldwide has increased dramatically over the past year, and poses a serious threat to individual and corporate systems. We are pleased to announce that as of today, all G-theory mail servers have been upgraded with server-side email antivirus filtering. All incoming mail will be scanned for viruses prior to delivery.

[May 04, 2004, 09:20 PM] Mail Server Upgrades
This evening after 9:30 p.m. Eastern time, the mail servers will be taken offline briefly for upgrades. Downtime per server should be less than 5 minutes.

[Apr 13, 2004, 11:42 PM] System Updates
Between 11:00 and 11:40 p.m. Eastern time, all user servers received security updates followed by a reboot. Downtime per server was less than 5 minutes.

[Apr 10, 2004, 01:52 PM] Server Maintenance
Beginning around 2:00 p.m. Eastern time, all www and mail servers will undergo system maintenance, followed by a reboot. Downtime per server should be less than 5 minutes.

[Mar 21, 2004, 10:56 PM] Critical Outage
This weekend our www server suffered catastrophic failure resulting in an extended outage for much of the day Saturday while the source of failure was identified. Due to the highly unusual mode of failure, a multi-stage recovery was necessary to correct the cause of the failure and to minimize the chance of customer data loss during recovery. The most recent backups were performed after 11:00 p.m. Eastern time on Friday, prior to the outage which began around 8:20 a.m. Saturday. Services were tentatively restored around 7:00 p.m. Saturday, while we continued recovery during the day Sunday. Another brief outage of approximately 15 minutes occurred Sunday evening around 8:30 p.m. while we deployed the final stage of recovery and brought the affected server back to full normal service.

[Mar 06, 2004, 09:00 AM] www Server Maintenance
Beginning at 9:00 a.m. all user www servers will have their web services stopped briefly for a security configuration change. Downtime for each server should be less than 2 minutes.

[Mar 05, 2004, 03:16 PM] Virus-Generated Fake E-mails
A new worm, identified as "W32/Bagle.j@MM", is propagating through Windows systems via e-mail. It harvests e-mail addresses from archives and/or address books on infected systems, and generates vaguely plausible forged e-mails to other users based on the domain names it finds in those e-mail addresses.

The forged e-mails contain a password-protected ZIP file which is used to propagate the infection, as well as a variety of text in the body which is designed to look like legitimate mail from, for example, "gtheory.com". Needless to say, we are not sending out any such notifications, and we recommend that no user should ever run an attachment received via e-mail.

More details on this worm are available at http://vil.nai.com/vil/content/v_101071.htm.