There were two notable outages during the week. On March 3, UUNETVerizon experienced an outage that impacted many of its peers and customers, including, Bank of America, JP Morgan Chase, Nomura, Samsung, and Zoom. The outage, lasting around 36 minutes over a 75 minute period, was first observed around 9:00 a.m. EST and appeared to center on UUNETVerizon nodes in Philadelphia, PA, and Ashburn, VA. This initial part of the outage lasted around 4 minutes and appeared to have a cascading impact on Cogent infrastructure located in New York, NY, and affected Cogent’s path to the JP Morgan Chase network. Approximately five minutes after the initial outage cleared, a second was observed that lasted around 13 minutes. It was observed on UUNETVerizon nodes located in Seattle, WA and Dallas, TX, as well as appearing to have a cascading impact in Level 3 Communications infrastructure located in Seattle, WA, and affecting Level 3 customers and partners in Canada. Five minutes into this second period, the Level 3 infrastructure direct outage cleared and after another five minutes, the only UUNETVerizon nodes exhibiting the issue were located in Dallas, TX. Around 9:50 a.m. EST, the third occurrence of the outage was observed 20 minutes after the second. This outage lasted around 19 minutes and was initially focused on UUNETVerizon infrastructure in Dallas, TX. Five minutes into the third period of the outage, UUNETVerizon infrastructure exhibiting problems expanded to include Seattle, WA. Approximately 10 minutes into this third period of the outage, UUNETVerizon infrastructure located in San Jose, CA was added to those in Seattle, WA, and Dallas, TX. At around 10:10 AM ET, the UUNETVerizon infrastructure located in San Jose, CA, was the only infrastructure exhibiting issues. The outage was cleared around 10:15 AM ET. Click here for an interactive view of the outage.
On March 3, PCCW experienced an outage affecting some of its U.S. customers and networks, including Flagstar Bank, Target, Bloomberg, Morgan Stanley, and Dell. The outage lasted around 31 minutes and was divided into three periods over an hour and 20 minutes. The outage was first observed around 8:45 a.m. EST and appeared to center on PCCW infrastructure located in Ashburn VA. The first period of the outage lasted around 9 minutes, before recurring 15 minutes later, again centered on PCCW infrastructure located in Ashburn, VA. This second outage lasted around 19 minutes. The third period was observed 30 minutes after the second ended and lasted around 9 minutes. The outage was cleared around 10:05 a.m. EST.
Click here for an interactive view of the outage.
Update March 1
Global outages across all three categories jumped from 279 to 393, a 41% increase over the week before. In the US outages went from 138 to 184, up 33%.
ISP outages rose from 233 to 311, a 33% increase worldwide, and from 123 to 166 in the US, a 35% increase.
Cloud-provider network outages globally jumped from 5 to 22, a 340% increase. The US accounted for two of them, up from one the week before.
Collaboration-app network outages dropped from four to two globally and from two to zero in the US.
There were three notable outages this week.
On Feb. 23, LinkedIn experienced a service disruption affecting its mobile and desktop global user base. The outage was first observed around 1:50 p.m. EST, with users attempting to connect to LinkedIn receiving server-unavailable error messages. Around 45 minutes later, services to some regions began to return, although others were still unable to use the services. After another 45 minutes, the server unavailable messages were replaced with content not available errors. The total disruption lasted around two hours, during which no network issues were observed connecting to LinkedIn web servers, further indicating the issue was application related. Service was restored around 3:40 p.m. EST. Click here for an interactive view of the outage.
On Feb. 25, Hurricane Electric experienced an outage that affected users across inthe US, UK, Australia, South Africa, New Zealand, Germany, Canada, Japan, Spain and Brazil. The outage, lasting around 36 minutes over a 45-minute period, was divided into two events. The first was observed around 1:40 a.m. EST on Hurricane Electric infrastructure in Singapore and Hong Kong. It lasted 24 minutes and initially cleared around 2:05 a.m. EST, but five minutes later, around 2:10 a.m. EST, a node in Marseille, France, was affected and created issues for around 12 minutes. The outage affected access to customer networks including Credit Suisse, Proctor and Gamble, DBS, Shell, and Bank of America. The issue was cleared around 2:25 a.m. EST. Click here for an interactive view of the outage.
On Feb. 24, Comcast Communications experienced an outage that affected peers and customers in the US, Canada, and the Netherlands. The outage, lasting around 14 minutes, was first observed around 11 p.m. EST and appeared to be centered on Comcast nodes in Newark, NJ, and affected access to customers including CBS, NBC, Bloomberg, and JP Morgan Chase. Ten minutes into the outage, the radius of the disruption expanded to include Comcast nodes in New York, NY, Dallas, TX, Chicago, IL, and Boston, MA. It affected customers in more countries including the US, UK, Australia, Canada, France, and the Netherlands. The outage was cleared around 11:15 p.m. EST. Click here for an interactive view of the outage.
Update Feb. 22
Outages overall were up 35%, from 206 to 279, compared to the week before. In the US they were up 53%, from 90 to 138.
Globally, the number of ISP outages jumped from 154 to 233, a 51% increase, and in the US, they increased from 78 to 123, up 58%.
Cloud-provider network outages dropped from 24 to five, a 79% decrease worldwide. In the US, they dropped from five to one.
Collaboration-app network outages doubled from two to four globally and from one to two in the US.
On Feb. 16, Level 3 Communications experienced a notable outage that affected multiple downstream partners and customers in countries including the US, Canada, Argentina, Mexico, and the UK. First observed around 11:40 a.m. EST, the outage lasted around 36 minutes over a one-hour period and affected access to customers including Bank of America, TiVo, and Lending Tree. The first minutes of the outage appeared to center on Level 3 nodes in San Francisco, CA, affecting only the U.S. and Canada. Ten minutes into the outage, nodes in Boston, MA, became involved, and at this point the impact spread to include the other countries. Five minutes after that, the outage in the San Francisco node cleared leaving just the Boston infrastructure in an outage condition. Twenty-four minutes after the outage was first observed it appeared to clear. Fifteen minutes later the outage reappeared, this time centered on nodes in San Francisco, Salt Lake City, UT, and Portland, OR. This second episode lasted for four minutes and was followed by two more four-minute outages each five minutes apart. The outage was cleared around 12:45 p.m. EST. Click here for an interactive view of the outage.
On Feb.18, GTT Communications experienced an outage that affected some of its partners and customers in the US. It lasted around 14 minutes, was first observed around 4:30 a.m. EST, and appeared to center on GTT nodes in Los Angeles, CA, affecting customer networks including Ford Motor Company, Guaranteed Rate and Loanet. The outage was cleared around 4:45 a.m. EST. Click here for an interactive view of the outage.
Updated Feb. 15
Worldwide outages across all three categories decreased from 267 to 206, a 23% drop from the week before. They dipped 4% in the US, from 94 to 90.
ISP outages decreased 20%, from 192 to 154, worldwide, and they stayed the same in the US at 78.
Globally cloud-provider network outages increased from 21 to 24, up 14%, and in the US bumped up from three to five.
Global collaboration app network outages remained at two for the second week in a row, while US outages dropped from two to one.
Around 4:30 a.m. EST on Feb. 11, AT&T suffered an outage centered in Washington, D.C., followed by issues in Tulsa, OK, and San Antonio, TX. that affected customers including some in the US, Germany, and Australia. Five minutes later only Washington, DC, and Tulsa, OK, nodes were involved and five minutes after that, just those in Washington, DC. Customer networks affected included J.P. Morgan Chase, Jeffries Group, Travelers Property Casualty, and ConocoPhillips. The outage lasted 14 minutes and was cleared at around 4:45 a.m. EST. Click here for an interactive view of the outage.
On Feb. 11, Cogent Communications experienced a series of outages over a period of 3 hours and 38 minutes that affected downstream providers as well as Cogent customers globally. The outage lasted a total of 43 minutes and was first observed around 1:07 a.m. EST centered on Cogent nodes in San Francisco, CA. This initial outage lasted around two minutes, and the Cogent environment was then stable for 43 minutes before experiencing a series of four-minute outages observed on Cogent nodes in Dallas, TX and Phoenix, AZ. An hour and a half after the initial outage was observed, a four-minute outage was observed centering on Cogent nodes located in Newark, NJ. The outage reappeared 25 minutes later, extending the list of affected nodes to locations including San Francisco, CA, Atlanta, GA, Cleveland ,OH, New York, NY, and Washington D.C. Less than an hour later, a 19-minute outage affected nodes including Miami, FL, Austin, TX, El Paso, TX, Houston, TX, San Jose, CA, Los Angeles, CA, and Las Vegas, NV. Customer networks affected included Ford Motor Company, Oracle, Home Depot, and TikTok. The outage was cleared at around 4:45 a.m. EST. Click here for an interactive view of the outage.
Updated Feb. 8
Globally outages in all three categories decreased from 278 to 267, a 4% decrease. In the US, outages decreased from 119 to 94, a 21% decrease.
ISP outages worldwide decreased from 214 to 192, a 10% drop. In the US, ISP outages decreased from 102 to 78, down 24%.
Cloud-provider network outages worldwide dropped from 30 to 21, down 30%. In the US they dropped from 10 to three.
Collaboration-app network outages globally decreased from five to two and stayed the same, at two, in the US.
There were two notable outages this week. Around 1:40 a.m. on Feb. 1, Hurricane Electric experienced an outage affecting countries including the US, New Zealand, and Brazil. The outage was centered on Hurricane Electric infrastructure in Los Angeles, CA. After five minutes, the number of interfaces affected there reduced and appeared to affect users in the US only. The outage lasted around nine minutes and affected customers including Disney Streaming and LinkedIn. The issue was cleared around 1:50 a.m. ET. Click here for an interactive view of the outage.
On Feb. 2, TATA Communications (America), experienced an outage that affected some of its downstream partners and customers in countries including the US. It was first observed around 1:30 a.m. ET as TATA nodes located in Los Angeles, CA, appeared to show outage conditions. After five minutes, other nodes located in Seville, Spain, and Singapore, were affected. As the number of affected nodes increased, so did the number of customer networks affected, including Wells Fargo, Reuters, Twitter, and Salesforce. The outage lasted around 18 minutes across a half hour period and was cleared around 2 a.m. ET. Click here for an interactive view of the outage.
Update Feb. 1
Outages in all three categories globally ticked up three from 275 to 278 during the week and dropped from 132 to 119 in the US, an 11% decrease.
Globally, the number of ISP outages decreased from 217 to 214. In the US, levels slightly decreased too, from 106 to 102.
Cloud-provider network outages jumped from 18 to 30 worldwide, a 67% increase. US outages increased from nine to 10.
Collaboration-app network outages remained flat both globally and in the US, with five worldwide and two in the US.
Two notable outages affecting Comcast Cable and Verizon occurred during the week.
On Jan. 26, Comcasts suffered a 24-minute outage first observed around 12 a.m. ET that appeared to be centered at Comcast nodes in Newark, NJ, affecting access to customer networks including Amazon, Bloomberg and CBS. At 20 minutes into the outage, disruption was also observed in a New York, NY, node. The outage affected multiple Comcast peers and customers, and it was cleared around 12:25 a.m. ET. Click here for an interactive view of the outage.
On Jan 12 Verizon experienced an outage that affected East Coast customers abilty to access services including Slack, Zoom, Amazon and Google Traffic disruption was observed around 11:30 a.m. ET across multiple nodes concentrated along the US Verizon backbone. During the outage Verizon indicated a fiber cut affected service delivery in the Brooklyn, NY, area but is not believed directly related to the larger outage. Network services started to stabilize around 12:30 p.m. ET. Click here for an interactive view of the outage.
Update Jan. 26
Outages in all three categories worldwide rose from 215 to 275, a 28% increase compared to the week before. In the US they rose from 105 to 132, a 50% increase.
Globally, the number of ISP outages increased from 160 to 217, a 36% increase and in the U.S., they increased from 88 to 106, a 20% increase.
Cloud provider network outages increased from 14 to 18 globally, a 29% increase, while in the US they jumped from two to nine, a 350% increase.
Collaboration app network outages worlwide increased from two to five and from one to two in the US.
There were two notable outages during the week. On Jan. 20, Level 3 Communications experienced an outage affecting downstream partners and customers in countries including the US, South Africa, the U.K., Turkey, Russia, New Zealand, and Australia. The outage was first observed around 12:20 p.m. ET and lasted about 34 minutes. It appeared centered on Level 3 nodes in Washington, DC, and affected customers including J.P. Morgan Chase, Visa International, and Oracle. IT was cleared around 12:55 p.m. ET. Click here for an interactive view of the outage.
On Jan.18, TATA Communications (America), experienced an outage affecting many of its downstream partners and customers in multiple countries including the US, the UK, Australia, India, Singapore, Germany, Hong Kong, Japan and Canada. The outage was first observed around 7:40 a.m. ET. TATA nodes located in Newark, NJ; New York, NY; Frankfurt, Germany; London, England; Singapore; Paris, France; and Buckinghamshire, England all appeared to show outage conditions. After five minutes, TATA nodes located in Chicago, IL; San Jose, CA; Seville, Spain; Tokyo, Japan; and Hong Kong were affected. As the number of TATA nodes affected increased, so did the number of customer networks affected, including Wells Fargo, Reuters, Oracle, and Amazon. The outage lasted around 34 minutes and was cleared around 8:15 a.m.ET. Click here for an interactive view of the outage.
Update Jan. 19
Worldwide outages in all three categories increased from 157 to 215 over the previous week, an increase of 37%, and were up from 88 to 105 in the US, a 19% increase.