The Iowa City Community School District has experienced multiple, short-lived internet outages this afternoon, and we’ve worked with our internet service provider – the Iowa Communications Network – to identify the cause as a denial-of-service attack targeting our network.
Direct denial-of-service (DDoS) attacks are relatively common, and we have resources in place to mitigate the impact of this type of attack on our network. That said, we are working with the ICN to ensure that those mitigation strategies are fully and properly implemented, given the downtime we’ve experienced today.
We do not expect substantial additional disruption, and will be working with the ICN to avert future problems.
Portions of the district will experience a network outage beginning at approximately 6 PM on Monday, November 19th, and lasting for 5-6 hours. This outage is due to work being done by the City of Iowa City on a fiber route along Dubuque Street that will allow for construction of a new building in the area.
Affected buildings will include schools in North Liberty and Coralville, schools in the western portion of Iowa City, and Longfellow Elementary School. For reference, users of portions of the City of Iowa City, Johnson County, City of Coralville, and University of Iowa networks may be affected by this work as well.
We don’t expect district-wide internet access to be impacted, and expect that the affected locations will come back up with no lingering effects when work is completed on Monday night.
Internet access within the Iowa City Community School District was interrupted this Sunday morning, August 26th, when Iowa Communications Network (ICN) engineers accidentally cut a district fiber optic connection in Tiffin, IA.
The cut has been repaired and service restored as of Sunday, August 26th at 2:15 PM. We will continue to monitor connectivity for issues going forward, but do not anticipate additional service interruptions.
In advance of winter break, I wanted to share a handful of dates when we’ll have network maintenance that will affect network, wireless, and/or internet availability within the district during the break.
January 2nd, 2018
Switch maintenance at Liberty High and Northwest Junior High will cause network disruptions in those buildings between 6 AM and 8 AM. Network and internet services may be unavailable during this time period.
Further work will be done throughout the day in the following buildings, but we do not expect the work to result in network outages or downtime: Alexander ES, West HS, Liberty HS, Educational Services Center, South East JH, and North Central JH.
There will be a brief – less than 5 minutes – internet outage district-wide on Wednesday, October 25th at 5:30 PM. This outage is related to maintenance being performed by our ISP to resolve a problem we’ve identified.
Connectivity should be restored for all devices immediately after the maintenance is completed, with no action required on the part of users.
After a couple of false starts, we successfully completed an upgrade of our internet service through the Iowa Communications Network (ICN) just over a week ago. The most substantial impact of this change is that our internet bandwidth has been expanded from 1Gb to 3Gb, a critical improvement (more on that below).
Why was this upgrade necessary?
Once we completed the upgrade, issues we had been facing with internet performance almost immediately vanished. Prior to the upgrade, we were seeing our bandwidth max out immediately at the beginning of the school day, and it would remain that way until school was out for the day. This means that – at any given time – we had more demand for bandwidth than our service could accommodate, resulting in slow performance and intermittent drops / loading errors. You can see this in the chart at right, which shows our demand and usage prior to the upgrade. Continue reading Internet Service Upgrade Completed→
The power outage at the physical plant headquarters has been resolved, and internet/phone services should now be working as normal at the physical plant, West High School, Weber Elementary, Horn Elementary, and TREC.
Most users shouldn’t have to do anything in order to get back on the network, but if you have problems getting network access on your computer after the outage, I’d recommend that you restart your computer in order to reset your network configuration.
If you are unable to access network resources or if larger-scale problems return, please contact the Helpdesk at x4357 or firstname.lastname@example.org
Thanks for your patience as we worked through this issue.
Our internet service upgrade – scheduled for last night – has been delayed until next week after we encountered an unexpected problem. We do not have a specific date/time for the next upgrade window, but I’ll post that here as soon as we’ve completed a conference call with the involved parties.
At 5:00 PM, ICCSD network administrators, working with Iowa Communications Network (ICN) and Lightedge Solutions engineers, switched our internet connection from our current 1Gb ICN connection to the new 3Gb ICN connection. A component of this switch also includes moving from our current internally-managed Palo Alto firewall to a Lightedge-managed Fortinet firewall; staying on our existing Palo Alto hardware is not an option, as it cannot handle the 3Gb throughput required for the new connection.
The internet connection worked properly, as did all but one of the mission critical services that we had previously identified as potential points of failure during the internet service upgrade. One service, however – a VPN tunnel to a particular site – was not successfully brought online after implementation of the new firewall. We attempted troubleshooting until the close of our announced upgrade window, but with no success were forced to revert to the previous settings and service.
What should I expect moving forward?
As previously mentioned, we’ll be rescheduling the upgrade window as soon as possible, likely for next week. We’ve already identified the steps necessary to solve the issue with the site that didn’t come up, and now need to coordinate availability on the part of ICN and Lightedge. I’ll post a specific date/time as soon as we have one.
Since the upgrade was not successful, I expect that internet performance issues will continue for the coming (approximately) week. Our network administrators have been tuning quality-of-service settings over the past couple of weeks in order to prioritize critical or interruption-intolerant traffic, and will continue to do so in order to make the most of the 1Gb service we currently have.
As mentioned in my previous post on this topic, we expect all internet performance problems to be immediately resolved following the successful upgrade to 3Gb service. We appreciate your patience as we work through this process.
UPDATE: This upgrade was unsuccessful (details here); we’ll be rescheduling the upgrade for next week.
Tonight is a big night for our internet service in the Iowa City Schools; a long-planned upgrade will take place beginning at 5:00 PM (on September 27th) that will expand our service from 1Gb bandwidth to 3Gb. Here’s what you need to know: