Hope College CIT - Server Hosting System Outage – Incident details

All systems operational

Server Hosting System Outage

Resolved
Degraded performance
Started about 1 month agoLasted about 1 month

Affected

Hope Infrastructure

Major outage from 5:16 PM to 8:02 PM, Degraded performance from 5:16 PM to 8:02 PM, Major outage from 8:02 PM to 7:28 PM, Degraded performance from 8:02 PM to 1:31 PM, Operational from 1:31 PM to 1:31 PM, Degraded performance from 1:31 PM to 9:00 PM, Operational from 9:00 PM to 3:55 PM

On-Campus Server Infrastructure

Degraded performance from 5:16 PM to 1:31 PM, Operational from 1:31 PM to 1:31 PM, Degraded performance from 1:31 PM to 9:00 PM, Operational from 9:00 PM to 3:55 PM

Concerto (signs.hope.edu)

Major outage from 5:16 PM to 7:28 PM, Degraded performance from 7:28 PM to 1:31 PM, Operational from 1:31 PM to 3:55 PM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Identified
    UTC
    Identified

    We will be performing additional maintenance on the server with issues starting at 2pm today, for 24 hours. We don't anticipate any user-facing outages, but please reach out to the Help Desk if you experience any issues today or tomorrow with services that are hosted on-campus.

  • Update
    UTC
    Update

    For now, the problematic server is behaving as expected, but we are continuing to actively investigate the root cause of why it intermittently acts up.

  • Monitoring
    UTC
    Monitoring

    Most services have returned to normal and we are continuing to investigate the cause of this issue.

  • Identified
    UTC
    Identified

    We are continuing to work on a fix for this incident. Specific server VMs that are affected are able to be migrated to other infrastructure as needed. If a service that you rely on continues to experience issues, please reach out and we will prioritize a fix for that service.

  • Investigating
    UTC
    Investigating

    We're looking into reports that several services that run on one of our on-campus servers aren't working as expected. This includes our digital signage software, signs.hope.edu. We'll update here when we know more.