[US Server] Phoenix server urgent maintenance - NVMe cable replacement - 2 Dec - 2AM SGT

  • November 28, 2024 by Tech #1

Dear Customer,

Our monitoring has recently picked up a new trend of IO error across multiple VM that runs on different OS in our node xen-az1.sosys.net

We have consulted with our vendor and they requested for NVMe data cable replacement to mitigate this risk of data corruption. 

As such, we are planning for 30 mins maintenance window as follow:

Datetime: 2 December 2024 - 2AM SGT

Prior to our activity, we will perform offsite backup of the data and safely shutdown the server for physical inspection.

We apologize for the inconvenience caused


Kind Regards,

SOSYS.NET

====== 

28 Nov: notification sent out

2 Dec 1.50AM: VM shutdown initiated

2 Dec 2AM: work for cable replacement has commenced

2 Dec 2.30AM: DC engineer informed on some unexpected boot up and OS stuck in POST. Currently the team still investigating onsite

2 Dec 3.07AM: DC engineer informed that server failed to boot up when 2 NVMe , a replacement server is undergoing and exisitng NVMe will be moved over

2 Dec 4.07AM: New server provisioning is ongoing

2 Dec 4.15AM: New server provisioning completed and data checked completed. In the process of racking 

2 Dec 4.50AM: Maintenance completed and all VMs are up. We apologize for prolong delay.