Due to the recent server instability incidents, we have decided to immediately migrate all accounts from the Sodium (US) server to our new Platinum (US) cloud server.
The new cloud server has been deployed with a different cloud provider, and we believe this will be the permanent resolution to the recent instability issues.
Your account, currently hosted on the Sodium (US) server is being migrated to the Platinum (US) server.
Our automated system will migrate each cPanel account to the new server. We expect the entire process to take 4 to 6 hours.
The migration is designed to be seamless for your websites, and traffic to the old server will be automatically proxied or forwarded to the new server, allowing you time to update your DNS records if the domain is not using our nameservers.
You will receive another email from us as soon as the entire migration process is completed.
If you did not receive the second email from us within the next 6 hours, please do refer to our Network Status page for updates (Client Portal login required).
To see if your website has been migrated to the new server, you can try to login to the old server's cPanel or WHM:
If you are able to login, that means your account is still in the migration queue and it is not migrated yet.
If you are not able to login, that means your account is already migrated, and you can login to the new server's cPanel or WHM with your existing username and password:
There's nothing that you will need to do, and the migration will be seamless for you.
Please update your DNS records to change from the old IP to the new IP after the migration is completed:
Your website will continue to load from the new server even if you have not updated your DNS records, so you will have time to update the DNS records.
We do highly recommend that you update your DNS records as soon as possible after the migration, in case of instability of the old server.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP is: 66.135.3.51
Please do not update your DNS records with the new IP until the migration has been completed for your account.
The following are the new URLs:
Your username and password will remain the same, but they will not work until the migration for your cPanel account is completed.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website is at risk of being inaccessible due to the instability of the old server. Please update your DNS records after the migration is completed.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns4.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (66.135.3.51).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
The Sodium (US) server is experiencing filesystem corruption.
We are working on a three prong solution to deal with the problem:
We will provide updates within the next 1 hour by email, and additional updates may be provided in our Server Status page.
UPDATE:
UPDATE #2:
Our new server is ready, and we will be migrating all accounts from Sodium (US) to the new server, Platinum (US) immediately.
We will create a new case to track the migration updates and we will send out emails about the details as well.
The Sodium (US-DAL) server is currently inaccessible and we are working on it.
More updates to follow.
UPDATE: We'll take the cloud server down for an emergency maintenance.
UPDATE #2: We've brought the server back up after the maintenance, and are monitoring it closely.
UPDATE #3: As a precaution, we are going to perform a graceful reboot of the server to have it boot into the latest kernel which includes the latest stability improvements.
UPDATE #4: The server is now running on the latest kernel. We are monitoring it closely to ensure that it is stable.
UPDATE #5: The server is inaccessible again. We are investigating.
UPDATE #6: We've booted the server back up and adjusted a configuration which may be causing the server to freeze up. We are monitoring it for stability. In the meantime, we have also taken a latest backup of all website data with JetBackup in case we need to restore the entire server back to a previous Disaster Recovery (DR) Backup.
UPDATE #7: We believe we've found the cause of the system instability. We're rebooting the server to have the configuration change take effect and will monitor it further to confirm if that is the case.
UPDATE #8: We are still seeing intermittent instability. We're troubleshooting further, and will provide updates.
UPDATE #9: We have decided to restore the entire server back to a previous Disaster Recovery (DR) Backup from October 12, 2024. During the restoration process, the server may be inaccessible for 1 to 2 hours. Once we have a working server, we will restore the JetBackup Daily Backup that we took today to ensure any data loss is kept to a minimal.
UPDATE #10: The Disaster Recovery (DR) backup restore has been completed and the server is up now. We will start the JetBackup restore soon.
UPDATE #11: We've started the JetBackup restore. It'll restore the website and cPanel data of each account one by one with the latest data from today. The process can take a few hours to complete, but websites will be up and accessible throughout the process.
UPDATE #12: The JetBackup restore has been completed, and all websites should have their latest data restored.
Due to the recent instability incidents, we have decided to proactively migrate all clients from the Xenon (US) server to our new Sodium (US) cloud server to ensure a more stable and reliable hosting environment.
The new cloud server offers improved performance and resource allocation, including a significant upgrade from 64 GB to 128 GB of RAM. Additionally, unlike the existing cloud server, the new cloud server is not affected by the underlying infrastructure issues that contributed to the recent instability and intermittent disk I/O problems.
Your account, currently hosted on the Xenon (US) server, will be migrated to the Sodium (US) server.
The migration is designed to be seamless for your websites, with zero downtime expected, regardless of whether you use our nameservers or not.
For 30 days after the migration, we'll automatically proxy or forward website traffic from your old server to the new server, giving you ample time to update your DNS records if the domain is not using our nameservers.
There's nothing that you will need to do, and the migration will be seamless for you.
We will send the new cPanel and WHM URL to you, and you can access them with your existing username and password after the migration is completed.
You will receive the new IP address before the migration, and you'll have 30 days to update your DNS records.
Your website will continue to load from the new server for up to 30 days even if you have not updated your DNS records.
We do still recommend that you update your DNS records as soon as possible after the migration, especially if you use our servers to receive emails or if you access cPanel or Webmail with your own domain, as these cannot be forwarded automatically to the new server.
We expect zero downtime, as the process is designed to be seamless.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP will be emailed to you before and after the migration.
The new cPanel and WHM URL will be emailed to you before and after the migration.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website will become inaccessible. Please update your DNS records promptly.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns4.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.UPDATE: We're getting ready to start the migration.
UPDATE #2: The migration is now in progress.
UPDATE #3: The migration progress is at 10%.
UPDATE #4: The migration progress is at 20%.
UPDATE #5: The migration progress is at 30%.
UPDATE #6: The migration progress is at 40%.
UPDATE #7: The migration progress is at 50%.
UPDATE #8: The migration progress is at 60%.
UPDATE #9: The migration progress is at 70%.
UPDATE #10: The migration progress is at 80%.
UPDATE #11: The migration progress is at 90%.
FINAL UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (72.9.154.190).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are aware of the database service issue in the Xenon (US) server and are actively working on it.
Further updates will be posted here.
UPDATE: The database service crashed and may be corrupted. We will start the recovery process immediately.
UPDATE #2: We have identified the potential cause of the issue, and we are working on it now.
UPDATE #3: We've confirmed the cause, and managed to resolve the issue permanently. We're monitoring the database service now to ensure that it is stable.
FINAL UPDATE: We've sent out the Reason For Outage (RFO) email to all affected clients, outlining the incident, resolution steps, preventive measures, and information about our Uptime Guarantee SLA compensation.
We are currently addressing an intermittent Disk I/O issue on the Xenon (US) server. This issue is primarily observed during nightly backups and may result in slower I/O performance.
While the issue's impact on website operations is currently minimal and occurs during off-peak hours, we are actively working towards a permanent resolution. The most effective fix requires a brief server shutdown, which could take anywhere from 15 minutes to an hour. We are delaying this maintenance as much as possible due to the minimal current impact, but we want to ensure you are aware of this potential disruption.
In some scenarios, it's also possible that we may need to schedule an emergency maintenance at short notice if the issue worsens unexpectedly. We will make every effort to minimize any disruption this may cause and provide timely updates if such a situation arises.
We appreciate your patience and understanding as we work diligently to resolve this matter.
UPDATE: In light of the recent issue with the Xenon server which has since be resolved, we will be arranging for a swapping of the underlying hardware.
This migration will not impact your server's IP address, and no action is required on your part.
As there's no immediate urgency for this hardware swap, we will schedule a maintenance window for an appropriate date during off-peak hours. We will send an email with the specific details of this maintenance when the time comes.
We will be performing scheduled maintenance on the Xenon (US) server to address the underlying operating system bug that caused the recent instability issues.
During this maintenance, we will also be upgrading the server's resources to enhance its performance and stability.
The maintenance will involve transferring the server's storage and data to new underlying hardware. This process will be conducted in stages to minimize disruption. However, a brief period of downtime is expected towards the end of the maintenance while the final transfer and switchover occur.
Maintenance Window
Impact
During the maintenance window, the Xenon server will remain operational for most of the time. However, there may be a brief period of downtime towards the end of the maintenance while the final transfer and switchover occur.
We will make every effort to minimize the downtime and complete the maintenance as quickly as possible.
No Actions Required
No action is needed on your part.
There will be no changes to the server's IP address, and your services should resume automatically once the maintenance is complete.
Improvements
We understand that any downtime can be disruptive, and we apologize for the inconvenience. We appreciate your understanding and patience as we work to improve the reliability and performance of our services.
All updates will be provided here at this Server Status page.
UPDATE: We're getting ready to start the maintenance.
UPDATE #2: We have started the scheduled maintenance.
UPDATE #3: The maintenance is progressing smoothly and remains on track for completion within the estimated timeframe. We appreciate your continued patience.
UPDATE #4: We're near the end of the maintenance where a brief period of downtime, ranging between 15 minutes to an hour, is expected while the final transfer and switchover occur.
UPDATE #5: As previously mentioned, the server is currently undergoing the final switchover phase, resulting in a brief period of downtime. Due to some technical complexities during this process, the downtime will unfortunately be slightly longer than initially anticipated, lasting between 1.5 to 2 hours. We apologize for any inconvenience this may cause. Our team is diligently working to bring the server back online as quickly as possible within the original maintenance window. Thank you for your continued patience and understanding.
UPDATE #6: Due to the ongoing complexities, we have made the decision to abort the maintenance. We will reschedule the maintenance at a later date using a different approach. Further updates and the new maintenance schedule will be communicated via email. We apologize for the extended downtime and appreciate your understanding.
We are aware of the issue with the Xenon (US) server and are working on it.
Further updates will be provided here.
UPDATE: While the server is stable right now, we are keeping a very close eye and are working with our data center on this, as we suspect that it could be some form of DoS attack which originates within the datacenter, and thus may not be filtered by the network layer DDoS Protection system. We will provide further updates here.
UPDATE #2: We're seeing some problems again, we are going to take the server down for an emergency maintenance right now. This is to resolve the Disk I/O issue, which could be the cause of the problem.
UPDATE #3: Maintenance complete, and server is back online. We are monitoring it now.
UPDATE #4: We're still seeing issues, updates to follow.
UPDATE #5: We've deployed a solution, and we are monitoring the server for stability.
UPDATE #6: We are still seeing issues, checking further.
UPDATE #7: We are taking the server down again, for a filesystem check.
UPDATE #8: Filesystem check complete, booting up server, and we will monitor it closely.
UPDATE #9: The server appears to be stable after the filesystem check. We will continue to monitor it closely.
UPDATE #10: We have not observed any additional issues with the server, and we can confirm that it is now stable. We can confirm that the issue is related to the filesystem within the server software, and it is not a hardware or network related issue. We will be sending out a Reason For Outage (RFO) email detailing the entire incident and the steps taken to resolve and prevent future issues.
We are happy to announce an upcoming upgrade to the infrastructure of our cPanel cloud servers that will deliver significantly improved performance and reliability for your NoFrillsCloud services.
We're consolidating our Asia Pacific servers into a single, state-of-the-art location in Singapore. Your account, currently hosted on our Mumbai server, will be migrated to this new Singapore server.
Your account, currently hosted on the Titanium (IN) server, will be migrated to our new, state-of-the-art Neon (AP) server.
The migration is designed to be seamless for your websites, regardless of whether you use our nameservers or not.
For 30 days after the migration, we'll automatically proxy or forward website traffic from your old server to the new server, giving you ample time to update your DNS records if needed.
There's nothing that you will need to do, and the migration will be seamless for you.
We will send the new cPanel and WHM URL to you, and you can access them with your existing username and password after the migration is completed.
You will receive the new IP address before the migration, and you'll have 30 days to update your DNS records.
Your website will continue to load from the new server for up to 30 days even if you have not updated your DNS records.
We do still recommend that you update your DNS records as soon as possible after the migration, especially if you use our servers to receive emails or if you access cPanel or Webmail with your own domain, as these cannot be forwarded automatically to the new server.
We expect minimal to no downtime, as the process is designed to be seamless.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP will be emailed to you before and after the migration.
The new cPanel and WHM URL will be emailed to you before and after the migration.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website will become inaccessible. Please update your DNS records promptly.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns4.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.
We're excited for you to experience the significant performance boost and improved features this upgrade will bring to your NoFrillsCloud services.
Thank you for your continued support as we strive to provide you with the best possible hosting experience.
UPDATE: We're getting ready to start the migration.
UPDATE #2: The migration is now in progress.
UPDATE #3: The migration progress is at 10%.
UPDATE #4: The migration progress is at 20%.
UPDATE #5: The migration progress is at 30%.
UPDATE #6: The migration progress is at 40%.
UPDATE #7: The migration progress is at 50%.
UPDATE #8: The migration progress is at 60%.
UPDATE #9: The migration progress is at 70%.
UPDATE #10: The migration progress is at 80%.
UPDATE #11: The migration progress is at 90%.
FINAL UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (89.187.162.81).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are happy to announce an upcoming upgrade to the infrastructure of our cPanel cloud servers that will deliver significantly improved performance and reliability for your NoFrillsCloud services.
The new server's Ultra-High-Frequency CPU and pure NVMe drives are engineered to deliver up to double the speed and responsiveness compared to our previous servers.
Your account, currently hosted on the Magnesium (SG) server, will be migrated to our new, state-of-the-art Neon (AP) server.
The migration is designed to be seamless for your websites, regardless of whether you use our nameservers or not.
For 30 days after the migration, we'll automatically proxy or forward website traffic from your old server to the new server, giving you ample time to update your DNS records if needed.
There's nothing that you will need to do, and the migration will be seamless for you.
We will send the new cPanel and WHM URL to you, and you can access them with your existing username and password after the migration is completed.
You will receive the new IP address before the migration, and you'll have 30 days to update your DNS records.
Your website will continue to load from the new server for up to 30 days even if you have not updated your DNS records.
We do still recommend that you update your DNS records as soon as possible after the migration, especially if you use our servers to receive emails or if you access cPanel or Webmail with your own domain, as these cannot be forwarded automatically to the new server.
We expect minimal to no downtime, as the process is designed to be seamless.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP will be emailed to you before and after the migration.
The new cPanel and WHM URL will be emailed to you before and after the migration.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website will become inaccessible. Please update your DNS records promptly.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns4.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.
We're excited for you to experience the significant performance boost and improved features this upgrade will bring to your NoFrillsCloud services.
Thank you for your continued support as we strive to provide you with the best possible hosting experience.
UPDATE: We're getting ready to start the migration.
UPDATE #2: The migration is now in progress.
UPDATE #3: The migration progress is at 10%.
UPDATE #4: The migration progress is at 20%.
UPDATE #5: The migration progress is at 30%.
UPDATE #6: The migration progress is at 40%.
UPDATE #7: The migration progress is at 50%.
UPDATE #8: The migration progress is at 60%.
UPDATE #9: The migration progress is at 70%.
UPDATE #10: The migration progress is at 80%.
UPDATE #11: The migration progress is at 90%.
FINAL UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (89.187.162.81).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are happy to announce an upcoming upgrade to our cPanel server infrastructure that will deliver significantly improved performance and reliability for your NoFrillsCloud services.
The new server's High-Frequency CPU and pure NVMe drives are engineered to deliver up to double the speed and responsiveness compared to our previous servers.
Your account, currently hosted on the Oxygen server, will be migrated to our new, state-of-the-art Silver server.
The migration is designed to be seamless for your websites, regardless of whether you use our nameservers or not.
For 30 days after the migration, we'll automatically proxy or redirect website traffic from your old server to the new server, giving you ample time to update your DNS records if needed.
There's nothing that you will need to do, and the migration will be seamless for you.
We will send the new cPanel and WHM URL to you, and you can access them with your existing username and password after the migration is completed.
You will receive the new IP address before the migration, and you'll have 30 days to update your DNS records.
Your website will continue to load from the new server for up to 30 days even if you have not updated your DNS records.
If you use our servers to receive emails, we recommend that you update the MX records as soon as possible after migration, as cannot be forwarded automatically from the old server to the new server.
We expect minimal to no downtime, as the process is designed to be seamless.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP will be emailed to you before and after the migration.
The new cPanel and WHM URL will be emailed to you before and after the migration.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website will become inaccessible. Please update your DNS records promptly.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns2.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.
We're excited for you to experience the significant performance boost and improved features this upgrade will bring to your NoFrillsCloud services.
Thank you for your continued support as we strive to provide you with the best possible hosting experience.
UPDATE: We're getting ready to start the migration.
UPDATE #2: The migration is now in progress.
UPDATE #3: The migration progress is at 35%.
UPDATE #4: The migration progress is at 45%.
UPDATE #5: The migration progress is at 55%.
UPDATE #6: The migration progress is at 65%.
UPDATE #7: The migration progress is at 75%.
UPDATE #8: The migration progress is at 85%.
FINAL UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (148.251.90.41).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are happy to announce an upcoming upgrade to the infrastructure of our cPanel cloud servers that will deliver significantly improved performance and reliability for your NoFrillsCloud services.
The new server's High-Frequency CPU and pure NVMe drives are engineered to deliver up to double the speed and responsiveness compared to our previous servers.
Your account, currently hosted on the Lead (AU) server, will be migrated to our new, state-of-the-art Gold (AU) server.
The migration is designed to be seamless for your websites, regardless of whether you use our nameservers or not.
For 30 days after the migration, we'll automatically proxy or redirect website traffic from your old server to the new server, giving you ample time to update your DNS records if needed.
There's nothing that you will need to do, and the migration will be seamless for you.
We will send the new cPanel and WHM URL to you, and you can access them with your existing username and password after the migration is completed.
You will receive the new IP address before the migration, and you'll have 30 days to update your DNS records.
Your website will continue to load from the new server for up to 30 days even if you have not updated your DNS records.
We do still recommend that you update your DNS records as soon as possible after the migration, especially if you use our servers to receive emails or if you access cPanel or Webmail with your own domain, as these cannot be forwarded automatically to the new server.
We expect minimal to no downtime, as the process is designed to be seamless.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP will be emailed to you before and after the migration.
The new cPanel and WHM URL will be emailed to you before and after the migration.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website will become inaccessible. Please update your DNS records promptly.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns4.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.
We're excited for you to experience the significant performance boost and improved features this upgrade will bring to your NoFrillsCloud services.
Thank you for your continued support as we strive to provide you with the best possible hosting experience.
UPDATE: We're getting ready to start the migration.
UPDATE #2: The migration is now in progress.
UPDATE #3: The migration progress is at 25%.
UPDATE #4: The migration progress is at 40%.
UPDATE #5: The migration progress is at 60%.
UPDATE #6: The migration progress is at 80%.
UPDATE #7: The migration progress is at 95%.
FINAL UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (103.108.229.195).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are happy to announce an upcoming upgrade to the infrastructure of our cPanel cloud servers that will deliver significantly improved performance and reliability for your NoFrillsCloud services.
The new server's High-Frequency CPU and pure NVMe drives are engineered to deliver up to double the speed and responsiveness compared to our previous servers.
Your account, currently hosted on the Zinc (US) server, will be migrated to our new, state-of-the-art Xenon (US) server.
The migration is designed to be seamless for your websites, regardless of whether you use our nameservers or not.
For 30 days after the migration, we'll automatically proxy or redirect website traffic from your old server to the new server, giving you ample time to update your DNS records if needed.
There's nothing that you will need to do, and the migration will be seamless for you.
We will send the new cPanel and WHM URL to you, and you can access them with your existing username and password after the migration is completed.
You will receive the new IP address before the migration, and you'll have 30 days to update your DNS records.
Your website will continue to load from the new server for up to 30 days even if you have not updated your DNS records.
We do still recommend that you update your DNS records as soon as possible after the migration, especially if you use our servers to receive emails or if you access cPanel or Webmail with your own domain, as these cannot be forwarded automatically to the new server.
We expect minimal to no downtime, as the process is designed to be seamless.
Our system is designed to pick up the changes, but we do recommend holding off on any website updates if possible.
The new IP will be emailed to you before and after the migration.
The new cPanel and WHM URL will be emailed to you before and after the migration.
No, your username and password for all services remain the same.
No, you should only update your DNS records after the migration is completed.
Your website will become inaccessible. Please update your DNS records promptly.
Visit https://www.nslookup.io/ns-lookup/
Enter your domain name and click "Lookup"
Under the results, look at the nameservers listed:
ns1-ns4.nfserver.net or ns1-ns4.yourdomain.com, you are using our nameservers and there's nothing that you will need to do.
We're excited for you to experience the significant performance boost and improved features this upgrade will bring to your NoFrillsCloud services.
Thank you for your continued support as we strive to provide you with the best possible hosting experience.
UPDATE: We're getting ready to start the migration.
UPDATE #2: The migration is now in progress.
UPDATE #3: The migration progress is at 15%.
UPDATE #4: The migration progress is at 55%.
UPDATE #5: The migration progress is at 70%.
UPDATE #6: The migration progress is at 95%.
FINAL UPDATE:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (72.9.154.188).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are troubleshooting a bug with the CloudLinux software on the Magnesium (SG) server.
The bug resulted in the CPU and RAM limits for some accounts to not be applied properly, which can negatively affect some websites.
We will be performing an emergency maintenance which includes a server reboot as part of the troubleshooting process.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your understanding as we work to provide the best possible service.
UPDATE: We are rebooting the server right now.
UPDATE #2: The server is up and running again.
UPDATE #3: The bug should be resolved now. We will be monitoring this to ensure that it remains stable.
UPDATE #4: We haven’t observed any additional issues, and will now consider this incident resolved.
Affecting Other - All Servers
We are scheduling an emergency maintenance to proactively address a recently discovered security vulnerability (CVE-2024-1086) in the Linux kernel.
This maintenance includes a server reboot to implement the latest security patch for this vulnerability.
As your Managed Cloud Hosting provider, we take full responsibility for server-level security. Rest assured, there's nothing you need to do on your end, we're handling this update to ensure your server remains protected.
What to Expect
Please refer to this Network Status page for all updates related to this maintenance.
We appreciate your patience and cooperation in maintaining a secure hosting environment.
UPDATE:
We've started the maintenance. The reboot will be done sequentially (one server after another).
If you experience downtime with your website, your server is being rebooted. The downtime should only take no more than 15 minutes.
UPDATE #2:
The maintenance has been completed for all cPanel servers.
We will be performing scheduled maintenance on the Magnesium (SG) server to address minor software issues and enhance overall stability. This maintenance involves a reboot with a new kernel.
Maintenance Window:
This update is necessary to resolve a few minor bugs with the CloudLinux operating system software. The new kernel will provide increased stability for services hosted on the server.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your understanding as we work to provide the best possible service.
UPDATE: We're getting ready to start the maintenance.
UPDATE #2: The server reboot is in progress. Further updates to follow.
UPDATE #3: The server is back up and running.
We will be performing scheduled maintenance on the Lead (AU) server to address minor software issues and enhance overall stability. This maintenance involves a reboot with a new kernel.
Maintenance Window:
This update is necessary to resolve a few minor bugs with the CloudLinux operating system software. The new kernel will provide increased stability for services hosted on the server.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your understanding as we work to provide the best possible service.
UPDATE: We're getting ready to start the maintenance.
UPDATE #2: The server reboot is in progress. Further updates to follow.
UPDATE #3: The server is back up and running.
We will be performing scheduled maintenance on the Zinc (US) server to address minor software issues and enhance overall stability. This maintenance involves a reboot with a new kernel.
Maintenance Window:
This update is necessary to resolve a few minor bugs with the CloudLinux operating system software. The new kernel will provide increased stability for services hosted on the server.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your understanding as we work to provide the best possible service.
UPDATE: We're getting ready to start the maintenance.
UPDATE #2: The server reboot is in progress. Further updates to follow.
UPDATE #3: The server is back up and running.
We will be performing an emergency maintenance on the Titanium (IN) server to address software bug and enhance overall stability. This maintenance involves a reboot with a new kernel.
This update is necessary to resolve a bugs with the CloudLinux operating system software. The new kernel will provide increased stability for services hosted the server.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your understanding as we work to provide the best possible service.
UPDATE: We are getting ready to start the maintenance.
UPDATE #2: We have started the maintenance and the server is being rebooted right now.
UPDATE #3: The server is back up and running.
We will be performing scheduled maintenance on the Oxygen (UK) server to address minor software issues and enhance overall stability. This maintenance involves a reboot with a new kernel.
Maintenance Window:
This update is necessary to resolve a few minor bugs with the CloudLinux operating system software. The new kernel will provide increased stability for services hosted on the Oxygen (UK) server.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your understanding as we work to provide the best possible service.
UPDATE: We're getting ready to start the maintenance.
UPDATE #2: The server reboot is in progress. Further updates to follow.
UPDATE #3: The server is back up and running.
We are currently investigating performance degradation on the Oxygen (UK) server. We've identified high Disk I/O for the /home5 storage drive. This could be caused by a number of factors, including potential network connectivity issues within the data center between the Cloud Servers and storage.
While we're investigating the network as a possibility, it's important to note that there is no indication of any risk to the data itself. Our primary focus is to identify the root cause and restore optimal performance for all users.
We are actively working with the data center to troubleshoot and may implement server-side optimizations to mitigate the impact in the meantime.
Clients on the /home5 storage drive may experience a more significant performance impact.
We apologize for any inconvenience and will provide further updates as soon ASAP.
UPDATE: We've implemented some mitigation measures within the server at the meantime, so any performance impact should be kept to a minimum right now.
UPDATE #2: The data center has implemented changes to address the performance issue on the /home5 drive of the server. We will be closely monitoring the situation for further improvements.
UPDATE #3: We haven't observed any additional issues with the storage performance, and will now consider this incident resolved.
We are scheduling an emergency maintenance window for the Oxygen (UK) server to enhance its resources.
This upgrade is crucial not only for immediate performance enhancement but also to prepare our server for the upcoming holiday season, ensuring it can handle increased demand and future growth.
The maintenance window is scheduled for the following date and time:
We expect service interruptions during the maintenance window to be between 15 to 45 minutes but we will schedule a full 2 hours window just in case.
Updates will be available on our Network Status page.
Thank you for your patience as we continuously improve our services to serve you better.
UPDATE #1: We are getting ready to start the upgrade.
UPDATE #2: We have started the upgrade.
UPDATE #3: The upgrade is still ongoing. We will share updates when it has been completed.
UPDATE #4: The upgrade is taking longer than usual due to a separate maintenance at the data center. We're extending our maintenance window by an additional 1 hour to cater to this.
UPDATE #5: The upgrade has been completed and the server is booting up now.
UPDATE #6: The maintenance ended up being completed within the initial estimated window of 2 hours so the extension was not necessary.
We've been battling a HTTP Flood DDoS Attack against the Magnesium (SG) server over the weekend.
The attacker aimed to overwhelm our web server with a multitude of connections. Nonetheless, given the robust capacity of our server, the scale of this attack was insufficient to activate our DDoS Protection system, as it didn't significantly impact our operations. However, customers may have received "downtime" notifications from monitoring tools like UptimeRobot during automatic web server restarts, aimed at shedding the surplus connections created by the attacker.
We've adjusted configurations on our web server and firewall to better manage such lesser-scale attack attempts and implemented a reCAPTCHA protection mechanism which will only be engaged automatically when attacks are detected to ensure no disruption during regular operation and no inconvenience to your website visitors.
However, today (Monday, 9 October 2023), the attack escalated, potentially rendering the reCAPTCHA protection insufficient if the surge in connections occurs abruptly.
As such, we have decided to activate the reCAPTCHA protection mechanism at all times. This means every visitor to your website will encounter a reCAPTCHA challenge before gaining access. While this may inconvenience some visitors, it is a necessary measure to thwart the attacker.
Further tightening of web server and firewall configurations has also been done to filter out potentially malicious connections.
We will continue to monitor the server and will share future updates here.
UPDATE: We have also scaled up our web server capacity to better handle any threats should the attacker further intensify the attack.
UPDATE #2: To clarify, the activating of the reCAPTCHA protection mechanism at all times is a temporary measure until this incident is completely resolved. We do not intend to keep it permanent to prevent inconveniencing website visitors.
UPDATE #3: The attack is still ongoing, but we have devised a solution to mitigate its effects. As such, we have deactivated the reCAPTCHA protection mechanism for now. It remains on standby to auto-engage should an intensive attack be detected.
UPDATE #4: While the attack is ongoing, its intensity has significantly lessened thanks to our successful mitigation strategies. You can remain at ease as our measures ensure continuous, disruption-free service.
UPDATE #5: As the situation remains stable and services are no longer impacted, we are closing this update. We will remain vigilant and continue to monitor the server closely.
We are scheduling a maintenance window to upgrade the MariaDB version of the Oxygen (UK) server.
The maintenance window has been scheduled for:
During the maintenance window, websites relying on databases, such as WordPress, may experience service interruptions lasting between 15 minutes to an hour. Please note that other server operations, including emails, static websites, file access (FTP/SFTP/File Manager) and DNS resolution, will remain unaffected.
To ensure data integrity, we will perform multiple backups of the databases prior to the upgrade.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your cooperation and understanding as we work to enhance our services.
UPDATE: We're getting ready for the maintenance by taking multiple backups of the databases. There may be a few quick restarts to the database service when this is going on.
UPDATE #2: The backups are now completed and we will start the upgrade now.
UPDATE #3: The upgrade has been completed successfully. Please get in touch if you are experiencing any issues with your website's database.
We are scheduling a maintenance window to upgrade the MariaDB version of the Magnesium (Singapore) server.
The maintenance window has been scheduled for:
During the maintenance window, websites relying on databases, such as WordPress, may experience service interruptions lasting between 15 minutes to an hour. Please note that other server operations, including emails, static websites, file access (FTP/SFTP/File Manager) and DNS resolution, will remain unaffected.
To ensure data integrity, we will perform multiple backups of the databases prior to the upgrade.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your cooperation and understanding as we work to enhance our services.
UPDATE: We're getting ready for the maintenance by taking multiple backups of the databases. There may be a few quick restarts to the database service when this is going on.
UPDATE #2: The backups are now completed and we will start the upgrade now.
UPDATE #3: The upgrade has been completed successfully. Please get in touch if you are experiencing any issues with your website's database.
We are scheduling a maintenance window to upgrade the MariaDB version of the Zinc (US) server.
The maintenance window has been scheduled for:
During the maintenance window, websites relying on databases, such as WordPress, may experience service interruptions lasting between 15 minutes to an hour. Please note that other server operations, including emails, static websites, file access (FTP/SFTP/File Manager) and DNS resolution, will remain unaffected.
To ensure data integrity, we will perform multiple backups of the databases prior to the upgrade.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your cooperation and understanding as we work to enhance our services.
UPDATE: We're getting ready for the maintenance by taking multiple backups of the databases. There may be a few quick restarts to the database service when this is going on.
UPDATE #2: The backups are now completed and we will start the upgrade now.
UPDATE #3: The upgrade has been completed successfully. Please get in touch if you are experiencing any issues with your website's database.
We are scheduling a maintenance window to upgrade the MariaDB version of the Titanium (India) server.
The maintenance window has been scheduled for:
During the maintenance window, websites relying on databases, such as WordPress, may experience service interruptions lasting between 15 minutes to an hour. Please note that other server operations, including emails, static websites, file access (FTP/SFTP/File Manager) and DNS resolution, will remain unaffected.
To ensure data integrity, we will perform multiple backups of the databases prior to the upgrade.
Please refer to this Network Status page for all updates related to this maintenance.
Thank you for your cooperation and understanding as we work to enhance our services.
UPDATE: We're getting ready for the maintenance by taking multiple backups of the databases. There may be a few quick restarts to the database service when this is going on.
UPDATE #2: The backups are now completed and we will start the upgrade now.
UPDATE #3: The upgrade has been completed successfully. Please get in touch if you are experiencing any issues with your website's database.
The Magnesium (Singapore) server was unresponsive and we are rebooting it at the moment.
UPDATE: The reboot is in progress and a File System Check is running.
UPDATE #2: The filesystem check has been completed and the server is accessible now.
We are scheduling a maintenance window to complement the emergency upgrade earlier today for the Oxygen (UK) server.
The maintenance will allow us to implement key changes in the server which will improve overall performance.
The maintenance window is scheduled for the following date and time:
We expect service interruptions during the maintenance window to be between 10 to 30 minutes but we will schedule a full 1 hour window just in case.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance soon.
UPDATE #2: We've started the maintenance.
UPDATE #3: The first phase of the maintenance has been completed. We are now starting the second phase.
UPDATE #4: The maintenance has been completed.
We are scheduling an emergency maintenance window to add more resources to the Oxygen (UK) server.
This will increase our server's capacity to provide better performance and allow us to handle future growths.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes but we will schedule a full 2 hours window just in case.
All updates will be provided here at this Network Status page.
UPDATE #1: We are getting ready to start the upgrade.
UPDATE #2: We have started the upgrade.
UPDATE #3: The upgrade has been completed and the server is booting up now.
We are scheduling a server migration for the Lithium (AU) server to a more powerful server, Lead (AU).
The new server runs on our latest Cloud platform with newer generation processors, memory and storage which will give your website performance a nice boost.
We are also upgrading the CPU and RAM allocations of your account for free as soon as your account is migrated to the new server.
We have scheduled for the server migration to start at the following date and time:
We expect the server migration to take 3 to 5 hours to complete at most and it will be queued according to the alphabetical order of the username of WHM accounts (so if your username starts with "a" or "b", it will be higher in the queue while if your username starts with "y" or "z", it will be lower in the queue).
Recommended: Arrange for an Earlier Migration
If you would like to be migrated earlier, please do click this link to get in touch with our Customer Service team and we can get it arranged for you. Please do allow up to 24 hours for our Customer Service team to make the arrangement for you.
The main benefit of arranging your migration earlier is that we will only be migrating your accounts at the date and time of your choice (before the scheduled date) so it will complete much faster for you.
What do I need to do?
If you are using our nameservers (ns1-ns4.nfserver.net) or your own Private Nameservers (based on our nameserver IPs), there is nothing that you will need to do and the server migration should be seamless for you.
If you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS, you will need to update your DNS records with the new server IP once the server migration is complete. We will provide the new IP to you by email as soon as the migration is complete.
UPDATE #1: We are getting ready to start the migration.
UPDATE #2: We've started the migration.
UPDATE #3:
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (194.195.126.222).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are running some optimizations to the MySQL of the Lead (AU) server.
During the next hour or two the MySQL service may be restarted and as such inaccessible for between 5 to 15 minutes at a time.
We will provide updates here once this has been completed.
UPDATE: This has been completed.
We are scheduling a server migration for the Iron (US) server to a more powerful server, Zinc (US).
The new server runs on our latest Cloud platform with newer generation processors, memory and storage which will give your website performance a nice boost. We are also adding an additional CPU core to your account for free as soon as your account is migrated to the new server.
We have scheduled for the server migration to start at the following date and time:-
We expect the server migration to take 3 to 5 hours to complete at most and it will be queued according to the alphabetical order of the username of WHM accounts (so if your username starts with "a" or "b", it will be higher in the queue while if your username starts with "y" or "z", it will be lower in the queue).
Recommended: Arrange for an Earlier Migration
If you would like to be migrated earlier, please do click this link to get in touch with our Customer Service team and we can get it arranged for you. Please do allow up to 24 hours for our Customer Service team to make the arrangement for you.
The main benefit of arranging your migration earlier is that we will only be migrating your accounts at the date and time of your choice (before the scheduled date) so it will complete much faster for you.
What do I need to do?
If you are using our nameservers (ns1-ns4.nfserver.net) or your own Private Nameservers (based on our nameserver IPs), there is nothing that you will need to do and the server migration should be seamless for you.
If you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS, you will need to update your DNS records with the new server IP once the server migration is complete. We will provide the new IP to you by email as soon as the migration is complete.
UPDATE #1: We are getting ready to start the migration.
UPDATE #2: We've started the migration.
UPDATE #3:-
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (170.187.155.144).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are scheduling a maintenance window to upgrade the Magnesium (Singapore) server to newer generation processors and memory which will further improve the performance of the server.
The maintenance window is scheduled for the following date and time:
We expect service interruptions during the maintenance window to be between 15 to 45 minutes but we will schedule a full 3 hours window just in case.
All updates will be provided here at this Network Status page.
UPDATE #1: We are getting ready to start the upgrade.
UPDATE #2: We have started the upgrade.
UPDATE #3: Based on the current progress, we expect the upgrade to be completed within the next 53 minutes.
UPDATE #4: Based on the current progress, we expect the upgrade to be completed within the next 45 minutes.
UPDATE #5: Based on the current progress, we expect the upgrade to be completed within the next 25 minutes.
UPDATE #6: Based on the current progress, we expect the upgrade to be completed within the next 15 minutes.
UPDATE #7: Based on the current progress, we expect the upgrade to be completed within the next 10 minutes.
UPDATE #8: Based on the current progress, we expect the upgrade to be completed within the next 5 minutes.
UPDATE #9: The upgrade has been completed and the server is booting up now.
We are scheduling a maintenance window to add more resources to the Titanium (India) server.
We do apologize for the short notice of this scheduled upgrade as it is needed to resolve the more frequent MySQL restarts that this server is experiencing lately.
The maintenance window is scheduled for the following date and time:
We expect service interruptions during the maintenance window to be between 15 to 45 minutes but we will schedule a full 3 hours window just in case.
All updates will be provided here at this Network Status page.
UPDATE #1: Apologies, our email about the upgrade being in progress went out earlier than scheduled. We will start the upgrade at the scheduled time.
UPDATE #2: We have started the upgrade.
UPDATE #3: The upgrade has been completed.
We are investigating an issue with the Copper (SG 2) server.
We will provide more updates once we have them.
UPDATE: There appears to be a file system issue. We completed a file system check and booted up the server now.
UPDATE #2: We are rebooting the server again to complete some tasks.
UPDATE #3: Server is running smoothly now. We will continue to monitor it for stability.
UPDATE #4: It appears that the server is up but networking is inaccessible. We are investigating this.
UPDATE #5: The networking of the server is accessible now. It appears that at some point AWS modified the server networking files during one of the reboots and that resulted in issues with the networking. We made sure to correct it and to get it to work properly. We will be monitoring the server now for stability.
We're rebooting the Copper (SG) server for a quick maintenance.
It will be back up soon.
UPDATE: The server is up and running now.
We are scheduling a server migration for the Carbon (UK) server to a more powerful server, Oxygen (UK).
This should result in better performance for your websites while also allowing us to scale up more seamlessly in the future.
We have scheduled for the server migration to start at the following date and time:-
We expect the server migration to take 3 to 5 hours to complete at most and it will be queued according to the alphabetical order of the username of WHM accounts (so if your username starts with "a" or "b", it will be higher in the queue while if your username starts with "y" or "z", it will be lower in the queue).
Recommended: Arrange for an Earlier Migration
If you would like to be migrated earlier, please do click this link to get in touch with our Customer Service team and we can get it arranged for you. Please do allow up to 24 hours for our Customer Service team to make the arrangement for you.
The main benefit of arranging your migration earlier is that we will only be migrating your accounts at the date and time of your choice (before the scheduled date) so it will complete much faster for you.
What do I need to do?
If you are using our nameservers (ns1-ns4.nfserver.net) or your own Private Nameservers (based on our nameserver IPs), there is nothing that you will need to do and the server migration should be seamless for you.
If you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS, you will need to update your DNS records with the new server IP once the server migration is complete. We will provide the new IP to you by email as soon as the migration is complete.
UPDATE #1: We are getting ready to start the migration.
UPDATE #2: We've started the migration.
UPDATE #3:-
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (109.74.194.213).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are scheduling a maintenance window to upgrade the Titanium (India) server to newer generation processors and memory which will further improve the performance of the server.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes but we will schedule a full 3 hours window just in case.
All updates will be provided here at this Network Status page.
UPDATE #1: We're running a snapshot backup before starting the upgrade.
UPDATE #2: We are starting the upgrade now.
UPDATE #3: The upgrade has been completed.
The Copper (SG) server is unresponsive and we are checking it.
More updates to follow.
UPDATE #1: We're rebooting to the server.
UPDATE #2: The server is now up and running. We are monitoring it for stability.
UPDATE #3: We have observed no additional issues with the server and will now consider this incident resolved.
The Copper server is unresponsive and we are rebooting the server.
More updates to follow.
UPDATE: The reboot has been completed and we are monitoring the server.
We are performing an emergency reboot on the Titanium (IN) server.
The reboot is necessary to address an abnormal Memory spike as a result of an experimental backup that we were running.
More updates to follow.
UPDATE: The reboot has been completed and we are monitoring the server.
We are performing an emergency reboot on the Copper (SG) server.
The reboot is necessary to address an abnormal IO spike.
More updates to follow.
UPDATE: The reboot has been completed and we are monitoring the server.
We are running an emergency maintenance for the Titanium (India) server.
More updates will be provided here.
UPDATE: The maintenance is now completed.
We are scheduling a maintenance window to add more resources to the Magnesium (Singapore) server.
This will increase our server's capacity to provide better performance and allow us to handle future growths.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're running a snapshot backup before we start the process so there will be a slight delay before we start the upgrade.
UPDATE #2: The snapshot backup is taking slightly longer than expected. We will provide an update once we are ready to start the upgrade.
UPDATE #3: The scheduled upgrade will start soon.
UPDATE #4: The scheduled backup has started.
UPDATE #5: We expect the upgrade to take another 30 minutes to complete.
UPDATE #6: The upgrade has been completed and the server is booting up now.
UPDATE #7: The server has booted up successfully.
The Magnesium server is unresponsive and we are rebooting the server.
More updates to follow.
UPDATE #1: The server is up and running now. It appears to be a kernel panic and we will devise a solution to prevent it from happening again.
UPDATE #2: We have implemented a fix to this and we will be monitoring the server stability over the next few weeks.
The Copper server is unresponsive and we are rebooting the server.
More updates to follow.
UPDATE #1: There appears to be a network issue at the server level. We are working on it now.
UPDATE #2: Network connectivity to the server has been restored. It appears that a recent cPanel update broke the networking of the server. We are getting in touch with cPanel support to determine the cause of the problem to prevent it from happening again.
The Magnesium server is unresponsive and we are rebooting the server.
More updates to follow.
UPDATE: Server is up and running smoothly now.
We are scheduling a server migration for the Helium (SG) server to a more powerful server, Copper (SG).
This should result in better performance for your websites while also allowing us to scale up more seamlessly in the future.
We have scheduled for the server migration to start at the following date and time:-
We expect the server migration to take 3 to 5 hours to complete at most and it will be queued according to the alphabetical order of the username of WHM accounts (so if your username starts with "a" or "b", it will be higher in the queue while if your username starts with "y" or "z", it will be lower in the queue).
Recommended: Arrange for an Earlier Migration
If you would like to be migrated earlier, please do click this link to get in touch with our Customer Service team and we can get it arranged for you. Please do allow up to 24 hours for our Customer Service team to make the arrangement for you.
The main benefit of arranging your migration earlier is that we will only be migrating your accounts at the date and time of your choice (before the scheduled date) so it will complete much faster for you.
What do I need to do?
If you are using our nameservers (ns1-ns4.nfserver.net) or your own Private Nameservers (based on our nameserver IPs), there is nothing that you will need to do and the server migration should be seamless for you.
If you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS, you will need to update your DNS records with the new server IP once the server migration is complete. We will provide the new IP to you by email as soon as the migration is complete.
UPDATE #1: We are getting ready to start the migration.
UPDATE #2: We've started the migration.
UPDATE #3:-
The migration has been completed.
If you have yet to receive the "Server Migration Completed" email from us, please check the Email History page in our Client Portal.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare or QUIC.Cloud DNS and if you are, do update the IP to point to the new server IP (54.255.16.27).
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are working on some instability with the new storage system with AWS and that involves a few server reboots.
UPDATE #1: Final reboot is running now.
UPDATE #2: This is completed now. We will be monitoring the server to make sure that it is stable.
UPDATE #3: We have monitored the server for some time and it is stable.
We will be performing a quick server reboot to the Iron (US) server.
Any service interruptions should only last for 5 to 15 minutes at most.
UPDATE #1: Reboot is in progress.
UPDATE #2: Reboot completed.
UPDATE #3: We're going to do another quick reboot and it shouldn't take more than 15 minutes.
UPDATE #4: Reboot completed.
We will be performing a quick server upgrade to the Iron (US) server.
Any service interruptions should only last for 5 to 15 minutes at most.
UPDATE: Server is booting up now.
We are aware that the Iron (US) server is unresponsive.
We are working on it and will provide further updates.
UPDATE: Server has been rebooted and it works properly now.
Our data center team is investigating a connectivity issue in the Mumbai data center affecting the Titanium server.
During this time, users may experience connection timeouts and errors for your website.
We will share additional updates as we have more information.
UPDATE #1: Our data center team has identified the issue affecting connectivity and are working quickly to implement a fix. We will provide an update as soon as the solution is in place.
UPDATE #2: At this time our data center team have been able to correct the issues affecting connectivity. We will be monitoring this to ensure that it remains stable. If you are still experiencing issues, please open a support ticket for assistance.
UPDATE #3: We haven’t observed any additional connectivity issues and will now consider this incident resolved. If you continue to experience problems, please open a support ticket for assistance.
We scheduled an emergency upgrade for the Hydrogen (US) server about two weeks ago but due to some complications during the maintenance, we decided to defer the upgrade.
After careful evaluations and plannings, we have deployed a brand new server - Iron (US) and we will migrate the accounts in Hydrogen over to the new server.
The new Iron server is a lot more powerful than the existing server and this should result in better performance for your websites while also catering for future growths.
We have scheduled for the server migration to start at the following date and time:-
We expect the server migration to take 6 to 8 hours to complete and it will be queued according to the alphabetical order of the username of WHM accounts (so if your username starts with "a" or "b", it will be higher in the queue while if your username starts with "y" or "z", it will be lower in the queue).
Recommended: Arrange for an Earlier Migration
If you would like to be migrated earlier, please do get in touch with our Customer Service team and we can get it arranged for you. Please do allow up to 12 hours for our Customer Service team to make the arrangement for you.
The main benefit of arranging your migration earlier is that we will only be migrating your accounts at the date and time of your choice (before the scheduled date) so it will complete much faster.
What do I need to do?
If you are using our nameservers (ns1-ns4.nfserver.net) or your own Private Nameservers (based on our nameserver IPs), there is nothing that you will need to do and the server migration should be seamless for you.
If you are using third-party nameservers like CloudFlare, you will need to update your DNS records to the new server IP:-
Old Server IP: 3.132.120.160
New Server IP: 18.223.200.157
UPDATE #1: We're getting ready to start the migration.
UPDATE #2: We've started the migration.
UPDATE #3: The main migration process is now complete. We're cleaning up minor processes here and there so if you see anything not right, please check back later.
UPDATE #4:-
The migration has been fully completed.
If you are seeing any issues with your website, please first make sure whether you are using third-party nameservers like CloudFlare and if you are, do update the IP to point to the new server as stated above.
Please do feel free to get in touch if you have any questions or require any assistance.
Thank you.
We are aware of the successive connectivity issues in Singapore over the past week, and we want to acknowledge the impact they have had.
As an update, our data center believe these issues are stemming from a software bug in the redundant aggregate switches.
An initial emergency maintenance was just completed to address this bug, and our data center will continue to work with the vendor to identify a long term fix.
We will keep this case updated with more information as we have it, and we will provide a post-mortem once the issue is resolved.
UPDATE:-
We have received the postmortem report from our data center for the successive connectivity issues in Singapore during the month of December 2020.
In short, the issue is caused by a software bug with the redundant networking switches. After an emergency maintenance to upgrade the software for each switch, we have not experienced any technical issues since then and we believe the issue to be fully resolved at this point.
This incident is claimable under our Uptime Guarantee Service Level Agreement (SLA) where we will pay you 10 times the actual downtime. Affected clients can get in touch with our Customer Service team to make a claim and we will process all requests within 72 hours.
We do apologise for the inconvenience caused to you and while we cannot promise that problems will never occur, we will do our very best to resolve them quickly while maintaining transparent and prompt communication.
The full technical report is included below:-
On 12-16-2020, at approximately 11:45 UTC, our monitoring systems alerted us to a connectivity issue in the Singapore data center. These monitoring systems indicated a large drop in traffic to and from the DC, however, no clear cause was apparent so the Network Operations team continued to troubleshoot. In our Singapore data center, we have a core switch pair in an active/active redundant configuration. The team was able to isolate the issue to the B-side switch, which had stopped forwarding traffic but did not trigger a failover.
The team then decided to force a failover to the A-side switch and this immediately restored full connectivity to the data center. The networking team then proactively rebooted the B-side switch.
When the B-side switch completed its power cycle, diagnostics checked out, so it was returned to service. The network operations team engaged the vendor for further investigation into the cause of the issue but there wasn’t anything that presented out of the ordinary other than the switch was discarding traffic.
On 12-18-2020 at 16:32 UTC, again, our monitoring systems alerted us to connectivity issues in the Singapore data center. This time, though, the A-side switch had stopped forwarding traffic.
A forced failover and power cycle of the affected switch brought the data center back to full service. We continued to work with the vendor, however, we were still unable to determine a root cause for these switches to stop forwarding traffic.
We experienced the same issue two more times, on 12-20-2020 at 11:01 UTC, and again on 12-21-2020 at 16:38 UTC.
Since there wasn't a clear solution as a result of our investigation with the vendor, we began operating under the premise that this was an unidentified bug in the code version we were currently using. Our team decided to schedule an emergency maintenance and proactively upgrade the code version of the switch to another recommended version.
On 12-21-2020 at 21:00 UTC, an emergency maintenance was performed to upgrade the software of each switch. This maintenance was completed without downtime.
We have not experienced any technical issues since the completion of this maintenance and we believe this issue to be fully resolved at this point. We are still working with the vendor to identify the underlying bug, however, a final cause has yet to be identified.
We are scheduling an emergency maintenance window right now to add more resources to the Hydrogen (US) server.
This will increase our server's capacity to provide better performance and allow us to handle future growths.
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We've started the upgrade.
UPDATE #2: Due to some complications during the maintenance, we are going to defer the upgrade for this server. We will re-schedule the maintenance to a later date and we will send out an email beforehand.
Our data center team is investigating a connectivity issue in the Singapore data center affecting the Magnesium server.
During this time, users may experience connection timeouts and errors for your website.
We will share additional updates as we have more information.
UPDATE #1: Our data center team has identified the issue affecting connectivity and are working quickly to implement a fix. We will provide an update as soon as the solution is in place.
UPDATE #2: At this time our data center team have been able to correct the issues affecting connectivity. We will be monitoring this to ensure that it remains stable. If you are still experiencing issues, please open a support ticket for assistance.
UPDATE #3: We haven’t observed any additional connectivity issues and will now consider this incident resolved. If you continue to experience problems, please open a support ticket for assistance.
We're seeing a spike in Disk I/O in the Magnesium server.
We are rebooting the server right now to rule out a server level issue.
We are also getting in touch with the data center to determine if it is a network connectivity issue like the one experienced the day before.
More updates will be provided here as we have them.
UPDATE #1: Server reboot in progress.
UPDATE #2: The server is running now and we will monitor it further. We do still see high Disk I/O activity but nothing within the server that is contributing to it. We are also not seeing anything at the network level. We suspect that it could be another Cloud server in the same physical host node causing issues and the data center is checking it at the moment.
UPDATE #3: Disk I/O appears to be between manageable and healthy at the moment. Our data center is also taking a further look into this to determine the cause of it. We are also going to shift the server to a different physical host node during the the scheduled upgrade tomorrow and the issue should be fully resolved by then. In the meantime we will keep a close eye on the server to make sure it runs smoothly.
UPDATE #4: We have decided to bring forward the scheduled upgrade to today in order to also resolve the Disk I/O issue we are facing.
UPDATE #5: As the upgrade has been completed, we will be monitoring the server closely now to ensure that the issue is resolved.
UPDATE #6: We're seeing the issue re-occur and we are working with our data center to investigate it.
UPDATE #7: The data center confirmed a network connectivity issue. We suspect that it could be related and will wait for their further updates.
UPDATE #8: The data center is still investigating the root cause of the connectivity issue in Singapore. We will continue to share updates on this situation as it develops.
UPDATE #9: Our data center has identified the issue affecting connectivity and are working quickly to implement a fix. We will provide an update as soon as the solution is in place.
UPDATE #10: The data center continues to work on the connectivity issues. We will share additional updates as more information are gathered.
UPDATE #11: At this time our data center has been able to correct the issues affecting connectivity in Singapore. We will be monitoring this to ensure that connectivity remains stable. If you are still experiencing issues, please open a support ticket for assistance.
UPDATE #12: We haven’t observed any additional connectivity issues in our Singapore data center, and will now consider this incident resolved. If you are still experiencing issues, please open a support ticket for assistance.
We are scheduling a maintenance window to add more resources to the Magnesium (Singapore) server.
This will increase our server's capacity to provide better performance and allow us to handle future growths.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We've decided to bring forward this upgrade to resolve the Disk I/O issue.
UPDATE #2: The upgrade is running at the moment.
UPDATE #3: The upgrade has been completed.
Our datacenter is scheduling a maintenance for the Magnesium (Singapore) server.
They will be performing maintenance on the network switches of the Singapore data center.
The maintenance will be performed during the following date and time:-
We do not expect any downtime during this maintenance.
However, a brief period of packet loss or increased latency may be observed.
Our datacenter is investigating a connectivity issue.
During this time, users may experience connection timeouts and errors for their website although the server is up and running.
We will share additional updates as we have more information.
UPDATE #1: Our datacenter has identified the issue affecting connectivity and are working quickly to implement a fix. We will provide an update as soon as the solution is in place.
UPDATE #2: Our datacenter corrected the issues affecting connectivity. We will be monitoring this to ensure that it remains stable. If you are still experiencing issues, please contact us for assistance.
UPDATE #3: We haven’t observed any additional connectivity issues and will now consider this incident resolved. Our data center is scheduling an Emergency Network Maintenance to further improve our network and reduce the possibility of incidence like this from happening.
We are scheduling a maintenance window to add more resources to the Helium (Singapore) server.
This will increase our server's capacity to provide better performance and allow us to handle future growths.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We have started the maintenance.
UPDATE #3: Due to some complications during the maintenance, we are going to defer the upgrade for this server. We will re-schedule the maintenance to a later date and we will send out an email beforehand.
We are scheduling a maintenance window to add more resources to the Magnesium (Singapore) server.
This will increase our server's capacity to provide better performance and allow us to handle future growths.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We have started the maintenance.
UPDATE #3: The maintenance has been completed.
We are scheduling a maintenance window to add more resources to the Carbon (UK) server.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We have started the maintenance.
UPDATE #3: The maintenance has been completed.
We are scheduling a maintenance window to add more resources to the Lithium (Australia) server.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We have started the maintenance.
UPDATE #3: The maintenance is taking slightly longer than expected. We will be powering on the server in the meantime so sites can continue to work.
UPDATE #4: We're continuing with the maintenance and the server will be inaccessible.
UPDATE #5: The maintenance has been completed.
We are aware of the sluggishness of the Magnesium (Singapore) server.
We are looking into it and will provide more updates.
UPDATE #1: We are rebooting the server.
UPDATE #2: We're notified by the Data Center that there's a network issue affecting the Singapore region. This is being investigated and more updates will be provided.
UPDATE #3: We're seeing some improvements but the issue is still being worked on and it is not fully resolved yet.
UPDATE #4: The reboot that we did just kicked in and completed.
UPDATE #5: The network appears to be stable at the moment but we will be awaiting for confirmation from our Data Center.
UPDATE #6: The network issue has been corrected. Our Data Center is monitoring it closely to ensure connectivity remains stable.
UPDATE #7: Our Data Center haven’t observed any additional connectivity issues in the Singapore region, and will now consider this incident resolved.
Our datacenter is scheduling a maintenance for the Magnesium (Singapore) server.
They will be performing network maintenance on the gateway routers of the Singapore data center.
The maintenance will be performed during the following date and time:-
We do not expect any downtime during this maintenance.
However, a brief period of packet loss or increased latency may be observed.
Our datacenter is scheduling a maintenance for the Titanium (India) server.
They will be performing network maintenance on the B side gateway routers of the Mumbai data center.
The maintenance will be performed during the following date and time:-
We do not expect any downtime during this maintenance.
However, a brief period of packet loss or increased latency may be observed.
We are scheduling a maintenance window to add more resources to the Titanium (India) server.
The maintenance window is scheduled for the following date and time:-
We expect service interruptions during the maintenance window to be between 15 to 45 minutes with our Cloud platforms allowing us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We have started the maintenance.
UPDATE #3: The maintenance has been completed.
Our datacenter is scheduling a maintenance for the Titanium (India) server.
They will be performing network maintenance on the gateway routers of the Mumbai data center.
The maintenance will be performed during the following date and time:-
We do not expect any downtime during this maintenance.
However, a brief period of packet loss or increased latency may be observed.
We will be scheduling a maintenance reboot for the Helium (Singapore) server to boot into a new kernel.
The maintenance will be performed during the following date and time:-
Any service interruptions during the maintenance should only last for 5 to 15 minutes at most.
We do not expect any issues but for precaution sake we will be taking a Snapshot of the server before the reboot.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We're starting the maintenance now.
UPDATE #3: The maintenance has been completed.
We will be scheduling a maintenance reboot for the Hydrogen (US) server to boot into a new kernel.
The maintenance will be performed during the following date and time:-
Any service interruptions during the maintenance should only last for 5 to 15 minutes at most.
We do not expect any issues but for precaution sake we will be taking a Snapshot of the server before the reboot.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We're starting the maintenance now.
UPDATE #3: The maintenance has been completed.
The Lithium (Sydney, Australia) server became unresponsive during a routine operation.
We have rebooted the server and is awaiting for it to boot back up.
More updates to follow.
UPDATE #1: The server is now back up and running.
We are scheduling a maintenance window to add more resources to the Magnesium (Singapore) server.
The maintenance window is scheduled for the following date and time:-
Service interruptions during the maintenance window should only last for 10 to 30 minutes at most as our Cloud platforms allows us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: The maintenance is starting now.
UPDATE #2: The maintenance has been completed.
Our datacenter is scheduling a maintenance for the host node of the Magnesium (Singapore) server.
The host node will be rebooted and a number of critical updates will be installed. These include general performance improvements, enhancements to networking throughput and a BIOS upgrade. These updates affect the underlying infrastructure that our Cloud Server reside on and will not affect your data.
The maintenance will be performed during the following date and time:-
A 2 hours maintenance window is allocated but the actual downtime is expected to be around 45 - 60 minutes at most.
All updates will be provided here at this Network Status page.
UPDATE #1: The maintenance has started.
UPDATE #2: The maintenance has been completed.
We are scheduling a maintenance window to add more resources to the Boron (India) server.
The maintenance window is scheduled for the following date and time:-
Service interruptions during the maintenance window should only last for 5 to 15 minutes at most as our Cloud platform allows us to scale seamlessly.
All updates will be provided here at this Network Status page.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We are taking a Snapshot of the server.
UPDATE #3: We're starting the upgrade right now. During this time there may be service interruptions of between 5 to 15 minutes.
UPDATE #4: Everything should work properly now. We're checking through to make sure everything is good before closing this maintenance.
UPDATE #5: Everything appears to be working properly so we will be closing this maintenance.
We are scheduling a maintenance window to add more resources to the Hydrogen (US) server.
The maintenance window is scheduled for the following date and time:-
Service interruptions during the maintenance window should only last for 5 to 15 minutes at most as our Cloud platform allows us to scale seamlessly.
We will provide updates here before, during and after the maintenance.
UPDATE #1: We're getting ready to start the maintenance.
UPDATE #2: We are taking a Snapshot of the server.
UPDATE #3: We're starting the upgrade right now. During this time there may be service interruptions of between 5 to 15 minutes.
UPDATE #4: Everything should work properly now. We're checking through to make sure everything is good before closing this maintenance.
UPDATE #5: Everything appears to be working properly so we will be closing this maintenance.
We are scheduling a maintenance to add more resources to the Helium (Singapore) server to meet growing demand.
The maintenance will be performed during the following date and time:-
Any service interruptions during the maintenance should only last for 5 to 15 minutes at most.
We will provide updates here before, during and after the maintenance.
UPDATE #1: We're getting ready to start the maintenance. For added precautions we will first take a Snapshot of the server.
UPDATE #2: We're starting the upgrade to add more resources to the server. During this time there may be service interruptions of between 5 to 15 minutes.
UPDATE #3: Everything should work properly now. We're checking through to make sure everything is good before closing this maintenance.
UPDATE #4: Everything appears to be working normally now so we will be closing this maintenance.
There appears to be some high server load with the Boron (India) server.
We're looking into it and will provide updates here.
UPDATE: Everything appears to be in order and the spike lasted for a few minutes only. We'll be closing this case for now.
UPDATE 2: Re-opening this as the issue is happening again. We're investigating.
UPDATE 3: The load spike happened for a few minutes and then goes back to normal. We're trying to determine what is causing the behavior.
UPDATE 4: Everything appears to be normal at the software level. We're going to rule out a Hypervisor level issue by shifting the server to a different hardware. This should take a few minutes for a server reboot.
UPDATE 5: The server has been shifted to a different Hypervisor and hardware. We will continue to monitor this.
UPDATE 6: Looks like we're seeing this again. Investigation in progress.
UPDATE 7: We're going to run an emergency maintenance to add more resources to the server. We will first shutdown the server and take a Snapshot of it to be safe.
UPDATE 8: We've added more resources to the server. We will continue to monitor this.
UPDATE 9: We've monitored the server for many hours appears to be good so we'll be closing this case.
There appears to be some network interruptions at the AWS Mumbai network.
This lasted for around 4 minutes and everything appears to be working properly now.
We will be performing a scheduled upgrade to the Helium server.
Any service interruptions should only last for 5 to 15 minutes at most.
We do not expect any issues but for precaution sake we will be taking a Snapshot of the server before the server upgrade.
More updates to follow.
UPDATE (15:27 UTC): We are starting the server upgrade now.
UPDATE (15:35 UTC): The upgrade has been completed successfully with minimal interruptions.
We will be performing a maintenance reboot for the Helium server to boot it into a new kernel.
We do not expect any issues but for precaution sake we will be taking a Snapshot of the server before the reboot.
More updates to follow.
UPDATE (16:19 UTC): We are initiating the reboot right now.
UPDATE (16:21 UTC): The server is back up.
We will be performing a maintenance reboot for the Lithium server to boot it into a new kernel.
We do not expect any issues but for precaution sake we will be taking a Snapshot of the server before the reboot.
More updates to follow.
UPDATE (15:58 UTC): We are initiating the reboot right now.
UPDATE (16:05 UTC): The server is back up.
Due to an emergency, we will be rebooting the Alpha server within the next 15 minutes.
The downtime should only be around 15 to 30 minutes at most, unless the server needs to run a File System Check (FSCK) where it can take an hour or more to complete. We will try not to have the server run FSCK after the reboot.
Update 1#: Server Rebooted.
Update 2#: Server reboot completed and server is back online.
Hello,
The Server was inaccessible and an FSCK is now running on the same. We expect the server to be back on the run very soon.
Update 1# Server was inaccessible.
Update 2# Server failed to get back Online after a reboot.
Update 3# FSCK is running on the server which is 44% complete at the moment.
Update 4# The Server is back online after a successfull File System check.
The server "NFAlpha" is inaccessible and we are investigating the issue.
UPDATE #1: The server is inaccesible via Console, hence it is rebooted by DC.
UPDATE #2: DC is currently working on the server
UPDATE #3: KVM attached, the server is rebooting. We will UPDATE #the progress.
UPDATE #4: Server reboot haven't completed successfully. Still working on it.
UPDATE #5: System rebooted again as it stalls while loading OS
UPDATE #6: Virtual drive on the server is degradded, we are still working on it.
UPDATE #7: RAID rebuild is in progress
UPDATE #8: Rebuild process reached 42%
UPDATE #9: Rebuild process reached 55%
UPDATE #10: Rebuild process reached 70%
UPDATE #11: Rebuild process reached 80%
UPDATE #12: Rebuild process reached 91%
UPDATE #13: The Rebuild process is complete, the server is being rebooted
UPDATE #14: Operating System failed to load, the Server is being booted to the Rescue mode and we are working on bringing it online.
UPDATE #15: Unfortunately, another HDD also failed. RAID array is being rebuild. Now at 52%.
UPDATE #16: RAID array is being rebuild. Now at 68%
UPDATE #17: RAID array is being rebuild. Now at 85%
UPDATE #18:RAID array build completed..
UPDATE #19: Server is online. Sites should be loading fine. We have replaced the failed drives. The Grub needs to be re-installed and it will be performed on a (later) schedule with prior notice.
UPDATE #20: Seems like the network interface is not properly initalized, we are working on it. Sorry for the inconvenience caused.
UPDATE #21: We are still working to resolve it.
UPDATE #22: The server is online now. Change in ethernet device conflicts with the server firewall, we are working on the firewall issue. The sites should be loading fine.
UPDATE #23: We are experiencing Raid issue now, asked DC to check Raid Controller. Awaiting response.
UPDATE #24: The server was rebooted to see whether its booting without the Rescue CD and then the Second Drive One Mirror of the stripe has failed. Both drives in one of the two mirrors has failed which killed the RAID0 of the RAID10. The Techs at DC attempted to force-online the RAID10 to no avail. We regret to say that the Data has been lost. We are working with the DC to reload the OS. Once the OS Reload is finished, all the data from our backups will be restored.
UPDATE #25: System Booted with Rescue CD. Backup restoration will be started as soon as possible
UPDATE #26: Encountered issue when initializing Raid, DC techs are working on it in order to rebuild the server. Restoration process will be started once the Rebuild has been completed.
UPDATE #27: Restoration process started.
UPDATE #28: Restoration in progress expected ETA 8+ hours.
UPDATE #29: 30% of the restoration has been completed.
UPDATE #30: 45% of the restoration has been completed. ETA around 8 hours
UPDATE #31: 70% of the restoration has been completed. ETA around 4 hours
UPDATE #32 : 90% of the restoration complete. ETA around 2 hours
UPDATE #33 : 1/3 Devices Restoration is complete. Moving on to the Device 2/3, ETA 2 hours
UPDATE #34: Restoration is complete. We are now rebooting the server.
UPDATE #35: The server is under a file system check(fsck) now.
UPDATE #36: The server is back online and the sites should be working fine.
UPDATE #37: We have monitored the server for some time and it is stable. A mass mail will be sent out on RFO and SLA Credits.
The server "NFAlpha" is inaccesibale and we are working on it.
UPDATE 1: DC rebooted the server but it is still not accesiable.
UPDATE 2: DC is currently checking on this.
UPDATE 3: DC updated that they need to replace the RAM and the sever is booting up now.
UPDATE 4: There sever is under a filse system check(fsck).
UPDATE 5: The file system check was completed and we are still not able to ping to the server. We have requested KVM access to check into the server.
UPDATE 6: The sever is online now. We are now checking all the services on the server.
UPDATE 7: The server is loading properly now.
Dear Clients,
We will be scheduling a physical server move for our server to our new Data Center building (within the same block) at the date and time listed below:-
====
Day: Saturday Night
Date: May 25th 2013
Time: 9.00PM PDT (Los Angeles, USA Time) - Time Zone Converter
====
The physical server move should be completed within 1 to 3 Hours but we are scheduling a full 5 hours window just in case.
Some benefits of moving to the new Data Center building include:-
- Better Network Connectivity and Redundancy
- Improved Power Redundancy
- Faster Backups for Our Servers
We will be moving the IP addresses along with the server so there will be no IP changes, no DNS propagation delays and nothing will need to be done on your end.
You will receive emails from us before and after the server move is completed. We will also provide further updates in this page.
Thank you.
----
UPDATE #1: The server move will start soon.
UPDATE #2: The server move has started.
UPDATE #3: The server has been shut down and is in the process of being moved.
UPDATE #4: The server move is still in progress.
UPDATE #5: Server move completed and server running fine now.
As communicated previously, we will continue with the maintenance on the Alpha server to provide better security and introduce new features which will be announced soon.
We will be rebooting the Alpha server within the next 15 to 30 minutes.
The downtime should only be around 15 to 30 minutes at most, unless the server needs to run a File System Check (FSCK) where it can take an hour or more to complete. We will try not to have the server run FSCK after the reboot.
UPDATE #1: The server is online now. Checking further.
UPDATE #2: Server is loading properly now.
As communicated previously, we will continue with the maintenance on the Alpha server to provide better security and introduce new features which will be announced soon.
We will be rebooting the Alpha server within the next 15 to 30 minutes.
The downtime should only be around 15 to 30 minutes at most, unless the server needs to run a File System Check (FSCK) where it can take an hour or more to complete. We will try not to have the server run FSCK after the reboot.
UPDATE #1: The server is online now. Checking further.
UPDATE #2: Server is loading properly now.
As communicated previously, we will continue with the maintenance on the Alpha server to provide better security and introduce new features which will be announced soon.
We will be rebooting the Alpha server within the next 15 to 30 minutes.
The downtime should only be around 5 to 15 minutes at most, unless the server needs to run a File System Check (FSCK) where it can take an hour or more to complete. We will try not to have the server run FSCK after the reboot.
UPDATE #1: We will be rebooting the server now.
UPDATE #2: Server booting up but network still inaccessible.
UPDATE #3: We will be booting back to the previous kernel and refer this further to CloudLinux's support.
UPDATE #4: Server is booting up now.
UPDATE #5: Server is loading properly now.
As communicated previously, we will continue with the maintenance on the Alpha server to provide better security and introduce new features which will be announced soon.
We will be rebooting the Alpha server within the next 15 tp 30 minutes. There will be a total of three reboots where the first two reboot will be performed within a short period while the third reboot will be performed later.
The downtime should only be around 5 to 15 minutes at most for each reboot, unless the server needs to run a File System Check (FSCK) where it can take an hour or more to complete. We will try not to have the server run FSCK after the reboot.
We will provide updates here.
UPDATE #1: Maintenance is starting now.
UPDATE #2: Server is rebooted (first reboot).
UPDATE #3: Server is booting up now.
UPDATE #4: Server booted up but without network as we need to install the network module for the new kernel.
UPDATE #5: It looks like we hit with another snag where the network module for the new kernel cannot be installed. We will be switching back to our previous kernel and work with CloudLinux's support to resolve this.
UPDATE #6: Server booting up on previous kernel.
UPDATE #7: Server back online. We will work with CloudLinux's support to resolve this before we proceed and as such the maintenance will need to be re-scheduled.
We will be rebooting the Alpha server within the next 30 minutes to boot into the latest kernel.
This will provide better security as well as new features which will be announced soon.
The downtime should only be around 5 to 15 minutes at most, unless the server needs to run a File System Check (FSCK) where it can take an hour or more to complete. We will try not to have the server run FSCK after the reboot.
We will provide updates here.
UPDATE #1: The server is being rebooted now.
UPDATE #2: Server is booting up now.
UPDATE #3: FSCK is currently running. We will try to speed up the process.
UPDATE #4: FSCK completed but server still not accessible. There appears to be some network configuration issue with the new kernel. We are working on making the server accessible ASAP.
UPDATE #5: We reverted back to the old kernel and rebooted the server again. Server is now up and running.
We will leave the server at the old kernel for now while we investigate the issue further.
We will re-schedule this to another date and we will announce this later.
There seem to be some file system issue with Alpha and an emergency maintenance will need to be conducted immediately which will include a reboot as well as a FSCK (File System Check).
The downtime may take anywhere from 30 minutes to 2 hours but we will schedule a full 4 hours window just in case.
More updates will be posted here.
> We had done the FSCK and rebooted the sever to a new kernel but still the sever is not online. We are working on this and will update you soon.
>The issue has been fixed. Server is up and running fine now.
We are facing a DDoS on the NFAlpha server and our DC has currently null route the Server IP to prevent it from affecting the overall network.
We will provide further updates when we have them.
Update: Our DC reports that the DDoS attack is more than 2Gbit.
Update 2: We are shifting all accounts away from the Server IP. There may be some DNS propagation delay but your website should be loading soon. To know which IP you are on, you can check it under the left bar of your cPanel.
Update 3: We are splitting groups of accounts into separate IPs to make it easier for us to identify the attack. If the attack shifts, we will know which group of accounts it will be targeting and as such we will be able to better track down which domain is being attacked.
Update 4: The attack seem to have subsided and the null route has been removed now. All accounts will be switched back to the Server IP and there may be some DNS propagation delay.
The server "NFAlpha" is inaccessible and we are working on it.
Update 1: DC rebooted the server but it is still not accessible.
Update 2: A KVM is being hooked up to check into the server.
Update 3: The Operating System (OS) refuses to boot somehow. We have loaded the system rescue CD to the server.
Update 4: It looks like the File System is corrupted and we are performing an FSCK (File System Check).
Update 5: FSCK seem to not work somehow.
Update 6: We tried another round of FSCK and it is currently running.
Update 7: Fscking is completed and the server is up now.
A detailed Reason For Outage (RFO) will be sent to all clients once we have full details about this.
Powered by WHMCS.
