XILO News and Status

We have received the following notification from one of our transit providers.

Impact: Temporary service breaks throughout maintenance period.
Service Impacted: Customers in Sovereign House, Meridian Gate, Harbour Exchange 6&7, Harbour Exchange 8&9, Global Switch 1, and Telehouse East and any customers with interconnects traversing this network segment.

March 2008
Maintenance Window: 22:00 28th March 2008 until 06:00am 29th March 2008
Expected service outage: maximum 10 minutes downtime per site, however each site connection shall be moved over in 2 stages so downtime is expected to only be a few seconds, additional 30 minutes downtime for router upgrade in HEX8&9.

During this time there maybe short bursts of network interruption which may result in slow or unresponsive connections for upto 10 minutes as above.

Internal monitoring has alerted our engineers of a possible problem with one of our core switches. We are conducting an emergency reboot of the device to see if the errors continue.

There maybe a small 1-2 minute outage for some hosted customers.

Apologies for the short notice but immediate action is needed to ensure that a switch is replaced immediately if necessary.

An arbitrary file inclusion vulnerability has been discovered in the Horde webmail application. At present, we can confirm that this security vulnerability in question affects Horde 3.1.6 and earlier. Based on incomplete information at this time, we also believe this affects Horde Groupware 1.0.4 and earlier as well (cPanel does not use Horde Groupware at this time).

cPanel customers should update their cPanel and WHM servers immediately to prevent any chance of compromise. The patch will be available in builds 11.18.2 and greater (or 11.19.2 and greater for EDGE systems). The updated builds will be available immediately to all fast update servers. The builds will be available to all other update servers within one hour of this posting.

To check which version of cPanel and WHM is on your server, simply log into WebHost Manager (WHM) and look in the top right corner, or execute the following command from the command line as root:

/usr/local/cpanel/cpanel -V

You can upgrade your server by navigating to  ‘cPanel’ -> ‘Upgrade to Latest Version’ in WebHost Manager or by executing the following from the command line as root:

/scripts/upcp

It is recommended that all use of Horde 3.1.6 and earlier be stopped (on cPanel and non-cPanel systems alike) until Horde updates can be applied.

You can disable Horde on your cPanel system by unchecking the box next to ‘Server Configuration’ -> ‘Tweak Settings’ -> ‘Mail’ -> ‘Horde Webmail’ within WHM, and saving the page with the new settings.

We’re currently looking into DoS (denial of service) attack against one site on Indigo. This has caused the web server to flood with connections and some pages may not currently load.

The site concerned has been suspended and we’re attempting to block the attack at the edge network.

After a systems upgrade by our card processor this weekend, some customers regular payments were not debited correctly resulting in a decline message being sent via email.

This issue was raised with the card processor who investigated the issue and re-configured our account correctly which appears to have resolved the problem.

Please contact billing via my.xilo.net if you are having continued problems in making payment.

After requests from customers, we have altered the way our blog sends out email notifications.

Whereas before it would send one single email, it will now send emails on every update (comment) that is posted on the topic in question.

If you have any questions regarding this, please contact support via My XILO.

We’ll be conducting routine maintenance on Indigo this evening. During the maintenance period we’ll be removing it from the cluster that it is currently located on.

The reason for this is that after the issue 2 weeks ago with our cluster, we have decided to revert back to our standard configuration of hosted servers so that such an outage is avoided in the future.

We’ll be starting the work at 11pm GMT this evening and we expect to be completed by 3am. Although the window is 4 hours, we expect the work to be completed in a much shorter period.

During the period, we will be disabling the following services to prevent any differences in data during the migration;

• FTP
• SMTP
• POP3/IMAP
• SCP(SSH)

Please accept our apologies for the short notice for the work. This is due to a few residual issues that are causing higher than normal loads on Indigo.

If you have any questions, please do contact us via My XILO.

Due to a customer script, the KILO server had two unplanned and necessary reboots today. The script in question saturated the CPU and as such, we were unable to login via SSH or console.

After the reboot, we identified the customer and have suspended their account until we can resolve their issues.

Unfortunately, due to the unclean reboot - the RAID array is having to initialise the data on the array. This will take a few hours and during this period, the server will respond slightly slower than normal.

Apologies for the two small outages and the slower than normal response times whilst the RAID initialises

We’d like to remind all our customers about using secure passwords on their hosting accounts. This includes FTP, mail box and MySQL (database) accounts. Also, for our resellers, creating accounts for your clients.

We’re seeing more and more clients and resellers using simple passwords similar to the below;

• 123456
• test
• password1
• testtest
• 123

This can lead to serious security issues on your account which may lead to the suspension of your account(s) if the password is guessed or brute forced.

We’ll be activating a new feature in cPanel that will only allow the creation of slightly harder password combinations to prevent any accounts being accessed without authorisation.  This is in addition to the brute force detection that we currently have in place which blocks most attacks.

If you feel you are having problems changing your password, please contact support via my.xilo.net.

We’ll be changing the way some of our older reseller accounts were assigned nameservers and IP addressing.

This will affect any reseller that has private nameservers (such as ns1.mydomain.co.uk etc) or is using any of the following;

• ns1.alpha.*.net (and ns2)
• ns1.bravo.*.net (and ns2)
• ns1.charlie.*.net (and ns2)
• ns1.delta.*.net (and ns2)
• ns1.echo.*.net (and ns2)
• ns1.foxtrot.*.net (and ns2)
• ns1.kilo.*.net (and ns2)
• ns1.india.*.net (and ns2)

(* - we have removed the domain to protect our resellers)

Also, any customers that are using the following IP for any of their private nameservers will need to update to the new, permanent value.

• Old : 72.21.47.31
• New: 67.225.140.31

This should be changed in your zone via WHM as well as at your domain registry. If you are unsure on how to do either of these, please contact support via http://my.xilo.net/ and we’ll take care of it for you.

The old nameservers will cease to work after 1st March 2008 so it is imperative that you change these immediately. Please contact support for the new nameservers for all of your accounts if you do not have private nameservers assigned. (Once again, not posted for the protection of resellers)

If you have any questions, please do contact us.