Archive for January 2011

Windows Server Says, “Network Cable Unplugged” When It’s Not?!?

Once again, stuck managing a Windows box. Yeah, I know, I’ll whine, bitch, moan, and cry you a river another time.

The Problem: Using the secondary NIC (PNET/VLAN), I found a lock of packet collision during negotiation, handshaking, and identification, causing Windows to give up and basically say, “well, since it’s not working, the cable must physically have been removed, because there’s no way I could ever be wrong.”

Wro…. err…. incorrect, Windows. (You’re wrong.)

The Discoveries: The truth was, at least in my case, that it wasn’t properly handling the gigabit capabilities of the card on the box. I’m not the administrator for these machines (though they’re housed in our datacenter), so I can’t be certain that nothing had changed recently, but their staff said nothing at all had been modified. Perhaps that really was the case, and nothing had been changed — Windows has been known to do stranger things than this, of course, sometimes out of the blue.

The Solution (for my case): Go to the screen where you can view your network adapters (your version of Windows dictates the path of navigation, hence the ambiguity). Next, right-click the adapter with the “Network Cable Unplugged” message and click “Properties.” Click the appropriate button to configure the network adapter. Then click the tab on that dialog for “Settings” or something of the like (sorry, but I logged out in a hurry, so this is from memory), and you’ll see a list of parameters on the left, with their values on the right. Find one related to speed and duplex, and if you see it set to “Auto” or similar, drop it to “100Mbps Full Duplex” and click OK. Close the properties dialog by clicking “OK” and see if the settings are already bringing the network adapter back online. If not, disable and re-enable the adapter, and – if it was indeed the same issue – you should be back online within a few seconds.

Distributing’s Synchronization Infrastructure

Several days ago, the primary server hosting all of the data comprising the site for synchrony with all of the mirrors around the world became completely inaccessible. Due to security policies with the provider hosting the server, it was some time before we were able to have the machine returned to normal operational status. As a result, network content became stale, and automated tests on the mirrors saw them as outdated and deactivated them. It pointed out a flaw that, though this time was just an inconvenience, has the potential to grow into something more serious – including a sort of self-denial-of-service, if you will, if it went unnoticed for several days and all mirrors were seen as outdated.

Mark Scholten from Stream Service, an ISP based in the Netherlands and provider of an official mirror for their countrymen at, offered to put up a second rsync server, which gave me an idea: take the load off the primary server by distributing it across three regions.

(Click the image to view the full size version.)

Mark set up the European (EU) box in their Amsterdam datacenter, we (Parasane) had already set up an emergency rsync mirror in case the primary dropped out again which would be repurposed for the Americas (AA), and I contacted Chris Chan at CommuniLink in Hong Kong for what would become the Asia-Pacific (AP) region. Chris had submitted an application to the official waiting list to become an official PHP mirror back in February of 2010.

Compiling data over the course of the last 12 months from mirrors in our network which had it readily available, accurate, and up to date, I drew out a plan for the regions so as to limit the load and stress on each new mirror. Thus, the tri-colored map above. I also learned in the process that we will have served roughly 223 gigabytes of data over HTTP, network-wide, by the end of January, 2011, which averages out to about 1.9GB per mirror, per day, with the 115 active mirrors we have worldwide as of right now.

Setting myself an arbitrary date of 30 April, 2011, the goal is to have all existing official mirrors flipped over to using the rsync server designated for their country. Visitors to should see no difference and should experience no negative impact during the transition, but the benefits will be great: far less of a likelihood of a mirror being automatically dropped from rotation due to stale content; the ability of the maintainer to decrease the amount of time to synchronize their mirror to hourly, providing the freshest content as it becomes available; less latency and greater speeds for many of those who are far from the current central rsync server; far, far less stress on our own network.

The immediate goal is ensuring that there are no snags, and that we can successfully synchronize all of the data to the website mirrors without omission. Beginning right away, I’ll be coordinating privately and directly with a few mirrors around the world to beta test the new layered design according to the rsync distribution plan. By 12 February of this year – a bit more than two weeks from now – I hope (and expect) to have all of the kinks straightened out. After that, we’ll begin migrating the rest of the network in its entirety to the new design.

All new mirrors from that point forward will be instructed to use their local rsync mirror as well, as defined by the map above.

It’s no large task, of course, but I’m hoping that the addition of just three new servers will help to ensure the health and stability of the network as a whole for years to come. While I don’t expect anyone to notice any difference – good or bad – in the user experience, behind the scenes I think we’ll not only see some differences in operations, but also begin to come up with even more ways to improve performance in the future.