Archive for *NIX

Importing Existing Keys and SSL Certificates Into Apache Tomcat

I rarely use Tomcat, but one of my clients is a Java guy and, as makes logical sense, uses Tomcat to serve the applications he writes. One of which required an SSL certificate. It’s no problem to create a new key, CSR, and import the certificate and certificate authority chains, but what if we already have an existing key and certificate for the same domain?

In our case, we had Apache serving the non-application stuff (in PHP, natch) on ports 80 and 443, with Tomcat on 8000 and 8443 (take that, Plesk!), and already had a certificate issued for the domain on the Apache side. Since the stuff used by Apache was in PEM format, I’ve added one of the steps required to convert it to PKCS12, which is what we’ll use for the Java keystore. These instructions are taken from a CentOS box, so you may need to make some modifications for other operating systems. It’s only here to serve as a guideline (and for my own future reference, primarily, because I know damned well I’ll forget again next year).

First, we need to concatenate the key, certificate (granted us by the CA) and the CA bundle into one single file. This is done most simply like so:
cat your_domain.key your_domain.crt your_ca_bundle.crt > your_domain.key_crt_bundle.pem

Next, we convert the concatenated PEM data into PKCS12:
openssl pkcs12 -export -out your_domain.key_crt_bundle.p12 -in your_domain.key_crt_bundle.pem

Create a password for the resultant PKCS12 file, and remember the password for a moment. Because you’ll need it when you import this PKCS12 into your Java keystore using the following command:
keytool -importkeystore -srckeystore your_domain.key_crt_bundle.p12 \
-srcstoretype pkcs12 -destkeystore your_domain.key_crt_bundle.jks -deststoretype jks

You’ll need to create a new password for the keystore, and then enter the password for the PKCS12 you created two steps back.

Then, edit your Tomcat server.xml file and define the full path and filename of the newly-created keystore, as well as your keystore’s password. In our case, the default location was /etc/tomcat6/server.xml. If you don’t know how to configure Tomcat6 for SSL at all, that’s beyond the scope of this particular post, and you will need to do some research. Also, do not pass GO!. Do not collect $200. And may God have mercy on your soul.

Finally, restart Tomcat doing the good ol’-fashioned service tomcat6 restart (or equivalent), and you should be good to go. And, if not…. sucks to be you.

Ubuntu Not Recognizing Changes To /etc/hosts

ah-ha

A moment ago, I finally figured out why changes to /etc/hosts on my local Ubuntu desktop were not being honored. In the past, it worked just fine, as expected, but this morning, it refused to recognize changes. I searched all over the web and found lots of people with the same problem, but no solutions. Plenty of helpful suggestions, mind you, but nothing would work for the folks who tried them. So, the solution? My NSCD was caching it. Perhaps there was a default value change recently, or maybe I just somehow never noticed it before because I’d add the entry prior to trying to work with the host. Not sure the ultimate reason, but the fix is in:

sudo vim /etc/nscd.conf

Change:
enable-cache hosts yes
…. to:
enable-cache hosts no

And then restart NSCD:

sudo service nscd restart

Voila! Finally, I can get on with my work for the day.

Horribly Slow Speeds On USB Stick, Ubuntu 12.04LTS (100KB/s?!?)

23426115i_01

I just finished building a new server for the house here and downloaded the latest build of Ubuntu Server 12.04LTS. My desktop is running an upgraded version of the same (but Desktop, not Server edition). Trying to create a USB boot disk to install on the new box was painfully slow: it was going to take 2.5 days.

After searching all over the web to see what others thought, checking the USB settings in my BIOS, and even rebooting for the sake of a potential fix chalked-up to voodoo, I realized the answer. Checking the USB stick’s partition, it was – unsurprisingly – FAT32. Once I dropped the partition (the stick was brand-new, just opened the package) and created a new ext4 partition in its place, I created my new USB boot disk in 38 seconds. That’s much more like it.

Custom sudo Login Prompt: Confuse Your Coworkers and Friends!

obey_sudo_card-p137691123849347174bh2r3_400

Quick way to have fun with Bash and sudo on a boring day. Insert the following into your /etc/bashrc, /etc/bash.bashrc, or similar file (as is appropriate for your distro and version):


alias sudo='sudo -p "Congratulations, %p! You are the one-millionth user to attempt to sudo to %U! Enter your password to see what you've won. "';

More info, from man sudo:

%H  expanded to the host name including the domain name (on if the machine's host name is fully qualified or the fqdn option is set in sudoers(5))
%h  expanded to the local host name without the domain name
%p  expanded to the name of the user whose password is being requested (respects the rootpw, targetpw and runaspw flags in sudoers(5))
%U  expanded to the login name of the user the command will be run as (defaults to root unless the -u option is also specified)
%u  expanded to the invoking user's login name
%%  two consecutive % characters are collapsed into a single % character
The prompt specified by the -p option will override the system password prompt on systems that support PAM unless the passprompt_override flag is disabled in sudoers.

British PHP? Cheers!

british_php

Several minutes ago, one of the regular and long-term contributors to the PHP community, a gentleman by the name of Daevid Vincent, posted a link to a blog post on the PHP General mailing list. (Enough links yet? Hang on, there’s more to come.)

The blog post, If PHP Were British, was something I enjoyed — despite my apparently inferior dialect of English, the bastardized American version. So I figured, what the hell? We got the land (after we, as Redcoats, knocked off a few million of those pesky Indians), so why not offer a peace treaty in the form of a few lines or reworked PHP core?

After about fifteen minutes of work, the result: BPHP (lifted straight from their comments section). It’s based upon the latest stable of the 5.4 branch as of this writing (5.4.4), and has the changes requested specifically by the main article, as well as few other related changes. Nothing serious, but it shows that, yes, most Americans are willing to reach out and be friendly and helpful world citizens, regardless of how we may appear to the rest of the nations around the globe.

Want to give it a test drive? Go ahead and download it in .tar.gz or in .tar.bz2 format.

You’ll no doubt see errors and such, but have no fear — I have absolute no intention of supporting this release, nor providing bug fixes, or really even acknowledging that I did, in fact, spend several minutes of my evening doing this.

I should probably mention that I did this in my own free time (I have about an hour before another client is re-running on the ABC network TV show “Shark Tank,” and finished some other work ahead of time), and not as part of the PHP team. And of course, it is licensed under the actual PHP license, is intended only for entertainment purposes, and neither myself (acting alone) nor the PHP Group, community, or anyone else is responsible for any damage, incompatibilities, et cetera. Just in case there are any future BPHP users out there who are lawyers. ;-P

Happy Friday…. mates.

Bash ‘for’ Loop and Filenames With Spaces

A quick post for my own future reference, primarily.

After banging me face off the desk for a while trying to figure out how to batch-convert a heaping spoonful of space-laden-named Excel files to CSV for a project I’m doing for my wife, I found a solution in the $IFS environment variable. Thus:

#!/bin/bash
IFSTMP=$IFS;
IFS=$(echo -en "\n\b");
for i in $(ls -1 *.xls); do
xls2csv $i > $i.csv 2>/dev/null;
done;
IFS=$IFSTMP;

Easy as pie. And I don’t mean like a mince pie or something that many folks don’t even like, but like convincing toddler to eat a chocolate cream pie. Yeah, that easy.

Announcing the Release of the System Detonation Library for PHP

As discussed somewhat at length in a rapidly-devolving thread on the PHP General mailing list, I am in favor of a function that, when called, will initiate on the host system a self-destruct sequence.  Well, being a nice, sunny, spring Friday morning, I decided to offer just that:

Introducing the first public release of the System Detonation Library for PHP.

This useless extension provides one function with one purpose: to cause your server to explode.  Due to the obvious hazards involved, including (but not limited to) loss of hardware, limbs, and potentially life and liberty, this has only been tested on one single occasion, using a PC with Ubuntu 10.10 and a heavily-modified SVN version of PHP 5.3.6.  Thankfully, as the test was successful, there were no serious injuries.

Firstly, you may download the package here.

Second, as a very basic course on the compilation and installation of this unofficial PHP extension, here are some simple instructions for Linux users.  All others are on their own, and this may (read: probably) will not work anyway…. which is a shame, because I know plenty of Windows boxes that should have the right to self-destruct as well.

  1. Download the package above.
  2. Extract it: tar -zxf detonate-0.2.tar.gz
  3. Change to the newly-created directory where the files are located: cd detonate-0.2/
  4. Build the wrappers for your version of the Zend/PHP API: phpize (NOTE: on Ubuntu-built packages, this command may be: phpize5)
  5. Build the necessary makefiles for your system: ./configure –with-detonate
  6. Compile the code: make
  7. Install the binary (as root, or using sudo): make install
  8. Edit your php.ini to load the newly-installed extension by adding this line: extension=detonate.so
  9. If you plan to use it via the CLI, you’re done.  For use on the web, remember to reload/restart your web server.
  10. Create a basic PHP script with the following: <?php detonate(); ?>
  11. Check your insurance coverage.
  12. Run the script created in Step #10.

And that’s all there is to it.  Feel free to install this on all of your systems and use it as a replacement for exit or die() in your scripts.  Because, unlike die(), this function will absolutely get the point across, once and for all.

Replacing One Character In A String With A Random Character

Just an hour or so ago, Ron Piggot asked a question on the PHP General mailing list. The original question was how he could replace a single matching character in a string containing multiple matches with another random character.  I mocked up a working example in about five minutes or so.  It’s far from perfect, and not very elegant, but it’ll work as a starting point of reference.

The code I sent back in reply follows:


Distributing php.net’s Synchronization Infrastructure

Several days ago, the primary server hosting all of the data comprising the php.net site for synchrony with all of the mirrors around the world became completely inaccessible. Due to security policies with the provider hosting the server, it was some time before we were able to have the machine returned to normal operational status. As a result, network content became stale, and automated tests on the mirrors saw them as outdated and deactivated them. It pointed out a flaw that, though this time was just an inconvenience, has the potential to grow into something more serious – including a sort of self-denial-of-service, if you will, if it went unnoticed for several days and all mirrors were seen as outdated.

Mark Scholten from Stream Service, an ISP based in the Netherlands and provider of an official mirror for their countrymen at nl.php.net, offered to put up a second rsync server, which gave me an idea: take the load off the primary server by distributing it across three regions.


(Click the image to view the full size version.)

Mark set up the European (EU) box in their Amsterdam datacenter, we (Parasane) had already set up an emergency rsync mirror in case the primary dropped out again which would be repurposed for the Americas (AA), and I contacted Chris Chan at CommuniLink in Hong Kong for what would become the Asia-Pacific (AP) region. Chris had submitted an application to the official waiting list to become an official PHP mirror back in February of 2010.

Compiling data over the course of the last 12 months from mirrors in our network which had it readily available, accurate, and up to date, I drew out a plan for the regions so as to limit the load and stress on each new mirror. Thus, the tri-colored map above. I also learned in the process that we will have served roughly 223 gigabytes of data over HTTP, network-wide, by the end of January, 2011, which averages out to about 1.9GB per mirror, per day, with the 115 active mirrors we have worldwide as of right now.

Setting myself an arbitrary date of 30 April, 2011, the goal is to have all existing official mirrors flipped over to using the rsync server designated for their country. Visitors to php.net should see no difference and should experience no negative impact during the transition, but the benefits will be great: far less of a likelihood of a mirror being automatically dropped from rotation due to stale content; the ability of the maintainer to decrease the amount of time to synchronize their mirror to hourly, providing the freshest content as it becomes available; less latency and greater speeds for many of those who are far from the current central rsync server; far, far less stress on our own network.

The immediate goal is ensuring that there are no snags, and that we can successfully synchronize all of the data to the website mirrors without omission. Beginning right away, I’ll be coordinating privately and directly with a few mirrors around the world to beta test the new layered design according to the rsync distribution plan. By 12 February of this year – a bit more than two weeks from now – I hope (and expect) to have all of the kinks straightened out. After that, we’ll begin migrating the rest of the network in its entirety to the new design.

All new mirrors from that point forward will be instructed to use their local rsync mirror as well, as defined by the map above.

It’s no large task, of course, but I’m hoping that the addition of just three new servers will help to ensure the health and stability of the network as a whole for years to come. While I don’t expect anyone to notice any difference – good or bad – in the user experience, behind the scenes I think we’ll not only see some differences in operations, but also begin to come up with even more ways to improve performance in the future.

(Finally) Announcing the Public Release of FileConv for PHP

Almost exactly two years ago, on New Year’s Day, 2009, I sent an email describing a new PHP extension I’d finished, and was interested in submitting to the PECL repository. The package, entitled FileConv, would natively handle file conversion for line endings, back and forth, between *NIX (LF: \n) and DOS (CRLF: \r\n). At that time, Ilia Alshanetsky recommended that I also add Mac conversion (CR: \r). Legacy MacOS, that is, prior to becoming a sort of ClosedBSD, if you will.

Somehow, as often happens in our busy lives, I forgot to follow through with submitting it to the PECL repo. Last night I was using one of the functions and found a datestamp bug, where – in certain situations – it would give the converted file a date approximately 430,000 years in the future. That’s actually almost 253-times the estimated duration for the complete decomposition of a modern PC, which is figured to be a paltry 1,700 years. That said, once I patched the file and recompiled, I was reminded of my discontinued effort to release the code to the public as an open source package. Well, time to change that, I suppose.

So today, you can download the FileConv extension right here:
FileConv-2.2.6.tar.bz2 (7,073 bytes)
FileConv-2.2.6.tar.gz (6,636 bytes)
FileConv-2.2.6.zip (10,531 bytes)

MD5 Hashes:
– d6200f0693ae63f9cc3bb04083345816 FileConv-2.2.6.tar.bz2
– c2b0db478628e0a4e2ce66fb06d19593 FileConv-2.2.6.tar.gz
– b3ff103424e4be36151a1c5f9cadd58d FileConv-2.2.6.zip

SHA1 Hashes:
– 0521fdeaa8bfb250c8c50bc133b355872fa70cad FileConv-2.2.6.tar.bz2
– 08e2c361fc41f925d0b4aa3a0bbdd7e0884b24d6 FileConv-2.2.6.tar.gz
– 9eb9355555dd8e6e6b6b7f3dc7464c7a6107b187 FileConv-2.2.6.zip

Keep in mind: this has only been tested on Linux (CentOS, Mandriva, and Ubuntu), as I have neither the ability nor desire to play on Windows (but feel free to try it). In a future release, the code will be compacted more, as well; right now, every conversion function has its own function within the source. This isn’t necessary: it could be a single master function with a simple switch for one small section of the code. I’ll get to that another day, when I have some time to hack it up again.

This distribution comes with a very simple automated installer for now. If/when it moves to PECL, that will be phased out, of course, as PECL will handle that itself. If you have root/sudo access on the box, you can just run ./install.sh from the package directory and follow the instructions from there. Manual installation instructions are included as well.

This package provides functions that somehow never made it into the PHP core: dos2unix(), unix2dos(), mac2unix(), mac2dos(), unix2mac(), and dos2mac(). It does not, however, do any checking or validation prior to conversion. If you decide to use this library, I’d highly recommend employing some basic checking in your code. Something like this should be used at a minimum:

<?php
function get_info($filename) {
  if (!function_exists('version_compare') || version_compare(phpversion(),'5.3.0','<')) {
    return trim(`file {$filename}`);
  } else {
    $finfo = finfo_open();
    $format = finfo_file($finfo, $filename);
    finfo_close($finfo);
    return $format;
  }
}

if (strpos(strtolower(get_info($filename)),' crlf line')) {
    // File is DOS (\r\n)
} elseif (strpos(strtolower(get_info($filename)),' cr line')) {
    // File is legacy Mac (\r)
} else {
    // File is *NIX (\n)
}
?>

NOTE: this does not ensure that it is a text file. You are strongly advised to address that as well. The included test.php file has a line that checks to see if the file is binary or text, so feel free to plagiarize that — or, better yet, build a better mousetrap.

If you come across any bugs/issues/whatever, let me know.