Elliot Loves Perl » upgrading

One thing I’ve never understood about the perl core: binary compatibility.

elliotlovesperl — Thu, 02 Jul 2009 02:12:24 +0000

So, I was reading chromatic’s latest, wherein he links to a p5p post which mentions the whole bit about @INC including multiple point releases in it, which makes having them around expensive. In case you didn’t know it, you can install perl point releases in the same directory and the default @INC will include all the releases in descending order, meaning that you can use binary modules from prior point releases without recompiling them.

But why?

If you’re going to go to the trouble of compiling a new perl executable, how much more effort is it to recompile everything else in your install? If you’re using continuous integration, don’t merely use it on your code, use it on the CPAN modules you use as well.

The whole idea of mixing point releases seems like it’s a nightmare waiting to happen.

Actually, why are you upgrading an existing perl installation at all? What if there’s some code that can’t deal even with a point release upgrade? You shouldn’t force that upgrade on others in your organization. Put the point release into a different directory and lock it down. Don’t use site_perl; install all of your modules into some private area and use PERL5LIB. (Learn to use INSTALL_BASE with ExtUtils::MakeMaker and install_base with Module::Build.)

Tagged: maintenance, upgrading

Dealing with perl binary compatibility issues as a user of XS modules.

elliotlovesperl — Mon, 25 May 2009 22:46:37 +0000

chromatic discusses binary compatibility with the perl executable. I’m someone who only uses XS modules and hopes to never have to write one. Yet, unlike what seems to be the prevailing fear, I don’t worry about binary compatibility. I don’t worry about upgrades breaking things. I don’t worry that I’m going to have mismatches in executables/libraries.

Why? Continuous integration.

Don’t just grab modules from the CPAN and install them somewhere never to be upgraded until you’re forced to at a huge amount of time and expense. Regularly rebuild everything you use, preferably daily or hourly.

Install a perl binary and then lock it down. Don’t use site dirs. Have the perl core be inviolate.
Instead, install to some other directory and point PERL5LIB at it. Both CPAN and CPANPLUS allow you to configure what you want to be passed to the configuration program. Pass a value for INSTALL_BASE to Makefile.PL and install_base to Build.PL that points to your directory.
Set up a build server to use CPAN or CPANPLUS to “install” and build everything. Since your build machine will be rebuilding everything regularly, when you upgrade your perl binary or an XS module, any problems will be quickly identified. Everything being rebuilt in one go means that all binary code will match up and you don’t have to worry about them getting out of sync. Because installing a distribution causes all of that distribution’s tests to be run, you’ll know that that distribution works in your environment.

(If you’re using taint mode and thus cannot use PERL5LIB, have your build machine build a custom perl executable along with everything else.)

Details for doing this belong in another post. But the point is, if you regularly rebuild all of the CPAN code you use, you don’t need to worry about binary issues.

Tagged: administration, continuous integration, modern perl, upgrading, xs

Managing perl

elliotlovesperl — Thu, 11 Sep 2008 17:22:15 +0000

(The lowercase “p” perl is what I mean in the title.)

There’s a great new page on the Perl 5 wiki about administering your perl installation. It includes quite a number of things I believe in. Some of the more significant bits:

First and foremost thing I can say is if you depend heavily on Perl (or any single piece of technology) build it yourself. This will allow you finer control and faster upgrades than your OS vendor can. Windows is the exception, Strawberry Perl does a much better job packaging Perl and tweaking it for Windows than you can (and if you think you can better, please contribute to Strawberry).

This is especially true on Red Hat. Ignoring general management issues, you really ought to have a threaded and a non-threaded perl so that single-threaded code doesn’t pay the overhead for threads.

Fourth, keep up to date. This means keeping CPAN modules and perl itself at or near the latest versions. This will incur the occasional breakage as you encounter either intermittent bugs in new versions of CPAN modules, or API changes, or project code that accidentally depends on certain versions of CPAN modules. But it’s worth it, because the farther you fall behind the harder it becomes to catch up. As your perl gets older and older, fewer and fewer CPAN modules will work requiring more and more patching to get them working. As your CPAN modules get older, it will require a greater leap to bring them up to date, and more upgrades simultaneously. The sudden upgrade will cause many bugs to happen simultaneously making them take far more time and effort to track down then if they could be dealt with individually. Large upgrades require large effort, time and risks. Small upgrades require small effort, time and risks. The looming risk of a large upgrade will make developers settle for an older, buggier, less capable version and write a work around rather than risk the upgrade. This creates a hidden drag on your project and creates more and more complex code to maintain.

Another point: when you discover a bug in a CPAN module, there may be a fix for it, but if you’re very out of date with it, it may cause significant problems with other modules, making your upgrade of that one module really painful.

At Barclay’s, it took me 7 months to upgrade them from perl 5.6 to 5.8 and the latest version of other modules. Way too long.

If you’ve got good test coverage, you should be able to easily find any problems with upgrades.