Skip navigation

Category Archives: Uncategorized

One thing I’ve never understood about the perl core: binary compatibility.

So, I was reading chromatic’s latest, wherein he links to a p5p post which mentions the whole bit about @INC including multiple point releases in it, which makes having them around expensive. In case you didn’t know it, you can install perl point releases in the same directory and the default @INC will include all the releases in descending order, meaning that you can use binary modules from prior point releases without recompiling them.

But why?

If you’re going to go to the trouble of compiling a new perl executable, how much more effort is it to recompile everything else in your install? If you’re using continuous integration, don’t merely use it on your code, use it on the CPAN modules you use as well.

The whole idea of mixing point releases seems like it’s a nightmare waiting to happen.

Actually, why are you upgrading an existing perl installation at all? What if there’s some code that can’t deal even with a point release upgrade? You shouldn’t force that upgrade on others in your organization. Put the point release into a different directory and lock it down. Don’t use site_perl; install all of your modules into some private area and use PERL5LIB. (Learn to use INSTALL_BASE with ExtUtils::MakeMaker and install_base with Module::Build.)

Dear LazyCPAN, please create these documentation modules.

Ideas that I’ve got floating around, but which I haven’t gotten to yet:

Pod::Coverage::PPI
Like Pod::Coverage, but rather than looking at the symbol table, uses PPI to discover subs that need documenting.
Pod::Coverage::Moose
Like Pod::Coverage, but, in addition to regular subroutines requiring documentation, reports on Moose generated accessors, delegations, etc. that are undocumented.
Pod::Coverage::PPIMoose
Combines the two others.

Perl::Critic supports autodie.

In chromatic’s session this morning, there was a comment that Perl::Critic didn’t support autodie. It actually has supported it since New Year’s day of this year.

Let me say emphatically that one of the core Perl::Critic developers (i.e. me) loves autodie. If he could, he would marry it.

If you aren’t using autodie, please do so.

Please use autodie. PLEASE use autodie. PLEASE “use autodie;”!!!

(Or, even better “use autodie qw< :all >;”.)

A reasonable approach for Modern::Perl.

At YAPC::NA this morning, chromatic answered my question about Modern::Perl, “What happens when perl 5.12 comes out?”, with what I think is a good solution.

The proposed answer is for Modern::Perl to take a date parameter to indicate what “modern” means. This is brilliant because it explains to even the unknowledgeable programmer when the code was current. Say the syntax is like:

    use Modern::Perl as_of => '2009-06-23';

What could be more clear?

What I got out of the Parrot Workshop.

I previously wrote about what I wanted out of the Parrot workshop. So, here’s what I actually got.

  • There are some amazingly smart people working on Parrot/Rakudo who really believe in the promise of both.
  • I need to actually start learning Perl 6. My wrote my first baby Perl 6 programs Saturday. It’s nice to have something that’s like (for some definition of “like”) Perl 5, but organized from the start so that things are consistent. Right now, all I see are the problems from Perl 5 that have been fixed; I don’t see any of the problems that exist in Perl 6 yet.

  • A lot of the introspection functionality isn’t there yet. Since things are still in flux, there isn’t a lot of concrete documentation on what each class does. Thus, I figured the easiest way to learn what’s there, as opposed to what’s in the spec, was to first figure out the introspection mechanisms and use those to poke around. It turns out that the $foo.HOW().^attributes() method doesn’t exist yet, so looking for the structure of classes isn’t possible yet. However, the $foo.HOW().^methods() method does exist, so I started exploring that. My first try at simply listing the methods on an object (say $_ for 1.^methods()) resulted in some expected stuff, but there were a bunch of “1”s and “2”s in there. It turns out that those are collections of multi methods (which are like overloaded methods in other languages, but the determination of which variant to use is made at runtime, not compile time). This is when I learned that a bunch of Rakudo classes (e.g. the multi method classes) are actually Parrot classes (currently) and thus they don’t act like Perl 6 classes are supposed to, so it’s hard to introspect those.
  • With the goal of starting on Perl6::Critic, I learned that I can get a parse tree via my $parse_tree = Perl6::Grammar.parse($source_code); or my $parse_tree = Perl6::Compiler.compile($parse_tree, :target('parse'));, or even get the abstract syntax tree via my $ast = Perl6::Compiler.compile($parse_tree, :target('past'));.
  • I had heard that Perl 6 supported classes and prototypes, but I had no idea where the prototypes came in. It turns out that Perl 6 classes are actually implemented as prototypes. In fact, classes are “instances” of themselves, specifically the undef instance. So, when you declare a typed variable, but don’t provide an initial value, the undef that the variable gets initialized to is the class itself. Given 
        my SomeClass $x;
    my $y = SomeClass;

    $x and $y refer to the identical thing. This is seriously cool stuff.

  • Parrot is far enough along that people are actually discussing cross-language issues such as IPC and standard Parrot libraries.
  • Some people aren’t happy writing code in “assembly language” (PIR) and are thus creating “C” for Parrot (Close, as in “close to C”).
  • Rakudo cannot handle the official Perl 6 grammar, STD.pm, yet, so I need to work with the Rakudo grammar.
  • I can actually see an approach to tackling Perl::Critic for Perl 6. The first thing I want to do is create a ppidump for Perl 6. Parrot has a means of doing this, but I want something in regular Perl 6, not only for the functionality, but to learn how to walk the parse tree. The next step is to implement a ProhibitMagicNumbers program; no configuration, no options, just take a set of file names and spit some stuff out. Patrick suggested that I could subclass the grammar and use actions on it to find the numbers and look at the surroundings; that would work for a specific policy, but this initial program would be an exploration in how to do the real Perl::Critic and I don’t plan on a grammar subclass per policy. (That would mean that each file would need to be parsed individually for each policy.)

Moose as documentation.

Forgetting all the type constraint, role, and attribute handling that Moose does, I like it simply for what it’s contributed to documentation.

I’ve been using the constraint syntax for describing complicated data structures in POD and comments in cases where Moose isn’t actually involved. It’s nice to be able to simply state that a subroutine returns “ArrayRef[HashRef[Maybe[Str|HashRef[Foo::Bar]]]”. That’s a lot more comprehensible than a paragraph of description.

What I want out of the Parrot Workshop.

I’m going to the Parrot virtual machine workshop next weekend. The suggestion has been made that “anyone who is coming from out of town to attend the Parrot talks is already involved with Parrot” and the schedule is too introductory. I beg to differ.

I am not presently involved with Parrot, nor do I expect to be in the future; I’m already behind on the projects that I’m involved with or want/need to start. At most, I expect that I’ll be filing bug reports.

I’m expecting that I’ll be using things running on Parrot. I want to understand what’s going on underneath in order to make my life in understanding higher level things easier. As a point of comparison, there are some edge cases in Java’s behavior that you cannot understand without knowing how the Java virtual machine works.

I want to know how to drop to a lower level, if the higher level code running on the VM isn’t working out for some reason.

When dealing with Perl 6 specifically, I want to know what I can get at the Parrot bytecode level and what I need to do with STD.pm. I’m probably going to be involved in a Perl 6 version of Perl Critic and I’d like to know what would be possible in terms of a generic higher-level-language-running-on-Parrot critic. (Yes, I don’t know how to use a programming language that’s still not finished being designed and I’m already planning on to tell you that you’re using it wrong.)

Using NYTProf to understand why not to do something.

Mr. Bunce is asking for examples of how NYTProf told you what to fix with poorly performing code.

Instead, I’m going to give an example of how NYTProf told me why I shouldn’t fix something a particular way.

I’ve got an app that needs to analyze a bunch of logs. Having used NYTProf in the past to identify that creation of DateTime objects is my biggest issue, I was thinking of ways to improve the situation. One thing I thought of is being able to create a DateTime based upon an epoch time instead of individual date components (year, month, etc.), with the idea that the cost of deriving the epoch time from the date components was what was expensive (I can affect the format of some of the logs, and thus could get the epoch time emitted). So, I quickly threw together a little Benchmark program that compared the costs of constructing DateTimes from each type of data. I was shocked to discover that creating objects based upon epoch times was over 20% more expensive than creating them out of date components.

What was going on? NYTProf provided me with the answer. I created two simple programs that created 100,000 DateTime objects, one for each kind of constructor arguments. I profiled both of them and compared the NYTProf output for them. This led me to looking at the from_epoch() code in the NYTProf generated HTML, and finding that DateTime passes the epoch time to gmtime to get the date components and then calls the regular constructor with those. The extra cost isn’t really in the gmtime calls, but a good part of it is in the use of Params::Validate in all of the public methods to check arguments and the epoch time path takes the code path through an additional public method.

Benchmark told me that I shouldn’t make a change and NYTProf allowed me to understand why that that result was correct and there wasn’t something wrong with my Benchmark use.

Dealing with perl binary compatibility issues as a user of XS modules.

chromatic discusses binary compatibility with the perl executable. I’m someone who only uses XS modules and hopes to never have to write one. Yet, unlike what seems to be the prevailing fear, I don’t worry about binary compatibility. I don’t worry about upgrades breaking things. I don’t worry that I’m going to have mismatches in executables/libraries.

Why? Continuous integration.

Don’t just grab modules from the CPAN and install them somewhere never to be upgraded until you’re forced to at a huge amount of time and expense. Regularly rebuild everything you use, preferably daily or hourly.

  1. Install a perl binary and then lock it down. Don’t use site dirs. Have the perl core be inviolate.
  2. Instead, install to some other directory and point PERL5LIB at it. Both CPAN and CPANPLUS allow you to configure what you want to be passed to the configuration program. Pass a value for INSTALL_BASE to Makefile.PL and install_base to Build.PL that points to your directory.
  3. Set up a build server to use CPAN or CPANPLUS to “install” and build everything. Since your build machine will be rebuilding everything regularly, when you upgrade your perl binary or an XS module, any problems will be quickly identified. Everything being rebuilt in one go means that all binary code will match up and you don’t have to worry about them getting out of sync. Because installing a distribution causes all of that distribution’s tests to be run, you’ll know that that distribution works in your environment.

(If you’re using taint mode and thus cannot use PERL5LIB, have your build machine build a custom perl executable along with everything else.)

Details for doing this belong in another post. But the point is, if you regularly rebuild all of the CPAN code you use, you don’t need to worry about binary issues.

DarkPAN setup

Gabor is looking at setting up corporate CPANs. While it doesn’t cover the web interfaces that he wants, brian d foy has been working on MyCPAN::Indexer and MyCPAN::App::DPAN.

The former has been in progress for a while as part of brian’s BackPAN indexing work. The latter is a more recent effort at creating a tool for managing CPAN-like repositories. It can copy archives into a repository and update the standard index files. It’s being used on a daily basis on both Windows and Linux to maintain a single DarkPAN at present, and, once some more work on automated building of Perl distributions under TeamCity is done, will be used for maintaining a lot more private repositories.

The way that a repository is updated is to have a copy of it checked out of Subversion, put the distribution archive in place, run dpan on it to update the indexes, and check in the result. CPAN.pm can be pointed at the Subversion HTTP URL and use it just as if it were one of the public CPAN mirrors. (Though of course, the DarkPAN repository has a restricted set of distributions in it, not even a mini-CPAN.)