Lexical and Anonymous Classes

[leonerd]

Way back in ancient perl, all named entities (variables and functions) lived in packages; their names were globally accessible, they exist once. Lexical variables were added later, followed by lexical subroutines. I believe the time has arrived to allow for lexical packages (especially packages that are used as object classes).

For example, consider if we could use my class ... to create a package that has lexical visibility. These might be handy for small structure-like classes, perhaps to return neater values to callers:

my class User {
  field $name :param :reader;
  field $uid  :param :reader;
}

sub list_users {
  return map { User->new( name => ..., uid => ... ) } @users;
}

This would basically act the same as if you'd used class Some::Local::Package::Name::_Here, as is often the convention now (especially with an underscore in the final component), but provides the same benefits of true lexical-only visibility, in the way that my variables and my sub functions already do.

In addition, it would be useful to permit anonymous packages; again by analogy to subroutines. These would be classes that literally don't have a name, and likely just exist in order to invoke the new constructor on them. For example, this is often handy in unit tests, when you need to construct an instance of some small class just as a one-off, to test something specific.

class Greeter {
  method greet { return "Hello, " . $self->name }
}

my $obj = (class :isa(Greeter) {
    method name { "unit test" }
})->new;
is( $obj->greet, "Hello, unit test" );`

Finally, it would be useful to note that these constructions should be able to lexically capture outside variables that appear in their scope. This is analogous to the way that lexical and anonymous subroutines can do so, except that here for an entire class, the outside variables can be captured by all of the methods and other expressions within the body of the class.

my @balls;

foreach my $colour (qw( red green blue yellow )) {
  push @balls, (class {
    method describe { return "This is a $colour ball"; }
  })->new;
}

say $_->describe for @balls;

How would ref work?

All of the above feels like it makes sense, up until you wonder what the ref() function would do on any of these instances. I don't have a good answer to that, but I will proceed to give one I think is reasonable and justified, and the best one I can currently think of.

Without any of these additions, in existing perl it is the case that the class that any object is blessed into is named on the symbol table, so the ref() function can return a plain string giving its name. This is useful for identification and printing purposes, as well as acting itself as a package name for method dispatch. You can at least attempt to invoke class methods on it, by code like (ref $obj)->new.

If we had these abilities to create anonymous or lexical classes, I don't think we could preserve all of the properties of ref(). In the case of lexical classes, while they do have a name, that name is only meaningfully unique within the scope that declared that class; it is not globally unique across the entire process. Thus, it wouldn't be a good idea for ref() to just return that short name, as it would lead to the possibility of two different instances of two different classes appearing to have the same ref() result, simply because those two classes had the same local name.

my @objects;
{
    my class Datum { field $x = 123; }
    push @objects, Datum->new;
}
{
    my class Datum { field $y = 456; }
    push @objects, Datum->new;
}

say ref($_) for @objects;

The situation is even worse with instances of truely anonymous classes, because those don't have any form of human-readable string name at all. It would at first appear that ref() would have to yield just some opaque string like __ANON__ for these; but if we are willing to permit a slight extension of semantics, I believe there is a better plan.

This extension would be to say that actually ref() should always return "the class" of the instance that it is invoked on, but simply that for backward compibility, if that class is package-named, it can be returned as a plain string. If invoked on a lexical or anonymous class, then ref() returns the class itself, or at least, some Perl-level representation of it - exactly the same value that the anonymous class { ... } syntax would have to yield anyway. Therefore, we wish to preserve the identity that, if $c contains something that can act like a class, then

ref( $c->new ) eq $c

should remain true. In order to specify how ref() works on instances of these other kinds of classes, it's simply sufficient to consider how those types of class expression can stringify.

Finally, it must be noted that there is a small compromise in this arrangement. It necessarily involves breaking the current assumption in Perl that ref() itself returns a plain string. I.e. currently

ref( ref( $x ) ) eq ""

is true for any possible value of $x, but would no longer be true if $x were an instance of a lexical or anonymous class. I feel in practice this is not a problem because I don't expect much code currently relies on that fact.

How would stringification work?

In the example of anonymous classes given above I glossed over considering what the Perl-visible value of the anonymous class expression should be. In order to fully explain how ref() would work with these instances, we must consider that class value itself.

I believe by analogy to things like anonymous array and hash references, there is clear precedent in suggesting the following behaviour:

my $anonclass = class { method m { return "hello" } };
my $obj = $anonyclass->new;

say ref($anonclass);  # CLASS (or maybe PACKAGE)
say ref(ref($obj);    # the same as above

say $anonclass;       # CLASS(0x12345678) for some memory address
say ref($obj);        # the same as above

say $obj;             # CLASS(0x12345678)=OBJECT(0x2345678A)

This preserves the invariant that ref() returns the same kind of thing that you could call the ->new method on to get another instance, as well as the invariant that the default stringification of an object reference prints its (stringified) class name, then an equals sign, then the reftype and memory address.

I am in two minds about whether these should stringify to CLASS or simply to PACKAGE. While all of the above discussion has been in terms of the use feature 'class' keywords, I don't necessarily feel that lexical or anonymous packages should be restricted to only being classes; I'm sure there are many valid use-cases for plain packages as well. I could be convinced in either direction.

A final thought on this section is to observe that if anonymous class syntax can yield these new "CLASS" type references to the classes, how would you create references the existing style of symbol-named packages? There are already other ideas floating about that would want to put stronger, more distinct behaviour onto references to the symbol table 1, so this idea may fit in as part of that.

Considerations for Object::Pad

As usual with object system ideas, I have attempted to start experimenting with some of these in the Object::Pad module. However, many parts of this idea go beyond what is possible for a CPAN module to do, without extending Perl core. In particular, to fully create this in a CPAN module would require adding to Perl:

• truely anonymous packages that have reliable reference counting
• a perl-visible SV type to represent a package or class reference; distinct from simply being a hash or glob reference
• stringification rules for these package-reference SV types
• a ref() function that can yield those true package or class reference values

I believe that a project to add this list of abilities into core perl would in fact be most of the work required to implement this full idea anyway, so it may not be worth considering adding just those to create it in Object::Pad, without also adding it to core.

I have begun attempting to implement lexical class syntax in Object::Pad currently, though at the moment these are simply shorthands for some globally-unique package name. They don't have the lexical-closure properties explained above. Already these limitied abilities have had some useful effect on module code, but I feel that a more complete implementation with all of these properties will prove far more useful again.

[book]

I vaguely remember some PSC discussions with @haarg about anonymous packages (or was it typeglobs?), and how they were necessary for some other, possibly desirable, features. Sadly, the exact details escape me.

Having more lexical things is great (I really like lexical subs, for example). On the other hand, I have this feeling that, as time passes, Perl is becoming less flexible (in that we used to be able to munge packages quite a lot). The new classes, for example, are (on purpose) very opaque to the Perl level.

Actually, are anonymous package actually anonymous typeglobs, or are they something else?

Getting back on topic: I seems to me that, to be maximally useful and flexible, anonymous packages will need some meta-protocol to be available: for example, how can one add a method to a package, when it's not accessible by name (e.g. to be able to use something akin to the *foo{THING} syntax)?

[leonerd]

I vaguely remember some PSC discussions with @haarg about anonymous packages (or was it typeglobs?), and how they were necessary for some other, possibly desirable, features. Sadly, the exact details escape me.

We already have anonymous globs - they're used for IO handles. It's only stashes that currently the implementation requires to have a name.

Having more lexical things is great (I really like lexical subs, for example). On the other hand, I have this feeling that, as time passes, Perl is becoming less flexible (in that we used to be able to munge packages quite a lot). The new classes, for example, are (on purpose) very opaque to the Perl level.

Actually, are anonymous package actually anonymous typeglobs, or are they something else?

Packages are stashes; specialised forms of hashes. ("Symbol Table hASH" => "STASH"). But currently the implementation of them requires them to have a name. Allowing a stash to not have a name is one of the big gating problems of trying to create this in a CPAN module, which requires core perl support.

Getting back on topic: I seems to me that, to be maximally useful and flexible, anonymous packages will need some meta-protocol to be available: for example, how can one add a method to a package, when it's not accessible by name (e.g. to be able to use something akin to the *foo{THING} syntax)?

That's what meta would be for. You can still obtain a metaclass object to refer to an anonymous package, and then just call methods on that.