Yet Another Python Multimethod Decorator, Part 2: Respecting Scope

Last time, I was complaining how Guido’s five-minute multimethod implementation isn’t production-ready (without explaining why not). A bit churlish, you may say – it clearly wasn’t intended to be that. It’s just a starting-point showing off some fancy syntactic sugar. Fair point.

So, what’s wrong with it, and how can we fix it? The first point is a real doozy, and one which Guido doesn’t even reference in his post.

It doesn’t obey sensible scoping rules.

Let’s say you have the following program:

def bar(x, y):
  @multimethod(string, string)
  def foo(x, y): return "inner string and string"

  return foo(x, y)

@multimethod(int, int)
def foo(x, y): return "int and int"

@multimethod(string, string)
def foo(x, y): return "string and string"

print(foo(2, 4))
print(foo("good", "news"))

print()

print(bar("hello", "world"))

print()

print(foo(42, 3))
print(foo("oh", "dear"))

You’d expect the following output:

int and int
string and string

inner string and string

int and int
string and string

What you actually get is:

int and int
string and string

inner string and string

int and int
inner string and string

That’s because any time you create a multimethod target, it goes into a global registry, and overwrites whatever was there before. Whenever you invoke a multimethod target, it looks up the most recently defined implementation for the arguments, and invokes that. The local function foo in bar isn’t local anymore. This makes it very, very hard to reason about code using multimethods – you’ve got a feature which will implicitly monkey-patch code you didn’t know you were monkey-patching.

How can we get around it? The simplest way is not to use decorators: create an instance of a dispatcher and add all the invocations to it. Basically, remove the registry and the syntactic sugar that the decorator provides. That’s well and good, but it’s a bit clunky. Assuming the syntactic sugar is considered a good thing, is there a way to make it work and respect scope?

Respecting scope, in this case, means one of two things:
a) defining all multimethod targets for a given multimethod as a single syntactic construct, such that they create a single result which is then added to the environment, or:
b) determining whether a multimethod of that name already exists in the environment, and adding to it if it does, or creating a new multimethod dispatcher if it doesn’t.

I couldn’t work out how to do a). I could work out how to do b), and it’s a nasty, dirty, ugly hack. Specifically, reflect on the environment, look up its parent’s parent, search for a member with the given name, and if it’s a dispatcher, modify that and return it from the decorator, otherwise create a new dispatcher, add the target, and return that from the decorator. Reflecting over environments is not the sort of thing anyone should ever be doing, normally: implementing language features is one of those exceptions.

That said, if there’s a cleaner way, I’d love to hear it. I still feel a bit dirty.

OK, so that’s one issue sorted. Next time: why can’t I have multimethod methods?

Advertisements

One thought on “Yet Another Python Multimethod Decorator, Part 2: Respecting Scope

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s