Ruminations on a system of units in measure in Scala, part 5: Infix and suffix notation vs punctuation.

this is the fifth part of an ongoing series of posts discussing my implementation of units of measure in scala and the various thoughts about programming language design it prompted this particular post follows on from the fourth post on syntax which in turn really requires the first post to make sense at the end of the last post i ended up with more questions than answers and the first of those questions was on how nice it is sometimes to not have excess punctuation in order to create a more english like syntax and why scalas support for infix and suffix notation might not always be a good thing this is the point i will address today

At this point I can be reasonably confident about two things: firstly, you read and understood the above stream of text. Secondly, it was painful to do so. Let’s try again:

This is the fifth part of an ongoing series of posts discussing my implementation of units of measure in Scala, and the various thoughts about programming language design it prompted. This particular post follows on from the fourth post on syntax, which, in turn, really requires the first post to make sense. At the end of the last post I ended up with more questions than answers, and the first of those questions was on how nice it is sometimes to not have excess punctuation (in order to create a more english like syntax) and why Scala’s support for infix and suffix notation might not be a good thing. This is the point I will address today.

The punctuation immediately makes the text more readable, breaking it down into manageable sub-units and emphasising what words belong with what other words. Wait, aren’t I supposed to be arguing that punctuation makes text less readable – wasn’t I arguing that println(area in acres) is superior to println(area.in(acres))?

Well, that depends on the audience. println(area in acres) is more pleasantly readable to me than println(area.in(acres)), but the latter is more readable to what really matters here: the parser. To be fair, in this case it doesn’t matter – but if we can use this notation in simple examples, then we’ll probably want to use it in some more complex examples as well – and we very quickly run into a whole host of issues.

Part of the root of this is in how programming languages are understood by a computer. English is a particularly horrendous language to understand, full of irregularities and overloading and context – so much context. Programming languages are simple, regular, and rigid.

In English, if you have a basic sentence structure of:

x flies like a y

You can substitute different sets of nouns, and end up with a completely different parsing of the sentence. Contrast the meanings of the following sentences:

Time flies like an arrow.
Fruit flies like a banana.

The first sentence is talking about the manner in which something flies. That’s also a valid parsing of the second sentence – but it’s not what it will commonly be taken to mean. The word “flies” can be either a verb (to fly) or part of a noun phrase (fruit flies), and what determines that is context: not even the word types can determine that alone.

Programming languages have very tightly defined grammars. They’re unambiguous (which rules out phrases like “Fruit flies like a banana”, which has two valid, meaningful parsings in English), but furthermore they’re (generally) context-free. What this boils down to is that the language grammar determines what “part of speech” a given symbol is based entirely on syntax, and when it’s parsing, it’ll see something along the lines of word word word word word and try to work out what the parse tree is.

Of course, if instead we have the sentences:

Time.flies(like(an_arrow));
Fruit_flies.like(a_banana);

Then, that’s something the parser can address on a structural basis. Time’s a value, flies is a method, like is a function, an_arrow is a value. Fruit_flies is a value, like is a method, a_banana is a value. It’s obvious even if you remove the meaningful names:

a.b(c(d));
e.f(g);

That’s how the parser sees them, in fact. This parsing, deciding what part of speech each identifier represents, is done prior to any semantic analysis (at which point the compiler starts attaching types and implementations to identifiers). As far as the parser is concerned, Time and a are equally meaningful. Is this a fundamental requirement of compiling programming languages – could we have syntax be semantically aware? That’s a question which, for now at least, is well beyond my pay grade.

So, what does the Scala parser do when it sees something like area in acres? It doesn’t have any fixed symbols from the grammar to latch on to, either in terms of punctuation or keywords (which, ultimately, are glamorous punctuation as far as the parser is concerned). All it sees is identifier identifier identifier. Well, it has specific rules to deal with that case. Effectively, identifier identifier identifier is converted into (or treated the same as) identifier.identifier(identifier). Similarly, identifier identifier is converted into identifier.identifier().

Is there anything wrong with this approach? Well, first of all, we’ve got special cases which hold for common operations but not less common ones. We can omit puncutation if we have an arity-0 method. We can omit punctuation if we have an arity-1 method. What about arity-2?

object Multiplier { def multiply(x: Int, y: Int) = {x * y} }
object Main { def main(args: Array[String]) { println(Multiplier multiply 3 4); } }

That’s a syntax error: the grammar doesn’t handle that at all. It’s fine, however, if instead you ask for:

object Main { def main(args: Array[String]) { println(Multiplier multiply (3, 4)); } }

So we can omit punctuation when Scala doesn’t require it, but it’ll handily complain when it needs punctuation which isn’t there? Well, yes and no.

Buffalo buffalo buffalo buffalo buffalo buffalo buffalo

The above is a meaningful English sentence (as is, in fact, any number of repetitions of the word buffalo greater than one). How does Scala handle such a thing?

object Buffalo {
  def Buffalo() = { println("Arity zero method called"); this }
  def Buffalo(a: Any) = { println("Arity one method called"); this }
  def Buffalo(a: Any, b: Any) = { println("Arity two method called"); this }
}

object Main {
  def main(args: Array[String]) {
    Buffalo Buffalo Buffalo Buffalo Buffalo Buffalo Buffalo
  }
}

How does Scala parse this? In principle, any of the following could make sense:

Buffalo.Buffalo(Buffalo).Buffalo(Buffalo).Buffalo(Buffalo)
Buffalo.Buffalo().Buffalo().Buffalo().Buffalo().Buffalo().Buffalo()
Buffalo.Buffalo(Buffalo.Buffalo(Buffalo, Buffalo), Buffalo))
Buffalo.Buffalo((Buffalo.Buffalo((Buffalo.Buffalo(Buffalo)))))

Obviously, it has to be handled in an unambiguous manner: only one parsing can be valid. Run this, and the output is “Arity one method called” times 3. It’s being parsed into Buffalo.Buffalo(Buffalo).Buffalo(Buffalo).Buffalo(Buffalo) – three invocations of the arity 1 method being called. Infix notation will never use the arity-2 method, and it will attempt to construct calls to the arity-1 method until there aren’t enough identifiers to do so.

When we run this:

object Main {
  def main(args: Array[String]) {
    Buffalo Buffalo Buffalo 
    Buffalo Buffalo Buffalo
  }
}

The output is “Arity one method called” twice: we’ve got two lines, each of which is Buffalo.Buffalo(Buffalo). Cool. With this:

object Main {
  def main(args: Array[String]) {
    Buffalo Buffalo 
    Buffalo Buffalo 
    Buffalo Buffalo
  }
}

We’d expect three invocations of “Arity zero method called”, right? Well, we don’t get that. We get “Arity one method called”, “Arity one method called”, “Arity zero method called”. So, it’s parsing that as Buffalo.Buffalo(Buffalo).Buffalo(Buffalo).Buffalo().

How’s that happening? We’ve got three separate statements here, right? Well.

object Main {
  def main(args: Array[String]) {
    Buffalo Buffalo;
    Buffalo Buffalo;
    Buffalo Buffalo;
  }
}

That does give us three invocations of “Arity zero method called”, but without the semicolons it’s just pasting them all together as a single line. So it’s not inferring semicolons correctly in the case of suffix notation. but it is in the case of infix notation? That’s because infix notation is special, and the compiler will treat expressions as infix if at all possible – turning suffix expressions into infix expressions if it can, to the point where this breaks:

object Main {
  def main(args: Array[String]) {
    Buffalo Buffalo
    while(true) { Buffalo Buffalo Buffalo }
  }
}

It’s trying to parse it into Buffalo.Buffalo( while(true) { Buffalo Buffalo Buffalo } ). Silly me, that’s not what I meant at all.

There’s also one niggling issue about the syntax which are relevant to Minstrel but not Scala. In Scala, if you have an object with an arity-0 method, the following are equivalent:

def a = object method
def b = object method()
def c = object.method
def d = object.method()

Whereas in Minstrel (assuming type inference for a moment), the following are both valid, but different:

a is object.method[]
a is object.method

In Minstrel, the first would invoke object.method and return the result, and the second would return the method, uninvoked, as a function (bound over the object’s environment). When methods can have side effects, you want it to be clear that methods are being invoked.

Note that in Scala the style guide is quite clear that these notations should only be used in side-effect-free contexts. Well, that’s a whole can of worms (what about when classes are subclassed with new behaviour?) but all I’ll say for now is – if we’re implying that with syntax, wouldn’t it be nice if the compiler could assert those implications were actually true?

Interestingly, were this Haskell, the two would be equivalent: a zero-arity function is a value, but that’s a consequence of a world without side-effects. For that matter, in Haskell, you can just string terms together without punctuation – but when all functions are arity-1 and anything which implies otherwise is syntactic sugar, there’s no ambiguity to handle.

This is speculation on my part, but I suspect that the presence of these notations in Scala was as an attempt to make it possible to write functional code that looks like established functional languages (ie Haskell) whilst glossing over the syntactic differences of those languages that make doing so straightforward in Haskell and can-of-wormsy in Scala.

So. That’s a lot of rambling around the subject. Conclusions?

We’ve got a whole host of problems here, opened up by trying to be more elegant than we can actually support. We’ve got a special case syntax which doesn’t scale, which can lead to unintuitive breakages of other special-case syntax deemed less important, and at the end of the day – what do we gain for it? Certain operations look nicer. Being able to write area in acres instead of area.in[acres] would be lovely, true, but it’s not worth the consistency costs that Scala imposes.

It’s a problem of the same category as the problems with numeric representations: coming up with an approach which is fine in simple cases but breaks down at some point in real usage is a poor foundation for a language. Consistency and robustness are not a negotiable goals.

Maybe there’s a way of doing it which scales better, but in the meantime, maybe a few dots and brackets aren’t all that horrible after all.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s