More about QL

The QL programming language was described at the high level in Introduction to QL; you should review that first. You can do an awful lot with just the capabilities described there: the vast majority of QL expressions are just a series of stages, transforming the data a little bit before displaying it. But there's a lot more to it, if you need access to more power.
Note that this page is aimed mainly at experienced programmers. Everyone is welcome to play with this stuff, but there are some assumptions here about how much you know.

Current Functionality

This section talks about QL as it stands today. Down at the bottom, we'll talk briefly about features that will probably be added down the road.

Concepts

To begin with, a recap: QL is a functional programming language. It is aimed strictly at starting with a context (typically the Thing that you're looking at), and then transforming that context by passing it through a series of Stages. We say that each Stage receives an incoming context from the left, and produces an outgoing context to the right.
QL is, for the moment, used entirely for display in Querki -- you use QL Expressions to describe what data to show on the page. It will eventually be enhanced to let you change information in the Space, but for now that doesn't exist.

Expressions, Phrases and Stages

Here's the detail on what goes inside of [[ ... ]]:
  • Everything inside of the double-square-brackets is an Expression.
  • An Expression consists of one or more Phrases, which are separated by newlines or semicolons.
  • A Phrase consists of one or more Stages, which are separated by arrows (->). You may have newlines within a Phrase, so long as there is an arrow at the beginning or end of the line. If a line ends without an arrow, and the next line doesn't start with one, then it is beginning a new Phrase.
The result of an Expression is the value of all of its Phrases, concatenated together. (In principle: there is an exception for value bindings, as discussed below. Also, there are currently a number of cases that only produce the result of the last Phrase; this is a bug, and should be fixed eventually.)
In most cases, you only need one Phrase. Multi-Phrase Expressions mainly become useful when you are using named value bindings or local function definitions, as discussed below.

Kinds of Stages

There are several different sorts of Stages:
  • Function calls, most often to the various system functions, such as ... -> _sort -> ...
  • Bindings, such as ... -> +$myThingy
  • Binding references, such as $myThingy -> ...
  • Local Function definitions, such as _def $myFunc($thing, $prop) = $thing -> $prop
  • Calls to local functions, such as $myFunc(Book, Title)
  • QText Stages, such as ""#### Title: ____""
  • Properties, which produce the value of that Property on the received Thing, such as ... -> Name -> ...
  • A List Literal, such as <Thing 1, Thing 2, Thing 3>
  • Property Setter, such as My Property(42)
  • A Sub-Expression
All of these are described in more detail in either the Introduction to QL or below.

Sub-Expressions

Occasionally, you want to use a full Expression as a Stage. This isn't often needed, but you can do it simply by enclosing it in parentheses. This is mainly useful in Local Function definitions.

_apply, and Creating your own Global Functions

One of the cute bits of magic in QL is the way that the same syntax can mean different things. For example, if you say [[My Thing]], that produces the named Thing, but if you say [[My Property]], it produces the value of that Property on the current Thing.
The way this works is through the _apply function. This is a crucial function, which everything in Querki makes use of. It is exactly, "what should I do when I am named in a QL expression?".
For ordinary Things (Models and Instances), _apply just produces the named Thing. For Properties, _apply takes the received Thing, looks up the named Property in it, and produces the value that was found. Querki's system-level functions all work by having specialized definitions of _apply that do particular things.
You can do this latter bit yourself. If you want to create a global Function -- a Function that can be called from anywhere in your Space -- just create it as a Thing, add the _apply Property to it, and fill that in. Any time you name that Thing in QL, it will execute the body of _apply. Keep in mind that the value of _apply is a Function, so its contents should be a QL Expression, without the [[ ]]. (The double-square brackets are how you embed a QL Expression inside of QText, and should only be used when you are in QText.)

Calling Functions with Parameters

As with any serious programming language, you can call many functions with parameters. For example, the _sort function takes one parameter, which says what to sort on:
Stories -> _sort(Headline) -> ...
With rare exceptions, parameters may be full Expressions, including multiple lines with separate Phrases if necessary. (If you get to that point, I recommend using indentation and parentheses to help make the code clearer.)
Parameters are evaluated by the Function, not at the callsite, which is extremely unusual. This is discussed in more detail below.

Named and Optional Parameters

Many of the system functions have named parameters, sometimes quite a number of them; sometimes, some of these parameters are optional. For example, the _QLButton function (which displays a button that, when pressed, runs some QL code and displays the result) has half a dozen parameters, two required and four optional.
When you get into something that complex, it's often helpful to specify your parameters by name. This is never required -- you can always just provide the parameters in the correct order -- but in complex cases it's often clearer to say something like:
Bookcase ->
  _QLButton(
    label = ""List the books"",
    target = ""bookdiv"",
    ql = _sort -> _bulleted
  )
...
<div id="bookdiv"></div>
Note that, when you are specifying parameters by name, you may give them in any order. (But the required parameters are still required.)

Creating Your Own Instance Functions

There is also a Function Type in Querki, which is similar to Large Text Type but is "just the stuff inside the square brackets". This is the middle ground between Global Functions (as discussed above) and Local Functions (discussed below). If you have some code that you need from several places in your Thing, you can create a Function Property, and put that code into it.
Note that these Functions aren't scoped like methods in a typical OO language, though -- as with Properties, the declaration is global across your Space. However, you can create something method-like by declaring a Function Property, and then giving it different definitions on different Models.
These Functions can consume parameters positionally. The Function can use 1-based positional bindings named $_1, $_2, etc, to get at the parameters used in the call. (Occasionally useful: the same is true of Text and Large Text Properties -- you can "call" these with parameters, and use those parameters in QL Expressions inside the text.) There isn't yet a way to define named parameters for an Instance Function, but that will be coming eventually.

Locally-bound Values

Pretty much every programming language has some concept of "variable", or at least "named values". Querki got by without them for a surprisingly long time (several years) -- you can do an awful lot just passing contexts from Stage to Stage -- but when you're trying to do something complex, they start to become necessary.
QL has no concept of a "variable" -- as a pure-functional language, immutability is central to it. (And you get some important benefits from that.) But there is a notion of "binding" a value to a name for the duration of the current scope. Anywhere after the name is bound, you can use it to get that value back.
Bound names always begin with $ -- that's the signal to the system that this is the name of a local value, rather than the name of a Thing in the Space. After that, they may contain any of the characters that you can use in a Link Name: letters, digits, underscores, dashes and (unlike most programming languages) spaces.
Bindings persist for the rest of the QL Expression in which they are declared. This is important, because it is why the no-variables thing works: if you're in a situation that is running the Expression repeatedly over a number of values (such as the _foreach function), it gets bound again each time around. As soon as you get to the end of the Expression, the name is no longer bound, so it can be reused.
You bind a name with the +$ operator, and use it simply by naming it. So the "hello world" of binding examples is:
[[""Hello"" -> +$greeting
  ""world!"" -> +$greeting2
  ""[[$greeting]] [[$greeting2]]""]]
This displays "Hello world!", as you would expect.
When we discussed Phrases and Expressions earlier, we mentioned that the value of an Expression was the concatenation of all of the values of its Phrases. Bindings are an important exception to this rule: if a Phrase ends with a +$ binding, it produces nothing. So while the above example is trivial, it illustrates a common pattern: you have several Phrases that build up some values as bindings, and then the last one puts them together.

Query Parameters

Occasionally, you want to build a page whose display depends on its URL. My own Comic Book Space is a good example: its Show Titles page has a "start" parameter in the URL, and lists all of the titles starting from there alphabetically. So, to show the listing starting with "batman", I use the URL:
https://www.querki.net/u/jducoeur/comics/#!Show-Titles?start=%22%22batman%22%22
That looks a little odd, but makes more sense when you consider that "%22" is the URL encoding of " -- so the parameter is actually ""batman"".
You generally don't want to do that by hand, of course, so Querki has the _withParam function. This receives a URL or a Thing, a name parameter, and a value parameter; it adds the name/value pair, and produces a new URL. So to produce my "batman" URL, I would say something like:
Show Titles -> _withParam(""start"", ""batman"")
This produces the URL for the Show Titles page, with the "start" parameter added.
From inside the page, you can then use this query parameter as if it was a local binding. So in this case, the QL code can use $start, and it will produce ""batman"".

Local Functions

Binding a local value is all well and good, but in complex cases you sometimes want more. In particular, sometimes you find yourself repeating the same Phrase or Expression repeatedly, to get things to display the way you want. That's almost always bad. (A common motto of programming is DRY -- "Don't Repeat Yourself" -- because it tends to lead to bugs.) In order to help you avoid that, QL includes a concept of Local Functions.
A Local Function is a lot like a bound value -- it only exists for the rest of the Expression in which it is declared. The difference is that, whereas a bound value is just that -- a specific value -- the Local Function is an Expression that will be evaluated every time it is referenced.
Local Functions are probably the most advanced concept in QL, and they're one of the few cases that involves non-trivial syntax. A basic Local Function definition looks like this:
_def $My Function = ... a QL Phrase...
The keyword _def starts the function definition. (And yes, that's the first time you've encountered a keyword in QL.) Then comes the name of the Function -- as with bound names, this must start with $ and consists of the characters that are legal in Link Names. Then comes =, which currently must be surrounded by spaces. Finally, you have the guts of the Function, which is a Phrase. Note that it isn't a full Expression -- that's because you usually only want a single Phrase. If you need a full Expression for your Function, surround the body with parentheses.
Here's an example of a simple Function in action, from the QL unit tests. This is operating on a little CD-management Space, consisting of Albums, Artists, and Genre tags for the Artists.
[[Album._instances -> _groupBy(Artists) -> +$byArtist
  _def $Show First = _first -> Link Name
  $byArtist -> _groupGet(Blackmores Night) -> $Show First
]]
The first line fetches all of the Albums, groups them by Artist, and binds those groupings to $byArtist. Then there is the $Show First function, which expects to receive a list of Albums by a particular Artist, and shows the Link Name of that Album. Finally, the last line fetches the Albums from the group Blackmore's Night, and uses $Show First to display the name of the first one. (Which happens to be "Fires at Midnight", which I quite like.)
As you can see, the Function receives the context that it is called with -- the list of Albums from _groupGet is passed into $Show First, and thus gets passed to _first -> Link Name. That's sufficient for a number of simple cases. (In particular, it often works nicely if the body of your Function is a Text literal.) But what if you want something more interesting?

Function Parameters

Local Functions can have parameters -- indeed, they have named parameters, so they are in this way currently more powerful than Instance Functions. The declaration of the parameters looks much like most programming languages, but the parameter names must, as usual for local names, start with $. So the full syntax for a Function definition is actually:
_def $My Function($param1, $param2, ...) = ... a QL Phrase...
You can, of course, use those parameters anywhere inside the Function's body.
So let's look at a more interesting version of the previous example:
[[Album._instances -> _groupBy(Artists) -> +$byArtist
  _def $From First Album($artist, $field) = $byArtist -> _groupGet($artist) -> _first -> $field
  $From First Album(Blackmores Night, Link Name)
  $From First Album(Eurythmics, Artists -> Genres)
]]
It's similar, but it illustrates several things. First, note that the body of $From First Album uses the $byArtist binding. Functions have access to any values that have been bound in-scope before them.
The body of $From First Album, thus, uses _groupGet to fetch the specified Artist's Albums from $byArtist, grabs the first of those Albums, and produces the specified $field from that Album.
But here, things get more interesting. Note the last line. This doesn't actually pass a Property into $field, it passes an Expression. This call will get all the Albums by the Eurythmics, take the first Album, go from that to the Artists (which is back to the Eurythmics), and fetches that Artist's Genres. (In fact, it displays "Rock" in the unit test.)
At this point, it may be occurring to you that this looks kind of weird, and you're not wrong -- it illustrates the fact that Querki processes parameters differently from almost every other language out there. In most languages, parameters are processed before you call the function. If this were a language like Java, the call $From First Album(Eurythmics, Artists -> Genres) would try to compute Albums -> Genres at the call site, and it would be essentially meaningless.
What's going on here is that Querki passes parameters literally, and evaluates them when they are referenced. The result is that QL behaves a lot like a macro language, rather than like an ordinary programming language. You can think of function calls as being a lot like literally substituting the body of the Function into the call site. So the last line is actually processed as essentially:
$byArtist -> _groupGet(Eurythmics) -> _first -> Artists -> Genres
This is very weird, but very powerful, and turns out to be the essential for keeping QL code clear and concise. It actually wasn't part of the original design, but we found, as we went, that this was necessary in order to make things work well. In particular, this is why you can simply name a Property as a parameter, and then use it inside the Function's body, which is often very useful.
Note that this is true of Global and Instance Functions as well -- the positional parameters ($_1, $_2, etc) are evaluated when they are referenced, just like parameters in Local Functions. And it's also true of nearly all of the system functions. It is specifically why you can say something like _sort(Name) -- it applies the Name Property to each received Thing, and sorts on the result. (Note the implication: you can pass any arbitrary Expression to _sort, and it will apply that Expression to each received Thing, and sort on the result. This passing-Expressions thing turns out to be really powerful and useful.)
You can defeat this, if you need to. If you begin a parameter with ~!, that means "evaluate this parameter here, with the current context, and pass the result to the function call". Conceptually, that seems like it should be very necessary, but in practice I have yet to find a case that actually needs it.

Built-in bindings

Finally, there are a couple of predefined bindings that are available when they make sense.
$_context is always bound to the context that was originally passed in to the beginning of this Expression.
$_defining is bound to the defining context of this call. The defining context is a bit of special syntax, which you can think of as "the left of the dot" in a call. It is sometimes used to specify a Property that this call will operate upon. In general, though, it's now usually easier to use a conventional parameter.
We will probably add $_this somewhere down the road. That will produce the Thing that this Expression was fetched from, and is likely to be useful when a Model is trying to say something about the current Instance. Yell if you find that you need it. (In principle, this seems useful; in practice, we haven't come up with many actual use cases, so it's still up in the air.)

List Literals

Occasionally, you may find that you want to specify a List right there in your QL expression, instead of just having it come from a Property. For example, the Location Type consists of three Properties. By default, we display all of them, comma-separated, like this:
... a location... -> <_Location Street Address, _Location Town, _Location State> -> _join("", "")
That is, you define a List Literal by surrounding it with < and >. The elements are comma-separated, and each element is a full Expression. Each element receives the context that is received by the List as a whole. The values produced by the elements are concatenated together; if an element produces an empty value, it is simply omitted. (So the above expression is quite nice -- if, say, the _Location Street Address is empty, it will just be left out, and there won't be a hanging comma.)
Take care to make sure all of the elements in a List Literal produce the same Type. While it's technically legal to do otherwise, the results are likely to be confusing.

Processing Stages

In general, when a Querki stage receives a List, it processes that element-by-element; this is particularly true of Text stages. So for example, if you say:
[[<1, 2, 3> -> ""foo [[_commas]] bar""]]
the result will be foo 1 bar foo 2 bar foo 3 bar. Where did the commas go? The reason no commas show up is that it first processes the value 1, on its own -- since that has just a single element, it doesn't insert any commas. This is most often what you want QL to do.
But sometimes, you want the stage to receive the entire List, and handle it together. You handle that by putting an asterisk * in front of the stage. For example, if you tweak the above to:
[[<1, 2, 3> -> *""foo [[_commas]] bar""]]
the result will be foo 1, 2, 3 bar -- the Text stage works on the entire List as a whole, and therefore inserts the commas.
Similarly, if you construct an _if expression like this:
[[<1, 2, 3, 4, 5> -> _if(_isEmpty, ""empty"", ""full"")]]
you will get full full full full full -- it will run the _if once on each element. But if you say:
[[<1, 2, 3, 4, 5> -> *_if(_isEmpty, ""empty"", *""full"")]]
it will produce full -- the _if will run once on the entire List, as will the Text stage ""full"".

Property Setters

Occasionally, you need to be able to declare the concept of a Property with an associated Value. You do this with the Property Setter syntax.
A Property Setter is a stage that contains the Property's name, following by a parameter list, which contains the value that we will set the Property to. For example, Summary(""This does something"") produces a Property Value of exactly that.
Property Setters aren't often useful by themselves; they simply render as the given value, in whatever form this Property normally renders.
These are mainly used as parameters in functions that need to talk about Property Values, such as _createThing or _changeProperties. _createThing sets the Performed By property using a Property Setter, like this:
_createThing(model = Album, Performed By($_context))

Operators

QL currently has a few operators -- shorthand syntax for common logical functions. They are:
  • A & B -- which means exactly _and(A, B)
  • A | B -- which means exactly _or(A, B)
  • !A -- which means exactly _not(A)
These are simply "syntax sugar", to make some logical expressions a bit easier to write and read, and are turned into those functions internally.
Note that you must have spaces around & and |, and ! binds more tightly than either of the binary operators. The binary operators apply to exactly two parameters -- you must use parentheses if you want to combine more than two. (Which means that there are no precedence rules, since they aren't needed.)

Comments

QL Expressions can have comments. This follows the current trend in programming languages: anything following // on a line is ignored as a comment. (We will probably add /* ... */ style block comments a bit down the line.)
Note that, while you can't yet use comments in QText, you can work around this by using empty QL Expressions, that contain nothing but a comment. For example, you can do something like this:
[[// *****************************
  // Here is my comment block.  This lets me put big, interesting comments
  // into arbitrary Text, by using an empty QL Expression.
  // *****************************]]

Documenting QL Expressions

If you want to embed QL in a document, you have to be a bit careful -- if you just put the QL in the page, it will evaluate instead of displaying. So there is a special QL operator, "[[```" ... "```]]" for this purpose.
Normally, that triple-tick syntax in QText renders something as fixed-width text. But if you use it inside of a QL block, it will prevent anything inside there from being evaluated, and instead output it as the same fixed-width text. It is usually the easiest way to embed a block of QL on a page or in a comment.

Future Plans

  • Maps and comprehensions?
  • Ability to write to the Space, not just read it
  • Writing will still be pure-functional: produce a change description, which will be applied after evaluation
  • Define an input field and how to save it, in one expression -- show possible syntax
  • Changing all Functions to be strongly typed
  • Strong typing will include some version of Generics
  • Type inference for Functions
  • Duck-typing and Typeclasses
  • Pre-compilation to speed up Function evaluation
  • Much of this is a good ways down the line -- 1-2 years likely
<- Prev: Users, Identities and Persons | Up: Advanced Topics | Next: Security in Querki ->