Announcing whisker – easier way to do templates in V

TL;DR

I need a capable template engine in V. There aren’t many good options available, so I’m making one myself. I’ve drawn inspiration from Mustache but I don’t like some aspects of it. So I’m doing my own version and I’m calling it whisker. It is available on GitHub: https://github.com/hungrybluedev/whisker

Introduction

I enjoy thinking about problems, rolling up my sleeves and writing code to get stuff done. One such problem I’m facing is finding a reliable, developer-friendly way of generating HTML (or other presentation formats) from application data. Specifically, I’m talking about templates and template engines. Another requirement for me is that I want to do it in V.

Options Available

In Python, I’d use Jinja. Java has Apache Velocity, JS has Pug and Mustache, to name a few. But I want to keep all of my code in V. These are the limited options I have available:

V Web Templates: V has the vweb module in its standard library.
Regular string interpolation: V has a very convenient string interpolation that has a lot of handy features.

V Web templates are reminiscent of Jinja and have a lot of convenient shorthands. However, the only way to pass data to these templates is to declare the variables in the function’s scope that calls $vweb.html(). I believe there should be a more formal way to pass data to these templates instead of wrapping them with individual functions. There is a one-to-one correspondence of the function you want to call to use a template, unless there are two functions that provide the same set of parameters to the same template file. This makes it support a many-to-one mapping.

From vweb’s documentation, we have the following way of using templates:

['/']
pub fn (mut app App) page_home() vweb.Result {
  // all this constants can be accessed by src/templates/page/home.html file.
  page_title := 'V is the new V'
  v_url := 'https://github.com/vlang/v'

  list_of_object := [
    Object{
      title: 'One good title'
      description: 'this is the first'
    },
    Object{
      title: 'Other good title'
      description: 'more one'
    },
  ]
  // $vweb.html() in `<folder>_<name> vweb.Result ()` like this
  // render the `<name>.html` in folder `./templates/<folder>`
  return $vweb.html()
}

The actual templates are as follows:

src/templates/page/home.html

<html>
  <header>
    <title>${page_title}</title>
    @css 'src/templates/page/home.css'
  </header>
  <body>
    <h1 class="title">Hello, Vs.</h1>
    @for var in list_of_object
    <div>
      <a href="${v_url}">${var.title}</a>
      <span>${var.description}</span>
    </div>
    @end
    <div>@include 'component.html'</div>
  </body>
</html>

src/templates/page/component.html

<div>This is a component</div>

src/templates/page/home.css

h1.title {
  font-family: Arial, Helvetica, sans-serif;
  color: #3b7bbf;
}

What’s notable here is the use of @ as a prefix for directives, the ability to import other templates, and the implicit transfer of scope from the function page_home to the main template. The regular V string interpolation is used as well.

I’m not really comfortable with this way of going about it, though. I want a more well-defined way of passing data to the template. Using V to type-check the parameters is actually smart, but this limits the scope for this template system’s adoption. Having a separate type checker, or locking down the type of data available to templates, is a better way to go about it IMO.

V’s regular string interpolation is great for logging and small string generation. I’d argue it is indispensable. But for larger ‘templates’, I’d like to use external files and be able to import them in V in some manner.

This means I’ll have to make something myself.

Inspiration: Mustache

I asked the folks in the official V Discord Server what they would like as a templating language. Delyan (@spytheman) and a few other folks suggesting looking at Mustache. So I started investigating.

After spending some time looking through the specification and some examples, I was intrigued. It apparently uses logic-less templates. Note that the engine is not logic-less of course. Instead, the templates do not have programmatic if-else statements, for-loops, etc. Branching exists, but as declarative, skippable sections.

Here is an example from the documentation:

<h1>{{header}}</h1>

{{#items}}
  {{#first}}
    <li><strong>{{name}}</strong></li>
  {{/first}}
  {{#link}}
    <li><a href="{{url}}">{{name}}</a></li>
  {{/link}}
{{/items}}

{{#empty}}
  <p>The list is empty.</p>
{{/empty}}

And we pass this JSON dataset to the template:

{
  "header": "Colors",
  "items": [
      {"name": "red", "first": true, "url": "#Red"},
      {"name": "green", "link": true, "url": "#Green"},
      {"name": "blue", "link": true, "url": "#Blue"}
  ],
  "empty": false
}

The output produced is:

<h1>Colors</h1>
<li><strong>red</strong></li>
<li><a href="#Green">green</a></li>
<li><a href="#Blue">blue</a></li>

If we set empty to true, then the output generated is:

<h1>Colors</h1>
<li><strong>red</strong></li>
<li><a href="#Green">green</a></li>
<li><a href="#Blue">blue</a></li>
<p>The list is empty.</p>

This is very interesting because of the following reasons:

The content to be interpolated is located inside the double curly braces ({{...}}). They look like ASCII art for mustaches, which I believe was the motivation for naming this template system like so.
We’re not limited to using these delimiters, though. They can be changed to something more convenient. This is useful for cases where we need to generate LaTeX, or C, or other languages with heavy use of curly braces.
Looping and branching are done using different sections. List sections are expanded for every item in a referenced list, map sections are expanded for every key. Regular and inverted sections test the truth value of a key and can be skipped.

Contrast this approach with Jinja. Definitely needs some getting used to, but I like the simplicity of it. In theory, the template processor will have an easier time with a more restrictive (but equally expressive) syntax.

Problems

Alright, Mustache looks good. Let’s implement the spec. That’s when we open the can of worms. My gripes are with the amount of wiggle room in the spec.

The querying using the labels inside the tags is quite peculiar. The engine will first look for a matching key in the closest context. If it can’t find anything, it will keep widening the context until the tag label is not present at all. IMO it is good for writing simpler templates. However, in case of absence, the engine returns an empty string. I disagree with this approach and think that we should handle a missing value error instead.
The entirety of JSON’s available data types is expected to be supported. I think that’s too much to support. Strings are absolutely important, so are lists and maps. Integers and other numbers make little sense because the program that calls the template processor can format them in the style preferred before sending the data over for interpolation. For branching though and test for truthy values, why not use Boolean values directly?
Other bad or unspecified behaviour. The additions feel like they were added after the fact without careful deliberation. For example, take lambdas: Albeit they’re a syntax extension, they fully depend on the language using their language’s Mustache engine. I’d like a language-agnostic approach to prevent lock-ins and sunk-cost fallacies.

Introducing: whisker

Despite the shortcomings, Mustache provides a very stable foundation that we can build on. We can leverage the test cases included in the official spec and use them to prototype our version quickly.

My version is called whisker because it’s close enough to the name of its inspiration and I think it sounds good personally. The source code is now available on GitHub: https://github.com/hungrybluedev/whisker

I’ll be writing about the various problems I’ve faced during development and how I eventually solved them. So keep watching this space and stay tuned for more updates!

Update: I’ve published a new blog post on whisker’s tokeniser.