MonadicT

I see dead objects!

Motivation

Writing HTML by hand is error-prone and just not fun. On the other hand, a simple templating language that integrates the power of a functional programming language with org-mode sounds like a fun weekend exercise!

Template language

Our template language is based on S-Expressions (Lisp syntax). We use S-expressions to represent HTML elements. An element has a name and optionally, a unique id, one or more class names, one or more attributes, and child elements. In S-expression form, this is represented as below.

(yatl-elem-name?my-id.cls1.cls2@atrr1\=val1@attr2\=val2
    (child-elem1...)
    ...)

A template defined using this form is passed to html5 macro which returns a string representation of equivalent HTML.

Our template language allows mixing standard Lisp code with templates. For most part, this mixing is achieved without any explicit notation. A form which looks like a function or macro invocation is evaluated as such. A list whose first element is neither a function nor a macro is treated as an element markup. When the name of an element clashes with a built-in or macro name, the normal processing can be overridden by quoting it. For example, (div x y) is interpreted as a Lisp expression, whereas, ('div (span)) is treated as an element. Also, (div?id) is also treated as an element definition due to the appended attributes.

Note that, we don't define any specific function or macros for various elements. We could certainly do that if there was some reason to eliminate duplication or error handling. This approach makes our template language infinitely extensible.

The template below generates a well-formed HTML document.

(yatl-html5
    (head)
    (body
        (div#content
            (concat "Hello, " World!"))))

That is all there is to our template language. It requires the user to know the element names and attributes. But, you can write helper functions in Lisp to address specific needs.

Template processor

Standard header

  ;;; yatl.el --- Yet another templating language

;; Copyright (C) 2017 Praki Prakash

;; Author: Praki Prakash <praki.prakash.gmail.com>
;; Created: 2 July 2017
;; Keywords: DSL, HTML, template
;; Homepage: http://MonadicT.github.io/

;; This file is not part of GNU Emacs.

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; Put a description of the package here

;;; Code:

Parsing key-value string

In our templating language, attributes are introduced with @ character, followed by the attribute name and its value. = separates the name and value.

This function tries to be robust to malformed strings.

(defun yatl-parse-kvp(s)
  (mapconcat
   (lambda (kv-str)
     (let* ((kv (split-string kv-str "="))
            (key (car kv))
            (val (cadr kv))
            (val (or val ""))
            (val (if (string-match "^\".*\"$" val)
                     val
                   (format "\"%s\"" val))))
       (concat key "=" val)))
   (split-string s "@" t)
   " "))

(assert (string-equal (yatl-parse-kvp "") ""))
(assert (string-equal (yatl-parse-kvp "@") ""))
(assert (string-equal (yatl-parse-kvp "@a") "a=\"\""))
(assert (string-equal (yatl-parse-kvp "@a=") "a=\"\""))
(assert (string-equal (yatl-parse-kvp "@a=b") "a=\"b\""))
(assert (string-equal (yatl-parse-kvp "@a=b@c=d") "a=\"b\" c=\"d\""))

Parse element name

Parses element name and returns the list of element name, id, class and attributes. Multiple class names are allowed but id must be unique. Id is introduced by ?, class name with . and attribute with @. The notation ? is chosen as a prompt for identity (who am i?) and @ for attribute.

(defun yatl-parse-elem-name(s)
  (let ((nm-id-cls (list '() '() '() '()))
        attrs idx buf escape string)
    (setq  idx 0)
    (mapcar
     (lambda (c)
       (cond
        (escape (progn
                  (setq escape nil)
                  (push c (nth idx nm-id-cls))))
        ((eq c ?\\)
         (setq escape t))
        ((eq c ?\")
         (setq string (not string)))
        (string
         (push c (nth idx nm-id-cls)))
        ((eq c ?.) (progn
                     (setq idx 2)
                     (push ?  (nth idx nm-id-cls))))
        ((eq c ??) (progn
                     (if (null (cadr nm-id-cls))
                         (setq idx 1)
                       (error "ID specified again!"))))
        ((eq c ?@) (progn
                     (setq idx 3)
                     (push c (nth idx nm-id-cls))))
        ((eq idx -1) (error "Expect one of \".,+,#'"))
        (t (push c (nth idx nm-id-cls)))))
     s)
    (list (concat (reverse (car nm-id-cls)))
          (concat (reverse (cadr nm-id-cls)))
          (concat (reverse (caddr nm-id-cls)))
          (yatl-parse-kvp (concat (reverse (nth 3 nm-id-cls)))))))

(assert (equal (yatl-parse-elem-name "img@foo=\"http://foo.bar/baaz\"")
               '("img" "" "" "foo=\"http://foo.bar/baaz\"")))

(assert (equal (yatl-parse-elem-name "div")
               '("div" "" "" "")))

(assert (equal (yatl-parse-elem-name "div")
               '("div" "" "" "")))
(assert (equal (yatl-parse-elem-name "div?id")
               '("div" "id" "" "")))
(assert (equal (yatl-parse-elem-name "div?id.c1.c2")
               '("div" "id" " c1 c2" "")))
(assert (equal (yatl-parse-elem-name "div?id.c1.c2@foo=bar@baz=qux")
               '("div" "id" " c1 c2" "foo=\"bar\" baz=\"qux\"")))

Return string representation

(defun as-string(o)
  (cond
   ((stringp o) o)
   ((numberp o) (format "%S" o))
   ((symbolp o) (symbol-name o))
   (t o)))

Convert a list to HTML element

This is the workhorse of our template processor. mk-elem inspects each form in the template and processes it as follows.

If a list passed to mk-elem is assumed to be an element specifications with the first element as the name, followed by attribute specifications and child elements. Element's name can include shorthand notation for id, class and attribute specifications. Also, an element's attributes can be specified separately from the element name by prefixing it with @. Child elements are processed recursively with mk-elem. The result is the string form of equivalent HTML.
If the child is an atom, its string representation is returned.

The following element specifications are all equivalent.

(div?id.cls@attr=val)
(div @id=id!class=cls!attr=val)
(div @id=id @class=cls @attr=val)

(defun yatl-mk-elem(o)
  (cond
   ((listp o)
    (multiple-value-bind (nm id cls attrs) (yatl-parse-elem-name (symbol-name (car o)))
      (let* ((children (cdr o))
             (children-s (mapconcat (lambda (x) (as-string x)) children " ")))
        (concat
         (format "<%s" nm)
         (unless (string-empty-p id) (format " id=\"%s\"" id))
         (unless (string-empty-p cls) (format " class=\"%s\"" cls))
         (unless (string-empty-p attrs) (format " %s" attrs))
         (if (not children)
             (format "/>\n")
           (format ">\n%s\n</%s>\n" children-s nm))))))
   ((symbolp o) (symbol-name o))
   ((stringp o) o)
   (t (format "%S" o))))


(assert (string-equal (yatl-mk-elem "a")
                      "a"))

(assert (string-equal (yatl-mk-elem '(div))
                      "<div/>\n"))

(assert (string-equal (yatl-mk-elem '(div?id))
                      "<div id=\"id\"/>\n"))

(assert (string-equal (yatl-mk-elem '(div?id.c1.c2))
                      "<div id=\"id\" class=\" c1 c2\"/>\n"))

(assert (string-equal (yatl-mk-elem '(div?id.c1@foo=bar@fit=bit))
                      "<div id=\"id\" class=\" c1\" foo=\"bar\" fit=\"bit\"/>\n"))

(assert (string-equal (yatl-mk-elem '(foo 1 2)) "<foo>\n1 2\n</foo>\n"))

(assert (string-equal (yatl-mk-elem '(img@src=\"http://example.com/images/fubar.png\"))
                      "<img src=\"http://example.com/images/fubar.png\"/>\n"))

Template processor

This is the implementation section of the template processor. This file can be processed using org-babel-tangle to produce a ~/.emacs.d/yatl.el file. The package is named yatl for "Yet Another Template Language" and (require 'yatl) to access it.

yatl-compile-fn

This is a helper function to examine each form and turn it into a form that can be passed to yatl-mk-elem. What we want is the ability to mix lisp code with our element markup code. We want this to be as seamless as possible. Consider the following example.

(html5 (head) (body (concat "Hello, " "World!")))

html5 will be defined as a macro later. We need to treat head, body as HTML elements and concat as a built-in function. For convenience, we would also want to be able to write our own functions and macros, if we so desire. To meet this requirement, we need a way to work with evaluated Lisp forms and modify it so that it can be evaluated to yield valid HTML content. A Lisp macro doesn't evaluate its arguments and is the perfect tool for this job. (Unfortunately, elisp has no support for reader macros which would made this task simpler.)

yatl-compile-fn looks for forms which might be function or macro invocations. It calls itself on the arguments and returns a potentially modified form. If the list is neither a function nor a macro invocation, then it is an element definition in our notation which is handled by invoking yatl-mk-elem.

The ability to mix Lisp code with our element description works, we need the ability to override the automatic recognition of function application. Consider the need to describe a 'div' element. div also happens to be Lisp function. When we want to use div as element, we override its meaning by writing it as 'div.

(defun yatl-compile-fn (form)
  (cond
   ((symbolp form) form)
   ((numberp form) form)
   ((stringp form) form)
   ((listp form)
    (cond
     ;; quoted form
     ((and (car form) (listp (car form)) (eq (caar form) 'quote))
      (yatl-mk-elem `(,(cadar form) ,@(mapcar #'yatl-compile-fn (cdr form)))))
     ;; lambda form
     ((and (eq (car form) 'lambda))
      (let* ((args-list (cadr form))
             (forms (cddr form))
             (new-forms (mapcar #'yatl-compile-fn forms)))
        `(lambda ,args-list ,@new-forms)))
     ;; special form
     ((and (special-form-p (car form)))
      ;; cond, let,let*, require special handling
      (cond
       ((eq (car form) 'let)
        (let* ((bindings (cadr form))
               (forms (cddr form))
               (new-bindings (mapcar
                              (lambda (binding)
                                (if (listp binding)
                                    `(,(car binding) ,(yatl-compile-fn (cadr binding)))
                                  binding))
                              bindings))
               (new-forms (mapcar #'yatl-compile-fn forms)))
          `(let ,new-bindings ,@new-forms)))
       (t `(,(car form) ,@(mapcar #'yatl-compile-fn (cdr form))))))
     ;; macro defn
     ((and (macrop (car form)))
      (eval `(,(car form) ,@(mapcar #'yatl-compile-fn (cdr form)))))
     ;; function
     ((and (symbolp (car form)) (fboundp (car form)))
      `(,(car form) ,@(mapcar #'yatl-compile-fn (cdr form))))
     ;; List of lists. Don't process?
     ((not (symbolp (car form)))
      form)
     (t `(yatl-mk-elem (list ',(car form) ,@(mapcar #'yatl-compile-fn (cdr form)))))))
   (t (throw 'Unhandled form))))

yatl-Compile macro

A macro which applies yatl-compile-fn to its arguments and concatenates the values returned. This must be a macro as the S-expressions can't be evaluated directly.

That is our ~~first~~ second attempt at designing this template language and its processor. Stay tuned for future posts where I will show its use with org-mode.

(defmacro yatl-compile(&rest forms)
  `(list
    ,@(mapcar
       #'yatl-compile-fn
       forms)))

(assert (equal (yatl-compile 1) '(1)))
(assert (equal (yatl-compile (+ 1 2)) '(3)))
(assert (equal (yatl-compile (span "foo")) '("<span>\nfoo\n</span>\n")))
(assert (equal (yatl-compile (span)) '("<span/>\n")))
(assert (equal (yatl-compile (if t (span))) '("<span/>\n")))
(assert (equal (yatl-compile (div (span))) '("<div>\n<span/>\n\n</div>\n")))

(assert (equal (yatl-compile (let (x) x)) '(nil)))
(assert (equal (yatl-compile (let ((x 1) (y 2)) (+ x y))) '(3)))
(assert (equal (yatl-compile (let ((x 1) (y 2) (z (span "foobar"))) z))
               '("<span>\nfoobar\n</span>\n")))
(assert (equal (yatl-compile (img?id@src=\"http://foo.bar/baaz.jpg\"))
               '("<img id=\"id\" src=\"http://foo.bar/baaz.jpg\"/>\n")))

yatl-compile-string
Occasionally, we need to create forms at run-time and yatl-compile macro doesn't evaluate its arguments. This function provides an escape hatch for these cases.
```
(defun yatl-compile-string(fmt &rest args)
  (let ((s (apply #'format fmt args)))
    (eval (yatl-compile-fn (list (make-symbol s))))))
```

html5 macro

Macro which wraps the yatl-compiled forms in HTML5 boilerplate.

(defmacro yatl-html5(&rest forms)
  `(concat
    "<!DOCTYPE html>\n<html>\n"
    (mapconcat (lambda (x) (format "%s" x)) (yatl-compile ,@forms) "")
    "</html>"))

HTML fragment generator

(defmacro yatl-html-frag(&rest forms)
  `(concat
    (mapconcat (lambda (x) (format "%s" x)) (yatl-compile ,@forms) "")))

yatl-Compile macro tests

We make sure our yatl-html5 works as expected when we mix our notation with Lisp function and macros. Let's define a few test fixtures.

(defmacro test-macro (&rest forms)
  `(concat
    "<div style=\"{display:flex}\">"
    ,@forms
    "</div>"))

(defun test-function (&rest forms)
  (apply #'concat forms))

(assert
 (equal
  (yatl-html5)
  "<!DOCTYPE html>\n<html>\n</html>"))

(assert
 (equal
  (yatl-html5 (head) (body))
  "<!DOCTYPE html>\n<html>\n<head/>\n<body/>\n</html>"))

(assert
 (equal
  (yatl-html5 (body (test-function "foo" "bar")))
  "<!DOCTYPE html>\n<html>\n<body>\nfoobar\n</body>\n</html>"))

(assert
 (equal
  (yatl-html5 (body (test-macro (test-macro))))
  "<!DOCTYPE html>\n<html>\n<body>\n<div style=\"{display:flex}\"><div style=\"{display:flex}\"></div></div>\n</body>\n</html>"))

(assert
 (equal
  (yatl-html5 (head))
  "<!DOCTYPE html>\n<html>\n<head/>\n</html>"))

(assert
 (equal
  (yatl-html5 (head (style)))
  "<!DOCTYPE html>\n<html>\n<head>\n<style/>\n\n</head>\n</html>"))

(assert
 (equal
  (yatl-html5 (let ((l '((a . 1) (b . 2)))) "foo"))
  "<!DOCTYPE html>\n<html>\nfoo</html>"))

Provide our module
Make this package available.
```
(provide 'yatl)
;;; yatl.el ends here
```