Merge pull request #123 from Omikhleia/feat-experiment-hook-symbols

feat(djot): Support programmatic symbols (extensible via Lua)

Author: Omikhleia

examples/sile-and-djot.dj

@@ -65,10 +65,11 @@ Actually, let's start with attributes, as they have the same general syntax for
 
 Attributes are put inside curly braces.
 On inline elements, attributes are placed immediately after the element they are attached to, with no intervening whitespace.
-They may contain line breaks, and may be "stacked," in which case they will be combined.
+They may contain line breaks, and may be "stacked," in which case they are combined.
 
 To attach attributes to a block-level element, put the attributes on the line immediately before the block.
-Block attributes have the same syntax as inline attributes, but they must fit on one line. Repeated attribute specifiers can also be used, and the attributes will accumulate.
+Block attributes have the same syntax as inline attributes, but they must fit on one line.
+Repeated attribute specifiers can also be used, and the attributes accumulate.
 
 Inside the curly braces, the following syntax is possible:
 
@@ -78,7 +79,7 @@ Inside the curly braces, the following syntax is possible:
 
  - `.foo` specifies a class, for styling purposes.
 
-   Multiple classes may be given in this way; they will be combined.
+   Multiple classes may be given in this way.
 
  - `key="value"` or `key=value` specifies a key-value attribute.
 
@@ -303,7 +304,7 @@ Attributes are optional, and are passed through to the underlying SILE package.
 You can notably specify the required image width and/or height, as done just above, by appending the `{width=... height=...}` attributes --- Note that any unit system supported by SILE is accepted, as well as percentages (see §[](#final-notes-units)).
 There are actually two kinds of image syntax: (direct) inline images and (indirect) reference images.
 Above, we used the former kind.
-The reference images are defined elsewhere in the document, and are referenced by their label:
+The reference images are defined elsewhere in the document, referred to by their label.
 
 {custom-style=CodeBlock}
 :::
@@ -321,11 +322,11 @@ Direct (inline) link's attributes override reference's attributes.
 {#djot-captioned-images}
 #### Implicit captioned figures
 
-An image with nonempty caption (i.e. "alternate" text), occurring alone by itself in a paragraph, will be rendered as a figure with a caption, as actually seen above.
+An image with nonempty caption (i.e. "alternate" text), occurring alone by itself in a paragraph, is rendered as a figure with a caption, as actually seen above.
 Otherwise, the caption is ignored.
 
-If your document class or previously loaded packages provide a `captioned-figure` environment, it will be wrapped around the image (and it is then assumed to take care of the caption, i.e. to extract and display it appropriately).
-Specifically, if the *resilient* book class is used, the caption will be numbered by default, and added to the list of figures. Specify `.unnumbered`, and `.notoc` respectively, if you do not want it.
+If your document class or previously loaded packages provide a `captioned-figure` environment, it is wrapped around the image (and it is then assumed to take care of the caption, i.e. to extract and display it appropriately).
+Specifically, if the *resilient* book class is used, the caption is numbered by default, and added to the list of figures. Specify `.unnumbered`, and `.notoc` respectively, if you do not want it.
 Otherwise, the converter uses its own fallback method.
 
 #### Extended image types
@@ -415,7 +416,7 @@ Another link to [the SILE website][sile].
 [sile]: https://sile-typesetter.org/
 
 The reference label should be defined somewhere in the document.
-If the label is empty, then the link text will be taken to be the reference label as well as the link text.
+If the label is empty, then the link text is taken to be the reference label as well as the link text.
 
 {#djot-cross-references}
 #### Cross-references
@@ -431,7 +432,7 @@ Empty local links (that is, without inline display content) are interpreted as c
 By default, they are resolved to the closest numbering item, whatever that might be in the hierarchical structure of your document.
 A pseudo-class attribute may be used to override the default behavior and specify which type
 of reference is expected (page number, section number or title text).
-Thus, the above example was obtained from the following input:
+Thus, the above example was obtained with:
 
 {custom-style=CodeBlock}
 :::
@@ -626,7 +627,7 @@ The contents of the block quote are parsed as block-level content.
 >
 > > Such quotes can be nested.
 
-If your document class or previously loaded packages provide a `blockquote` environment, it will be used.
+If your document class or previously loaded packages provide a `blockquote` environment, it is used.
 Otherwise, the converter uses its own fallback method, with hard-coded styling.
 
 #### Attributed quotes (epigraphs)
@@ -896,7 +897,7 @@ Djot supports the "pipe table" syntax, with its own way for marking the optional
 :::
 
 
-When using the *resilient* classes, the caption will be numbered by default, and added to the list of tables.
+When using the *resilient* classes, the caption is numbered by default, and added to the list of tables.
 Specify `.unnumbered`, and `.notoc` respectively, as table attributes, if you do not want it.
 
 ### Code blocks
@@ -946,7 +947,7 @@ This is a very naive approach to syntax-highlighting, until the converter possib
 
 #### Rendered code blocks
 
-If the converter knows how to render the content of a code block, it will do so by default.
+If the converter knows how to render the content of a code block, it does so by default.
 The `render` attribute can be set to `false` to prevent this behavior, and enforce the content to be rendered as raw verbatim text.
 
 : Mardown and Djot code blocks
@@ -1027,7 +1028,7 @@ The `render` attribute can be set to `false` to prevent this behavior, and enfor
 
   : Pie charts
 
-    Likewise, if you installed the optional *piecharts.sile* collection, then code blocks marked as "piechart" are automatically rendered, with all other attributes are passed to the underlying processor.
+    Likewise, if you installed the optional *piecharts.sile* collection, then code blocks marked as "piechart" are automatically rendered, with all other attributes passed to the underlying processor.
     Consider the following code block, consisting in a CSV table...
 
     {custom-style=CodeBlock}
@@ -1331,22 +1332,24 @@ Since it's possible to have unused footnote definitions, let's craft one as show
 When encountering a symbol, this converter looks for such a footnote and expands its content.
 It works with inline elements as shown above, but also with full blocks, provided the symbol is the only element in a paragraph of its own.
 
-Of course, these pseudo-footnotes[^djot-pseudo-footnotes] can in turn contain symbols, which will get replaced too.
+Of course, these pseudo-footnotes[^djot-pseudo-footnotes] can in turn contain symbols, which get replaced too.
 
 
 [^djot-pseudo-footnotes]: You may still use them as regular footnotes.
 Whether this is a good idea is another question...
 
 ### Predefined symbols
 
-This converter also comes with a few symbols predefined.
+This converter also comes with a few symbols predefined.[^djot-programmatic-symbols]
 
 - `:_TOC_:` must stand alone in its own paragraph. It inserts a table of contents.
    Attributes on the symbol are passed through, e.g. `:_TOC_:{depth=3}`.
 - `:U+xxxx:` (where `xxxx` is a hexadecimal value in upper case) is replaced by the corresponding Unicode character --- `:U+2122:` gives :U+2122:.
 
 The above variable substitution mechanism has precedence over these symbols, allowing you to possibly override them.
 
+[^djot-programmatic-symbols]: For 3d-party class and package designers, a Lua method is provided to add their own symbols.
+
 {#djot-metadata-symbols}
 ### Contextual metadata
 

inputters/djot.lua

@@ -682,25 +682,6 @@ function Renderer.en_dash (_)
   return("–")
 end
 
-local predefinedSymbols = {
-  _TOC_ = {
-    standalone = true,
-    render = function (node)
-      return createCommand("markdown:internal:toc", node.attr)
-    end
-  },
-  _FANCYTOC_ = { -- of course, requires having installed the fancytoc.sile module
-                 -- We are not going to check that here, so I won't document it.
-    standalone = true,
-    render = function (node)
-      return {
-        createCommand("use", { module = "packages.fancytoc" }),
-        createCommand("fancytableofcontents", node.attr)
-      }
-    end
-  },
-}
-
 function Renderer:getUserDefinedSymbol (label, node_fake_metadata)
   local content
   if self.metadata[label] then -- use memoized
@@ -758,33 +739,11 @@ function Renderer:symbol (node)
       end
       return text
     end
-    -- Let's finally look for predefined symbols
-    local symbol = predefinedSymbols[node.alias]
-    if symbol then
-      if symbol.standalone and not node._standalone_ then
-        SU.error("Cannot use " .. label .." as inline content")
-      end
-      return symbol.render(node)
-    end
-    local pos = node_pos(node)
-    if node.alias:match("U%+[0-9A-F]+") then
-      local content = {
-        createCommand("use", { module = "packages.unichar" }),
-        createCommand("unichar", {}, node.alias, pos)
-      }
-      if node.attr then
-        -- Add a span for attributes
-        return createCommand("markdown:internal:span", node.attr, content, pos)
-      end
-      return content
-    end
-    SU.warn("Symbol '" .. node.alias .. "' was not expanded (no corresponding metadata found)")
-    local text = ":" .. node.alias .. ":"
-    if node.attr then
-      -- Add a span for attributes
-      return createCommand("markdown:internal:span", node.attr, text, pos)
-    end
-    return text
+    -- Not a metadata symbol, pass our internal properties and delegate
+    local options = node.attr or {}
+    options._symbol_ = node.alias
+    options._standalone_ = node._standalone_
+    return createCommand("markdown:internal:symbol", options, nil, node_pos(node))
   end
 end
 

packages/markdown/commands.lua

@@ -121,6 +121,20 @@ end
 
 function package:_init (_)
   base._init(self)
+  if not SILE.scratch._markdown_commands then
+    -- NOTE:
+    -- I don't like those scratch variables, but package reinstancing
+    -- may occur in SILE. I never liked it, but it still occurs in SILE 0.15.5
+    -- when silex-x is not present, so we have to deal with it.
+    -- For putative readers:
+    --    load the djot package, cause you want to use Djot.
+    --    load the markdown package, cause you want to use Markdown too.
+    -- Both load the markdown.commands package.
+    -- Guess what? The markdown.commands package is instanciated twice.
+    -- It's supposed to be a SILE feature...
+    SILE.scratch._markdown_commands = {}
+  end
+  self.predefinedSymbols = SILE.scratch._markdown_commands
 
   -- Only load low-level packages (= utilities)
   -- The class should be responsible for loading the appropriate higher-level
@@ -151,6 +165,40 @@ function package:_init (_)
     self:loadPackage("resilient.epigraph")
     self:loadPackage("resilient.defn")
   end
+
+  -- Register some predefined symbols
+  -- Later we'll have packages or classes possibly register their own
+  -- predefined symbols.
+  self:registerSymbol("_TOC_", true, function (options)
+    return {
+      createCommand("markdown:internal:toc", options),
+    }
+  end)
+  self:registerSymbol("_FANCYTOC_", true, function (options)
+    -- Of course, it requires having installed the fancytoc.sile module
+    -- We are not going to check that here, so I won't document it.
+    return {
+      createCommand("use", { module = "packages.fancytoc" }),
+      createCommand("fancytableofcontents", options),
+    }
+  end)
+end
+
+-- Register a predefined symbol for use in Djot (as of now)
+-- The symbol is a leaf inline command that will be expanded when encountered.
+-- The symbol is registered with
+--  - a name
+--  - a boolean indicating if must be standalone (i.e. alone at block-level)
+--  - a function that will be called to render the symbol
+-- The function will receive as options the attributes set on the symbol,
+-- and must return a table of AST elements.
+-- Note that a span will also be created around an inline symbol if it has
+-- attributes, so styling can be applied to the symbol.
+function package:registerSymbol(name, standalone, render)
+  -- Multiple package reinstancing is may occur in SILE, see comment above
+  -- on scratch variables... So we do not warn if a symbol is already registered,
+  -- and just overwrite it silently.
+  self.predefinedSymbols[name] = { standalone = standalone, render = render }
 end
 
 local UsualSectioning = { "part", "chapter", "section", "subsection", "subsubsection" }
@@ -777,6 +825,36 @@ Please consider using a resilient-compatible class!]])
     end
   end, "Definition item in Markdown (internal)")
 
+  self:registerCommand("markdown:internal:symbol", function (options, _)
+    local symbol = SU.required(options, "_symbol_", "symbol")
+    local standalone = SU.boolean(options._standalone_, false)
+    local content
+    local predefined = self.predefinedSymbols[symbol]
+    if predefined then
+      if predefined.standalone and not standalone then
+        SU.error("Cannot use " .. symbol .. " as inline content")
+      end
+      content = predefined.render(options)
+    elseif symbol:match("^U%+[0-9A-F]+$") then
+      content = {
+        createCommand("use", { module = "packages.unichar" }),
+        createCommand("unichar", {}, symbol)
+      }
+    else
+      SU.warn("Symbol '" .. symbol .. "' was not expanded (no corresponding metadata found)")
+      local text = ":" .. symbol .. ":"
+      content = { text }
+    end
+    options._symbol_ = nil
+    options._standalone_ = nil
+    if next(options) and not standalone then
+      -- Add a span for attributes
+      SILE.call("markdown:internal:span", options, content);
+    else
+      SILE.process(content)
+    end
+  end, "Symbol in Djot (internal)")
+
   -- B. Fallback commands
 
   self:registerCommand("markdown:fallback:blockquote", function (_, content)
@@ -952,6 +1030,8 @@ package.documentation = [[\begin{document}
 A helper package for Markdown and Djot processing, providing common hooks and fallback commands.
 
 It is not intended to be used alone.
+
+For class or package designers, it provides a method \code{registerSymbol} to define their own symbols, which can be used in the Djot syntax.
 \end{document}]]
 
 return package