new-ryuslash.org/literate-build.org

ryuslash's website's build's files

• Makefile
• Eldev
• Publishing project
  • Projects
    • Pages
    • Posts
    • RSS
    • Assets
    • All
  • File footer
• Footnotes

I'm a big fan of Literate Programming, so I figured I'd make the builds for my website a literate configuration file as well.

First I need to build the build files. This is the smallest make file that I can think to make to enable me to build the rest out. This make file is duplicated in both this file and the source code repository since I don't know of a way not to.

I specify that build.mk file should depend on this file ({{{input-file}}}) and that it is generated by running org-babel-tangle-file on it. The $< is the first dependency and $@ is the current target file (whatever .mk file we're generating).

build.mk: literate-build.org
	emacs -quick -batch \
	    -eval "(package-initialize)" \
	    -load ob-tangle \
	    -eval "(org-babel-tangle-file \"$<\"))"

I can't use Eldev in this particular recipe because it needs a file with proper package headers and all, and that doesn't get generated until we run this. I'm cheating a little bit because I'm expecting this command to also generate the other files the build needs without explicitly saying so.

After that it's just a matter of including the file I want.

  include build.mk

GNU Make (I don't know about other makes) will see if there is a recipe to make the file it wants to include and will try and run it before trying to include the file. This combined with our %.mk target ensures that make will always try to recreate the build.mk file when {{{input-file}}} is updated.

Makefile

This is the actual make file that builds and deploys my site. It's all put into the build.mk file and executed from there. The %.mk pattern rule thankfully doesn't get recognized as a make target, so the first target define in the included file is assumed to be the default target.

First off I specify the help target. This target parses the make files and extracts targets that include some comment on what they do. This target should come first so that it automatically becomes the default target. This way when I run just make I can see which targets I have available. I got this awesome trick from Victoria Drake’s article How to create a self-documenting Makefile.

help:							## Show this help
	@grep --extended-regexp --no-filename '^[^#].*?\s##\s' $(MAKEFILE_LIST) \
	    | sort \
	    | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

The build target converts everything from whatever source files they are to html and css. The build target has 2 other targets it depends on, not surprisingly html and css. The html and css targets don't have any comment because they're not really meant to be executed directly.

build: html css			  ## Build the site and copy it to the staging directory

The html target calls Emacs. It depends on the Eldev file having been generated to specify any additional dependencies and which package archives should be used and this file will be generated by the Eldev target.

html: Eldev
	@echo "Publishing..."
	eldev emacs --quick --batch --load publish.el --funcall org-publish-all

The css target does specify its dependencies. This is both an exercise in writing make files (which I generally quite enjoy), and also to make sure that my builds don't take too long unless they actually have to. Ultimately any .css file gets created from a .less file by calling the lessc program. I'm intentionally not using recursive make in this project because it slows make down a lot, and I don't have to manage several make files this way.

css: public/assets/css/main.css public/assets/css/tekuti.css public/assets/css/cgit.css

public/assets/css/main.css public/assets/css/tekuti.css public/assets/css/cgit.css: \
    src/less/include/common.less \
    src/less/include/components.less \
    src/less/include/colors.less \
    src/less/yoshi.css

public/assets/css/%.css: src/less/%.less
	lessc $< $@

The deploy target first makes sure that build has been executed at least once and then uses rsync to upload all of the files. This intentionally doesn't depend on the build target so that I can upload whatever I happen to have generated without being forced to rebuild.

deploy:							## Deploy the site to live
	@[[ -e public/index.html ]] || (echo "Run 'make build' before deploy" && exit 1)
	rsync --verbose --checksum --recursive --delete \
	    --exclude '*~' --exclude '.eldev' --delete-excluded \
	    public/ ryuslash.org:ryuslash-next/

The clean target makes sure that everything that is generated gets cleaned up again. This is important if I need to start with a clean slate.

clean:							## Remove all of the build files
	@echo "Cleaning up..."
	@rm -rvf *.elc
	@rm -rvf public
	@rm -rvf .org-timestamps
	@rm -rvf posts/index.org build.mk Eldev publish.el

The serve target is a convenience target for when I'm writing or making modifications to the build and publish processes. It just starts a simple php web server in the public/ directory so that I can easily load it in my browser.

serve:						   ## Run a simple web server to look at the results
	@cd public && php -S "localhost:8000"

The theme target is another convenience target. I generate the colors for the source code blocks on my site from my Emacs theme. This target exports the colors from my theme so that the code blocks can use them. This file is then included by the less files. There is no good dependency here, because there is no file for the export of my theme to depend on right now, just occasionally I have to run it. It does depend on the Eldev file having been generated.

I keep this particular target around for playing with, but right now it doesn't actually work. Just because even though I load and enable yoshi-theme it doesn't appear to apply the theme in a batch session. Seems understandable because no UI actually gets loaded, but that does mean that it can't figure out which faces it sets and it just outputs the colors for the default theme.

theme: Eldev					## Generate the theme CSS
	eldev emacs --quick --batch --load htmlize --load ox-html \
	    -eval "(setq org-html-htmlize-output-type 'css)" \
	    -funcall org-html-htmlize-generate-css \
	    -load yoshi-theme \
	    -eval "(enable-theme 'yoshi)" \
	    -load make-mode \
	    -eval "(kill-whole-line)" \
	    -eval "(kill-whole-line)" \
	    -eval "(goto-char (point-max))" \
	    -eval "(forward-line -2)" \
	    -eval "(kill-whole-line)" \
	    -eval "(kill-whole-line)" \
	    -eval "(css-mode)" \
	    -eval "(indent-region (point-min) (point-max))" \
	    -eval '(write-file "src/less/yoshi.css")'

Finally, as a precaution, I specify that all of the main targets are phony targets. This way if I ever introduce any file with the same name as these targets they will still build and not assume that because the file exists, everything is up-to-date.

.PHONY: publish deploy html css help theme

Eldev

With Eldev I can install dependencies of my /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/Publishing%20project locally. Really the only reason I chose to use Eldev is because it is available in the Guix repository.

Since this is an actually full-fledged Emacs Lisp file and I don't know what I'm going to be doing in the future, I should be sure to enable lexical binding. Otherwise Emacs defaults to using dynamic binding, and that might cause some surprises in the future, since I'm quite used to using lexical binding everywhere.

; -*- lexical-binding: t; -*-

All I'm doing here, really, is enable the various Emacs Lisp Package Archives.

(eldev-use-package-archive 'gnu)
(eldev-use-package-archive 'nongnu)
(eldev-use-package-archive 'melpa)

Publishing project

Before anything else, here too I need to add a file header and enable lexical binding. If I don't add the file header Eldev won't work right.

;;; publish.el --- Publish project for ryuslash.org -*- lexical-binding: t; -*-

And then I have to keep a list of required packages so that Eldev can figure out which to download. I also need the Version header to keep Eldev happy.

;; Version: 1
;; Package-Requires: (<<required-packages>>)

I install dockerfile-mode because some of my posts include examples of docker files. And I use the latest org mode package.

Some packages must be loaded to make sure their features can be used by the export process.

(require 'dockerfile-mode)
(require 'ob-dot)
(require 'ox-publish)
(require 'ox-rss)
(require 'rainbow-delimiters)
(require 'subr-x)

After that I define a constant for the current root so that it's easy to refer to. I use defconst here instead of defvar for 2 reasons:

1. It's a constant, and shouldn't be changed during execution.
2. Re-evaluating a defvar doesn't do anything¹, but re-evaluating a defconst updates the value.

(defconst publish-root
  (file-name-directory
   (or load-file-name
       (buffer-file-name)))
  "The directory where ‘oni-org’ was loaded from.")

Keep the timestamp cache (used to determine which pages need to be cleaned up) local. This prevents publishing from relying on global state on my PC and makes it easy to reset everything.

(setq org-publish-timestamp-directory
      (expand-file-name ".org-timestamps/" publish-root))

I add the rainbow-delimiters-mode to the prog-mode-hook so that delimiters are highlighted like they are in my usual Emacs sessions.

(add-hook 'prog-mode-hook 'rainbow-delimiters-mode)

This requires that I have the rainbow-delimiters package installed.

Anything that is marked with either the noexport or draft tag shouldn't be exported.

(setq org-export-exclude-tags '("noexport" "draft"))

Don't ask for my confirmation whether babel blocks should be evaluated. If I've added them I want them to run.

(setq org-confirm-babel-evaluate nil)

Reset the org-mode style, I don't want the default style interfering with my own.

(setq org-html-head-include-default-style nil)

Just add CSS class names to code blocks, I'll make sure that the CSS selectors exist.

(setq org-html-htmlize-output-type 'css)

Use HTML5 and all its fancy elements.

(setq org-html-html5-fancy t)
(setq org-html-doctype "html5")

Projects

Here come the actual projects.

(setq org-publish-project-alist `(<<projects>>))

Pages

The pages project is the main publishing project. It exports all of the .org files except for README.org and anything found in the posts/ directory. The README.org is only relevant when you're looking at the source code for this website and posts/ has its own project.

It also loads the stylesheet into the HTML head, and add a link to my Mastodon account in the postamble, for verification purposes.

("pages"
 :base-directory "."
 :base-extension "org"
 :publishing-directory "public/"
 :recursive t
 :exclude ,(rx string-start
               (or "posts/"
                   (and "README.org" string-end)))
 :publishing-function org-html-publish-to-html
 :html-head "<link rel=\"stylesheet\" href=\"/assets/css/main.css\" type=\"text/css\"/>"
 :html-postamble t
 :html-postamble-format (("en" "<p class=\"social social-mastodon\">Find me on <a href=\"https://fosstodon.org/@ryuslash\" rel=\"me\">Mastodon</a></p>
<p class=\"date\">Date: %C</p>
<p class=\"creator\">%c</p>")))

Posts

The posts project is an experiment right now to see if I can find a way I'm happy with to actually publish my blog through plain old org files. This goes together with the /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/RSS project. This also calculates a rough estimate of how long the reading time for the page will be and adds it to the preamble of each post.

First I need a few of functions. Calculate how many minutes it would take to read the given buffer. I don't remember where I got this particular algorithm from, and I haven't tried out how accurate it is, but my experience with other websites doing this is that it's generally not very accurate anyway.

(defun publish-calculate-reading-time (buffer)
  "Calculate the amount of minutes it would take to read the contents of BUFFER."
  (with-current-buffer buffer
    (max 1 (/ (count-words (point-min) (point-max)) 228))))

I decided to split the calculations and formatting functions, so here is just a simple function that format the given reading time.

(defun publish-format-reading-time (time)
  "Return a string describing TIME."
  (format "Reading time: %d minute%s"
          time
          (if (= time 1) "" "s")))

And then I need a function to actually generate the preamble. This function ignores the index.org file, because that is not a page to read but just and index of my blog posts.

(defun publish-generate-post-preamble (_project)
  "Generate the preamble for a post with the estimated reading time."
  (unless (string= (file-name-nondirectory (buffer-file-name)) "index.org")
    (publish-format-reading-time
     (publish-calculate-reading-time (current-buffer)))))

Finally the actual project. It publishes files inside the posts/ directory.

("posts"
 :base-directory "posts/"
 :base-extension "org"
 :publishing-directory "public/posts/"
 :recursive t
 :publishing-function org-html-publish-to-html
 :html-head "<link rel=\"stylesheet\" href=\"/assets/css/main.css\" type=\"text/css\"/>" ; (ref:html-head)
 :html-preamble publish-generate-post-preamble) ; (ref:html-preamble)

At /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/(html-head) I again load the CSS style sheet for this site. /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/(html-preamble) calculates the page's reading time and puts it on the page.

RSS

The rss project works together with the posts project. This project very specifically generates and targets a specific file and exports it as an RSS feed. For this I require the ox-rss package.

Before I can define my project I still need to have some helper functions. The first I'll make is the publish-empty-time. It's just a dumb little function that takes the current time and subtracts it with itself. This is used as an empty value, so that if compared to another time it'll always be less than the other time.

(defun publish-empty-time ()
  "Get an empty time value."
  (let ((current-time (current-time)))
    (time-subtract current-time current-time)))

Next up is publish-get-latest-modified-time which takes a list of files and finds the latest time at which any of the files were modified.

(defun publish-get-latest-modified-time (files)
  "Get the latest modification time of any file from the list FILES."
  (car
   (last
    (sort (mapcar #'org-publish-cache-mtime-of-src files) #'time-less-p))))

I don't know why exactly, but there is a time-less-p and a time-equal-p, but no time-greater-p or any other. Just for clarity I decided to implement publish-time>= which is just the complement of time-less-p (because if the time is not less, it has to be either greater than or equal to).

(defun publish-time>= (a b)
  "Check if time A is greater than or equal to time B."
  (not (time-less-p a b)))

Now a bigger function with a bit more meat to it. This function opens an org file and tries to get some information out of it to construct a headline from it with a link to the actual post. This is used to generate the index of all of the posts. It goes through the file trying to find any headings tagged with the tag summary and extracts the contents from the first one. This gives us something to put on the index so it's not just a plain list with links.

(defun publish-extract-summary-from-file (file props)
  "Extract a summary from FILE.
PROPS is used as an aid in getting the right information from the file."
  (format "* %s\n:PROPERTIES:\n:CUSTOM_ID:  %s\n:PUBDATE:  %s\n:RSS_PERMALINK:  %s\n:END:\n\n%s\n\n[[file:%s][Read More]]\n\n"
          (car (org-publish-find-property file :title props))
          (file-name-nondirectory file)
          (format-time-string "[%Y-%m-%d %a %H:%M]" (org-timestamp-to-time (car (org-publish-find-property file :date props))))
          (file-name-nondirectory file)
          (car (org-map-entries (lambda () (let ((element-data (cadr (org-element-at-point))))
                                             (buffer-substring-no-properties
                                              (map-elt element-data :contents-begin)
                                              (map-elt element-data :contents-end))))
                                "summary"
                                (list file)))
          (file-name-nondirectory file)))

Finally the last function mushes it all together and actually generates an index.org file. I just looks through the posts/ directory and finds any files that start with a date. It compares the last modification time of the index.org and that of the latest modified post, and skips generating if the index.org is newer. Then it takes the 30 latest posts and extracts the necessary information from them and writes it all into index.org.

(defun publish-generate-index (props)
  "Generate an index from my posts.
Argument PROPS
."
  (let* ((index-file (expand-file-name "posts/index.org"))
         (index-generated-time (or (and (file-exists-p index-file)
                                        (org-publish-cache-mtime-of-src index-file))
                                   (publish-empty-time)))
         (files (directory-files "posts/" t (rx bos (= 8 digit) "-" (= 4 digit) "-" (one-or-more nonl) (not "~") eos)))
         (latest-modification-time (publish-get-latest-modified-time files)))
    (if (publish-time>= index-generated-time latest-modification-time)
        (message "Not generating index...")
      (progn
        (message "Generating index...")
        (with-temp-buffer
          (insert "#+title: ryuslash's blog\n")
          (insert "#+options: num:nil\n")
          (insert "#+html_link_up: /")
          (insert "#+html_link_home: /")
          (insert "\n")
          (apply 'insert
                 (mapcar (lambda (file) (publish-extract-summary-from-file file props))
                         (take 30 (reverse files))))
          (write-file "posts/index.org"))))))

With that all out of the way I can finally assemble my project.

("rss"
 :base-directory "posts/"
 :base-extension "org"
 :rss-extension "xml"
 :preparation-function publish-generate-index ; (ref:preparation-function)
 :publishing-directory "public/posts/"
 :publishing-function (org-rss-publish-to-rss) ; (ref:publishing-function)
 :html-link-home "https://ryuslash.org/posts/"
 :html-link-use-abs-url t
 :section-numbers nil
 :exclude ".*"                          ; (ref:exclude)
 :include ("index.org")
 :table-of-contents nil)

The /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/(preparation-function) uses my publish-generate-index function to create the index.org before trying to turn it into an RSS feed. The /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/(publishing-function) exports the org file to an RSS feed, this is from the ox-rss package. And /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/(exclude) excludes everything, and then we include the index.org again so that it only tries to turn the index.org into an RSS feed, not anything else.

Assets

This is a simple project that just goes around and copies over any asset files (.svg, .jpg, etc.) from the source directory into the publish directory.

("assets"
 :base-directory "."
 :recursive t
 :exclude "^public/"
 :base-extension "svg\\|png\\|jpg\\|ico"
 :publishing-function org-publish-attachment
 :publishing-directory "public/")

All

This is a convenience project so that I can publish everything all at once. It also establishes the right order in which to do so. Really the only order is that /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/RSS has to go before /ryuslash/new-ryuslash.org/src/commit/409897a83085edbe36211f1fea47e42bb8417d2b/Posts because the RSS preparation function needs to happen before the index.org it generates can be included in the Posts export.

("all" :components ("pages" "rss" "posts" "assets"))

File footer

Finally I need to add a file footer, again to keep Eldev happy.

(provide 'publish)
;;; publish.el ends here

Footnotes

---

¹

You have to evaluate it in a special way. Either with .