Arcology: An Org Roam Publishing System [ref]

Tags:  Arcology

The Arcology is a system for publishing my org-mode documents to the web, an important component of Ryan Rix's Complete Computing Environment. Rather than a pure static-site design, a directory full of "compiled HTML", the Arcology aims to provide just enough dynamicity to make publishing on the web simple, transparent, and powerful. Arcology uses the same database caching that Org Roam uses with some custom extensions to provide a web-view of a subset of documents in my knowledge base, and "indieweb" facilities around this like microformats, webmentions, indieauth, and some day a full ActivityPub implementation for my Journal and a "link blog" of my Archive posts.

This project is a repository of Org Mode formatted documents which describe and implement a web service written in Elixir using the Phoenix Framework. Rather than directly editing the source code, these documents are edited instead – source code, justification and the links between disparate modules and data-flows expressed in text as well as code.

Table of Contents


There is a Makefile to make it easier to tangle the files. You will need Emacs and Org-mode installed for this to work. Some day all of this will be in Mix tasks that can contain the whole set of dependencies.

make tangle will tangle any org document which is newer than a handle left in _build. It's important to keep your tangled documents up to date with the file system, but for development it's likely that you're sending the Elixir code to a running IEx session, there will be an org-mode documentation providing for this at some point soon.

make init will tangle, download dependencies, and compile them.

Developing in a REPL and a Babel doc

Arcology development requires a postgres database with a database called arcology_dev. run shell:mix ecto.setup (or click on that link!)

Executing this code will allow you to send the org-babel block under point to the REPL. I don't know how that handles noweb, blocks but it kinda works. Most of the time, I rely on automatically tangling and recompiling.

(use-package eval-in-repl
  (require 'eval-in-repl-ielm)

  ; i disable transient mark mode
  (defun eir-eval-in-iex ()
    "Provides eval-in-repl for Elixir."
    ;; Define local variables
    (let* ()
      (eir-repl-start "\\*Alchemist-IEx\\*" #'alchemist-iex-run t)
      (eir-send-to-iex (buffer-substring-no-properties (point) (mark)))))

  (setq eir-repl-placement 'left)
  (setq eir-jump-after-eval nil)
  (setq eir-ielm-eval-in-current-buffer t))

(use-package org-babel-eval-in-repl
  :after ob
  (add-to-list 'ober-org-babel-type-list '("elixir" . (eval-in-repl-iex eir-eval-in-iex)))
  (evil-define-key 'normal 'org-mode-map (kbd "C-<return>") #'ober-eval-in-repl)
  (evil-define-key 'normal 'org-mode-map (kbd "M-<return>") #'ober-eval-block-in-repl))

(use-package alchemist)

Alchemist can run an IEx shell with iex -S mix phx.server, and run tests with mix

Run this with Alt-Return to disable colors and see the IEx shell in action. Many "naked" code blocks can be evaluated in this fashion, but EIR doesn't support noweb blocks.

alias Arcology.{Repo,Roam,Page}
IEx.configure(colors: [enabled: false])

To trigger a tangle after org-src blocks, run this code. Opening an org-src block will do a bunch of window-modification which isn't restored properly by winner-mode, so stepping in and out of org-src blocks kind of sucks. I have C-x s bound to org-edit-src-save, and attach this function to it with the Emacs Advising Functions.

(defun arcology/tangle-on-block-save ()
  (org-babel-tangle-file org-src-source-file-name))
(advice-add 'org-edit-src-save :after #'arcology/tangle-on-block-save)

To trigger a recompile in iex after tangle, execute this block. It'll also touch the files which the Makefile checks for "up-to-date"-ness in the tangle task which'll need to be run by build scripts. This code is kind of cursed; you see, org-babel has a pre-tangle hook which isn't so useful, but current-buffer is the org-mode doc. The post-tangle hook is ran in each output file. So this cursed code runs an idle timer!

(defun arcology/idle-file-for-make-command ()
  (run-with-idle-timer 1 nil #'arcology/recompile-iex))

(defun arcology/recompile-iex ()
  (let* ((file (or org-src-source-file-name (buffer-file-name)))
         (touch-file (replace-regexp-in-string "\\([^/]+\\)\\.org$" "_build/\\1.touch" file)))
    (when (s-contains? "arcology" file)
      (message "touch %s" touch-file)
      (call-process-shell-command (format "touch %s" touch-file))
      (eir-send-to-iex "recompile"))))

(add-hook #'org-babel-pre-tangle-hook #'arcology/idle-file-for-make-command)


This is Referenced


in "The Page You Requested is Not Available"

For more information, see the Arcology Project README.

in "Arcology Sitemap"

That said, I've always had a visual streak, I like being able to see the forest for the trees, as it were. Being able to see the full set of relationships is a useful thing; org-roam provides a server process which can provide a relationship graph for a page or my entire Knowledge Base, but I would like to have something similar directly in the Arcology. That is what this page describes, a sitemap which rather than listing the pages in a heirarchy lists them in a connected graph.

in "Arcology Sitemap"

Navigating the Arcology is supposed to be a somewhat curated experience. I either send out a direct link to a page, and people can read and explore from there, or they go to one of the index pages like The Arcology Garden, The Complete Computing Environment, or The Lion's Rear. Those index pages are carefully curated, if not to send people on a romp through the respective garden, then to at least not get lost if they go back to that page.

in "The Arcology Project"

in "The Arcology Project"

in "Project Configuration for arcology"

in "Arcology Page Controller"

in "Arcology Page Controller"