Jorge Israel Peña

Dots

Sun, 16 Feb 2014 00:00:00 +0000

It’s become a common practice to keep one’s dotfiles version controlled, oftentimes mirrored somewhere like Github. A lot of people start their own dotfile collection based (forked) off of someone else’, but that never felt right to me. In my view, dotfiles are personal, sensitive configuration files that are explicitly defined to one’s own taste. It never made sense to me to want to use a giant wad of files with who knows what configuration directives in there. It seemed a lot like cargo culting to me.

I do enjoy looking at other peoples’ dotfiles from time to time to see what I can pick out and adapt to my own, but otherwise I like to grow mine organically—that is, only configure what I need, as I need it—to ensure that I really understand my configuration. As a result, I created my own dotfiles from scratch.

File Structure

The file structure is pretty simple. There’s a folder for every type of dotfile collection, for example: zsh, git, vim, and so on. Each of these can contain hidden files and folders that can be deployed by the deploy script.

Deployment

My deploy script is called sprinkle, which is a heavily modified fork of the deploy script from holman’s dotfiles. I chose this script to start from because I liked that it was a shell script, unlike most other deploy scripts I had seen which were Rakefiles, naturally requiring Ruby to be installed on the system before being able to deploy the dotfiles.

I also liked that the deploy script had a naming convention such that files and folders with a .symlink suffix were those that were deployed. However, I didn’t like that vim and Github wouldn’t detect the filetype—and therefore wouldn’t highlight—due to the misleading file extension. So I ended up heavily customizing the script.

Now, instead of the .symlink-suffix naming convention, those files and folders that should be deployed are themselves hidden. This allows vim, Github, and others to detect the file type and provide highlighting. Running the deploy script for the first time yields something like this:

My zsh files have an alias for the sprinkle script so that it can be run from anywhere, though in this case I was already in my dots directory. Files that haven’t been deployed are immediately deployed (symlinked) unless there’s an existing file in the destination. In that case, there are options to backup, overwrite, or remove (without deploying) the existing file, as well as skip that file altogether.

These commands are entered when prompted by simply entering the first letter of the action, i.e. o for overwrite. A capitalized letter performs that action for all remaining files as well. This is what the prompt looks like:

tmux

My tmux configuration is pretty simple I think. I keep the bind at C-b though it kind of interferes with C-f/C-b scrolling, in which case there’s a bit of lag for C-b unless I tap it twice. Instead I’m getting used to scrolling with C-u/C-d though it’s pretty disorienting starting out.

My window list is pretty subtle I think. Active windows are underlined and the current window name is emboldened.

I have binds for creating new windows with M-n and renaming a window with M-r:

bind -n M-r command-prompt 'rename-window %%'
bind -n M-n command-prompt -p "Name of new window:" "new-window -n '%%'"

I also created simple binds for navigating and moving windows around. M-h/M-l moves to the left and right window respectively, and M-j/M-k moves the current window left and right, respectively.

# switch between windows left/right
bind -n M-h previous-window
bind -n M-l next-window

# move windows left/right
bind -n M-j swap-window -t -1
bind -n M-k swap-window -t +1

zsh

My zsh configuration files are created from scratch as well, I don’t use something like oh-my-zsh for the same reasons that I stated in the opening paragraph. I do use antigen, which is similar to vundle but for zsh, mainly to avoid having to either keep stale snapshots or juggle git submodules of zsh plugins I use. I only use two zsh plugins: syntax-highlighting and extra/community completions.

I then have a separate zsh sub-folder that stores zsh files that configure different aspects of zsh, such as aliases, completions, functions, bindings, and so on.

Aliases

One of my most used aliases is :q which is simply aliased to exit, making it very natural for me to exit shells. If pacman is present on the system, I create many aliases to different kinds of pacman commands, such as pacup for pacman -Syu, pacin for pacman -S, and so on.

Completions

There’s nothing really special about my completions configuration. I do set it up to use my dircolors setup, so that the completion menu uses the correct colors for the different kinds of files. I also set it up for case-insensitive substring completion, so that I can type a bit of text from anywhere in the filename, regardless of case, and have it tab-complete correctly.

Functions

I do have a few functions I find to be very useful. The first is one that opens up a man-page directly to a given flag, i.e. manf ls -l opens the man-page for less directly to the point that describes the -l switch.

function manf() {
  man -P "less -p \"^ +$2\"" $1
}

Another one of note is one I found on stackoverflow which basically prepends a column to ls -lh which contains the permissions of the files in octal/numerical form. I let this take the place of the regular ls command, which can still be run for whatever reason by prepending it with command:

function ls() {
  command ls -lh --color=always $@ |\
    awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/)\
         *2^(8-i));if(k)printf("%0o ",k);print}'
}

Example output:

total 32K
755 drwxr-xr-x 2 jorge users 4.0K Feb 15 20:05 git
644 -rw-r--r-- 1 jorge users    0 Jan 27  2013 README.md
755 drwxr-xr-x 2 jorge users 4.0K Feb 13 00:54 ruby

I also created a wrapper around the built-in cd function that accepts parameters of the form b... where the number of dots is arbitrary. If the wrapper detects a parameter of that form, it removes the b prefix and expands each of the dots to ../. This makes for very quick navigation up an arbitrary amount of directories. The actual change-directory work is delegated to the original built-in cd function:

function cd() {
  emulate -LR zsh

  if [[ $1 == 'b.'* ]]; then
    builtin cd ${${1/"b"}//"."/"../"}
  else
    builtin cd $*
  fi
}

Example usage:

user@host some/dir/here $ cd b..
user@host some/ $ # went up two directories

I also have a function for listing the pacman orphan packages on my system, i.e. those that aren’t required by any other package. I found a command for doing this but it just dumped a list of every package, so I modified it to also list the description of the package. This requires the expac package, a utility to query the pacman database:

function pacorphans() {
  expac "%n:%N:%d" -Q $(expac "%n %G" | grep -v ' base') |\
    awk -F: '$2 == "" {printf "%s: %s\n", $1, $3}'
}

Example output:

yasm: A rewrite of NASM to allow for multiple syntax supported (NASM, TASM, GAS, etc.)
zsh: A very advanced and programmable command interpreter (shell) for UNIX

Finally, another function I recently created that I find very useful is one to fetch my external IP address and both copy it to my clipboard and print it out to the terminal. This is very useful because my IP address does change from time to time, usually if I restart my router. Considering that I host a mumble and syncplay server, whenever this happens I have to inform my friends of the change, which usually requires me to manually determine my external IP by going to some website that provides the information.

So I decided to create a command that gets the IP address from ipinfo.io, copies the response to my clipboard, and also prints it out to the terminal.

In the past, one would use a command such as ifconfig to list the computer’s local addresses. Recently there has been a shift to use the new ip command which houses many sub-commands such as ip addr which is now the preferred method to list addresses.

So what I did was create a wrapper for this ip command and create a fake sub-command called get which performs this task of retrieving my external IP address. If the sub-command get wasn’t provided, then the wrapper delegates the work to the actual ip command, if it exists.

function ip() {
  emulate -LR zsh

  if [[ $1 == 'get' ]]; then
    res=$(curl -s ipinfo.io/ip)
    echo -n $res | xsel --clipboard
    echo "copied $res to clipboard"
  else
    # only run ip if it exists
    if (( $+commands[ip] )); then
      command ip $*
    fi
  fi
}

Example output:

$ ip get
copied 123.45.678.90 to clipboard

Cursors

I setup mode-aware cursors in zsh, to better emphasize when I’m in vi mode and not. This is pretty straightforward, simply sending the correct terminal control sequence:

function zle-keymap-select {
  zle reset-prompt

  if [[ $KEYMAP = "vicmd" ]]; then
    echo -ne "\033]12;10\007"
  else
    echo -ne "\033]12;6\007"
  fi
}

function zle-line-finish {
  echo -ne "\033]12;6\007"
}

zle -N zle-keymap-select
zle -N zle-line-finish

This works perfectly fine in urxvt, but tmux must be configured to allow this because otherwise the setting of the cursor color above by zsh bypasses tmux, applying to tmux as a whole. This means that if one tmux window is in vi mode, the cursor will change, but if one then switches to another tmux window that is in insert mode, the cursor color for that window will remain the same as in the one in vi-mode. That is, the changed cursor color applies to every screen in tmux.

tmux did implement functionality for it to remember the cursor color on a per-window basis back in 2011, but this is only configured out of the box for xterm, since every terminal’s control sequences may vary.

The cursor color is inherently global, so what happens is that tmux remembers the cursor color for every window. When switching to another tmux window, tmux checks if that window’s cursor color had been previously changed. If so, tmux sets the global cursor color to that window’s saved cursor color. Otherwise, it means that that window’s cursor color hasn’t been changed, in which case it needs to reset the cursor color to the “default” cursor color, in case the previous window did change the color.

For this, two terminal escape sequences have to be defined, or overridden: the first tells tmux how to set the cursor color and the other tells tmux how to reset it to the “default” color.

The sequence for setting the color is the same in xterm and urxvt: \033]12;color\007. However, there is no sequence I know of—after looking at man 7 urxvt—for resetting the cursor color to the default cursor color. For xterm, it is \033]112\007. So instead what I decided to do was tell tmux that the sequence was simply the one to set the color, but with the default cursor color explicitly defined, which for me is the 6th ANSI color code (cyan).

set -g terminal-overrides ',rxvt*:Cs=\E]12;%p1%s\007:Cr=\E]12;6\007'

Misc

When one runs a command that doesn’t exist, it generally gives an error pointing out that fact. However, the pkgfile package provides a zsh script that, when sourced, provides information about which package such a command may be found in.

[[ -e /usr/share/doc/pkgfile/command-not-found.zsh ]] &&\
  source /usr/share/doc/pkgfile/command-not-found.zsh

Example output:

$ clojure
clojure may be found in the following packages:
  community/clojure 1.5.1-2     /usr/bin/clojure

I also configured highlighting for the less pager.

My prompt is pretty involved and it’s discussed more in-depth in my customization post, though it’s slightly outdated. See my dotfiles for the latest configuration.

One noteworthy thing however is that I highlight the path separator in a subtle cyan.

SLASH="%{$fg[cyan]%}/%{$reset_color%}"
echo "${${PWD/#$HOME/~}//\//$SLASH}"

My vi-binds are pretty straightforward. One noteworthy thing is that I bound _ and g_ to go to the beginning and end of line, respectively, to reflect what I use in vim.

bindkey -M vicmd "_" beginning-of-line
bindkey -M vicmd "g_" end-of-line

I also created some binds specific to the completion menu. I bound shift-tab to go in the reverse direction of tab. I also changed Enter to accept and enter the command, instead of the default, which only accepts the completion and allows the user to continue typing the command. Instead, I bound C-g to perform the accept-only:

bindkey -M menuselect "^M" .accept-line
bindkey -M menuselect "^G" accept-line
bindkey -M menuselect "^[[Z" reverse-menu-complete

vim

My vim configuration is discussed in other posts, such as this one. Some noteworthy things are my mode-aware cursors which you can see here, where my statusline is also discussed. They’re basically color-coded based on the mode. I also made all of the cursors be the block cursor, rather than I-beam for insert mode as is default. I also disabled cursor blinking:

set gcr=a:block

" mode aware cursors
set gcr+=o:hor50-Cursor
set gcr+=n:Cursor
set gcr+=i-ci-sm:InsertCursor
set gcr+=r-cr:ReplaceCursor-hor20
set gcr+=c:CommandCursor
set gcr+=v-ve:VisualCursor

set gcr+=a:blinkon0

hi InsertCursor  ctermfg=15 guifg=#fdf6e3 ctermbg=37  guibg=#2aa198
hi VisualCursor  ctermfg=15 guifg=#fdf6e3 ctermbg=125 guibg=#d33682
hi ReplaceCursor ctermfg=15 guifg=#fdf6e3 ctermbg=65  guibg=#dc322f
hi CommandCursor ctermfg=15 guifg=#fdf6e3 ctermbg=33  guibg=#268bd2

I also enable the cursorline only on the current window:

augroup cursorline
  autocmd!
  autocmd VimEnter,WinEnter,BufWinEnter * setlocal cursorline
  autocmd WinLeave * setlocal nocursorline
augroup END

Instead of letting CtrlP generate the list of files, which can be slow, I delegate this work to find and dir on unix and windows respectively. If we’re within a git repository, then I take advantage of git ls-files to do this instead.

let s:ctrlp_fallback =
  \ has('win32') ?
    \ 'dir %s /-n /b /s /a-d' :
    \ 'find %s -type f'

let g:ctrlp_user_command = [
  \ '.git',
  \ 'git --git-dir=%s/.git ls-files -co --exclude-standard',
  \ s:ctrlp_fallback
  \ ]

Conclusion

All in all it’s a pretty simple dotfiles system I think. Feel free to take a look through it to see what you may like to adapt to your own configuration.

Live Editing with Hakyll

Fri, 20 Dec 2013 00:00:00 +0000

A recent trend in developer-oriented blogging is to use static site generators, perhaps the most popular being Jekyll written in Ruby. The general workflow when writing posts with static site generators is the classic edit, save, reload.

A while back, a friend was telling me about his Jekyll setup and in passing mentioned how indispensable some “LiveReload” tool was. I previously did use Jekyll myself, but I hadn’t heard of this LiveReload tool. For me, reloading didn’t feel like such a hassle, so having a tool that “only” automatically reloaded the page for me didn’t feel entirely beneficial.

For this reason, I assumed that this was not what the LiveReload tool did, and instead I assumed it somehow replaced the content (i.e. post body) in-place. It turns out I was incorrect in my assumption and ended up implementing what I thought LiveReload did in Hakyll, the static site generator I use written in Haskell.

Conception

I had to come up with a way to intercept the post body content when it was altered, and send that to the client, who would then simply replace the content div with the received content. Conceptually, it was a pretty straightforward process.

I decided that the communication channel between the Hakyll preview server and the client would be a WebSocket, so I looked for websocket packages on Hackage and found one by the author of Hakyll himself. There are backends for wai and snap, but starting up a wai or snap server seemed too heavy for what I was going to do.

Having written Hakyll and Pandoc compilers before, I knew that the best place to get the changes to a post would be within the Hakyll compiler pipeline, before templates are applied.

compile $ getResourceBody
  >>= pandocCompiler (storeDirectory conf)
  >>= pushToClient
  >>= loadAndApplyTemplate itemTemplate context
  >>= loadAndApplyTemplate "templates/layout.html" layoutContext

Inserting a compiler at this appropriate location would give me access to changes to posts as they were applied, allowing me to then send them to the client via WebSocket.

Server Architecture

The WebSocket server that comes with the websockets package forks separate threads for every client that connects. I fork the WebSocket server itself within the site binary to avoid interrupting the main Hakyll compilation process. Therefore there needs to be a way for the Hakyll compiler to communicate with the WebSocket client threads, specifically to send each client the changes as they’re made to a post.

main thread
- hakyll
- WebSocket server
  - client thread 1
  - client thread 2
  - client thread N

State

First, however, I had to think about how the WebSocket server would communicate with the clients. I decided that the most straightforward thing to do would be to communicate on a per-route basis. A route in this case refers to a post, such as posts/live-editing-with-hakyll.markdown. The clients use that path to establish the connection, which tells the WebSocket server that they’re interested in data about that particular post. This way, the client doesn’t get updates about posts it doesn’t care about.

An alternative to this would’ve perhaps been to use a single communication channel, where the server would send data about every post that was changed, and clients decided which applied to them. While this seems simpler, it has the consequence that every post’s data is sent to the client, even posts no one is actually paying attention to. This is inefficient, and to avoid this inefficiency would again require some form of “interest-registration” which is implicit in the aforementioned method.

For this reason, the server needs a bit of state to keep track of which connections care about which routes, best represented by a Map of routes to connections, something like:

Map.Map Route Connections

Communication

Haskell provides a variety of concurrency primitives, such as MVars and Channels. MVars are mutable locations in memory and Channels are simply FIFO channels. It made sense to use a channel between the Hakyll compiler and the WebSocket client threads, so that WebSocket client threads would subscribe/listen to the channel and the Hakyll compiler would publish/write to the channel with the new post data.

However, we’re actually going to be using the stm package’s variants of these concurrency primitives: TVars and TChans. The Software Transactional Memory package allows us to perform transactional, atomic operations to avoid race conditions. We’re going to use a TVar for the server state, which will allow the Hakyll and WebSocket server to perform transactional read/write operations on the server state without stepping on each others’ toes (i.e. race conditions).

I chose a TVar and not a TMVar because we won’t have a concept of an “empty state,” which is what the TMVar variant allows. The state will always contain something, even if that’s just an empty Map. So let’s wrap our state up in a TVar:

type State = TVar (Map.Map Route Connections)

There’s a slight problem with our idea of using channels, however. Reading a value from a channel “consumes” that value, i.e. it’s no longer available for reading from that channel. This complicates things because it means that I couldn’t easily have a single channel to write every post’s changes to because that would mean that only the first client thread to read the data would receive the data.

Fortunately, the channel type supports a duplication operation. If you have channel A and you duplicate it, yielding channel B, writing to any one of those channels makes the data available to read from both channels:

Duplicate a Chan: the duplicate channel begins empty, but data written to either channel from then on will be available from both. Hence this creates a kind of broadcast channel, where data written by anyone is seen by everyone else.

Remember that our server state consists of a Map of routes to connections. Instead, we could make this a Map of routes to broadcast channels. When the Hakyll compiler updates a post, it’ll see if there’s a broadcast channel associated with the post’s route and if so, it’ll pipe the data through that broadcast channel. On the WebSocket client thread side, they will duplicate the broadcast channel for that route they’re interested in, or if it doesn’t already exist, create it first.

One final point is that we would like to avoid writing to a channel if no one is listening, to avoid unnecessary work. A simple way to achieve this is through simple reference counting. When a new client listens in on the broadcast channel, we increment the reference count, and accordingly decrement it when they disconnect, removing the channel from the Map altogether if no one is listening anymore.

To be precise, we will be creating the broadcast channel with newBroadcastTChan, which is recommended when creating a broadcast channel. This is because creating a broadcast channel with newTChan has the consequence that it’s treated as any other channel that can be read from, so data begins to pile up as it’s written since it’s never read from, only the duplicate channels are read from. Creating one with newBroadcastTChan closes the read stream which allows the garbage collector to dispose of items once they’re read from the duplicate channels.

Implementation

Now that we’ve planned stuff out we can get to the implementation. As a summary, this is what the WebSocket server will be doing:

client opens post which initiates connection to WebSocket server for that post
WebSocket server forks off a thread to handle that client
1. client checks server state to see if broadcast channel already exists for the post
  - exists
    1. duplicate the broadcast channel
    2. increment reference count
  - doesn’t exist
    1. create a broadcast channel for this post
    2. put it in server state with initial reference count of 1
    3. duplicate it
2. listen in on duplicate channel for changes to the post
3. when data is received from the channel, pipe it to WebSocket client
4. repeat steps 2-3 forever

And here’s what the Hakyll compiler will be doing:

retrieves item body
checks if there’s a broadcast channel available for this item
- yes: pipe the item body through the channel
- no: do nothing

First we have our server state, which consists of a map from routes to pairs of channels and their reference counts. Once again, we’ll store this state in a TVar so that it can be read and written to from Hakyll and the WebSocket server in a transactional manner.

type Channels = TVar (Map.Map String (TChan String, Integer))

WebSocket Server

The WebSocket server is pretty straightforward. We’ll print a message out and listen in on port 9160, specifying a client connection handler wsHandler that needs access to the server state:

wsServer :: Channels -> IO ()
wsServer channels = do
  putStrLn "WebSocket Server Listening on http://0.0.0.0:9160/"
  WS.runServer "0.0.0.0" 9160 $ wsHandler channels

The client handler starts out in a straightforward manner. We begin by getting the request data, from which we yield the path that the client connected to. This path, after all, signifies the post the client is interested in. Then we proceed to accept the client connection:

wsHandler :: Channels -> WS.ServerApp
wsHandler channels pending = do
  let request = WS.pendingRequest pending
      path    = tail . BC.unpack $ WS.requestPath request

  conn <- WS.acceptRequest pending

Next we need to get the channel we’ll be listening in on. Notice that this is performed atomically within the STM monad, since we don’t want to have race conditions between the time we read the map and when we update the server state with either an incremented reference count or a new entry.

We begin by retrieving the server state from the TVar. We then perform a lookup in the Map to determine if a channel already exists for the given path. If a channel doesn’t exist, we create a new broadcast channel using newBroadcastTChan and insert it into the server state Map with an initial reference count of 1. Finally we duplicate this broadcast channel using dupTChan, which will be our result. If a broadcast channel already does exist, then we simply increment the reference count and duplicate it.

  chan <- liftIO $ atomically $ do
    chans <- readTVar channels

    case Map.lookup path chans of
      Just (ch, refcount) -> do
        modifyTVar' channels $ Map.insert path (ch, refcount + 1)
        dupTChan ch
      Nothing -> do
        ch <- newBroadcastTChan
        modifyTVar' channels $ Map.insert path (ch, 1)
        dupTChan ch

Now that we have the correct channel to listen from, we can forever perform the same loop:

read from the channel; this blocks until there’s something to read
pipe the data to the WebSocket

  handle catchDisconnect . forever . liftIO $ do
    atomically (readTChan chan) >>= WS.sendTextData conn . T.pack

Finally we need to gracefully handle the case where the client leaves by either decrementing the reference count or outright removing the channel from the Map. This is also an atomic operation since we’re performing a read followed by a write:

  atomically $ do
    chans <- readTVar channels
    case Map.lookup path chans of
      Just (ch, refcount) -> do
        if (refcount - 1) == 0
          then modifyTVar' channels $ Map.delete path
          else modifyTVar' channels $ Map.insert path (ch, refcount - 1)
      Nothing -> return ()

Hakyll Compiler

Remember that we also need a Hakyll compiler to insert into the Hakyll compiler pipeline. This compiler will read the server state Map and determine if if there’s a channel to send the item body to, and if there isn’t, does nothing.

We will need two pieces of data relevant to the Item being compiled: the path to the file responsible for this Item as well as the Item’s body. The path is what we’ll use as the key into the Map, and the body is what we’ll pipe through the channel:

webSocketPipe :: Channels -> Item String -> Compiler (Item String)
webSocketPipe channels item =
  unsafeCompiler $ do
    let path = toFilePath . itemIdentifier $ item
        body = itemBody item

We fork off another thread to atomically retrieve the server state and determine if there’s a channel associated with the path, and if so, pipes the body through the channel:

    void . forkIO $ atomically $ do
      chans <- readTVar channels

      case Map.lookup path chans of
        Just (ch, _) -> writeTChan ch body
        Nothing -> return ()

The final and very important thing to do is to return the item as it was passed to us. This in effect makes this Hakyll compiler transparent, just “observing” the data that’s passing through it.

    return item

Main Thread

The main thread should begin by initializing the server state:

  channels <- atomically $ newTVar Map.empty

You’ll need a way to determine whether the WebSocket server should run. In my case I have a previewMode variable that’s only true when the Hakyll action is either watch or preview; everything else defaults to deploy-mode.

  let previewMode = action == "watch" || action == "preview"
  when previewMode $ void . forkIO $ wsServer channels

Finally, don’t forget to insert the Hakyll compiler into the pipeline, passing it the server state:

    compile $ getResourceBody
      >>= pandocCompiler (storeDirectory conf)
      >>= webSocketPipe channels
      >>= loadAndApplyTemplate itemTemplate context
      >>= loadAndApplyTemplate "templates/layout.html" layoutContext

Client Side

Now we need to wire stuff up from the client side. First though there’s one last thing we need to do in the back-end. We need to create a Hakyll Context that inserts the client-side JavaScript only if we’re in preview mode, otherwise when you deploy your site every visitor will be attempting to connect to the WebSocket server.

In my setup I have a postCtx that specifies the Context to use for posts. I’ve changed it to be a function that takes as argument a Bool specifying whether or not the site is in preview mode. This indicator is further passed on to a function called pushJS that will embed the JavaScript if it’s True:

postCtx :: Bool -> Context String
postCtx preview = mconcat
  [ pushJS preview "pushJS"
  -- ...
  , defaultCtx ]

Before we get to pushJS, consider that we might want to disable this feature on a per-post basis. Personally I’d like this functionality to be on by default, but there are some posts I have such as this one that are ridiculously long and take a very long time to load, so I’d like to be able to set a metadata push: off option in that particular post to disable it. Let’s define a function that gets the metadata value and assumes its True unless it’s explicitly set to false or off:

pushOn :: (MonadMetadata m) => Item a -> m Bool
pushOn item = do
  pushMeta <- getMetadataField (itemIdentifier item) "push"
  return $ case pushMeta of
             Just "false" -> False
             Just "off" -> False
             _ -> True

Now we can get to the pushJS function. We only generate the JavaScript code if the site is in preview mode and the option isn’t disabled for this particular Item. The way this will work is that it’ll load the contents of the file templates/push-js.html into the key tag, which is "pushJS" in my case as defined above. So in my layout template I’ll have $pushJS$ , which will be replaced by the contents of push-js.html or it’ll be an empty string if the requirements for the feature aren’t met. One last thing is that we pass the path of the file responsible for the Item into the push-js.html template as the $path$ tag so that the WebSocket knows what path to connect to:

pushJS :: Bool -> String -> Context String
pushJS preview key = field key $ \item -> do
  push <- pushOn item
  if preview && push
    then do
      path <- fmap toFilePath getUnderlying
      tmpl <- loadBody "templates/push-js.html"
      itm <- makeItem "" :: Compiler (Item String)
      gend <- applyTemplate tmpl (constField "path" path) itm
      return $ itemBody gend
    else return ""

The contents of the file push-js.html are pretty straightforward. We use the $path$ that we were passed by pushJS to connect to the WebSocket server. Then we define an onmessage handler. This handler does a couple of things. First it finds the element that contains my post body. Once it has the element, it replaces the contents of this element with the data received through the WebSocket.

We then have to perform some house cleaning, essentially re-running JavaScript functionality that ran on DOM load, such as creating links out of the headers in the post. I wrapped this stuff up in a global refresh function. This allows me to simply call it again in this handler. The last thing I do in this handler is re-run MathJax $\LaTeX$ typesetting on the post body element, since it originally ran on DOM load as well.

<!-- preview push -->
<script async="true" type="text/javascript">
  jQuery(function (){
    var ws = new WebSocket('ws://localhost:9160/$path$');
    ws.onmessage = function (e) {
      var content = jQuery('article .entry-content');
      content.html(e.data);

      window.refresh();

      MathJax.Hub.Queue(["Typeset", MathJax.Hub, content[0]]);
    };
  });
</script>

Conclusion

So I added this functionality to Hakyll that I thought others had through LiveReload. Of course, I showed it to the friend that told me about LiveReload to begin with and he was amazed, as was I when he told me that all LiveReload did was refresh the page for you.

Of course, the advantage of LiveReload over this is that it handles any asset, such as style sheets, and reloads the entire page so that you can see those changes, rather than just the post body. But like I mentioned in the beginning of this post, I don’t feel like I need that functionality in particular. Perhaps I’ll implement it later on as well anyways though, for convenience. I have a feeling it’ll be more straightforward than this.

A Simpler Vim Statusline

Sat, 14 Dec 2013 00:00:00 +0000

Everyone that uses Vim and their grandmothers have apparently adopted the use of statusline plugins like vim-powerline or vim-airline. The latter, more recent vim-airline has gained popularity due to the fact that vim-powerline was deprecated in favor of a still-in-development rewrite that aims to provide a more general foundation for people to use the same kind of statusline in other areas, such as tmux and shell prompts (cringe). So now everyone can have the same look: solarized and some powerline-like status, everywhere.

There’s of course nothing wrong with using these plugins, but I’ve come to realize that they’re overdone for my particular needs, and I don’t even really care for how they look.

Patched Fonts

First there was the annoyance of having to use patched fonts to achieve the “powerline look”. This isn’t a huge problem since it’s pretty straightforward to track down, download, and install pre-patched fonts. This has only become somewhat more annoying due to the fact that the new powerline rewrite patches fonts differently, so you have to find the patched font for the version of powerline you use, and different plugins adopt different versions. Now do this for every OS you use, in my case, Windows, Linux, and OS X.

The point is that in the grand scheme of things, when you take a step back, you realize you’re doing all of this for a mere statusline in a particular application (or two).

Noise

By default, powerline shows the mode you’re currently in, git branch if applicable, file name, file format, file encoding, file type, percentage through file, line number, column number, as well as flags that show whether you’re in paste mode, if the file is modified, or if it’s read-only. The plugins also sport an entirely custom CtrlP statusline.

When I switched over to airline recently I decided to take advantage of the fact that it’s somewhat customizable. At first I only did so to manually define a colorscheme I wanted, since the predefined one I had been using kept being changed around by upstream.

Then I came to realize that I didn’t need all of that information. This is a similar conclusion I came to when I recently redesigned my shell prompt. I started by shortening the Vim mode part of the statusline, e.g. NORMAL became N.

I rarely ever need to care about the file format, encoding, or type. If I do, I can simply set ff, fenc, or ft respectively to have them printed. The file type in particular I can usually deduce from the file extension or contents, unless I’ve explicitly set it myself.

Similarly, I don’t need the current line number to be in the statusline since it’s already in the number line on the left which I always have visible due to relativenumber. This has the effect of numbering the current line as line 0. Setting number as well changes this so that the current line is numbered with the actual, absolute line number.

Finally, I’ve rarely cared to know how far I am in the file, but I admit that sometimes it’s useful to get an idea of how much more of the file there is, usually in something like a configuration file. For the rare occasion in which this is useful, this information is a simple Ctrl-G away.

Signal

After taking all of this away, what remains is the file name, column number, git branch, and status indicators for whether or not the file is modified, read-only, or in paste mode.

The column number has become indispensable for those language compilers/interpreters that output the column number on which an error was found, such as Haskell’s GHC. In fact, I’m now constantly annoyed by those that don’t do this. Given an error on line 30 column 24, I go to the line number with :30 and the column with 24| and I’m instantly at the problem location.

The file name is of course useful, specifically shown relative to the current working directory. The git branch is also useful and provided by the fugitive plugin.

Finally, a select few markers are very useful, such as whether or not the file has been modified since the last write, if a file is read-only, and if one is in paste mode.

After removing what I didn’t need, I ended up with this very customized airline installation, where the number on the right side is the column number:

Redesign

This statusline was fine, but I couldn’t shake the thought that it felt out of place with those colors and arrows; it felt overdone. I ended up commenting out my airline configuration—which was already pretty long—and attempted to create a statusline from scratch with the goal of simplicity. I ended up with this, where the number on the left under the number line is the column number:

I completely got rid of any mode designation in the statusline since I realized that I had the showmode option set, which already shows the current mode in the message line under the statusline. Further still, I created mode-aware cursors that change color based on the mode using the gcr option. Top-to-bottom: normal, insert, visual, replace, command:

Inactive Statuslines

One piece of functionality that I wanted to preserve from airline was support for different active and inactive statuslines. In my case, I wanted this to be a subtle difference as you can see in the image below, where some things lose their color and the angle quotes become inverted:

This feature isn’t built into Vim, but it can be emulated by defining Vim auto commands on window focus events which refresh every window’s statusline:

function! s:RefreshStatus()
  for nr in range(1, winnr('$'))
    call setwinvar(nr, '&statusline', '%!Status(' . nr . ')')
  endfor
endfunction

augroup status
  autocmd!
  autocmd VimEnter,WinEnter,BufWinEnter * call <SID>RefreshStatus()
augroup END

The function that actually constructs the statusline is called Status. It takes an argument that corresponds to the window for which the statusline is to be constructed. I then detect whether the statusline being constructed is going to be active by checking it against the current/focused window:

let active = a:winnum == winnr()

I set the statusline using the %!SomeFunc() syntax so that the result of the function is the value that’s interpreted to construct the statusline. One problem is that when done this way, the function is evaluated in the context of the currently focused window and buffer, so if you use the value of &modified in your statusline somehow, and you have various windows but only the currently focused one is modified, all of the other windows will show the same modified marker, because the query as to whether or not they’re modified was done within the context of the focused window.

The documentation presented the solution to this problem:

Note that the “%!” expression is evaluated in the context of the current window and buffer, while %{} items are evaluated in the context of the window that the statusline belongs to.

Vim 7.4’s documentation

This means that if we want to perform behavior specific to the window or buffer for which the statusline is being created, we should wrap that in a %{} expression block:

let stat .= "%{&modified ? ' +' : ''}"

Defining a function to construct the statusline isn’t as complicated as it may seem. Once the boilerplate is defined, it’s a very simple way of constructing a statusline, and in my opinion more organized and manageable than typical, densely packed statusline declarations. For example, here’s the code for showing the file-modified marker, where Color is a helper function that conditionally colors the passed content based on whether or not the window is active:

let stat .= Color(isactive, 2, "%{&modified ? ' +' : ''}")

Here’s what the modified, paste mode, and read-only markers look like:

I didn’t want the read-only marker to be RO, so instead I got the inspiration from when one does sudo !! to re-do the previous command with sudo. In fact, I have a mapping for this that I found online, w!!:

cmap w!! %!sudo tee > /dev/null %

I finally had to redo the CtrlP theme from scratch as well, because I had previously customized it through airline’s API. This turned out to be pretty straightforward, since CtrlP itself exposes a function for this. With airline, I had this, which I specifically made to look like how it looked in powerline by default:

This actually looked pretty nice, in my opinion, but this look doesn’t mesh well with what I’ve created so far. I completely got rid of everything but the current mode and search scope path. Whether regex is turned on and/or if searching by file name is reflected in the CtrlP prompt already:

Conclusion

Like I said when I started out this post: there’s nothing wrong with using one of these statusline plugins. They certainly look nice and seem like a step up from the default statusline. It’s just that I took a step back and realized I didn’t need the things they offered, and they didn’t look as great as I had originally thought, back when I compared it to a default statusline.

For as much as some of these plugins claim to be “light,” it’s generally difficult to be lighter than using no plugin at all ¹. It’s not that this extra code will make your Vim unusably slow with today’s hardware, but that it seems pointless to carry around if you’re not even using most of it, especially considering how simple it seems to be to define your own statusline, and—if you care to—personalize your setup.

lightline is one statusline plugin I know of—but haven’t used—whose philosophy it is to have the user define as much of the statusline as possible in the hopes of remaining light. I haven’t used it, so I can’t comment on it, but in my case defining my own from scratch was simple enough already. That said, this might be useful for others who find themselves in a similar position but who would rather use something maintained by someone else. ↩︎

Custom XKB Options with Gnome

Fri, 08 Nov 2013 00:00:00 +0000

As I’ve explained before, I use a keyboard without media keys, so I rebind three otherwise-unused keys to act as my media keys. I accomplish this on Gnome by using a custom XKB option that I created. I have to do this because the Gnome keyboard settings binder ignores the keys I want to bind for this particular purpose. I used to enable this XKB option by simply running a script at startup that enabled it, but I noticed that it was subject to either a race condition or a precedence issue. I’m more convinced it was the latter, as if the option that I enabled was overridden by Gnome’s own initialization of XKB. This suspicion seems to be confirmed by the Arch wiki page on keyboard configuration:

This article describes low-level configuration using XKB which is effective in most cases, but some desktop environments like GNOME override it with its own settings.

Arch Wiki

As a result, I decided to look for a way to have my solution work alongside Gnome’s initialization of XKB. There was very little information on Gnome’s interaction with XKB short of reading Gnome’s source, but after searching around for a long time, I found that there is indeed a dconf option hidden away at /org/gnome/desktop/input-sources/xkb-options. Before I could use this though, I had to make my XKB option available system-wide.

First the symbol file has to be placed in the symbols directory /usr/share/X11/xkb/symbols. This file simply binds the keys I want to use to the XF86 designated media keys. This way the Gnome binder will pick them up without complaining when I bind them to the volume keys, since it otherwise seems to prevent the binding of these very keys as they are. In other words, with this symbol file, RCTRL will be interpreted by the system as if I had pressed XF86AudioRaiseVolume, which is what an actual volume-up media key would yield:

partial hidden modifier_keys
xkb_symbols "bottom_right" {
  key <RCTL> { [ XF86AudioRaiseVolume ] };
  key <MENU> { [ XF86AudioLowerVolume ] };
  key <RWIN> { [ XF86AudioMute ] };
};

Now an option has to be created for this symbol file and then listed in the system-wide options list for the rule set you’re using, in my case evdev, at /usr/share/X11/xkb/rules/evdev. Here, volume_keys corresponds to the filename of the symbol file, and bottom_right is the name I gave the group:

volume_keys:bottom_right = +volume_keys(bottom_right)

Now the option can be referenced system-wide, so it can be entered into the the Gnome XKB options dconf key at /org/gnome/desktop/input-sources/xkb-options. Put it inside the list as 'volume_keys:bottom_right', for example mine is set to:

['volume_keys:bottom_right', 'compose:ralt']

You can now go ahead and bind the appropriate keys to volume functions inside Gnome Setting’s keyboard section.

Given that it’s tucked away deep inside dconf, and given Gnome developers’ track record of pruning “cruft” or anything that the lowest common denominator user doesn’t use, the future availability of this option is in question. That said, it currently works perfectly fine.

Update: On archlinux, pacman -Qo shows that the evdev file is owned by the xkeyboard-config package. Whenever this package is updated, it overwrites this file, necessitating the change to be added once again. I’ll have to look into a more resilient way to have this setup.

Extra Dependencies in Hakyll

Fri, 28 Jun 2013 00:00:00 +0000

I use scss for my site’s stylesheets. scss is a language very similar to CSS that adds support for variables, nesting, mixins, selector inheritance, and more—while retaining a syntax very similar to CSS itself.

Split Stylesheets

A common practice I’ve noticed with the use of scss is to avoid having one monolithic stylesheet and instead opt to split it out into separate semantic files. For example, post.scss would concern styling for posts, syntax.scss would concern styling for Pygments syntax highlighting, etc. These files are then imported into one stylesheet, e.g., screen.scss, using the @import directive. It is this stylesheet that gets compiled by the scss compiler into the monolithic CSS.

Problem

In Hakyll, rules are generally designated by a pattern that matches a resource coupled with a route and a compiler. So this was the rule I originally had for scss/screen.scss:

match "scss/screen.scss" $ do
  route $ constRoute "css/screen.css"
  compile $ sassCompiler

The rule simply states that Hakyll should:

find the file scss/screen.scss
route it to css/screen.css
compile it using my custom sassCompiler.

This worked fine, but it meant that when I built or previewed the site, if I modified one of the split stylesheets, such as post.scss, it wouldn’t regenerate the monolithic stylesheet. It would only do so if scss/screen.scss itself was modified.

Solution

With the help of Hakyll’s creator, Jasper, I learned that the solution involves the use of makePatternDependency to create a Dependency from a given Pattern, and rulesExtraDependencies to associate the dependencies with a specific Compiler.

Advanced usage: add extra dependencies to compilers. Basically this is needed when you’re doing unsafe tricky stuff in the rules monad, but you still want correct builds.

A useful utility for this purpose is makePatternDependency.

Jasper on Hackage

Now when I’m previewing my site—or build the site in general—and I modify any scss file, it correctly regenerates the monolithic css/screen.css file. Here’s my new scss compiler rule:

match "scss/**.scss" $ do
  compile getResourceBody

scssDependencies <- makePatternDependency "scss/**.scss"
rulesExtraDependencies [scssDependencies] $ do
  create ["css/screen.css"] $ do
    route $ idRoute
    compile $ sassCompiler

Post Feed in Hakyll

Wed, 19 Jun 2013 00:00:00 +0000

When I made my site, specifically when I switched to Hakyll, I didn’t bother to include a syndication feed because I didn’t expect that anyone would care to want to subscribe to my site. However, someone filed an issue concerning this on github. I knew Hakyll exposed a module specifically for this: Hakyll.Web.Feed. It was more a matter of implementing it in a straightforward manner with the least duplication of work.

I’d like to subscribe to your blog, but I can’t seem to find an RSS feed (nor the Hakyll code to generate one). Would you consider adding one?

Nathan on Issue #1

Considerations

If I used my custom post compiler, it would include the table of contents and Pygments highlighted code. This was a problem because the table of contents didn’t work correctly in the feed reader I tested with and so just served to waste space. Worse, code blocks were completely absent from the feed reader. Finally, posts containing math type—which is rendered with MathJax on this site—did not render at all in the feed reader.

So it was obvious to me that I had to compile the posts meant for the syndication feed with a more vanilla Pandoc compiler. However, I did want to keep the abbreviation substitution filter as that seemed to work perfectly fine.

Because I needed to compile the posts with an entirely different Pandoc compiler, I knew that already I was duplicating some effort. Knowing this, I wanted to make sure to save as much work as possible to avoid further duplicate effort.

Drying Up

Since I wanted to use the abbreviation substitution filter in both the feed and regular post compiler, I knew that it was a potential location of duplicate effort. Both compilers would start something like this:

compile $ getResourceBody
  >>= withItemBody (abbreviationFilter)
  >>= pandocCompiler -- or pandocFeedCompiler

So it would have been preferable if I could save the state of the Item (i.e. post) as it was right after abbreviation substitution. Fortunately, Hakyll has support for this in the form of snapshots.

The solution was to save a snapshot of the post from the regular compiler for posts after it had been through the abbreviation substitution ¹:

match postsPattern $ do
  route $ niceRoute "posts/"
  compile $ getResourceBody
    >>= withItemBody (abbreviationFilter)
    >>= saveSnapshot "abbreviated"
    >>= pandocCompiler
    -- ...

Implementation

This meant that I could now refer to the “abbreviated” snapshot of any post. All I had to do now was to define a Rule to compile posts specifically for the syndication feed. Hakyll also has support for this in the form of versions, in which one can compile different versions of the same thing and refer to them later on.

So what I do in the “feed” version of the post compiler was to get the underlying Identifier for the given post and load the “abbreviated” snapshot of the version of that post that has no name, i.e. the version of the post compiled by the regular post compiler.

I then pass that snapshot to pandocFeedCompiler which is simply a more vanilla Pandoc compiler that removes the table of contents sentinel value I use, doesn’t generate the table of contents, doesn’t highlight code with Pygments, and uses regular superscripts etc. instead of MathJax:

match postsPattern $ version "feed" $
  compile $ do
    ident <- getUnderlying
    loadSnapshot (setVersion Nothing ident) "abbreviated"
      >>= makeItem . itemBody
      >>= pandocFeedCompiler

All that was left to do was to create the atom.xml file. An ephemeral Context is created to denote that the $description$ tag should be filled with the body of the post, as the syndication feed rendering functions in Hakyll expect. All “feed” versions of posts are loaded, sorted in reverse chronological order, and the first ten are taken. Finally the function renderAtom actually generates the XML from all of this information:

create ["atom.xml"] $ do
  route idRoute
  compile $ do
    let feedCtx = postCtx <> bodyField "description"
    posts <- fmap (take 10) . recentFirst
      =<< loadAll (postsPattern .&&. hasVersion "feed")
    renderAtom feedConf feedCtx posts

Caveat

Notice that we are using the “feed” versions of posts to render the syndication feed. This poses a problem, because the atom feed template requires access to the $url$ field, but notice that the “feed” version is not routed.

This means that a Route is not created for “feed” versions, and as a result the $url$ will be an empty string, so the link to individual stories in the feed will just link to the site root!

This becomes apparent when you look at the implementation of urlField, which is defined in Hakyll.Web.Template.Context:

urlField :: String -> Context a
urlField key = field key $
    fmap (maybe empty toUrl) . getRoute . itemIdentifier

First it gets the Item’s Identifier, and then it gets that Identifier’s Route. The problem is that since we’re using a different version of the post, the Identifier will be different, and there won’t be a Route associated with that Identifier.

I actually use a slightly different urlField-type function, which I called niceUrlField, it simply returns the URL without the index.html at the end. However, the solution to this problem is the same in both functions.

The solution is to get the no-name version of the Identifier that is retrieved, that is, the version of the post without an explicit version—the version that was compiled normally. This is done using the setVersion function. The function can be changed to this:

urlField' :: String -> Context a
urlField' key = field key $
    fmap (maybe empty toUrl) . getRoute . setVersion Nothing . itemIdentifier

This successfully retrieves the correct URL of the post, just make sure you mappend this alternate function in your feed’s Context.

Conclusion

It’s a shame that some duplicate work seems necessary when it comes to compiling the post. That is, I have to compile every post using my special Pandoc compiler, and then again using the more vanilla feed compiler I made. I tried to balance this by saving effort at the very least with the abbreviation substitution filter, so that it only runs once on every post.

If you’re wondering what postsPattern is, refer to my Drafts in Hakyll post, in which this value is used to determine from where to pull posts in, in order to facilitate a draft preview system. ↩︎

Drafts in Hakyll

Wed, 05 Jun 2013 00:00:00 +0000

In the post about my switch to Hakyll I talked about the various features I implemented in my Hakyll blog. One feature that was sorely missing was support for drafts: posts which aren’t supposed to be published when the site is deployed.

I usually take my time writing posts. Sometimes it can take me days, during which I might want to deploy other minor changes to the site, or perhaps even a shorter, quicker post. Without a draft system, I’m forced to manually move the draft post out of the provider directory so that it doesn’t get generated and subsequently deployed.

A draft system is able to clearly distinguish draft posts from regular posts. This way, when it comes to deploying the site, draft posts aren’t deployed along with it.

Considerations

I believe that the fundamental problem with draft systems in static site generators is that drafts, like regular posts, come to permeate the entire site. They accomplish this by showing up on index pages, tag pages, and any other place you might expect regular posts to show up in. This is something to keep in mind when creating a draft system because it means that simply deleting the compiled page won’t suffice, as there will still be traces in other pages.

One approach to this problem is to quarantine the draft posts such that they don’t show up on any of these things and instead only show them when you visit them directly. This is not an option for me because when I preview drafts I want to see how they will affect the entire site. I don’t preview drafts simply to check how my post is formatted.

Examples

The following two draft system implementations exemplify the two approaches I can think of for a draft system. These are approaches taken by static site generators. There are other ad hoc solutions, such as creating a separate drafts branch in git.

Octopress had support for drafts hacked onto Jekyll by way of a plugin that allowed a metadata field published to be set that, if set to false, would establish an environment variable that would be detected on site generation in order to regenerate the site without the draft posts. This consequently meant that draft posts were stored in the same directory as regular posts.

Jekyll implemented support for this in its 1.0 version by allowing a new directory, _drafts/, to store draft posts which could be previewed by specifying the --drafts flag to most operations. However, it was right after Jekyll 1.0 was released that I decided to switch to Hakyll.

Octopress’ draft system was pretty straightforward in my opinion, despite being a pretty hack-ish implementation. I would create drafts in the same directory as all of the other posts, and would simply set metadata published: false. This would allow the draft to show up when I previewed the site, but not when it was ultimately deployed. This was accomplished by regenerating the site on deploy, this time without the preview posts.

The other solution I could think of consisted of detecting when the site was being previewed, and if that were the case, establish a different output directory and a different posts pattern which would include the posts in a separate drafts/ directory. When the site wasn’t being previewed, the regular output directory would be used.

Implementation

Both approaches amount to hacks on top of Hakyll, but after some consideration, it seems to me that the second option is a lot less messy.

My solution consists of some code that runs before the Hakyll driver. The code extracts the first argument from the program arguments, which by convention is the action to perform, e.g. build, clean, preview, and checks to see if it’s the preview action.

main = do
  (action:_) <- getArgs

If the preview action is being run, the Hakyll configuration data structure’s destinationDirectory field, i.e. the output directory, is changed to a separate one for previewing purposes. This implies that the field is set to the deployable output directory by default. This is important because it means that all actions other than preview will ignore drafts.

Furthermore, if we are previewing, the pattern used to fetch posts is changed to also include the posts in the drafts/ directory. This is achieved by using the .||. function to compose two Pattern types.

  let previewMode  = action == "preview"
      hakyllConf   = if previewMode
                     then myHakyllConf { destinationDirectory = "generated/preview" }
                     else myHakyllConf
      postsPattern = if previewMode
                     then "posts/*" .||. "drafts/*"
                     else "posts/*"

Finally, we need to make one modification to an existing action. The clean action removes the provider, cache, and destination (output) directories. However, we now have two separate destination directories and by default every other action only knows of the deployable destination directory, i.e. the one without drafts. For this reason, we have to detect if the action being run is clean, and if so, remove the preview output directory.

Note that this depends on System.Directory.

  when (action == "clean") $ do
    putStrLn "Removing generated/preview..."
    removeDirectoryRecursive "generated/preview"

Now that this is done, you simply have to be sure to use the hakyllConf that was created above, as well as postsPattern wherever you would have simply put "posts/*" before. Two places that come to mind are tag generation and posts compilation:

  hakyllWith hakyllConf $ do
    tags <- buildTags postsPattern (fromCapture "tags/*.html")

    match postsPattern $ do
      -- etc.

Usage

This drafts system is pretty straightforward. When you run ./site preview it’ll serve the site with drafts as well. Deployment carries on as usual, i.e. you shouldn’t have to modify your deployment routine. With this system, you’ll never accidentally deploy drafts because they won’t ever show up in that output directory to begin with.

The Switch to Hakyll

Tue, 14 May 2013 00:00:00 +0000

This site was originally built with Jekyll. Technically I began with the pre-packaged distribution known as Octopress which offered a Rakefile for common tasks as well as an out-of-the-box directory structure. I didn’t use many of these features, however, so I had been wanting to shed traces of Octopress, partly motivated by the pursuit of increased speed in site generation. I found the opportunity to do this when Jekyll 1.0 was released recently.

To cut away the unnecessary components of Octopress, I decided to go through every file and keep only what I absolutely needed. This is evident in commits after 712168ec.

I was well on my way to making the site’s source a lot leaner when I remembered that I had been wanting to try Hakyll, a static site generator written in Haskell that I had heard about on Hacker News. Given that I was more or less starting the site from scratch, I figured it was the perfect opportunity to try it.

Ultimately, this site is now compiled with Hakyll. It took me about a week to implement every feature I wanted in Hakyll and Pandoc. The net effect is that the difference in speed and flexibility is highly appreciable.

File Structure

Oftentimes when new to a system, learning its directory structure can help one to get oriented. Unlike some other static site generators, Hakyll does not enforce any particular directory structure or convention. The one I have adopted for my repository looks like this:

Entry	Purpose
provider/	compilable content
src/	Hakyll, Pandoc customizations
Setup.hs	build type
blaenk.cabal	dependency management
readme.markdown	repository information

I build the site binary with cabal build which results in a new top-level directory dist/, which stores the object files generated by GHC. The site binary, stored at the top level, is the actual binary which is used for generating and manipulating the site. This binary has a variety of options, the ones I commonly use are:

Option	Purpose
build	Generate the entire site
preview	Generate changes on-the-fly and serve them on a preview server
deploy	Deploy the site using a custom deploy procedure

Build creates a top-level directory generated/ with two sub-directories: a directory cache/ for cached content and a directory site/ where the compiled site is stored.

Deploy puts the compiled site into top-level directory deploy/ which is git-controlled and force pushes the content to the master branch, effectively deploying (on GitHub).

Hakyll

As I mentioned earlier, Hakyll is a static site generator written in Haskell. Hakyll sites are fleshed out using a Haskell Embedded Domain Specific Language (EDSL). This EDSL is used to declare rules for different patterns which should be searched for within the provider directory and what should be done with them.

For example, in the following Hakyll program:

main :: IO ()
main = hakyll $ do
  match "images/*" $ do
      route   idRoute
      compile copyFileCompiler

match "images/*" is a Rule that states that the provider directory should match all files matching the glob images/*, Route them using the idRoute, and compile them using the Compiler copyFileCompiler.

Routing a file in the context of a static site generator like Hakyll refers to the mapping between the file as it sits in the provider directory and its name/path in the compiled directory; in this case, idRoute keeps the same name/path in the compiled directory.

Compiling a file in this context refers to the operations that should be performed on the contents of the file, for example processing through Pandoc for Markdown to HTML generation, or in this case, simply copying the file from the provider directory to the compiled directory.

Compiler is a Monad, which allows for seamless chaining of operations that should be performed on any given file. For example, here is my Rule for regular posts:

match "posts/*" $ do
  route $ nicePostRoute
  compile $ getResourceBody
    >>= withItemBody (abbreviationFilter)
    >>= pandocCompiler
    >>= loadAndApplyTemplate "templates/post.html" (tagsCtx tags <> postCtx)
    >>= loadAndApplyTemplate "templates/layout.html" postCtx

This states that the compilation process for any given post is as follows:

the post body (i.e. excluding post metadata) is read
the result is passed to an abbreviation substitution filter
the result is passed to my custom Pandoc compiler
the result is embedded into a post template with a so called “post context”
the result is embedded into the page layout

A post is routed using the nicePostRoute function which is largely borrowed from Yann Esposito. It simply routes a posts/this-post.markdown to posts/this-post/index.html so that the post can be viewed at posts/this-post/.

An interesting thing to note is that when templates are applied, they are supplied a Context. A Context is simply a Monoid that encapsulates a key (i.e. String identifier for the field) and an Item. During application of the template, if a field of the form \$key\$ is encountered, the supplied Context is searched for an appropriate handler (i.e. one with the same key). If one is found, the item is passed to that Context’s handler and the result is substituted into the template.

In the above Rule for posts, I pass a pre-crafted post Context, postCtx, and mappend to it a special tags context, tagsCtx which encapsulates tags information for that post.

SCSS

The first customization I made was to allow support for SCSS. This is usually possible with a simple line:

getResourceString >>= withItemBody (unixFilter "sass" ["-s", "--scss"])

This works fine in POSIX environments, of which Linux is my primary environment for development. However, it’s very useful to me to have Windows support as well. The problem is that on Windows, ruby gem binaries—such as scss—are implemented using batch file stubs. The underlying function used for creating the process in unixFilter is System.Process’ createProcess, specifically with the proc type. On Windows, this uses the CreateProcess function. Using this function, batch files are not run unless they are run explicitly with cmd.exe /c batchfile. The problem is that there is no simple way to find the file path of the batch file stub for scss.

The solution to this is to use the shell type with createProcess instead of proc. This has the effect of a system call, where the parameter is interpreted by the shell, in Windows’ case, cmd.exe. As a result, the program can simply be called as scss, leaving the shell to automatically run the appropriate batch file stub.

To accomplish this, I had to implement what was essentially a mirror copy of Hakyll.Core.UnixFilter with proc switched out with shell. I’ll be suggesting a pull request upstream soon which gives the user the option and removes the duplicate code. Now I can implement an SCSS compiler like the following, though I additionally pass it a few extra parameters in my actual implementation:

getResourceString >>= withItemBody (shellFilter "sass -s --scss")

Abbreviations

One feature I missed from kramdown that wasn’t available in my new markdown processor, Pandoc, was abbreviation substitution. It consists of writing abbreviation definitions which are then used to turn every occurrence of the abbreviation into a proper abbr HTML tag with an accompanying tooltip consisting of the definition.

I had hardly used regular expressions in Haskell before, so the method of using it was pretty confusing to me at first. There’s a base regex package called regex-base which exposes a common interface API, and then there are a variety of backend implementations. Hakyll happens to use regex-tdfa, a fast and popular backend, so I decided to use that one instead of introducing additional dependencies.

One way of using regular expressions in Haskell is through type inference, as is described in the Text.Regex.Base.Context documentation:

This module name is Context because they [sic] operators are context dependent: use them in a context that expects an Int and you get a count of matches, use them in a Bool context and get True if there is a match, etc.

Keeping this in mind, I explicitly annotated the [[String]] type since I wanted every match and sub-match. I created a function abbreviationReplace that takes a String, removes the abbreviation definitions, and then creates abbr tags out of every occurrence of the abbreviation using the parsed definitions.

The abbreviationReplace function begins like this:

abbreviationReplace :: String -> String
abbreviationReplace body =
  let pat = "^\\*\\[(.+)\\]: (.+)$" :: String
      found = body =~ pat :: [[String]]

Git Tag

In a previous post I talked about a liquid tag I created for Jekyll which inserts the SHA of the commit on which the site was last generated. I have come to like this small feature of my site. It’s not some tacky “Powered by blah” footer. It’s pretty unobtrusive. It seems unimportant to people who wouldn’t understand what it’s about, and those who would understand it might immediately recognize its purpose.

Update: I have stopped including the git commit in the footer of every page. The problem with doing this was that, in order to have every page reflect the new commit, I had to regenerate every page before deploy. This obviously doesn’t scale well once more and more pages are added to the site. Instead I have adopted a per-post commit and history link which I believe is a lot more meaningful and meshes perfectly well with generation of pages, i.e. if a post is modified, there’ll be a commit made for it and since it was modified it will have to be regenerated anyways. Now I simply include social links in the footer.

One thing I forgot to update the previous post about was that I ended up switching from using the Rugged git-bindings for Ruby to just using straight up commands and reading their output. The reason for doing this was that, while everything worked perfectly fine on Linux, Rugged had problems building on Windows. It turned out that taking this approach ended up being simpler and had the added benefit of decreasing my dependencies.

The equivalent of a liquid tag in Jekyll would be a field, expressed as a Context. For this reason I created the gitTag function that takes a desired key, such as git, which would be used as \$git\$ in templates, and returns a Context which returns the String of formatted HTML. One problem was that to do this I had to use IO, so I needed some way to escape the Compiler Monad. It turned out that Hakyll already had a function for something like this called unsafeCompiler, which it uses for UnixFilter for example.

Here’s what gitTag looks like:

gitTag :: String -> Context String
gitTag key = field key $ \_ -> do
  unsafeCompiler $ do
    sha <- readProcess "git" ["log", "-1", "HEAD", "--pretty=format:%H"] []
    message <- readProcess "git" ["log", "-1", "HEAD", "--pretty=format:%s"] []
    return ("<a href=\"https://github.com/blaenk/blaenk.github.io/commit/" ++ sha ++
           "\" title=\"" ++ message ++ "\">" ++ (take 8 sha) ++ "</a>")

Pandoc

Hakyll configuration is fairly straightforward. What took longer was the process of re-implementing some features that I had in kramdown when I used Jekyll that weren’t available in my new document processor, Pandoc.

Pandoc is a very interesting project that basically works by parsing input documents into a common intermediate form represented as an abstract syntax tree (AST). This AST can then be used to generate an output document in a variety of formats. In this spirit, I feel it’s a lot like the LLVM project. It seems to me that it has been gaining popularity especially from an end-user perspective (i.e. using the pandoc binary), commonly used to do things such as write manual pages in markdown or generate ebooks.

The very nature of how Pandoc transforms input documents into an AST lends itself to straight-forward AST transformations. I have created two such transformations so far: one for Pygments syntax-highlighting and another for fancy table of contents generation.

One of the things I needed to implement, however, was the abbreviation substitution described above. I would have implemented it as a Pandoc customization, but Pandoc has no representation for abbreviations in its abstract syntax tree. This was why I implemented it as a Hakyll compiler instead, using simple regular expressions.

There is actually work towards implementing abbreviation substitution according to the readme under the section “Extension: abbrevations” [sic] but it says:

Note that the pandoc document model does not support abbreviations, so if this extension is enabled, abbreviation keys are simply skipped (as opposed to being parsed as paragraphs).

Pygments

Update: This has been through two redesigns since this was written. The first involved an fs-backed caching system, but this was still too slow, since the bottleneck seemed to be caused by continuously spawning a new pygmentize process. Most recently I’ve created a pygments server that the site opens alongside it at launch, and this Pandoc AST transformer communicates with it through its stdout/stdin handles. It works perfectly and the site compiles a lot quicker. It also fully supports UTF-8:

¥ · £ · € · $ · ¢ · ₡ · ₢ · ₣ · ₤ · ₥ · ₦ · ₧ · ₨ · ₩ · ₪ · ₫ · ₭ · ₮ · ₯ · ₹

One of the first things I wanted to implement right away was syntax highlighting with Pygments. There are a variety of options for syntax highlighting. In fact, Pandoc comes with support for kate: a Haskell package for syntax highlighting written by the author of Pandoc. However, I don’t find it to be on par with Pygments. In the past, I simply posted code to gist and then embedded it into posts. This caused unnecessary overhead and more importantly, would break my site when github made changes to the service.

Eventually I realized that github just uses Pygments underneath, so I implemented a Pandoc AST transformer that finds every CodeBlock, extracts the code within it, passes it to Pygments, and replaces that CodeBlock with a RawBlock containing the raw HTML output by Pygments. I also implemented a way to specify an optional caption which is shown under the code block. I use blaze-html for the parts where I need to hand-craft HTML.

Ultimately, this all means that I can write code blocks like this in markdown:

``` haskell
testFunction :: String -> Integer
```

Or, with a caption:

``` ruby: some caption
args.map! {|arg| arg.upcase}
```

One thing I had to do was invoke unsafePerformIO in the function I created which runs the code through pygmentize, an end-user binary for the Pygments library. I’m not sure if there’s a better way to do this, but my justification for using it is that Pygments should return the same output for any given input. If it doesn’t, then there are probably larger problems.

pygmentize :: String -> String -> String
pygmentize lang contents = unsafePerformIO $ do

I don’t feel particularly worried about it, given my justification. It’s a similar justification used by Real World Haskell when creating bindings for PCRE with the foreign function interface:

It lets us say to the compiler, “I know what I’m doing - this code really is pure”. For regular expression compilation, we know this to be the case: given the same pattern, we should get the same regular expression matcher every time. However, proving that to the compiler is beyond the Haskell type system, so we’re forced to assert that this code is pure.

This is what the AST transformer I wrote looks like:

pygments :: Block -> Block
pygments (CodeBlock (_, _, namevals) contents) =
  let lang = fromMaybe "text" $ lookup "lang" namevals
      text = fromMaybe "" $ lookup "text" namevals
      colored = renderHtml $ H.div ! A.class_ "code-container" $ do
                  preEscapedToHtml $ pygmentize lang contents
      caption = if text /= ""
                then renderHtml $ H.figcaption $ H.span $ H.toHtml text
                else ""
      composed = renderHtml $ H.figure ! A.class_ "code" $ do
                   preEscapedToHtml $ colored ++ caption
  in RawBlock "html" composed
pygments x = x

The more sophisticated and complex of the AST transformers I wrote for Pandoc is table of contents generation. This is something that kramdown had out of the box, though not as fancy. Paired with automatic id generation for headers, this meant that simply placing {:toc} in my page would replace that with automatically generated table of contents based on the headers used in the page.

Alternatives

Pandoc actually does have support for table of contents generation using the --toc flag. In fact, Julien Tanguy recently devised a way to generate a separate version of every post which only included the table of contents, then re-introduced the table of contents as a Context field $toc$ .

I actually tried this approach, along with a metadata field that decided if the table of contents should be included in a given post or page. However, I ended up deciding against using it. One advantage would be that it took less code on my end, and possibly I would avoid re-inventing the wheel. One reason I didn’t keep it was because there was a tiny increase in compilation time which I fear might accumulate in the future as the number of posts grow. The reason for this is that the table of contents is generated for every post/page, instead of only the ones that should display it.

Another reason was that it would require me to implement the fancy section numbering in JavaScript, which I don’t think would be too difficult since in this case the table of contents already exists and I simply need to insert my numbering. The main reason I decided against it, along with the previous two reasons, is that there would be a noticeable delay between the time when the table of contents are shown plainly and when they are transformed into my custom table of contents.

Implementation

Implementing this involved many steps. In general terms, I had to make a pass through the document to collect all of the headers, then I had to make another pass to find a special sentinel marker I would manually place in the document to replace it with the generated table of contents. This effectively makes table of contents generation a two-pass transformer.

Gathering all of the headers and their accompanying information, i.e. HTML id, text, level, proved to be a pretty straight-forward task using queryWith from the pandoc-types package:

queryWith :: (Data a, Monoid b, Data c) => (a -> b) -> c -> b
-- Runs a query on matching a elements in a c.
-- The results of the queries are combined using mappend.

Once I collect all of the Header items’ information, I normalize them by finding the smallest header level (i.e. big header) and normalizing all headers based on that level. That is, if smallest header level is 3 (i.e. h3), every header gets its level subtracted by 2 so that all headers are level 1 and above. Note that I’m not actually modifying the headers in the document, just the information about them that I’ve collected.

Next, a Data.Tree is constructed out of the headers which automatically encodes the nesting of the headers. This is done by exploiting groupBy by passing it < as an equivalence predicate:

tocTree :: [TocItem] -> Forest TocItem
tocTree = map (\(x:xs) -> Node x (tocTree xs)) . groupBy (comp)
  where comp (TocItem a _ _) (TocItem b _ _) = a < b

This Tree is finally passed to a recursive function that folds every level of the Tree—known as a Forest—into a numbered, unordered list. While that may sound like an oxymoron, the point is that I wanted to have nested numbering in my table of contents. For this reason, I create an unordered list with a span containing the section number concatenated to the parent’s section number. This function generates the HTML.

The final problem was finding a way to insert the table of contents on-demand, in a location of my choosing. In kramdown, this is achieved by writing {:toc}, which gets substituted with the table of contents. Pandoc has no such thing, however. For this reason, I chose a list with a single item, “toc,” as the place holder for the table of contents. This means that I write the following wherever I want the table of contents to show up:

* toc

You can take a look at the beginning of this post to see what the generated table of contents looks like, especially the nested numbering I was referring to.

Deploying

I host my site using GitHub Pages. Such sites are deployed by pushing the site to the master branch of the repository. I wrote a quick shell script that accomplishes this in a pretty straightforward manner. It creates a git ignored directory, deploy/, which itself is under git control, associated with the same repository, but its master branch instead.

When I deploy the site with ./site deploy, the contents of deploy/ are removed—except for the .git/ directory—and then all of the new generated files are copied into it. A commit is then generated for the deployment, tagged with the SHA identifier of the commit from which the site was generated, to make it easy for me to track things down sometimes. An eight character, truncated SHA is used as follows:

COMMIT=$(git log -1 HEAD --pretty=format:%H)
SHA=${COMMIT:0:8}
git commit -m "generated from $SHA" -q

Finally, the commit is force pushed to the repository, replacing everything already there, effectively deploying the site.

Conclusion

Preliminary migration to Hakyll was pretty quick. This included porting all of my posts, pages, and other assets to the Hakyll and Pandoc Markdown formats. The rest of the week was spent implementing the various features, some outlined above, and refining the code base.

At first I was a little rusty with my Haskell and found myself at odds with the seemingly capricious compiler, trying to find one way or another to appease it. I quickly remembered that patience prevailed when concerning Haskell, and eventually came to really enjoy reasoning out the problems and solving them with Haskell.

The site binary which is in charge of generation, previewing, etc. is compiled. Once you have configured Hakyll to your liking, you have a very fast binary, especially compared to other site generators which are known not to scale well with the amount of posts. The Compiler Monad in Hakyll takes care of dependency tracking, allowing re-generation of only those items which are affected by those which were changed, instead of the whole site.

But perhaps my favorite aspect of Hakyll is that it’s more like a library for static site generation which you use as you see fit, and as a result, your site is entirely customizable.

xmonad Ignores Bindings

Sun, 24 Feb 2013 00:00:00 +0000

In my previous post I talked about how I spent a while configuring my system, specifically urxvt and zsh, in preparation for setting up xmonad. I’ve finally gotten around to setting up xmonad. One problem in particular stopped me from continuing with the rest of the configuration.

Update: Shortly after posting in the issue tracker entry for this issue relating my experience and affirming that the proposed patch fixed the problem, the gracious developers merged the patch into the main tree. This problem should no longer affect anyone!

Media Keys

I have a regular keyboard layout, Das Keyboard Model S Ultimate, which lacks media keys (i.e. volume up, down, etc). This wasn’t too much of a problem when I used headsets because most of them have dedicated volume controls. However, I got tired of headsets being rendered useless when any little thing messed up (e.g. microphone, a speaker, etc).

As a result I ended up buying a cheap standalone mic and now use my iPhone’s Shure SE215-K earbuds for sound on my computer. This is very easy to do given my computer case’ front panel audio connector. Of course, the problem now is that there aren’t any dedicated media keys and having to use a GUI to change the volume is cumbersome.

My solution to this problem in Windows and Mac is to bind the bottom right keys to media keys as follows:

Key	Purpose
Right Control	Volume Up
Menu	Volume Down
Right Windows	Volume Mute

Binding

Creating these binds is possible on Windows via a registry hack, facilitated using a program such as SharpKeys.

On Linux I initially did this using xmodmap:

remove Control = Control_R
keycode 105 = XF86AudioRaiseVolume
add Control = Control_R

keycode 135 = XF86AudioLowerVolume

remove mod4 = Super_R
keycode 134 = XF86AudioMute
add mod4 = Super_R

Binding to these XF86Audio* keys automatically adds support for these keys in different applications like mplayer2, but I wanted system-wide volume support. This is typically accomplished by wiring them up in your given Desktop Environment or Window Manager. So I went ahead and did so in xmonad.hs:

((0, xF86XK_AudioMute), spawn "amixer -q set Master,0 toggle"),
((0, xF86XK_AudioLowerVolume), spawn "amixer -q set Master,0 5%- unmute"),
((0, xF86XK_AudioRaiseVolume), spawn "amixer -q set Master,0 5%+ unmute")

The Problem

The problem was that xmonad would only react to the Right Control key (Volume Up). However, xev correctly interpreted the keys as having been bound to the XF86Audio* keys. I was really confused as to why the binds apparently did work at the system level but only one of them worked at the window manager level.

To rule out that it wasn’t something with the system-level (xmodmap) binds, I decided to check if it worked in Awesome:

awful.key({}, "XF86AudioLowerVolume", function () awful.util.spawn("amixer -q set Master,0 5%- unmute", false) end),
awful.key({}, "XF86AudioRaiseVolume", function () awful.util.spawn("amixer -q set Master,0 5%+ unmute", false) end),
awful.key({}, "XF86AudioMute", function () awful.util.spawn("amixer set Master,0 toggle", false) end),

Indeed it worked perfectly. So now I had narrowed down the problem to xmonad.

Bug Hunting

Eventually I decided to stop by #xmonad on freenode. There I found Paul Fertser who spent the next ~6 hours helping me track down what he figured to be a bug in xmonad. I told him that the system-level binds did work, but not in xmonad. I showed him my binds using xmodmap -pke.

He noticed that the XF86Audio* keys were bound twice: once by default by XKB (xmodmap’s more modern replacement) bound to the keycodes I would have if my keyboard had media keys, and bound again to the keys I chose (the bottom right keys). He then hypothesized that xmonad wasn’t grabbing the keys at all due to Xlib limitations. Specifically, the XKeysymToKeycode function only returns one keycode per key, biased towards lower keycodes, presumably due to an increasing iterative search of the keycodes for a match.

This theory accounted for why the Right Control (Volume Up) bind did work and not the others. What happened was that Right Control’s keycode was lower than the duplicate bind’s keycode. As a result, when xmonad used XKeysymToKeycode it retrieved the correct keycode. The other two binds, however, have higher keycodes than the default-bound ones, and so XKeysymToKeycode returned the first (lower) keycode it found and as a result xmonad never even knew of the other binds’ existence.

To test this theory, Paul had me run ltrace on xmonad to see which keys xmonad grabbed. The output of this clearly showed that xmonad only grabbed the keys with the lower keycodes.

Workaround

Now that we were pretty sure of the cause of this, the workaround was to remove the other keycodes (for keys I didn’t even have on my keyboard). At this time I decided I might as well switch over to XKB. The first order of business was to dump my XKB map:

$ setxkbmap -print > ~/.xkb/keymap/mymap

Then I created a ~/.xkb/symbols/volume_keys file to store my media key binds. It took me a long while to figure out how to remove/unbind the default-bound keys. One problem was that XKB sets different aliases for keys. For example, <I0D> (I guess that’s a media key) was aliased to <MUTE>. I looked around in /usr/share/X11/xkb/rules/evdev to see what was aliased and made sure to unbind those too. As for unbinding, at first Paul suggested to bind the keys to NoSymbol but that apparently had no effect. Eventually I found out it was possible with VoidSymbol.

partial modifier_keys
xkb_symbols "volume_keys" {
  // mute
  replace key <MUTE> { [ VoidSymbol ] };
  replace key <I0D> { [ VoidSymbol ] };

  // lower volume
  replace key <VOL-> { [ VoidSymbol ] };
  replace key <I0E> { [ VoidSymbol ] };

  // raise volume
  replace key <VOL+> { [ VoidSymbol ] };
  replace key <I0F> { [ VoidSymbol ] };

  replace key <RCTL> { [ XF86AudioRaiseVolume ] };
  replace key <MENU> { [ XF86AudioLowerVolume ] };
  replace key <RWIN> { [ XF86AudioMute ] };
  replace key <RALT> { [ Multi_key ] };
};

Now I loaded my XKB map in ~/.xinitrc:

$ xkbcomp -I$HOME/.xkb ~/.xkb/keymap/mymap $DISPLAY

I restarted xmonad with Mod-Shift-Q (so that ~/.xinitrc is rerun) and everything now worked perfectly.

Bug Report

Over the course of my transition to XKB, Paul found that there was already an issue opened back in 2009 concerning this. The issue report has a patch attached that fixes this, but the patch has yet to be applied to xmonad. Paul suggested I try the patch myself and communicate my results back to the issue report. So I went ahead and got xmonad and xmonadContrib from the darcs repository, ran a simple darcs apply keycode.dpatch, and installed each with a --prefix to prevent clashing with the ones already installed with pacman. Indeed, the patch worked perfectly.

Terminal Customization

Tue, 12 Feb 2013 00:00:00 +0000

A while back I switched over to zsh as my shell and used oh-my-zsh to get up and running quickly. I barely used any of the features it provided, so I recently decided to do away with it and get zsh setup from scratch. At the same time I decided it’d be a good idea to do the same for urxvt. These initiatives had the consequence that I ended up completely redoing the way I maintained my dotfiles which had the effect of greatly improving my overall setup.

Before I go any further I’d like to point out that all of the things that I’ll talk about in this post are available in my dotfiles repository. I’ll make an effort to link to the relevant individual files from the repository for each topic I cover.

Here is the end result:

dotfiles

I previously had a simple Rakefile that symlinked all of the files in the dotfiles directory into my home directory, except for some in an exception list. However, this had the consequence that I had to have ruby installed beforehand, and I didn’t like to install ruby through means other than something like rbenv. I preferred instead to be able to get my dotfiles up and running as soon as possible on a new system. As a result I opted to use a shell script to deploy my dotfiles.

After looking around in dotfile repositories I found hoffman’s bootstrap script. The simple script uses find to find files and directories whose names end in “.ln” and symlinks them into the home directory. I modified it a bit to use the “.ln” suffix instead of “.symlink”—purely cosmetic of course—as well as some other slight changes. I think I’ll change it later so that it can gracefully handle operating system-dependent dotfiles.

urxvt

My first goal was to get urxvt configured properly. I really didn’t like the way stock urxvt looked and operated (e.g. clipboard use), so I set out to learn its configuration format. I ended up defining my own color scheme as well as improving its clipboard support.

Colors

I initially attempted to replicate the color scheme I used in the OS X terminal, however I found that the very same colors didn’t look quite the same in the terminals I tried on Linux (GNOME’s or urxvt). I have an IPS monitor which I think has made me pretty sensitive to color. As a result I decided to tweak it a little, and I feel that I’ve come up with an even better color scheme than before by taking some of the colors from my vim theme.

I’ve come to recognize recently that it’s pretty easy to go overboard with the amount of colors used in anything, and that oftentimes things tend to look better with a more restricted color palette. My terminal uses more color than your common terminal or prompt, but I think it’s all in good taste and for semantic purposes. In fact, I’ve come to really like the color scheme I chose, it somehow reminds me of SNES game color palettes.

Clipboard

Stock urxvt uses the X Window Selection copy and paste mechanism. Paste can be done by middle clicking, and copying is on-select. I found a set of scripts that take the copy and paste system from barebones to awesome.

The clipboard script allows me to copy and paste using the Alt-keys, as in OS X where one can use the Cmd-keys to copy and paste. These are some nice binds to use without interfering with the terminal by sending signals to the current program.

The keyboard-select script allows me to go into “visual mode” on the terminal and use vi-bindings to do my copying. This lets me copy text from the terminal without having to leave the keyboard. This paired with my zsh vi-bindings means I never have to leave the home row.

zsh

This was the bulk of the work, but I quickly realized that it wasn’t all that difficult. I didn’t want to have one huge zshrc file. Instead I wanted to have specific files for different parts of the configuration, for example prompt.zsh would contain configuration for the prompt. I had seen such systems in oh-my-zsh as well as certain peoples’ dotfile repositories. I took inspiration from sunaku’s dotfiles in which he has a zsh file that sources all of the zsh files in a directory.

Prompt

My zsh prompt is actually pretty simple. I played around with multiline prompts but I really disliked the feel of them. The zsh file dedicated to defining the prompt is very clean in my opinion, which is something I strived for throughout its development.

Basic

The basic prompt consists of a lambda followed by the path—which auto-collapses $HOME to ~—with highlighted path separators followed by a right arrow. Pretty simple and easy on the colors in my opinion, aside from the unconventional highlighting of the path separators, which was something I had long wanted and took me a while to get right.

My prompt is inspired by the Haskell lambda syntax so that it looks like every line is a lambda operating on the current directory whose code is the command you type:

\dir -> command_in(dir)

Here’s what the actual prompt looks like:

Here it is in text:

λ ~/code/haskell ➜

Update: I have changed my prompt to be multiline. All that has changed is that the ➜ is on its own line.

git

Like any self-respecting modern prompt, mine incorporates git information when within a git repository. It shows the current branch, whether there are any untracked files (denoted by .), modified files (denoted by #), or staged files (denoted by +). Another nice little thing I added was a marker for how far ahead and/or behind we are from the remote branch. This is appended to the aforementioned information and only shows up when we are ahead or behind.

All of these features were done using zsh’s vcsinfo with the help of Arjan van der Gaag and, once again, sunaku. Here’s what it looks like ¹:

Again in text:

λ ~/.dots (master . # +){+1} ➜

The branch is master and apparently there’s unstaged files, modifed files, and staged files. Furthermore, the {+1} shows that my branch is one commit ahead of the remote branch. It would also show the number of commits we are behind, in red, if that were the case—which in my opinion can be very handy so that you can avoid conflicts by rebasing or merging before you push. This segment only shows up if either of those conditions is met.

SSH

The prompt also detects if it’s being viewed through SSH. I don’t like viewing hostname in my prompt on machines I’m on locally as I feel it’s pointless. However, when I’m connected to a remote server via SSH, it’s often handy to have the hostname around to differentiate between your computer and the remote host. For this reason my prompt only shows the hostname when it detects that it’s being viewed through an SSH connection. Here’s what it looks like:

Once again in text:

[someserver] λ ~/.dots (master) ➜

Update: October 28, 2013

I’ve actually done away with this component of the prompt. I didn’t like how the lambda no longer aligned with the arrow symbol. I’ve instead decided to add a pretty simple green R at the end of the current working path, to signify that I am on a “remote” machine, so it reads something like “currently on x path remotely.”

λ ~/.dots R (master)
➜

vi-Binds

One thing that I can’t live without now when using vi-bindings is binding jj to vi-mode. The default key for this is Escape, but Escape is used for other hotkeys that the terminal (or shell?) intercepts. For this reason, a single keypress of Escape introduces a bit of lag, which I imagine is required to differentiate a hotkey (i.e. Esc-C) from a simple Escape keypress.

Binding to jj has the consequence of being more accessible. In fact, this is a common bind that people tend to use in vim for this very reason. Before coming to this realization I mainly used Emacs-binds because they didn’t introduce lag. However, with this new bind I’m able to jump into vi-mode and edit commands very quickly and more intuitively (for a vim user like myself).

Highlighting

If you’ve been wondering how it is that my commands are highlighted, it’s made possible by this highlighting script. Simply sourcing that script into your zsh environment suddenly colors your commands. In my opinion it makes the terminal look much better without going overboard with the colors. I did have to tweak some settings though because I felt that the default configuration did go a bit overboard on the styling, such as underlining program names.

Conclusion

I have to say that I love the way my terminal looks and operates now. This has all been in preparation for setting up xmonad, which I intend to take a shot at soon. For the curious, I’m doing all of this on arch linux, but I didn’t say it earlier because nothing in this post depends on this. In fact, none of the content in this post is all that specific to Linux itself even. I’ll try to update this post whenever my setup changes, but I think I’m quite comfortable with the way it is, and will be for the foreseeable future.

In my opinion, the benefit of using such tried and proven, mature tools is that once you take the time to configure them to your liking you can use them without modification for a long time. Barring some imminent, majorly disruptive paradigm shift in software development, I can see myself using many of these tools throughout my career as a software developer.

Ignore the echo command. I used it to give the current line some padding to make taking a screenshot a little bit easier. ↩︎

Naive Convolution in Haskell

Fri, 04 Jan 2013 00:00:00 +0000

Convolution is a mathematical method of combining two signals to form a third signal. Passing the Dirac delta function (unit impulse) $\delta[n]$ through a linear system results in the impulse response $h[n]$ . The impulse response is simply the signal resulting from passing the unit impulse (Dirac delta function) through a linear system.

Principle

The properties of homogeneity and shift-invariance in Linear Time-Invariant System Theory hold that scaling and shifting an input signal in a linear system results in the same scaling and shifting in the output signal. Because of these properties, we can represent any impulse as a shifted and scaled delta function and consequently know what the impulse response, i.e. output signal in response to an impulse input signal, will be for that scaled and shifted impulse.

An impulse of $-3$ at the $8^{th}$ sample would be represented as a unit impulse $h$ by scaling the delta function $\delta$ by $-3$ and shifting it to the right by $8$ samples: $-3\delta[n-8]$ , where $n-8$ means the $8^{th}$ sample is now the $0^{th}$ . Due to homogeneity and shift invariance, we can determine the impulse response of this impulse by simply scaling and shifting the unit impulse response in the same manner. In other words:

$$-3\delta[n-8] \mapsto -3h[n-8]$$

What this means is that if we know the unit impulse response of a system, we consequently know how the system will react to any impulse, not just a unit impulse. These impulse responses can then be synthesized to form the output signal that would result from running the input signal through the actual system. An example of the powerful implications of this property is convolution reverb, in which an impulse response of a physical or virtual space is generated and then convolved with any input signal to simulate the effect of reverberation in that space.

In short, the input signal convolved with the unit impulse response results in the output signal. Convolution of input signal $x[n]$ with unit impulse $h[n]$ to generate output signal $y[n]$ is denoted as:

$$x[n] * h[n] = y[n]$$

Since convolution allows us to go from input signal $x[n]$ to output signal $y[n]$ , we can conclude that convolution involves the generation of the impulse response for each impulse in the input signal as decomposed by impulse decomposition, as well as the subsequent synthesis of each impulse response, to generate the output signal.

Definition

Convolution can be described by the so-called convolution summation. The convolution summation is pretty simple, and is defined as follows:

$$y[i] = \sum_{j=0}^{M-1} h[j]\ x[i-j]$$

Where the length of the output signal $y[n]$ is defined as $M + N - 1$ where $M$ is the length of the unit impulse response and $N$ is the length of the input signal.

All this says is that a given sample $y[i]$ in the output signal $y[n]$ is determined by the summation of every $i^{th}$ sample in every resultant impulse response. In effect, the summation above encodes how different samples in the resulting impulse responses contribute to a single output sample.

Natural imperative instinct might lead you to conclude that this can be easily implemented using nested iterations and arrays:

const int outputLength = M + N - 1;
int *y = new int[outputLength]();

for (int i = 0; i < outputLength; ++i) {
  for (int j = 0; j < M; ++j) {
    if (i - j >= 0)
      y[i] += x[i - j] * h[j];
  }
}

But wait up! We are using Haskell, a functional programming language which typically does without both arrays and iteration. This means that to implement convolution in Haskell without the use of Arrays or imperative iteration loops, we need to really understand the operation occurring in the convolution summation.

Convolution Machine

The book The Scientist and Engineer’s Guide to Digital Signal Processing uses a metaphor known as the Convolution Machine to help conceptualize the convolution operation at a granular level. The convolution machine is simply a theoretical machine in which the unit impulse response is:

wrapped onto a roller/cylinder
rolled over the input signal such that each sample lines up with one on the reversed impulse response
each lined-up pair of samples from input signal and impulse response is multiplied and each product is summed

If you’re wondering why step 3 mentions a reversed impulse response, imagine that you have a roller and that the impulse response is on a strip of tape. Now imagine that you apply the impulse response tape over and around the roller, such that the numbers are facing you and are in the correct order. Now, when you roll this roller over and across the input signal, from left to right, the numbers on the impulse response tape will make contact with the input signal in reverse order.

See this page for an illustration of the convolution machine in Figure 6-8.

Implementation

Implementing the convolution machine is pretty straightforward once we are able to conceptualize what it is actually doing.

Let’s start with the type signature. Since we’re not using arrays, we’ll represent the signals as lists of numbers. Convolution does something with two signals to produce a third signal, so the type signature is pretty straightforward:

convolve :: (Num a) => [a] -> [a] -> [a]
convolve hs xs = undefined

In the signature, xs refers to the input signal and hs refers to the impulse response.

Padding

Now for the implementation of convolve. First, consider this component of the convolution summation:

$$x[i-j]$$

When we are computing the first sample, such that $i = 0$ , in the output signal $y[n]$ , then at one point we need to refer to the $x[-(M-1)]$ sample where $M$ is length of impulse response. However, there are no samples to the left of the first sample.

So what we have to do is prepad the input signal with $M-1$ samples of value $0$ . This padding has the added benefit of allowing us to simply map over the padded input signal to generate the output signal. This is because the convolution operation’s output signal length is $M + N - 1$ where $M$ is the length of the impulse response and $N$ is the length of the input signal. The padding can be achieved with:

let pad = replicate ((length hs) - 1) 0
    ts  = pad ++ xs

Once we prepad the input signal with enough zero samples, we can pass the padded input signal and impulse response to a function which simulates the rolling of the convolution machine. This function will be nested within convolve and will simply be used as a recursive helper function.

Let’s Roll

roll :: (Num a) => [a] -> [a] -> [a]
roll _  [] = []
roll hs ts = undefined

The roll function is recursive and will simulate the actual rolling of the convolution machine over the input signal. As it rolls, it will consume the head of the input signal ts. Think of the consumption of the head as if the input signal is being wrapped around the roller as it rolls. The input signal ts will therefore eventually be empty, meaning the convolution machine has finished rolling over the entire input signal.

The roll function is run for every sample in the output signal. This is where the bulk of the implementation comes in. At any given sample in the input signal, we simulate the roll by zipping the input signal from that sample forward along with the impulse response. This generates a list of pairs each consisting of the input signal sample with its corresponding impulse response sample (which is being rolled over it).

If you have trouble conceptualizing this, imagine that the impulse response on the roller is tape, so that when you roll it over the input signal, the impulse response—which, remember, makes contact with the input signal in reverse—sticks to the input signal and is lined up such that each sample in the impulse response is directly over a sample of the input signal.

We then need to multiply the components of each pair with each other, i.e. the input sample multiplied by its corresponding impulse response sample. The act of zipping and multiplying the zipped up pairs can be done in one go with zipWith (*). We then gather all of these products and sum them up. This sum is the latest computed sample in the output signal.

We construct the complete output signal by cons’ing the sample with a recursive call to roll, however this roll will concern only the next sample forward, thereby simulating rolling across the input signal.

With this information, we can finish the definition of roll:

roll :: (Num a) => [a] -> [a] -> [a]
roll _  [] = []
roll hs ts = let sample = sum $ zipWith (*) ts hs
             in sample : roll hs (tail ts)

Here is the whole convolution function convolve put together:

convolve :: (Num a) => [a] -> [a] -> [a]
convolve hs xs =
  let pad = replicate ((length hs) - 1) 0
      ts  = pad ++ xs
  in roll ts (reverse hs)
  where
    roll :: (Num a) => [a] -> [a] -> [a]
    roll _  [] = []
    roll hs ts = let sample = sum $ zipWith (*) ts hs
                 in sample : roll hs (tail ts)

Reduction

Now that we understand the concept behind convolution, we can reduce the above implementation a bit further.

The observation we should make is that the roll function acts like map, specifically over ts. The only detail is that on every element mapped over, the result of that element’s mapping concerns the list ts from that element forward. If we are on the third element of ts, we only act on the third element forward. In other words, we are mapping over every tail of ts. Knowing this, we can change the roll function to a straight up map over tails ts.

However, tails considers [] to be a tail of any list—which is technically correct—so we’ll always have a trailing 0 element if we do it this way. That’s why we simply take the init of the result of tails, which returns every element in a list except the last one. We also still need to prepad the signal, so those lines remain:

convolve :: (Num a) => [a] -> [a] -> [a]
convolve hs xs =
  let pad = replicate ((length hs) - 1) 0
      ts  = pad ++ xs
  in map (sum . zipWith (*) (reverse hs)) (init $ tails ts)

Parallelization

There’s something to be said about how the various properties of the Haskell language come together to make certain algorithms trivially parallelizable. Green threads, single assignment, function purity and its consequent idempotence/referential transparency, among other things. I think it’s interesting to note how easy it can be to parallelize this naive convolution algorithm.

parMap

The parallel Haskell package contains various tools for parallelization. One of these is the Control.Parallel.Strategies module, which defines the parMap function, which maps over list elements in parallel, in essence, a parallel map:

parMap :: Strategy b -> (a -> b) -> [a] -> [b]

parMap takes an evaluation strategy which is used to actually perform the evaluation in parallel. We use the rdeepseq evaluation strategy, which fully evaluates the argument to Normal Form (i.e. fully evaluated), as opposed to rseq which merely evaluates the argument to Weak Head Normal Form (WHNF). The rdeepseq strategy can only operate on arguments it knows it can fully evaluate, those that conform to the NFData typeclass from the Control.Deepseq module. To conform to this, we add another type constraint to our convolution parameters:

parConvolve :: (NFData a, Num a) => [a] -> [a] -> [a]

Continuing forward, all we have to do now is make a drop-in replacement of map with parMap. Actually, it’s not quite a drop-in replacement, because we need to supply parMap with the rdeepseq evaluation strategy:

parConvolve :: (NFData a, Num a) => [a] -> [a] -> [a]
parConvolve hs xs =
  let pad = replicate ((length hs) - 1) 0
      ts  = pad ++ xs
  in parMap rdeepseq (sum . zipWith (*) (reverse hs)) (init $ tails ts)

Benchmark

The criterion Haskell package provides tools for benchmarking and analyzing code. The synthetic benchmark we will conduct will run each implementation with an impulse response of length 100 and an input signal of length 1,000.

In the following code, conv is the naive implementation, conv' is the reduced naive implementation, and parConv is the parallel implementation:

data ConvType = Naive | Reduced | Parallel deriving (Eq, Ord)
convTypes = Data.Map.fromList [(Naive, conv), (Reduced, conv'), (Parallel, parConv)]

main = defaultMain [
  bench "Naive Convolution" (runConv Naive),
  bench "Reduced Convolution" (runConv Reduced),
  bench "Parallelized Convolution" (runConv Parallel) ]
  where runConv ctype =
          let hs = [1..100 :: Int]
              ts = [1..1000 :: Int]
              convfn = fromJust $ Data.Map.lookup ctype convTypes
          in nf (convfn hs) ts

Compile the benchmark with:

$ ghc --make -O2 -threaded -o conv conv.hs

Run it with:

$ ./conv -o bench.html -r out.csv +RTS -N4

The -o parameter specifies an output file for generated charts and graphs. The -r parameter specifies a comma separated value (CSV) file to output relative statistics which we use to measure performance relative to the reference, non-reduced naive implementation.

The +RTS parameter is a delimiter which begins parameters to the runtime system. The -N# parameter specifies how many cores to utilize. The machine I was using has 6 cores, but I found that using less than that lowered the amount of statistical variance. I imagine this was because the computer was able to continue its own tasks on the other two cores.

The above benchmark yielded the following results:

Name	% faster
Naive (Reference)	0
Reduced	0
Parallel	54

The parallel version apparently really boosts performance. An important thing to realize is that when parallelizing things, it’s considered best to only parallelize when the benefits outweigh the relative overhead of managing the green threads.

For example, in my tests, changing the impulse response length to 5 and the input signal length to 10 shows the parallel version to be 27% slower than the naive implementation. Also notice that the reduced version is a bit slower, for what I can only imagine to be a GHC optimization that applies to the naive implementation but not to the reduced version.

Name	% faster
Naive (Reference)	0
Reduced	-16
Parallel	-27

On the other hand, increasing the impulse response length to 1,000 and the input signal length to 10,000 maintained a similar performance increase:

Name	% faster
Naive (Reference)	0
Reduced	0
Parallel	48

Fusion

As Christian pointed out in the comments, this naive implementation can really benefit from Stream Fusion. It’s embarrassingly straightforward to enable stream fusion on this naive algorithm, considering that all we’re doing is qualifying the list functions so that the ones from the stream-fusion package are used instead. Doing this yielded a 36% performance increase over the parallel algorithm showcased above, which was already fast. This is quite a bountiful optimization reward indeed.

import qualified Data.List.Stream as S

parConvolveSF hs xs =
  let pad = S.replicate ((S.length hs) - 1) 0
      ts  = pad S.++ xs
  in parMap rdeepseq (S.sum . S.zipWith (*) (S.reverse hs)) (S.init $ S.tails ts)

Conclusion

I’m new to Digital Signal Processing, so if you notice any glaring errors please feel free to correct me; I would appreciate it. If you are interested in this subject and would like to read a book to learn more, I wholeheartedly recommend The Scientist and Engineer’s Guide to Digital Signal Processing. If you would like to learn more about Convolution, you can check the relevant chapters in that freely available book.

Imperative Approach

You can also check out this page as well, which also covers multidimensional convolution with a concrete example of a Gaussian filter applied to an image for the purposes of blurring it. This specific application of the Gaussian filter is known as the Gaussian Blur. The Gaussian Blur is pretty popular in realtime image rendering, such as in video games, because of a property it has which allows it be applied in two dimensions, e.g. in an image, as two independent one-dimensional operations. This makes it dramatically faster and more efficient, and is trivial to implement in modern GPU Compute shaders ¹. Such shaders can then be used to implement effects such as motion blur in games ². The page also provides imperative implementations of convolution in C++.

Optimizations

Haskell is known for having many ways of doing any one thing, so if you come up with a better solution feel free to gist it and post it in the comments.

Of course, this post concerns a naive implementation of convolution. There are other more optimized implementations of convolution, such as FFT convolution which exploits the Fast Fourier Transform and the principle of duality—convolution in the time domain is equivalent to multiplication in the frequency domain—to perform convolution a lot faster in some cases.

As described in 3D Game Programming with DirectX 11 by Frank D. Luna in Chapter 12, page 450, § 12.7 ↩︎
Despite this optimization of Gaussian Blurring, many implementations optimize further. Blurring typically involves rendering the scene to a separate buffer (e.g. Render-to-Texture) at a scaled-down resolution. This speeds up the blurring operation as there are less pixels to operate on. Then the result is rendered to the actual screen. Since the point is to blur, the upscaling is usually hardly noticeable. ↩︎

I’m Back

Wed, 26 Dec 2012 00:00:00 +0000

It’s been a while since I last had a blog. In fact, I think it’s been two, maybe three years now. I previously had a WordPress blog that I updated regularly. However, the things I was studying on my own at the time consumed me and I considered writing about those things to be a low priority. After all, who want’s to hear what a noob thinks about something he’s barely beginning to learn about? So instead I figured I’d continue reading and studying, and maintaining a blog kind of fell into the backburner.

Blogging

For over a year now I had wanted to get back into blogging, at the very least to serve as a sort of journal for what I’ve learned, what I’m working on, etc. I didn’t want to bother maintaining a WordPress blog anymore, so I gave tumblr a shot. I felt that tumblr wasn’t as flexible as I would like and lost interest again. A few months ago I got the urge again to set up a blog, and having heard of static site generators on hacker news, I ended up giving Jekyll/Octopress a try. That’s what I’m running right now and so far it’s working out well. I actually set this up a few months ago, but hadn’t had the time to properly set it up and design myself a theme.

Haskell

I’ve done a lot during my absence these past few years. In fact, it’s all a big blur so I don’t know where to begin to talk about it. Most recently, for example, I worked on a music visualizer for Linux’ Pulse Audio in Haskell using OpenGL and FFTW. This was the first program I wrote in Haskell, aside from little programs I would write to learn the language. I learned Haskell primarily using a combination of Learn You a Haskell for Great Good and Real World Haskell. For certain things, such as understanding the concept of laziness in Haskell, I used the Haskell Wikibook. I also had a Professor at school who was familiar with Haskell and DSP as my mentor. Thanks to his previous experience, I was able to learn both things pretty quickly. Whenever I was unsure of my understanding of a given topic, such as Monads, I had him to discuss and refine my understanding with.

Digital Signal Processing

I worked on this project to learn both Haskell and Digital Signal Processing. The book I used for learning Digital Signal Processing is The Scientist and Engineer’s Guide to Digital Signal Processing. For the longest time—probably the same amount of time I’ve been without a blog—I’ve wanted to learn about Digital Signal Processing, but every book I found assumed some prior DSP knowledge, electronics/circuits knowledge, or hardcore mathematical understanding. This book seems to teach DSP from the perspective of a developer, showing code for algorithms discussed throughout, such as the Fast Fourier Transform.

I will probably write a post with more information about the project and what it was like to learn Haskell and DSP, and where I intend to go forward with this. For now I just wanted to push out a real post, unlike the test post which I use to test the design and markup of this site.

Test Post

Wed, 19 Sep 2012 00:00:00 +0000

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent dolor nisi, feugiat sit amet egestas sit amet, mattis sit amet velit. Proin quis metus id mi vulputate adipiscing eget et sapien. Sed iaculis auctor sem ac convallis ¹. Cras nisi libero, dignissim quis facilisis et, bibendum eget enim. Vivamus est leo, tristique ut adipiscing vitae, malesuada adipiscing tellus. Integer nec aliquam justo. Nam ac lacus erat. In hac habitasse platea dictumst. Donec pharetra, lectus sit amet ultrices fringilla, metus mi auctor ligula, in hendrerit dui arcu eu leo.

Hacker News

Milton_From_OfficeSpace moments ago [–]

Who needs DropBox when the average Joe can use tar and rsync? Your business is doomed.

Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nulla facilisi. Vestibulum consectetur nulla eget erat fermentum malesuada. Nullam ultricies nulla ut eros vulputate sed cursus tortor faucibus. Cras ac risus mauris. Suspendisse molestie elementum luctus. Vestibulum venenatis blandit velit ac pretium. Press CTRL + X.

$$ i\hbar\frac{\partial \psi}{\partial t} = \frac{-\hbar^2}{2m} \left( \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} + \frac{\partial^2}{\partial z^2} \right) \psi + V \psi $$

Nulla facilisi. Cras lectus velit, rhoncus et mollis eget, ultricies quis ligula. Integer ornare nibh in est molestie tempus. Praesent bibendum placerat diam id gravida. Fusce et velit eget dui laoreet interdum eget id arcu. Ut et massa nibh, ut mollis augue. In hac habitasse platea dictumst. Sed massa orci, venenatis vitae dapibus sit amet, varius ut nibh. Ut vel enim leo, at posuere nisi. In elementum sem vel nibh malesuada sagittis.

$$R_{ab} - {\textstyle 1 \over 2}R\,g_{ab} + \Lambda\ g_{ab} = \kappa\, T_{ab}$$

Nulla facilisi. Cras lectus velit, rhoncus et mollis eget, ultricies quis ligula. Integer ornare nibh in est molestie tempus. Praesent bibendum placerat diam id gravida. Fusce et velit eget dui laoreet interdum eget id arcu. Ut et massa nibh, ut mollis augue. In hac habitasse platea dictumst. Sed massa orci, venenatis vitae dapibus sit amet, varius ut nibh. Ut vel enim leo, at posuere nisi. In elementum sem vel nibh malesuada sagittis.

require 'base64'

File.open('favicon.png', 'wb') do |f|
  f.write(Base64.decode64(base64))
end

Donec vitae turpis non augue pulvinar imperdiet eu non erat. Integer mollis diam ac quam lobortis luctus. Suspendisse scelerisque sodales massa sit amet pharetra. $\Delta S = \pi$ ² Donec cursus, velit in mollis dapibus, nunc elit pharetra erat, et dictum sem sapien pretium ligula. Cras nibh elit, mollis eu eleifend sed, faucibus in velit. Aliquam gravida tristique molestie. Donec gravida, diam ac auctor auctor, nulla enim varius sem, non auctor urna mi vitae enim. Cras aliquam semper nibh, quis viverra urna ultrices et. Nunc rhoncus fringilla dapibus.

Immortal Words

As you can see, there’s lots of work to do! Aenean mauris augue, faucibus vel sagittis nec, viverra sit amet sapien. Etiam neque nisi, molestie vel lacinia a, scelerisque eget justo. Suspendisse imperdiet lectus eu metus malesuada vel vehicula libero rutrum. Nulla facilisi. Cras lectus velit, rhoncus et mollis eget, ultricies quis ligula. Integer ornare nibh in est molestie tempus. Praesent bibendum placerat diam id gravida. Fusce et velit eget dui laoreet interdum eget id arcu.

Too many people have opinions on things they know nothing about. And the more ignorant they are, the more opinions they have.

Thomas Hildern on Fallout Wiki

The Meaning of Life

Aenean tristique, magna a semper venenatis, orci odio tempor lorem, ac molestie ligula enim ac urna. Fusce egestas nisl id massa porta vitae pretium tellus aliquet. Morbi tempor arcu sit amet felis hendrerit aliquam. Nulla facilisi. Suspendisse diam tortor, pharetra ac porta eu, ullamcorper nec lectus. Sed nulla turpis, blandit et tincidunt vel, condimentum et sapien. Aliquam dignissim blandit pharetra. Mauris ornare accumsan libero at bibendum. Morbi dui nisl, aliquet sit amet gravida sit amet, mollis commodo nulla. Quisque eu felis purus.

$$ \forall x, y : \mathbb{Z}, x > 3 \land y < 2 \Rightarrow x^2 - 2y > 5 $$

Aenean mauris augue, faucibus vel sagittis nec, viverra sit amet sapien. Etiam neque nisi, molestie vel lacinia a, scelerisque eget justo. Suspendisse imperdiet lectus eu metus malesuada vel vehicula libero rutrum. Nulla facilisi. In lectus nisl, semper ac imperdiet eu, ultricies luctus erat. Sed ultrices lectus in massa tincidunt fringilla. Aenean lectus tortor, tempus a sagittis vitae, hendrerit eget lacus.

Things to do

first do this
then do this other thing
then maybe, if you’re not tired, do this last thing

Here it is numbered for you:

first do this
then do this other thing
then maybe, if you’re not tired, do this last thing

As you can see, there’s lots of work to do! Aenean mauris augue, faucibus vel sagittis nec, viverra sit amet sapien. Etiam neque nisi, molestie vel lacinia a, scelerisque eget justo. Suspendisse imperdiet lectus eu metus malesuada vel vehicula libero rutrum. Nulla facilisi. In lectus nisl, semper ac imperdiet eu, ultricies luctus erat. Sed ultrices lectus in massa tincidunt fringilla. Aenean lectus tortor, tempus a sagittis vitae, hendrerit eget lacus.

Cras nibh elit, mollis eu eleifend sed, faucibus in velit. ↩︎
This is the awesome constant ↩︎

Jorge Israel Peña

Dots

File Structure

Deployment

tmux

zsh

Aliases

Completions

Functions

Cursors

Misc

vim

Conclusion

Live Editing with Hakyll

Conception

Server Architecture

State

Communication

Implementation

WebSocket Server

Hakyll Compiler

Main Thread

Client Side

Conclusion

A Simpler Vim Statusline

Patched Fonts

Noise

Signal

Redesign

Inactive Statuslines

Conclusion

Custom XKB Options with Gnome

Extra Dependencies in Hakyll

Split Stylesheets

Problem

Solution

Post Feed in Hakyll

Considerations

Drying Up

Implementation

Caveat

Conclusion

Drafts in Hakyll

Considerations

Examples

Implementation

Usage

The Switch to Hakyll

File Structure

Hakyll

SCSS

Abbreviations

Git Tag

Pandoc

Pygments

Table of Contents

Alternatives

Implementation

Deploying

Conclusion

xmonad Ignores Bindings

Media Keys

Binding

The Problem

Bug Hunting

Workaround

Bug Report

Terminal Customization

dotfiles

urxvt

Colors

Clipboard

zsh

Prompt

Basic

git

SSH

vi-Binds

Highlighting

Conclusion