Xiang

Comparing the Actor Model and CSP Concurrency with Elixir and Clojure

2023-12-18T00:00:00+00:00

Introduction
BEAM vs. JVM
- BEAM
  - Per-process memory
  - Preemptive scheduling
- JVM
  - Shared Heap
  - OS threads and OS scheduler
The Actor Model vs. CSP
- The Actor Model on the BEAM
- CSP with core.async in Clojure
Conclusion
References and Further Resources

Introduction

This is a digest of the talk on this topic I gave at Code BEAM STO 2022 and at ElixirConf EU 2022. I also talked about this in an episode of the Elixir Wizards podcast this month.

Before diving into this topic, I would like to first add a disclaimer that I am far from an expert on JVM and Clojure (as I have been working with Elixir most of my professional career). I gave the talk as I was intrigued by the topic and would like to share my learnings. As the saying goes, “the best way to learn a topic you’re interested in is to submit a talk proposal. If the proposal is accepted, you really have to dig into it”. I will try to present the information to the best of my knowledge, but some inaccuracies might be inevitable, in which case feel free to point them out in the comments.

In my talk, I approached the comparison from two dimensions: BEAM/ERTS vs. JVM and the Actor Model vs. CSP. Theoretically speaking, the Actor Model and CSP might be considered the two sides of the same coin. However, practically speaking, there have been some major differences in their popular implementations, with implementations of the Actor Model such as BEAM and Akka focusing a lot on the no-shared-memory aspect, while implementations of CSP such as seen in Golang, Rust and Clojure’s core.async focus a lot on the “execution flow” aspect, with the implicit underlying assumption that the concurrency still takes place in a shared-memory environment. Therefore, I considered it helpful to also touch upon the differences between the BEAM and the JVM, which serves as a nice foundation for understanding the differences between the implementations.

BEAM vs. JVM

BEAM

Here I would like to emphasize these two aspects of the BEAM VM: per-process memory space and preemptive scheduling.

Per-process memory

The following is a simplified illustration of the BEAM/ERTS memory layout (Pool 2015). The key takeaway here is that each process has its separate heap, i.e. they share no memory with each other, which is key to its Actor Model, which we’ll discuss later.

A corollary of this is that there will be no global garbage collection that could potentially impact all processes at the same time.

(There are some optimizations, such as large binaries being stored separately, so that only pointers to them instead of full copies of them need to be passed around, which we will not explore in detail here.)

Preemptive scheduling

The mechanism with which schedulers work is a unique and sometimes overlooked feature of BEAM. This is what enables the soft real-time nature of BEAM and what makes it great at handling massive concurrency challenges (but maybe not the best at handling CPU-intensive tasks).

When you launch an iex session, the smp:20:20 part actually indicates the number of schedulers available (“smp” stands for “Symmetrical Multiple Processes”). It’s usually the same as the number of CPU cores.

Since OTP 13, each scheduler has its own process queue. All processes of the BEAM are load-balanced across all schedulers.

With preemptive scheduling, once a process runs for 4000 reductions (one reduction is approximately 1 function call), it will be put to the back of the queue, and replaced by the next process in the queue. This ensures highly consistent time-sharing across all processes. (Independent GC also helps.) Imagine if you have 100K connections being handled on the same VM via 100K processes, nobody will necessarily hog any more CPU time share than any other process, and everybody gets the same experience, which is great.

However, you can easily see how this might not be ideal if you need to handle a special number-crunching workload which requires intensive CPU usage over a sustained period of time (dynamic typing is another negative against this use case). Indeed, you would then usually resort to NIFs, i.e. Native Implemented Functions (which are run on dirty schedulers), or ports (a mechanism with which BEAM can communicate with external programs). Note that NIFs are not subject to preemptive scheduling however (thus “dirty”), and thus can actually block if you’re not careful on how it terminates.

JVM

In contrast to BEAM’s per-process memory model and preemptive scheduling, JVM takes a relatively conventional approach which many might be more familiar with.

Shared Heap

JVM has a heap which is shared by all threads, which facilitates shared-state concurrency, in which multiple threads can potentially read and write the same memory space and thus the same resource. This requires the use of concurrency mechanisms such as locks in Java or atom, agent, ref, var in Clojure to avoid race conditions, which however could result in deadlocks if not careful. At the same time, it does make some scenarios which require coordination between multiple threads to perform heavylifting during a CPU-intensive task easier.

(Theoretically, a stop-the-world garbage collection operation can also result in all threads being affected and halted at the same time in contrast to the independent processes run on the BEAM VM, though from what I understand, a lot of optimizations have been made on this front, so this is generally not such a big issue in recent versions of JVMs anymore.)

OS threads and OS scheduler

Conventionally, the JVM creates operating system threads and relies on the operating system scheduler to run the threads. Therefore, JVM threads are much heavier than BEAM’s lightweight processes, and the frequent blocking and unblocking of threads in IO-intensive tasks will be costly. On the other hand, they’re a great fit for CPU-intensive tasks.

(Note: With the formal introduction of Project Loom in JVM 23, Java has also made virtual threads available, which are much more lightweight and in a sense share similarities with BEAM processes. As it’s still in its early days, its impact on libraries such as Akka and core.async remains to be seen.)

The Actor Model vs. CSP

The Actor Model (Hewitt 1973) and Communicating Sequential Processes (Hoare 1978) were proposed close to each other and co-evolved over the years. (Anecdotally, the creators of Erlang independently arrived at an idea similar to what Hewitt proposed, without first having known about it.)

Theoretically, you can think of the main difference between those two as the distinction between processes as the first-class entities vs. channels as the first-class entities. Beyond that though, the real-world implementations of the Actor Model (e.g. BEAM, Akka) and CSP (e.g. Go, Rust, Clojure’s core.async) tend to also have different practical foci, which we’ll explore in more detail up next.

The Actor Model on the BEAM

As we have seen in the previous section on the BEAM, one key feature of the Actor Model is how the processes should share no memory with one another.

This goes hand-in-hand with the fact that processes are first-class: each process has a unique name; processes communicate by identifying each other by name, then sending messages directly to each other. This concurrency model directly eliminates a lot of race conditions which can lead to subtle bugs in a shared-memory language. Theoretically, all you need to trace are the messages sent and the behaviors of each process upon receiving certain messages. (Of course, as anybody who has worked with Erlang/Elixir can attest, the reality of debugging can still be much messier than what’s promised here, which is unavoidable.)

A related property for actors is that send is non-blocking by default, while receive is blocking. What this means is that a sender process can in a sense “fire and forget” every message it sends, while it’s up to the receiver process to handle messages that arrive at its mailbox via entering a receive call, which would block the process from further operations until a message matching one of the clauses specified in the receive block arrives. Here’s an example in iex:

iex(1)> iex_process = self()
#PID<0.113.0>
iex(2)> spawn(fn -> send(iex_process, {:hello, self()}) end)
#PID<0.114.0>
iex(3)> receive do
...(3)> {:hello, pid} -> "Got hello from #{inspect pid}"
...(3)> end
"Got hello from #PID<0.114.0>"
iex(4)> 1 + 1 # Now iex_process is unblocked and able to do something else
2

Note that if none of the receive block’s clauses match the incoming message, the process will keep waiting and thus be blocked:

iex(1)> iex_process = self()
#PID<0.113.0>
iex(2)> spawn(fn -> send(iex_process, {:hello, self()}) end)
#PID<0.114.0>
iex(3)> receive do
...(3)> :unmatched -> "Will not match here"
...(3)> end

At this point iex is blocked and you can’t do anything further except for e.g. interrupting it.

In addition, a process can never “refuse” to receive messages, and a mailbox’s size is theoretically unlimited if the messages arrive but remain unhandled. In fact, one way to crash a BEAM VM is to overflow the mailboxes! Here’s an illustration of this behavior:

iex(1)> iex_process = self()
#PID<0.113.0>
iex(2)> :erlang.process_info(iex_process, :messages) # Empty mailbox at this point
{:messages, []}
iex(3)> spawn(fn -> send(iex_process, {:hello, self()}) end)
#PID<0.114.0>
iex(4)> :erlang.process_info(iex_process, :messages) # The message is already in the mailbox
{:messages, [hello: #PID<0.114.0>]}

Note how the message is already in the mailbox even though the iex process didn’t specify any receive block to handle incoming messages at all, and may simply be going on with its business obliviously.

(Of course, we rarely need to spawn raw processes and write receive blocks directly in production Erlang/Elixir code. This is because OTP already offers a whole package of abstractions to make the life of developers much easier. A well-known abstraction is gen_server which for example lets you specify with a clear syntax callback functions (handle_call, handle_cast, handle_info) which spell out how the process should respond to incoming messages, and handles a lot of edge cases under the hood so that you don’t need to bother handrolling a receive block. OTP also offers many other tools such as process linking & supervision, node clustering, tracers and debuggers, which are out of discussion scope here.)

CSP with core.async in Clojure

Aside: Conventional approaches to concurrency in Clojure

Before beginning this section, I would first like to point out that core.async is a later addition to Clojure, while Clojure’s more conventional approach to concurrency involves tools such as ref, atom, agent, and var, which, similar to locks in Java, facilitate shared-memory concurrency, but can be much more ergonomic and pleasant to use due to Clojure’s functional nature, which enables the separation of identity and state via the widespread use of persistent data structures (“persistent” in this context refers to the fact that the data structure always preserves its previous version when modified. A classic example of this is a list, to which if you prepend a new head, the tail remains unchanged and a reference to which can be easily kept around as the previous version of the entire list. This is of course a feature shared by other functional languages such as Elixir). Another interesting concept is Software Transactional Memory (STM), which allows you to keep a series of reads and writes in one single transaction, somewhat similar to a database transaction. These ideas are not unique to Clojure: Various functional languages such as Haskell offer similar tools for shared-memory concurrency.

When recording the podcast, Owen raised a very good point that when he first started with Elixir, he thought the no-shared-memory nature of the Actor Model and the BEAM is something that automatically comes with functional programming, which turned out to not be the case. In fact most functional programming languages still handle shared-memory concurrency, just with a different (and some may argue more elegant) flavor compared to how procedural languages traditionally do it. We won’t have time to go into the details of these, though I’d encourage you to refer to the resources listed in at the end of the post if you’re interested in knowing more.

`core.async` and CSP

Different from the Actor Model, in CSP, processes are anonymous, while channels are named first-class entities. In Clojure’s core.async, you can return a channel from a function and pass it around. Readers and writers use channels to relay messages.

In core.async, the default channel is blocking and synchronous: Once a writer writes a message into the channel, it will be blocked until a reader reads the message.

(let [c (chan)]
  (thread (>!! c "hello") (println "writer unblocked"))
  (thread (println (<!! c))))

Output:

hello
writer unblocked

In this example, we create a channel named c and then spawn two JVM threads. The first thread writes "hello" into the channel, while the second thread reads the message off the channel and prints it immediately, after which the writer gets unblocked.

Similarly, once a reader decides to listen on a channel, it will be blocked until a writer delivers a message:

(let [c (chan)]
  (thread (println (<!! c)) (println "reader unblocked"))
  (thread (>!! c "hello")))

Here, the reader is blocked first, before it later gets unblocked once the writer puts a message "hello" into the channel.

(There are variants such as buffered channels, which enable non-blocking behavior to some extent: As long as the buffer is not full, a writer will be able to fire and forget and will not be blocked, similar to how a BEAM process sends its message. However, unlike the unlimited mailboxes in BEAM/OTP, there are no channels of unlimited size in core.async, meaning even a buffered channel will get full at some point. Sliding channels and dropping channels can also be used, which drops the oldest or the newest message when full.)

Unlike BEAM mailboxes, channels can be explicitly closed to further writes, after which remaining messages can still be read from the channel.

(let [c (chan)]
  (thread (>!! c "hello"))
  (close! c)
  (thread (println (<!! c))))

In terms of topology, each channel can potentially have zero to many readers and writers, and each thread can decide to write to and read from multiple channels at the same time:

When there are multiple readers for one channel, a message will only be read once by at most one reader.
When there are multiple writers for one channel, all other writers will be blocked after the channel is full.
You can use alts!! in core.async (in Go: select) to let a reader read from multiple channels. It will select the channel that first receives a message. Here’s an example from the official guide:

(let [c1 (chan)
      c2 (chan)]
  (thread (while true
              (let [[v ch] (alts!! [c1 c2])]
                (println "Read" v "from" ch))))
  (>!! c1 "hi")
  (>!! c2 "there"))

Output:

Read hi from #object[clojure.core.async.impl.channels.ManyToManyChannel 0x5cd1c0c4 clojure.core.async.impl.channels.ManyToManyChannel@5cd1c0c4]
Read there from #object[clojure.core.async.impl.channels.ManyToManyChannel 0xd59d3dd clojure.core.async.impl.channels.ManyToManyChannel@d59d3dd]

“`go` blocks” in core.async

There are actually two ways to run a CSP process for core.async: An actual JVM thread, which is initialized with thread as seen in the examples above, and an inversion of control “thread” initialized with the go macro.

Recall from the JVM discussion that blocking/unblocking each JVM thread is heavy, unlike the lightweight processes that live on the BEAM. In the case of the go blocks, a fixed-size thread pool is used under the hood. Some macro magic takes place to “park” and “unpark” the blocks and switch them in and out of the threads in the pool, instead of the thread variant where a real JVM thread will be blocked. This makes go blocks much more lightweight. Here’s an example taken from the core.async repo:

(let [n 1000
  	cs (repeatedly n chan)
  	begin (System/currentTimeMillis)]
  (doseq [c cs] (go (>! c "hi")))
  (dotimes [i n]
	(let [[v c] (alts!! cs)]
  	(assert (= "hi" v))))
  (println n "msgs in" (- (System/currentTimeMillis) begin) "ms"))

which output “1000 msgs in 194 ms”, i.e. we spawned 1000 channels and were able to read and write to each of them in mere 194ms!

Inversion of control with `go` blocks

Another interesting thing to note, as mentioned earlier, is how the use of CSP in Go, Clojure, Rust etc. still facilitates working with shared memory, unlike the Actor Model used on the BEAM and even in Akka to some extent:

“It should be noted that Clojure’s mechanisms for concurrent use of state remain viable, and channels are oriented towards the flow aspects of a system.” - Rich Hickey

“Messages should be immutable, this is to avoid the shared mutable state trap.” - Akka

This “flow aspects” mentioned by Rich Hickey is emphasized by the fact that not only can go blocks be more efficient than thread blocks on the JVM, they also serve as an inversion of control mechanism. This is particularly evident in its use with ClojureScript, which due to its runtime is singlethreaded by nature. The parking and unparking of go processes onto a single thread greatly simplifies the conventional challenge of callback hell. Consider the following example (adapted from Seven Concurrency Models in Seven Weeks):

(defn show [elem]
  (set! (.. elem -style -display) "block"))
(defn hide [elem]
  (set! (.. elem -style -display) "none"))

(defn start [])
  (go
    (let [page1 (dom/getElement "page1")
          page2 (dom/getElement "page2")
          next-page-button (dom/getElement "next")
          next-page (get-events next-page-button "click")]
      (show page1)
      (<! next-page) ;; Waiting for the user to click on the next-page-button and trigger the event.
      (hide page1)
      (show page2))))

(set! (.-onload js/window) start)

If you are familiar with ES6’s async/await, you may immediately see the parallels between the use of <! here and the await keyword. Back in the days, fewer languages had this “async” inversion of control construct. (async/await was only formally added to JavaScript in 2017.) Nowadays it’s very much widespread as it offers tangible benefits, as I’m sure everybody who has refactored some JS callback hell with async/await can attest to.

Conclusion

For many developers, concurrency concepts might sound like something rather removed from their day-to-day work. However, an understanding of the concurrency properties of various programming languages and common concurrency patterns can be an integral part of your general knowledge. I just talked to a developer who was tasked with building a system completely in NodeJS, but decided and convinced the team that it might be a better idea to build the payment processing component in Elixir, which is still working extremely well under heavy traffic a few years later. On the other hand, I also remember a blog post from Discord a few years ago on how they needed to delegate some CPU-intensive tasks to Rust in their server codebase mainly based on Elixir, which as mentioned earlier might not be a one-size-fit-all language especially when you’re faced with the need to handle heavy, sustained workloads.

Having a thorough understanding of the concurrency model of the language you work with daily also helps you have a much firmer grasp on your programs’ behaviors.

This topic is so big that it’s impossible to cover it with a talk, a podcast episode, or a blog post. If you’re interested in exploring further, I would highly recommend the book Seven Concurrency Models in Seven Weeks, which I greatly enjoyed, and the other resources listed below. Even if you won’t get to try out all the concurrency patterns and programming languages at work, this knowledge is intellectually very satisfying in its own right, and can aid you greatly in your journey to becoming a better programmer.

References and Further Resources

Seven Concurrency Models in Seven Weeks
Elixir in Action
Designing for Scalability with Erlang/OTP
The Joy of Clojure
Actor Model of Computation (Hewitt 1973)
Communicating Sequential Processes (Hoare 1978)
Comparison of Erlang Runtime System and Java Virtual Machine (Pool 2015)
Hewitt, Meijer and Szyperski: The Actor Model (Video discussion)
Clojure Concurrency - Rich Hickey’s 2008 Talk
Clojure core.async Channels - announcement in 2013
Clojure core.async - Rich Hickey’s 2013 Talk
JEP 425: Virtual Threads (Preview)

Displaying Org Mode Pomodoro Clock in the Menu Bar

2020-12-27T00:00:00+00:00

As I’ve written in previous posts^{(1, 2)}, I’m a huge fan of Emacs Org mode. One thing that I’ve been doing on-and-off is to use Org mode to also track my time and implement the pomodoro method. It’s not entirely perfect, though one thing that has helped me a lot is to display the current pomodoro status in the menu bar, which makes it much easier to keep an eye on things even when I’m not using Emacs. No matter how great an OS Emacs is, surely you can’t live your whole life within it? ;)

The following screenshots show this in action under four different pomodoro clock types, respectively:

Normal pomodoro:
Pomodoro in overtime (enabled with org-pomodoro-manual-break set to t):
Short break
Long break

I’ll describe my setup in MacOS. The method should be replicable in Linux and Windows with minimum adaptations. I also came across a post for Linux and an app for Windows achieving something similar using alternative approaches, which you might be interested in checking out.

There are two key pieces to this approach:

the function org-pomodoro-format-seconds, which outputs the pomodoro clock according to org-pomodoro-time-format (not necessarily in seconds, thus it’s somewhat of a misnomer).
the open-source app BitBar which evaluates an arbitrary command and displays the output in the menu bar.

The following is a step-by-step walkthrough:

Specify the format you want the time to be displayed via org-pomodoro-time-format. In my case I have a simple format of mm:ss:

(setq-default org-pomodoro-time-format "%.2m:%.2s")

Define a function that will produce the final output to be displayed on the menu bar for the script. In my case I made the following additions to the string output:
- Prepend prefix characters “P”, “O”, “B”, “LB” to indicate the exact state of the pomodoro.
- Display “N/A” when there’s no ongoing pomodoro.
The entire function definition is as follows:

(defun jx/produce-pomodoro-string-for-menu-bar ()
  "Produce the string for the current pomodoro counter to display on the menu bar"
  (let ((prefix (cl-case org-pomodoro-state
            (:pomodoro "P")
            (:overtime "O")
            (:short-break "B")
            (:long-break "LB"))))
          (if (and (org-pomodoro-active-p) (> (length prefix) 0))
            (list prefix (org-pomodoro-format-seconds)) "N/A")))

Install BitBar and ensure it runs normally. Open BitBar’s plugin folder. Create a shell script org-clock.1s.sh that calls emacsclient to evaluate the above-defined function:
- The 1s part indicates that BitBar should evaluate this script every one second.
- You may need to run chmod +x org-clock.1s.sh after creating it to make it executable.
- If the emacsclient command fails, you may need to ensure that (server-start) is run automatically every time Emacs starts, so that the Emacs server/client is available.
The script is as follows:

#!/bin/bash

export PATH='usr/local/bin:/usr/bin:$PATH'

# The default output of (org-pomodoro-format-seconds) contains parentheses, which I don't need. Sub them out with sed.
emacsclient --eval "(jx/produce-pomodoro-string-for-menu-bar)" | sed 's/["()]//g'

That’s it! Now, whenever you start a pomodoro in org-mode, you should be able to see the clock reflected in your menu bar :)

Differences between Vim Regex, Emacs Regex and PCRE

2020-02-24T00:00:00+00:00

When I first switched over from Vim to Spacemacs, one thing that really caught me off guard was the behavior of / search under evil-mode. evil-mode reproduces the Vim experience so well, it’s easy to forget the underlying Emacs base of it all.

Turns out, both Vim and Emacs have their own regex syntaxes that are slightly different from the one used by PCRE (Perl-Compatible Regular Expressions). vi (and its predecessor ed) and emacs are both older than Perl, so this is to be expected.

I would like to list some of the differences that are likely to be encountered in your day-to-day work. I will also list some resources that you can refer to for more details.

Differences between Emacs Lisp regex and PCRE regex

The following are some common gotchas of Elisp regex:

You need to escape some additional symbols such as:
- backslash \: \\
- alternation |: \|
- grouping ( and ): $ and $
- counting { and }: \{ and \}
\s begins a syntax class. Whitespaces are denoted as \s- instead of \s.
Use [0-9] or [:digit:] instead of \d to denote digits.
Use \1, \2 etc. instead of $1, $2 to refer to the results of capturing parentheses.

For more details, refer to The EmacsWiki and this SO question.

In addition, there is a tool which converts PCRE regex to Emacs regex.

Differences between Vim regex and PCRE regex

This SO answer by J-P summarizes some the common gotchas of Vim regex:

Perl    Vim     Meaning
---------------------------
x?      x\=     Match 0 or 1 of x
x+      x\+     Match 1 or more of x
(xyz)   \(xyz\) Use brackets to group matches
x{n,m}  x\{n,m} Match n to m of x
x*?     x\{-}   Match 0 or 1 of x, non-greedy
x+?     x\{-1,} Match 1 or more of x, non-greedy
\b      \< \>   Word boundaries
$n      \n      Backreferences for previously grouped matches

You can also get some additional information via :help perl-patterns.

There are some ways to change the behavior of Vim regex:

Use the “very magic regex mode” by prepending the pattern with \v. However, note that \b will still mean “backspace” instead of “word boundaries”.
Manually run perl: :perldo s/searchme/replaceme/g.
Use the plugin eregex.vim which performs the conversion automatically and allows you to search using PCRE regex by default.

Elixir Process Orchestration In Kubernetes With Libcluster And Swarm

2020-01-07T00:00:00+00:00

Update: horde is a library that is “built to address some perceived shortcomings of Swarm’s design.” (from the introductory blog post), and is currently (as of December 2020) more actively maintained than Swarm. It also works together with libcluster. We used horde for some new use cases and we recommend you to also check it out if Swarm doesn’t fit your need.

Collaborated with Rocky Neurock

We run Elixir on Kubernetes. While there were some rough edges for such a setup a few years ago, it has become much easier thanks to libraries such as libcluster and Swarm. Recently, we had a use case which those two libraries fit perfectly. However, since the libraries are relatively new, you might find the documentation and examples a bit lacking and it took us a while to get everything working together. We wrote this comprehensive walk-through to document our experience.

Note that you’d need to run your Elixir instances as releases, either with distillery or built-in releases (Elixir 1.9+), in order for this to work. This walk-through is based on our setup using Distillery.

libcluster and Swarm

First, a bit of background information:

The ability for different nodes to form a cluster and maintain location transparency has always been a huge selling point for the BEAM VM. libcluster is a library that helps with automatic node discovery and cluster formation, especially when the nodes are run in Kubernetes pods.

The main reason you might want a cluster is so you can organize and orchestrate processes across different nodes. This is where Swarm comes into the picture. Swarm maintains a global process registry and automates tasks such as process migration/restart after the cluster topology changes.

In our case, we have some permanent worker processes which should only have one instance of each running across all nodes at any given time. The problem is that we perform k8s rolling deployments with our CD pipeline, thus pods (and therefore nodes) will get destroyed and recreated throughout the day.

With Swarm, we register all above-mentioned worker processes globally, so whenever a pod gets destroyed, its worker processes will be automatically restarted on another healthy pod, ensuring that they are running exactly as envisioned all the time.

(Note: Initially, Swarm contained both the auto-clustering functionality and the process orchestration functionality. Later, the maintainer decided that it would be better to split them into two separate libraries, which become the libcluster and Swarm we see today.)

Clustering with libcluster

In our first step, we’re going to ensure that automatic cluster formation in Kubernetes takes place successfully.

Before we add libcluster to our project, we need to ensure that every Elixir node has a unique name. By default, in the rel/vm.args file generated by Distillery, we have a line just like:

-name <%= release_name %>@127.0.0.1

This means every node will be started as yourapp@127.0.0.1, which is not what we want.

We could first expose the Kubernetes pod IP as an environment variable in the kube configuration (e.g. kube/deploy.yaml) for the Elixir app:

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: your-app
  # ...
spec:
  # ...
  template:
    # ...
    spec:
      # ...
      containers:
      - name: your-app
        # ...
        env:
        - name: MY_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP

and then you can change the line in rel/vm.argsso the pod IP address will be substituted at runtime:

-name <%= release_name %>@${MY_POD_IP}

After ensuring unique node names, you can already test clustering manually via your console:

# On your local machine
~ kubectl exec -n your-namespace your-pod -c your-container -- sh

# Launch Elixir console after connecting to your k8s pod.
~ bin/console

iex(app@<POD_1_IP>)> Node.connect(":app@<POD_2_IP>")
iex(app@)> Node.list()

# If things are set up correctly, Node.list() should return [":app@<POD_2_IP>"]

Automatic Clustering

Having tested manual clustering, we can then move on to automatic clustering with libcluster.

You may notice that there are three clustering strategies for Kubernetes in libcluster and wonder which one you should use. In our experience, Cluster.Strategy.Kubernetes.DNS is the easiest to set up. All it requires is that you add a headless service to your cluster and no modifications are needed for the already existing (Elixir) pods.

In your k8s config:

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: your-app
  # ...
spec:
  template:
    metadata:
      # The selector for the headless service can
      # find the app by this label
      labels:
        app: your-app
        role: service
      # ...

---
apiVersion: v1
kind: Service
metadata:
  name: your-app-headless
spec:
  clusterIP: None
  # The selector is just a way to filter the pods on which the app is deployed.
  # It should match up with the labels specified above
  selector:
    app: your-app
    role: service

After adding the headless service, we’re finally ready for libcluster. You may add the ClusterSupervisor to the list of supervisors to be started with your app.

### In mix.exs
  defp deps do
    [
      ...
      {:libcluster, "~> 3.1"},
      ...
    ]
  end

### In config/prod.exs
config :libcluster,
  topologies: [
    your_app: [
      strategy: Cluster.Strategy.Kubernetes.DNS,
      config: [
        service: "your-app-headless",
        # The one as seen in node name yourapp@
        application_name: "yourapp",
        polling_interval: 10_000
      ]
    ]
  ]

### In lib/application.ex
  def start(_type, _args) do
    # List all child processes to be supervised
    children =
        [
          [
            {
              Cluster.Supervisor,
              [
                Application.get_env(:libcluster, :topologies),
                [name: Yourapp.ClusterSupervisor]
              ]
            }
          ],
          # Start the Ecto repository
          Yourapp.Repo,
          # Start the endpoint when the application starts
          YourappWeb.Endpoint,
        ]

    opts = [strategy: :one_for_one, name: Yourapp.Supervisor]

    Supervisor.start_link(children, opts)
  end

If everything goes well, your pods should already be automatically clustered when you start them. You can verify this by running Node.list() in the console, similar to what we did above.

Optional: Local cluster setup with docker-compose

During development, you will not want to deploy your changes every time to an actual k8s cluster in order to validate them or run a local k8s cluster for that matter. A more lightweight approach would be to make libcluster work together with docker-compose and form a local node cluster. We found Cluster.Strategy.Gossip the easiest to set up for this purpose.

The following assumes the app is started with docker-compose directly, without using Distillery releases.

First, we need to make sure that each Erlang node has a unique name, just as we did for the production environment. We will do it in our entrypoint script for docker-compose:

# In Dockerfile:
ENTRYPOINT ["/opt/app/docker-entrypoint-dev.sh"]

# In docker-entrypoint-dev.sh:
if [ -z ${NODE_IP+x} ]; then
    export NODE_IP="$(hostname -i | cut -f1 -d' ')"
fi

elixir --name yourapp@${NODE_IP} --cookie "your_dev_erlang_cookie" -S mix phx.server

Then, we need to scale the number of containers for our service to 2. You can easily do it by adding another service in your docker-compose.yml file:

services:
  your_app: &app
    build:
      context: ./app
    ports:
      - 4000:4000
    # ...

  your_app_2:
    <<: *app
    ports:
      - 4001:4000

(Note: Another way to achieve the same is to use the --scale flag of the docker-compose up command.)

Finally, we just need to specify our clustering strategy correctly:

### In config/config.exs
config :libcluster,
  topologies: [
    your_app: [
      strategy: Cluster.Strategy.Gossip,
      config: [
        port: 45892,
        if_addr: "0.0.0.0",
        multicast_addr: "230.1.1.251",
        multicast_ttl: 1
      ]
    ]
  ]

By default, the port and the multicast address should already have been available. If not, you can check your docker-compose configurations.

By this point, the two local nodes should be able to automatically find and connect to each other whenever you start your app via docker-compose.

Process registration with Swarm

After the foundation has been laid with libcluster, we may now move on to Swarm.

This sentence from the documentation is key to using Swarm to its maximum potential:

Swarm is intended to be used by registering processes before they are created, and letting Swarm start them for you on the proper node in the cluster.

Therefore, we found the example from the documentation, which shows Swarm being used together with a normal Supervisor, to be slightly confusing: a normal Supervisor must start with some initial child worker processes, which will not be managed by Swarm. DynamicSupervisor seems to suit Swarm’s use case the most: we can start a DynamicSupervisor without any children and ensure all child processes are registered with Swarm before they are dynamically started later.

We can write our DynamicSupervisor module as such:

defmodule Yourapp.YourSupervisor do
  use DynamicSupervisor

  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  def start_link(state) do
    DynamicSupervisor.start_link(__MODULE__, state, name: __MODULE__)
  end

  def init(_) do
    DynamicSupervisor.init(strategy: :one_for_one)
  end

  def register(worker_name) do
    DynamicSupervisor.start_child(__MODULE__, worker_name)
  end
end

Note that in the init function we didn’t need to provide any actual children to be started.

The register function is a convenience function that needs to be provided to Swarm.register_name/4 whenever we want to start a worker process with Swarm. It simply calls start_child and would return the pid of the started worker.

This is how you would dynamically start your worker process anywhere in your app:

Swarm.register_name(
  :unique_name_for_this_worker_process,
  Yourapp.YourSupervisor,
  :register,
  [Yourapp.YourWorker]
)

Finally, we come to the definition for the worker process itself. Below is a minimal working example which would simply restart a killed worker process on another node, without preserving its state:

defmodule Yourapp.YourWorker do
  use GenServer

  def start_link(state) do
    GenServer.start_link(__MODULE__, state)
  end

  def init(opts) do
    initial_state = initialize_worker(opts)

    {:ok, initial_state}
  end

  def handle_call({:swarm, :begin_handoff}, _from, state) do
    {:reply, :restart, state}
  end

  def handle_info({:swarm, :die}, state) do
    {:stop, :shutdown, state}
  end

  defp initialize_worker(opts) do
    # ...
  end
end

Optionally, if you need to make use of the state handover functionality, you would need to make your worker more complicated with these additions:

  # Change the handling of :begin_handoff
  # This is triggered whenever a registered process is to be killed.
  def handle_call({:swarm, :begin_handoff}, _from, current_state) do
    {:reply, {:resume, produce_outgoing_state(current_state)}, current_state}
  end

  # Handle :end_handoff
  # This is triggered whenever a process has been restarted on a new node.
  def handle_call({:swarm, :end_handoff, incoming_state}, _from, current_state) do
    {:noreply, end_handoff_new_state(current_state, incoming_state)}
  end

Now, if you kill a node, you should see all the workers that were originally running on it automatically restarted on another node in the cluster.

Thanks for reading. We hope this walk-through has been of some help to you.

Emacs Setup for Elixir and Vue

2019-10-22T00:00:00+00:00

I had been an avid user of Spacemacs for a long time. However, it could sometimes be time-consuming to get the setup just right in Emacs land. I would like to share some of my configurations here for a satisfactory editing experience for Elixir and Vue.

Language Server Protocol

Language Server Protocol (LSP) was created by Microsoft to define a common standard for providing editor-agnostic code intelligence support. It has become widely popular since its creation.

Emacs support for lsp comes with the lsp-mode package. Both of the following setups depend on it. Therefore, one needs to install it first, either directly or via the Spacemacs lsp layer.

      # In `dotspacemacs/layers`:
      lsp

Vue.js

The landscape surrounding Vue support on Emacs has been quite confusing.

There had been an old vue-mode package, which is likely to be the first result on Google. However, the package lacks many features compared to Vetur (a popular VSCode extension) such as autocompletion, and is not being actively developed anymore.

There is also a legacy lsp-vue package, which is supposed to leverage vue-language-server from Vetur. However, this package is not compatible with the the newest version of Emacs’ lsp-mode.

Actually, the new lsp-mode already ships with a lsp-vetur.el, which provides support for Vue via vue-language-server. There is even a non-official Vue layer for Spacemacs that integrates with ESLint and Prettier to provide a complete editing experience similar to Vetur in VSCode. I have been using this layer for a while and it works great. The author has mentioned submitting a MR to Spacemacs to incorporate it into the list of official layers, but hasn’t done so yet. That partly explains why it might take one some digging to find it.

To install the layer:

# Folder for private layers
cd ~/.emacs.d/private
git clone git@github.com:thanhvg/vue.git

# Install the necessary tools manually
# vue-language-server is the language server developed by vetur
npm install -g eslint prettier vue-language-server

To configure the layer in your .spacemacs file:

      # In `dotspacemacs/layers`
      (vue :variables
           vue-backend 'lsp)

       # Depends on whether you already configured this variable in the `javascript` layer
       (node :variable node-add-modules-path)

If you want to format the Vue file with prettier automatically upon saving, add the following:

  # In dotspacemacs/user-config
  (add-hook 'vue-mode-hook
            (lambda ()
              (add-hook 'before-save-hook #'prettier-js nil t)))

You can then enjoy Vetur-like editing capabilities in Emacs. You can find default shortcuts in the layer README.

Elixir

alchemist.el had been the go-to code intelligence tool for Elixir a couple of years ago. However, its development has stalled, and elixir-ls, which provides more complete code intelligence out of the box, is currently the recommended IDE server for Elixir.

While vscode-elixir-ls downloads and builds elixir-ls automatically, we can also manually clone and build the project, so that it can be used by Emacs (and other editors), similar to what we did with vue-language-server.

git clone git@github.com:elixir-lsp/elixir-ls.git
cd elixir-ls
mix compile
# For example, when the target Elixir version is 1.9.0
mix elixir_ls.release -o release-1.9.0

Since we have several projects with different Elixir versions, I also made sure to compile several versions of elixir-ls using asdf, each corresponding to one of the target projects. One just needs to change the .tool-versions file in elixir-ls folder before compiling and releasing again.

The newest lsp-modeprovides support for elixir-ls already. A recent PR to Spacemacs’ Elixir layer introduced support for lsp backend.

      # In `dotspacemacs/layers`:
      (elixir :variables
              elixir-backend 'lsp
              elixir-ls-path "path-to-elixir-ls/release-1.9.0"

To switch between the different versions of elixir-ls compiled for different Elixir versions, we can make elixir-ls-path a directory local variable, by having a .dir-locals.el file in the project root folder:

((elixir-mode
  (elixir-ls-path . "path-to-elixir-ls/release-1.8.1")))

To add auto formatting on save:

  # In `dotspacemacs/user-config`:
  (add-hook 'elixir-mode-hook
            (lambda ()
              (add-hook 'before-save-hook #'lsp-format-buffer nil t)))

You could also refer to the comprehensive FAQ on ElixirForum, should you have any questions regarding the setup.

Tweaks and Customizations

On MacOS, you might get an error about exceeding the maximum limit of file descriptors, since lsp-mode tries to watch the folder for changes. Some workarounds are described in this post. If none of them works, you can also disable the file watching altogether:

      (lsp :variables
           lsp-enable-file-watchers nil
           )

By default, lsp-mode displays a floating window that displays documentation following your cursor movement, which I found a bit intrusive. You can turn it off by setting lsp-ui-doc-enable to nil

      (lsp :variables
           lsp-ui-doc-enable nil)

Then one can display the documentation with , h h.

I don’t like the default behavior, where the documentation buffer pops up at the bottom of the editor. After some digging, it turns out that the *Help* buffer is controlled by popwin. By adding the following configuration, I was able to view the documentation buffer to the right of the screen:

  (setcdr (assoc "*Help*" popwin:special-display-config)
          '(:dedicated t :position right :stick t :noselect t :width 0.3))

My XMonad Configuration (with Screenshots of Common XMonad Layouts)

2018-11-19T00:00:00+00:00

From i3 to XMonad
Useful modules from xmonad-contrib
xmobar and trayer
Adding an entry in /usr/share/xsessions for startup applications
My full configuration files
- ~/.xmonad/xmonad.hs
- ~/.xmobarrc

XMonad offers unparalleled customizability, especially with the extensive xmonad-contrib library. However, as the modules in xmonad-contrib are simply listed in an alphabetical order, and there’s no voting mechanism to help differentiate the usefulness (to most users at least) of them, it took me some time to go through a few of them and find what could best benefit my workflow. Setting up xmobar and trayer for the status bar was also not that straightforward.

Here I’ll list some modules that were helpful to me (accompanied by screenshots), in the hope that some might find this article useful and save them some time. My full configuration files are posted at the end of the article.

This also serves as a note to myself as I keep exploring XMonad. I’m still a learner and I’d appreciate it if you point out mistakes in my configuration.

From i3 to XMonad

I’ve been using Thinkpad X1 Carbon with Arch Linux for a while and my experience has been great. The only two features I miss from MacOS are the built-in Dictionary and the seamless HiDPI support, but I can get by without them just fine. The pleasure of being able to harness the full power of Arch Linux together with a proper window manager far outweighs the inconvenience. The topic of X1 Carbon vs. Macbook is probably best left for another article though.

I started with i3, as it is undoubtedly the most popular WM out there, and perhaps the most beginner-friendly. However, some of i3’s inflexibility constantly gnawed at me. Eventually I decided that I’m comfortable enough with WMs to begin exploring something more customizable.

In comparison to i3, the mental model adopted by XMonad is (unexpectedly) much more intuitive in several aspects, out of the box:

The concepts of “screen” and “workspace” are cleanly separate, which is great. You are able to switch to/move an application to a particular screen, without worrying about which exact workspace is currently on that screen. This is a life-changer for me as I now almost exclusively use screen-switching keys (which I’ve conveniently defined as Mod+a, Mod+s, and Mod+d).
A screen merely “projects” a workspace. This is to say, any workspace that is not currently shown, does not “belong” to any particular screen either. You can easily put any other workspace onto the current screen just by Mod+workspaceid. This is a massive improvement from the tedious i3 procedure. This also ensures that disconnecting/reconnecting to external monitors remains a smooth experience, as workspaces get automatically projected to available screens.
You are free to customize the workspace layout to your heart’s content. The automatic layout algorithms ensure that you won’t need to perform manual splits like those in i3. The default Tall layout is already very sensible in that it is automatically able to use both axes of the screen.

Useful modules from xmonad-contrib

The above are only the beginning, as xmonad-contrib offers many ready-to-use modules which massively enhance the already great defaults.

Layouts

Layout algorithms are the fundamentals of any window manager. There are tons of layouts in xmonad-contrib, but save for a summary page without screenshots on the Wiki, there doesn’t seem to be much easily accessible information around. I’ve tried out each layout in there. IMO while most of them suit very specific needs and might not be very useful for most users’ daily workflow, a few of them could become indispensable. I’ll list such layouts below, complete with screenshots.

Tabbed (`XMonad.Layout.Tabbed`)

This layout adds tabs to the default Fullscreen layout. It’s in a sense similar to i3’s default fullscreen layout.

This is the essential layout for multi-monitor setups, where each application automatically occupies the whole screen.

TwoPane (`XMonad.Layout.TwoPane`)

This is a frequent use case I had in i3: Divide a window into two panes and cycle between applications within an individual pane. For example, I might have a tech talk playing in one pane, while alternatively programming with a code editor or taking notes with org-mode in the other pane. Another example is keeping Anki/an article open in one pane and cycling between different dictionary apps in the other. The TwoPane layout achieves this by fixing the application in the main pane while allowing you to cycle through other applications in the secondary pane.

By default the split is vertical. However, just like the case in Tall layout, by simply mirroring the layout you can also make the split horizontal, as shown in the screenshot.

There’s also a DragPane layout that allows you to additional resize the split ratio by mouse, and offers more configuration options. However it didn’t seem to work on my system as the pane borders constantly blink.

ResizableTall (`XMonad.Layout.ResizableTile`)

The default Tall layout only allows for adjusting the ratio of the main split, i.e. all the secondary panes will have the same size. However, there might be a use case where you want to have one relatively large secondary pane (e.g. Emacs) and a relatively small secondary pane (e.g. terminal). ResizableTall extends Tall by allowing for the layout to be extended just fine.

The screenshot shows both the ratio of the main split and that between the secondary panes adjusted.

emptyBSP (`XMonad.Layout.BinarySpacePartition`)

This layout will automatically split your focused window in two to make space for the newly created window.

Spiral, Dwindle (`XMonad.Layout.Dwindle`)

These two layouts imitate awesomeWM and produce increasingly smaller windows in fixed locations.

I find the above listed layouts able to satisfy almost all of my daily needs for now. However, you can create much more complicated custom layouts by using modules such as Xmonad.Layout.Combo or Xmonad.Layout.LayoutCombinators.

`XMonad.Actions.PhysicalScreens`

This is an essential module for multi-monitor setups. When multiple monitors are connected, the screen ids get assigned quite arbitrarily by default. However, we’d normally want the screens numbered in a left-to-right order according to their physical locations. This module provides the getScreen and viewScreen functions that help us do just that.

Example usage:

--
-- mod-{w,e,r}, Switch to physical/Xinerama screens 1, 2, or 3
-- mod-shift-{w,e,r}, Move client to screen 1, 2, or 3
--
[((modm .|. mask, key), f sc)
    | (key, sc) <- zip [xK_w, xK_e, xK_r] [0..]
    , (f, mask) <- [(viewScreen def, 0), (sendToScreen def, shiftMask)]]

`XMonad.Layout.Spacing`

This achieves the same thing as that by i3gaps. XMonad argues that the correct terminology for this should be “spacing” instead of “gaps”, since “gaps” should refer to the gap between a window and the edges, not between panes within a window.

This makes the layout a bit less crowded.

myLayout = avoidStruts $
  ...
  ||| tiled
  ||| twopane
  ...
  where
     tiled = spacing 3 $ ResizableTall nmaster delta ratio []
     twopane = spacing 3 $ TwoPane delta ratio

`XMonad.Hooks.EwmhDesktops`

You need to add an ewmh hook if you want to correctly use rofi to locate and switch to a running application by its name.

Just import the module and then add ewmh as such:

main = do
    ...
    xmonad $ ewmh def
        {
        ...
        }

`XMonad.Layout.NoBorders`

It would be silly to have a border around the window if the window always occupies the whole screen. Use noBorders to avoid that in such layouts (e.g. tabbed, Full).

myLayout = avoidStruts $
  noBorders (tabbed shrinkText myTabConfig)
  ...

`historyHook` (`XMonad.Actions.GroupNavigation`)

historyHook keeps track of your window history and allows for actions such as going back to the most recent window.

Just append >> historyHook to the end of your logHook, e.g.

        , logHook = dynamicLogWithPP myPP {
                                          ppOutput = hPutStrLn xmproc
                                          }
                        >> historyHook

xmobar and trayer

Normally one would want to have a status bar and an application/applet tray. The most popular choices for those seems to be xmobar and trayer.

The configuration options for xmobar is stored in .xmobarrc. They are relatively well-documented in the official README.

Note that one would need to manually leave some space to the side of the xmobar so that the trayer can be displayed:

In .xmobarrc:

Config {
       ...
       , position = TopW L 85
       ...
       }

I spawn xmobar in my xmonad.hs file:

main = do
    xmproc <- spawnPipe "xmobar"
    xmonad $ ewmh def
        {
        ...
        , logHook = dynamicLogWithPP myPP {
                                          ppOutput = hPutStrLn xmproc
                                          }
                        >> historyHook
        ...
        }

and spawn trayer in my startup script:

trayer --edge top --align right --SetDockType true --SetPartialStrut true \
       --expand true --width 15 --transparent true --alpha 0 --tint 0x283339 --height 20\
       --monitor 1 &

Note that by setting --transparent true, --alpha 0 --tint 0x283339, I was able to ensure that it has the same background color as what I set in .xmobarrc.

Since xmobar and trayer are completely separate processes, if one of them crashes you can just relaunch it individually without impacting the other one’s normal functioning.

Adding an entry in /usr/share/xsessions for startup applications

Finally, when logging in, one might want to launch some startup applications prior to launching xmonad itself, just as one would do in .i3/config with exec.

You can simply write a bash script run-xmonad which includes all the commands you want to run. For example:

#!/bin/bash

setxkbmap -option "ctrl:nocaps"
xset r rate 200 50 &

trayer --edge top --align right --SetDockType true --SetPartialStrut true \
       --expand true --width 15 --transparent true --alpha 0 --tint 0x283339 --height 20\
       --monitor 1 &
nm-applet &
xfce4-power-manager &
pactl load-module module-bluetooth-discover &
blueman-applet &

# User apps
dropbox start &
thunderbird &
goldendict &
pulseaudio &
pa-applet &

xmonad

Note that there is a file /usr/share/xsessions/xmonad.desktop already, which allows you to launch xmonad after logging into an xsession. You can simply create a copy and change the line

Name=Xmonad
Exec=xmonad

Name=Xmonad (with startup apps)
Exec=/home/jx/Dropbox/scripts/run-xmonad

You should then be able to choose this new entry from your dm at your next login.

My full configuration files

`~/.xmonad/xmonad.hs`

import XMonad hiding ((|||))
import qualified XMonad.StackSet as W
import qualified Data.Map        as M

-- Useful for rofi
import XMonad.Hooks.EwmhDesktops
import XMonad.Hooks.DynamicLog
import XMonad.Hooks.ManageDocks
import XMonad.Util.Run(spawnPipe)
import XMonad.Util.EZConfig(additionalKeys, additionalKeysP, additionalMouseBindings)
import System.IO
import System.Exit
-- Last window
import XMonad.Actions.GroupNavigation
-- Last workspace. Seems to conflict with the last window hook though so just
-- disabled it.
-- import XMonad.Actions.CycleWS
-- import XMonad.Hooks.WorkspaceHistory (workspaceHistoryHook)
import XMonad.Layout.Tabbed
import XMonad.Hooks.InsertPosition
import XMonad.Layout.SimpleDecoration (shrinkText)
-- Imitate dynamicLogXinerama layout
import XMonad.Util.WorkspaceCompare
import XMonad.Hooks.ManageHelpers
-- Order screens by physical location
import XMonad.Actions.PhysicalScreens
import Data.Default
-- For getSortByXineramaPhysicalRule
import XMonad.Layout.LayoutCombinators
-- smartBorders and noBorders
import XMonad.Layout.NoBorders
-- spacing between tiles
import XMonad.Layout.Spacing
-- Insert new tabs to the right: https://stackoverflow.com/questions/50666868/how-to-modify-order-of-tabbed-windows-in-xmonad?rq=1
-- import XMonad.Hooks.InsertPosition

--- Layouts
-- Resizable tile layout
import XMonad.Layout.ResizableTile
-- Simple two pane layout.
import XMonad.Layout.TwoPane
import XMonad.Layout.BinarySpacePartition
import XMonad.Layout.Dwindle

myTabConfig = def { activeColor = "#556064"
                  , inactiveColor = "#2F3D44"
                  , urgentColor = "#FDF6E3"
                  , activeBorderColor = "#454948"
                  , inactiveBorderColor = "#454948"
                  , urgentBorderColor = "#268BD2"
                  , activeTextColor = "#80FFF9"
                  , inactiveTextColor = "#1ABC9C"
                  , urgentTextColor = "#1ABC9C"
                  , fontName = "xft:Noto Sans CJK:size=10:antialias=true"
                  }

myLayout = avoidStruts $
  noBorders (tabbed shrinkText myTabConfig)
  ||| tiled
  ||| Mirror tiled
  -- ||| noBorders Full
  ||| twopane
  ||| Mirror twopane
  ||| emptyBSP
  ||| Spiral L XMonad.Layout.Dwindle.CW (3/2) (11/10) -- L means the non-main windows are put to the left.

  where
     -- The last parameter is fraction to multiply the slave window heights
     -- with. Useless here.
     tiled = spacing 3 $ ResizableTall nmaster delta ratio []
     -- In this layout the second pane will only show the focused window.
     twopane = spacing 3 $ TwoPane delta ratio
     -- The default number of windows in the master pane
     nmaster = 1
     -- Default proportion of screen occupied by master pane
     ratio   = 1/2
     -- Percent of screen to increment by when resizing panes
     delta   = 3/100

myPP = def { ppCurrent = xmobarColor "#1ABC9C" "" . wrap "[" "]"
           , ppTitle = xmobarColor "#1ABC9C" "" . shorten 60
           , ppVisible = wrap "(" ")"
           , ppUrgent  = xmobarColor "red" "yellow"
           , ppSort = getSortByXineramaPhysicalRule def
           }

myManageHook = composeAll [ isFullscreen --> doFullFloat

                          ]

myKeys conf@(XConfig {XMonad.modMask = modm}) = M.fromList $

    -- launch a terminal
    [ ((modm .|. shiftMask, xK_Return), spawn $ XMonad.terminal conf)

    -- close focused window
    , ((modm .|. shiftMask, xK_q     ), kill)

     -- Rotate through the available layout algorithms
    , ((modm,               xK_space ), sendMessage NextLayout)

    --  Reset the layouts on the current workspace to default
    , ((modm .|. shiftMask, xK_space ), setLayout $ XMonad.layoutHook conf)
    , ((modm .|. shiftMask, xK_h), sendMessage $ JumpToLayout "Mirror Tall")
    , ((modm .|. shiftMask, xK_v), sendMessage $ JumpToLayout "Tall")
    , ((modm .|. shiftMask, xK_f), sendMessage $ JumpToLayout "Full")
    , ((modm .|. shiftMask, xK_t), sendMessage $ JumpToLayout "Tabbed Simplest")

    -- Resize viewed windows to the correct size
    , ((modm,               xK_n     ), refresh)

    -- Move focus to the next window
    , ((modm,               xK_Tab   ), windows W.focusDown)

    -- Move focus to the next window
    , ((modm,               xK_j     ), windows W.focusDown)

    -- Move focus to the previous window
    , ((modm,               xK_k     ), windows W.focusUp  )

    -- Move focus to the master window
    , ((modm,               xK_m     ), windows W.focusMaster  )

    -- Swap the focused window and the master window
    , ((modm,               xK_Return), windows W.swapMaster)

    -- Swap the focused window with the next window
    , ((modm .|. shiftMask, xK_j     ), windows W.swapDown  )

    -- Swap the focused window with the previous window
    , ((modm .|. shiftMask, xK_k     ), windows W.swapUp    )

    -- Shrink the master area
    , ((modm,               xK_h     ), sendMessage Shrink)

    -- Expand the master area
    , ((modm,               xK_l     ), sendMessage Expand)

    -- Shrink and expand ratio between the secondary panes, for the ResizableTall layout
    , ((modm .|. shiftMask,               xK_h), sendMessage MirrorShrink)
    , ((modm .|. shiftMask,               xK_l), sendMessage MirrorExpand)

    -- Push window back into tiling
    , ((modm,               xK_t     ), withFocused $ windows . W.sink)

    -- Increment the number of windows in the master area
    , ((modm              , xK_comma ), sendMessage (IncMasterN 1))

    -- Deincrement the number of windows in the master area
    , ((modm              , xK_period), sendMessage (IncMasterN (-1)))

    -- Toggle the status bar gap
    -- Use this binding with avoidStruts from Hooks.ManageDocks.
    -- See also the statusBar function from Hooks.DynamicLog.
    , ((modm              , xK_b     ), sendMessage ToggleStruts)
    , ((controlMask, xK_Print), spawn "sleep 0.2; scrot -s")
    , ((0, xK_Print), spawn "scrot") -- 0 means no extra modifier key needs to be pressed in this case.
    , ((modm, xK_F3), spawn "pcmanfm")
    , ((modm.|. shiftMask, xK_F3), spawn "gksu pcmanfm")
    ]

    ++
      [((m .|. modm, k), windows $ f i)
      | (i, k) <- zip (XMonad.workspaces conf) [xK_1 .. xK_9]
      , (f, m) <- [(W.greedyView, 0), (W.shift, shiftMask)]]
    ++
      [((m .|. modm, key), f sc)
      | (key, sc) <- zip [xK_a, xK_s, xK_d] [0..]
      -- Order screen by physical order instead of arbitrary numberings.
      , (f, m) <- [(viewScreen def, 0), (sendToScreen def, shiftMask)]]

main = do
    xmproc <- spawnPipe "xmobar"
    xmonad $ ewmh def
        { modMask = mod4Mask
        , keys = myKeys
        , manageHook = manageDocks <+> myManageHook
        , layoutHook = myLayout
        , handleEventHook = handleEventHook def <+> docksEventHook
        , logHook = dynamicLogWithPP myPP {
                                          ppOutput = hPutStrLn xmproc
                                          }
                        >> historyHook
        , terminal = "lxterminal"
        -- This is the color of the borders of the windows themselves.
        , normalBorderColor  = "#2f3d44"
        , focusedBorderColor = "#1ABC9C"
        }
        `additionalKeysP`
        [
          ("M1-<Space>", spawn "rofi -combi-modi window,run,drun -show combi -modi combi")
          , ("C-M1-<Space>", spawn "rofi -show run")
          -- Restart xmonad. This is the same keybinding as from i3
          , ("M-S-c", spawn "xmonad --recompile; xmonad --restart")
          , ("M-S-q", kill)
          , ("M-'", windows W.swapMaster)
          , ("M1-<Tab>", nextMatch History (return True))
          , ("M-<Return>", spawn "lxterminal")
          -- Make it really hard to mispress...
          , ("M-M1-S-e", io (exitWith ExitSuccess))
          , ("M-M1-S-l", spawn "i3lock")
          , ("M-M1-S-s", spawn "i3lock && systemctl suspend")
          , ("M-M1-S-h", spawn "i3lock && systemctl hibernate")
        ] `additionalMouseBindings`
        [ ((mod4Mask, button4), (\w -> windows W.focusUp))
        , ((mod4Mask, button5), (\w -> windows W.focusDown))
        ]

`~/.xmobarrc`

Config {
       font = "xft:Noto Sans:size=9:antialias=true,Noto Sans CJK SC:size=9:antialias=true"
       , bgColor = "#283339"
       , fgColor = "#F9fAF9"
       , position = TopW L 85
       , commands = [
                    Run Battery [ "--template" , "B: <acstatus>"
                                , "--L" , "15"
                                , "--H" , "75"
                                , "--low"      , "darkred"
                                , "--normal"   , "darkorange"
                                , "--high"     , "#1ABC9C"
                                , "--" -- battery specific options
                                       -- discharging status
                                       , "-o"	, "<left>% (<timeleft>)"
                                       -- AC "on" status
                                       , "-O"	, "<fc=#dAA520>Charging</fc>"
                                       -- charged status
                                       , "-i"	, "<fc=#1ABC9C>Charged</fc>"
                                ] 50
                    -- , Run Cpu [ "--template" , "C: <total>%", "-L","0","-H","50","--normal","#1ABC9C","--high","darkred"] 10
                    -- , Run Memory ["-t","M: <usedratio>%"] 10
                    , Run DiskU [("/", "D: <free>")] ["-L", "20", "-H", "60"] 10
                    -- , Run Swap [] 10
                    , Run Date "%a %d.%m. %H:%M" "date" 10
                    , Run StdinReader
                    ]
       , sepChar = "%"
       , alignSep = "}{"
       , template = "%StdinReader% }{ %battery% | %disku% | %date%"
       }

Multitenant Phoenix application deployment with SSL using HAProxy

2018-08-01T00:00:00+00:00

Table of Contents

Installing HAProxy
Phoenix-specific HAProxy configurations
- Checking for errors in the HAProxy configuration file
Configuring Phoenix Apps
- Working with Distillery
Adding TLS support via HAProxy

Recently I needed to deploy two Phoenix applications on a CentOS server that I was provided with. A normal way to do so would be to use subdomains, i.e. using appA.domain.com and appB.domain.com. However, in this case I was not able to configure the DNS records for the server. I had to thus resort to appending path segments to the domain, i.e. domain.com/appA and domain.com/appB, and rely on a reverse proxy to route the request to the corresponding app.

HAProxy seemed an attractive option for the reverse proxy, as its configuration syntax is clean and straightforward. However, there is relatively few documentation for it compared with Nginx, and it took me a while to figure everything out. I’d like to share my experience in this article.

The commands are tested on a fresh CentOS 7.5 VM.

Installing HAProxy

This section documents the steps to install the newest HAProxy and get it running. You may also just choose to install it using the package manager of your system and skip ahead to the next section after verifying the installation. Part of this section is based on this online guide.

The HAProxy version in the CentOS repo is not up to date. To install the newest version of HAProxy, first download the source code from http://www.haproxy.org/#down.

wget http://www.haproxy.org/download/1.8/src/haproxy-1.8.13.tar.gz
tar -xzvf haproxy-1.8.13.tar.gz
cd haproxy-1.8.13

You may first need to install the dependencies with

sudo yum install gcc pcre-static pcre-devel openssl openssl-devel

before the compilation.

You can then compile HAProxy with

sudo make TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1

A few notes about the flags:

TARGET=linux2628: The build targets Linux 2.6.28, 3.x, and above. Such kernel versions should be the ones in use by default nowadays in most Linux systems.
USE_PCRE=1: Use libpcre for regex processing.
USE_OPENSSL=1: This is important for adding SSL/TLS support. Otherwise we would not be able to let HAProxy use our own SSL certificates in the later configurations.
USE_ZLIB=1: Enable native support for zlib to benefit from HTTP compression.

After the compilation finishes, run

sudo make install

to install it.

Then, some initialization steps need to be performed:

sudo ln -s /usr/local/sbin/haproxy /usr/sbin/haproxy
sudo mkdir -p /etc/haproxy
sudo mkdir -p /var/lib/haproxy
# Useful if you want to enable the default stats page for HAProxy
sudo touch /var/lib/haproxy/stats
sudo cp ~/haproxy-1.8.13/examples/haproxy.init /etc/init.d/haproxy
sudo chmod 755 /etc/init.d/haproxy
sudo systemctl daemon-reload
sudo chkconfig haproxy on
sudo useradd -r haproxy

Before we are able to use HAProxy, we may also need to open up the ports 80 and 443 to outside traffic, if they aren’t open already. You can either do it by directly modifying iptables, or use the firewall-cmd frontend that comes with CentOS 7. The following is the direct iptables command:

sudo iptables -A INPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo systemctl reload iptables

Now we can add a basic configuration file with sudo vi /etc/haproxy/haproxy.cfg:

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 2048

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000
    option forwardfor
    option http-server-close

frontend http_front
    bind *:80
    stats uri /haproxy?stats
    default_backend http_back

backend http_back
    balance roundrobin
    server web1 127.0.0.1:80 check

We can then start up HAProxy with

sudo systemctl start haproxy

You should be able to see the HAProxy stats page at http://example.com/haproxy?stats. This means you have successfully installed HAProxy.

Phoenix-specific HAProxy configurations

Now that HAProxy runs normally, let’s edit the configuration file /etc/haproxy/haproxy.cfg so that it is able to serve two different Phoenix apps, distinguishing them using path segments.

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 2048

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000
    option forwardfor
    option http-server-close

frontend http_front
    # We don't really need the stats page now.
    # stats uri /haproxy?stats
    bind *:80
    acl appA_url url_beg /appA
    acl appB_url url_beg /appB
    use_backend appA if appA_url
    use_backend appB if appB_url

backend appA
    http-request set-path %[path,regsub(/appA/?,/)]
    balance roundrobin
    server web1 127.0.0.1:4001 check

backend appB
    http-request set-path %[path,regsub(/appB/?,/)]
    balance roundrobin
    server web1 127.0.0.1:5001 check

acl appA_url url_beg /appA creates an access control list called appA_url that is activated whenever the URL path after the domain name begins with /appA.
use_backend appA if appA_url then tells HAProxy to select the backend called appA if the ACL appA_url is activated.
http-request set-path %[path,regsub(/appA/?,/)] modifies the HTTP request so that the path segment /appA/ is replaced with / before it goes through to the actual Phoenix app. This is important because we don’t want to modify the router.ex file of the app. That is to say, if you have

  scope "/", App do
    pipe_through :browser
    get "/", PageController, :index
  end

which expects to serve the homepage at http://example.com/, the router will still receive the incoming HTTP requests as if they were for http://example.com/ instead of for http://example.com/appA/ , and thus serve the pages correctly.

(We will also need to change a part of Phoenix configuration for this to work. This will be covered in the next section.)

server web1 127.0.0.1:4001 check assumes that appA is being served at port 4001 of this machine. The relevant Phoenix configuration will be covered in the next section.

Don’t forget to restart HAProxy with sudo systemctl restart haproxy after saving the changes.

Checking for errors in the HAProxy configuration file

N.B.: Sometimes the haproxy service might fail to start due to errors in `haproxy.cfg`. The following command checks for errors in the config file:

sudo haproxy -c -V -f /etc/haproxy/haproxy.cfg

Configuring Phoenix Apps

Now we need to also configure the relevant Phoenix apps so that they work in conjunction with HAProxy. For this we need to modify App.Endpoint in config/prod.exs:

config :app, App.Endpoint,
  url: [host: "example.com", path: "/appA"],
  http: [port: 4001],
  cache_static_manifest: "priv/static/cache_manifest.json"

Note how we added path: "/appA" under the :url key. This is necessary, because we want the links automatically generated by Phoenix (e.g. http://example.com/appA/js/app-01302d06e5b1534d67ec820fde5c1292.js?vsn=d) to still have the /appA path segment in them, so that HAProxy can properly identify the HTTP requests as going to appA. After identifying the incoming requests, the ACL appA_url will be activated and HAProxy will proceed to strip the path segment, before rerouting the HTTP request to the appA backend.

Think about it this way: If we don’t do this, the incoming HTTP requests by clients would directly ask for example.com/js/app.js instead of example.com/appA/js/app.js. However, since HAProxy doesn’t have any ACL defined that matches the former case, it won’t know how to handle the request, and an error will be returned as the result.
We are using HTTP since HTTPS connection between the server and the client will be handled by HAProxy. Of course, it can still be a good practice to use HTTPS for the connection between HAProxy and Phoenix, especially if your app and HAProxy actually live on different machines.
We manually specify port: 4001 to match up with the port specified in the HAProxy configuration. You can also use {:system, "PORT"} instead of 4001, and feed in the port as an environment variable when launching the app.

Needless to say, the same edits are to be performed on appB, with the path as "/appB" and the port as 5001.

To ensure that the applications automatically start on boot, create a systemd service for each app in the folder /lib/systemd/system:

app-a.service:

[Unit]
Description=AppA

[Service]
Type=simple
User=username
Group=groupname
Restart=on-failure
Environment=MIX_ENV=prod "PORT=4001"
Environment=LANG=en_US.UTF-8

WorkingDirectory=/path/to/appA
ExecStart=/usr/local/bin/mix phoenix.server

[Install]
WantedBy=multi-user.target

app-b.service:

[Unit]
Description=AppB

[Service]
Type=simple
User=username
Group=groupname
Restart=on-failure
Environment=MIX_ENV=prod "PORT=5001"
Environment=LANG=en_US.UTF-8

WorkingDirectory=/path/to/appB
ExecStart=/usr/local/bin/mix phoenix.server

[Install]
WantedBy=multi-user.target

and enable them with sudo systemctl enable app-a.service sudo systemctl enable app-b.service. Start them with systemctl start.

If everything is set up correctly up to this point, you should now be able to access the applications at http://example.com/appA and http://example.com/appB.

(The systemd configuration is taken from an ElixirForum thread by yurko, where further details and discussions can be found.)

Working with Distillery

Instead of the plain

WorkingDirectory=/path/to/appB
ExecStart=/usr/local/bin/mix phoenix.server

we might want to use distillery to package our Phoenix apps. This also allows us to stop and restart the apps more easily. The corresponding systemd config would be:

[Service]
Type=simple
User=user
Group=group
WorkingDirectory=/path/to/appB
ExecStart=/path/to/appB/_build/prod/rel/appB/bin/appB foreground
ExecStop=/path/to/appB/_build/prod/rel/appB/bin/appB stop
Environment=LANG=en_US.UTF-8
Environment=MIX_ENV=prod
Environment=RELEASE_MUTABLE_DIR=/path/to/appB/_build/prod/rel/appB/var/tmp
LimitNOFILE=65536
UMask=0027
SyslogIdentifier=appB
Restart=always
RestartSec=5

Adding TLS support via HAProxy

Normally we’d want to serve our apps via HTTPS, which HAProxy also supports.

First, we need to concatenate all the related keys/certs (.pem files) into one. The order required by HAProxy is:

server private key (without any password)
server certificate
intermediate certificate 1
intermediate certificate 2
Root trusted authority certificate (if any)

For example, if your cert files are stored under /etc/pki/tls/certs/, you would run

DOMAIN='example.com' sudo -E bash -c 'cat /etc/pki/tls/private/key_example.pem  /etc/pki/tls/certs/cert_example.pem /etc/pki/tls/certs/chain.pem > /etc/haproxy/certs/$DOMAIN.pem'

If you use letsencrypt, it’s likely that you already have a fullchain.pem that you can use directly.

DOMAIN='example.com' sudo -E bash -c 'cat /etc/letsencrypt/live/privkey.pem /etc/letsencrypt/live/fullchain.pem > /etc/haproxy/certs/$DOMAIN.pem'

There is also a letsencrypt plugin for HAProxy, though I haven’t used it myself.

Then, we need to modify /etc/haproxy/haproxy.cfg again:

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    maxconn 2048
    # TLS
    tune.ssl.default-dh-param 2048
    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
    # Disable SSL v3, which is insecure
    ssl-default-bind-options no-sslv3

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000
    option forwardfor
    option http-server-close

frontend http_front
    bind *:80
    http-request add-header X-Forwarded-Proto http
    acl appA_url url_beg /appA
    acl appB_url url_beg /appB
    use_backend appA if appA_url
    use_backend appB if appB_url

frontend https_front
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem
    http-request add-header X-Forwarded-Proto https
    acl appA_url url_beg /appA
    acl appB_url url_beg /appB
    use_backend appA if appA_url
    use_backend appB if appB_url

backend appA
    http-request redirect scheme https if ! { ssl_fc }
    http-request set-path %[path,regsub(/appA/?,/)]
    balance roundrobin
    server web1 127.0.0.1:4001 check

backend appB
    http-request redirect scheme https if ! { ssl_fc }
    http-request set-path %[path,regsub(/appB/?,/)]
    balance roundrobin
    server web1 127.0.0.1:5001 check

We added some TLS-specific configurations under global, after the comment # TLS
We added a HTTPS frontend. It binds to port 443, and uses the concatenated certificate file that we just produced in the previous step.
In the backends, we added a http-request redirect directive to make sure that even if a user visited the website via HTTP (port 80), they will still be redirected to the HTTPS site instead.

Restart haproxy with sudo systemctl restart haproxy. Now, the apps should be up and running under https://example.com/appA and https://example.com/appB.

(Here is an alternative configuration for the frontend and backend portions):

frontend http_front
    bind *:80
    http-request add-header X-Forwarded-Proto http
    http-request redirect scheme https

frontend https_front
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem
    http-request add-header X-Forwarded-Proto https
    acl appA_url url_beg /appA
    acl appB_url url_beg /appB
    use_backend appA if appA_url
    use_backend appB if appB_url

backend appA
    http-request set-path %[path,regsub(/appA/?,/)]
    balance roundrobin
    server web1 127.0.0.1:4001 check

backend appB
    http-request set-path %[path,regsub(/appB/?,/)]
    balance roundrobin
    server web1 127.0.0.1:5001 check

Circumventing the Great Fire Wall (GFW) in China with Shadowsocks

2016-06-08T00:00:00+00:00

Update March 2020

It’s been a while since this article was originally written (in 2016). Fortunately, Shadowsocks remains a reliable way to circumvent the GFW, especially considering the fact that even more paid VPN services are struggling now. There have been a few changes to Shadowsocks over the years, and I’d like to provide a more up-to-date and minimalist guide.

Server

There have been multiple Shadowsocks server implementations over the years. Currently the most active one is shadowsocks-rust, though the one I find easiest to use is go-shadowsocks2.

Set up a Digital Ocean VPS (Detailed explanation and alternatives to DO can be found in the section “Spinning Up a VPS” below). After setting up the VPS on DigitalOcean, we’ll want to download the go-shadowsocks2 binary:
Ensure that wget, gunzip and ufw are installed by running sudo apt install wget gunzip ufw.
Open the port 8488 with sudo ufw allow 8488; sudo ufw enable.
Download the release with wget https://github.com/shadowsocks/go-shadowsocks2/releases/download/v0.1.0/shadowsocks2-linux.gz. You can find the latest release version at the Github repo.
Extract the executable with gunzip shadowsocks2-linux.gz
Change the permission for the executable with chmod +x ./shadowsocks2-linux
Run the server with ./shadowsocks2-linux -s 'ss://AEAD_CHACHA20_POLY1305:your_password@:8488' -verbose

By now you should already have a running Shadowsocks server. If you want to make it more robust/”proper”, you could make it a systemd service on the VPS, which I’m not going to describe in detail here.

Client

There’s no need for the proxy switching plugins anymore. Native clients for Shadowsocks exist for all major platforms(Windows, MacOS, Linux, Android, iOS) and are fairly intuitive to use. I would just describe the necessary configuration, based on the way we started the server above:

Server: The IP address of the Digital Ocean VPS that you launched.
Remote port: 8488 (corresponds to the 8488 part above).
Password: your_password (corresponds to the your_password part above).
Encryption Method: CHACHA20-IETF-POLY1305 (corresponds to the AEAD_CHACHA20_POLY1305 part above).

That’s it. If everything’s configured correctly, you should now be able to access the internet without restrictions, via the VPS.

Note that for some applications, you may need to manually configure the proxy so that their traffic goes through the Shadowsocks client. The following is an example for Dropbox under MacOS. Under Dropbox Preferences’ “Network” tab, you can find a “Proxies” button. Configure the proxy with the following parameters:

Proxy type: SOCKS5
Server: 127.0.0.1:1086 (this is the port on which our local Shadowsocks client is running).

You should be able to configure other apps similarly.

Introduction

I’m not sure how great a need there is for another proxy to circumvent the Great Fire Wall (GFW) for expats living in China. You’ve probably done your homework and obtained a VPN before coming here, or have some VPN service already available from your company. Nevertheless, seeing that the government is increasingly tightening its grip on the Internet and killing a few previously functioning paid VPN services, and that there seem to be few posts about Shadowsocks out there available in English, I feel it wouldn’t hurt to contribute another entry here. The advantage of such a proxy is obvious: since you’ll be the one who operates your own server with presumably limited bandwidth usage, it would be very hard for the authority to find it out, target it and kill it off. It has become a kind of de facto GFW circumvention tool for many programmers. Hopefully this post would be of some use to you and help you access the Internet more happily in China, or whatever place with heavy Internet censorship. At least this would also serve as a note for myself if I were to perform a new installation some time later.

A word of caution before I proceed: Shadowsocks refers to a protocol and a family of open-source software which implement the protocol, originally conceived by one developer (@clowwindy). Its encryption relies on a pre-shared key and block cipher algorithms. It focuses on being lightweight and fast, however the security aspects of the Shadowsocks protocol likely haven’t been rigorously evaluated by security experts. It’s good for daily needs, but you probably won’t want to solely rely on it if you need total anonymity/a particularly strong level of protection.

Spinning Up a VPS (Virtual Private Server)

Shadowsocks is a proxy software, it means you’ll first need a server to proxy the traffic with. There are a handful of decent VPS (Virtual Private Server) providers out there with attractive pricing (starting from $5 per month), for example:

(You may notice that referral codes are included in some of the links. Signing up from a referral can get you $10 credit at DigitalOcean, while Vultr offers $5 credit currently).

There are also some other low-end options available at lower prices. However they don’t usually offer servers in Asia:

https://lowendbox.com/
http://buyvm.net/

If you’re in China, it would make sense to choose a server in the Asia-Pacific region. Vultr offers Tokyo and Sydney, while DigitalOcean offers Singapore and most recently, Bangalore. The Tokyo and Singapore servers are reported to work pretty well. However, if your network, like mine, is provided by China Telecom, it seems that the Bangalore/Sydney server’s performance far exceeds those of the previous two. I was only able to watch YouTube in 240p with the Tokyo server, but now 720p isn’t much of an issue. Or maybe it’s just because the region only has relatively few users by now. In any case, YMMV and it’s always best to try them out by yourself, either via the official speedtest or better, just set up the proxy and test it in action: The servers are billed per hour so just performing some tests won’t rake up huge costs.

I’ll demonstrate using the example of a server running Ubuntu 16.04 LTS on DigitalOcean. This process should be applicable to other VPS providers as well since they all provide Ubuntu as an option.

As shown in the picture below, Choose Ubuntu 16.04 LTS x32, $5/mo option, and the appropriate region.
Set up your user account. There is an excellent tutorial series on DigitalOcean detailing the whole process:
1. Step 1, connect to the droplet with ssh. You’ll use OpenSSH if you’re on Mac/Unix or PuTTY on Windows to ssh into the newly created droplet.
2. Step 2, add a new user. This is actually not mandatory for setting up Shadowsocks but it’s a part of the common procedure of setting up a Ubuntu server. It should make the system more secure.
For the record, the following are all the commands needed to add a user called “demo”. The $local prefix means the command is executed on your local machine:

local$ ssh root@SERVER_IP_ADDRESS
adduser demo
gpasswd -a demo sudo
local$ ssh-copy-id demo@SERVER_IP_ADDRESS # Optional, add public key authentication
vim /etc/ssh/sshd_config # Change PermitRootLogin to no. You may also use nano instead of vim
service ssh restart

Install the server application for Shadowsocks

After you’re able to log in as the new user you’ve just created (via ssh demo@SERVER_IP_ADDRESS), you can now install the server application for Shadowsocks. There are a few implementations in different languages available. I’m using the Go version by @cyfdecyf since

It’s easy to run. Just download the binary and you’re good to go.
It supports multiple clients with a single process. This means I can simultaneously let my computer and my phone use the proxy from the same server without further complication.

The steps:

mkdir shadowsocks && cd shadowsocks to make a directory for shadowsocks.
Find the latest release of Shadowsocks and download the archive. To download the currently newest release for 32-bit Linux, run curl -L -O https://github.com/shadowsocks/shadowsocks-go/releases/download/1.1.5/shadowsocks-server-linux32-1.1.5.gz
Extract the binary by running gzip -d shadowsocks-server-linux32-1.1.5.gz
Make the binary executable by chmod a+x shadowsocks-server-linux32-1.1.5
Make a configuration file for the server by executing vim config.json (or nano config.json) and pasting in the content. The following is an example. Note how in this example I specified two separate ports for computer and cellphone, with different passwords “computer” and “mobile”. The full documentation of the file format is available at Github
```
 {
     "port_password": {
         "8388": "computer",
         "8387": "mobile"
     },
     "method": "aes-128-cfb-auth",
     "timeout": 600
 }
```
Run the server by executing ./shadowsocks-server-linux32-1.1.5 You should see the server displaying startup messages. We’re done here.

(Should there be any abnormality on the server, you can just restart the VPS (via the command line or using control panel of DigitalOcean), ssh into the VPS by ssh demo@SERVER_IP_ADDRESS, and start this application again by cd shadowsocks && ./shadowsocks-server-linux32-1.1.5)

Install the client

The full range of clients available is documented at the official website. Choose the one that suits your platform. I recommend the Qt5 versions for their intuitive GUI. Besides, the alternative version shadowsocks-win didn’t seem to work for me on Windows and is only available in Chinese. The following are the steps for configuration on Ubuntu desktop. The process should be similar for any other system, including Windows and Android.

install the app from the corresponding ppa (on Windows you would just directly download and open the .exe file):
```
 sudo add-apt-repository ppa:hzwhuang/ss-qt5
 sudo apt-get update
 sudo apt-get install shadowsocks-qt5
```
Launch the app from command line. Note that although the package is called shadowsocks-qt5, the binary itself is called ss-qt5!
Create a new profile. The following is my configuration file. Please fill in your own server address and password. Ensure your encryption method matches that specified on the server.

Important: Remember to select the option “One-time authentication”

If the connection is successful, then “test latency” in the client app should give you a reasonable latency figure.
Configure proxy settings. There are some methods to set the global proxy for the system, but the more common use case is to configure on an app-by-app basis, which would also be more reliable and flexible.
- For Firefox, the setting is at Preferences -> Advanced -> Connection.
  
  Important: Remember to tick the option “Remote DNS”. Otherwise the DNS poisoning attack will still be in effect.
- For the stable version of Chrome, there is no built-in setting for proxy. A plugin called SwitchyOmega will help. Since you probably can’t reach Chrome Web Store without circumventing the GFW, you’ll have to search for and install the plugin manually. You may directly download it here. The blogger auooo has also made it available on a web storage. Drag the file into the Chrome “extensions” page (enter chrome://extensions/ in your Chrome address bar to access the page) to install it. Once installed, the configuration is as follows:
  
  Also remember to select the shadowsocks profile by clicking on the SwitchyOmega icon on the menu bar:
- For Dropbox, the setting is directly visible among the configuration tabs.
You should be able to apply the similar configuration procedure for other apps that need proxying.

The Android app should be able to proxy all traffic globally by default without the need of manually configuring proxy.

For those who can read Chinese, there is a very detailed post by auooo with step-by-step screenshots of client configuration on every operating system. You may want to refer to that post as well.

P.S.: For Mac OS X users, the most common GUI seems to have Chinese as the only available UI language. You may want to follow the step-by-step guide with pictures (in Chinese), or alternatively, build the Qt5 version of the client by yourself.

Solving CJK Font Rendering Issue in Chrome on Ubuntu 16.04

2016-04-29T00:00:00+00:00

Update: The fix for this bug has been released to xenial-updates.

(中文版本看这里)

There is a bug in Ubuntu 16.04 LTS’s default fallback CJK font package fonts-noto-cjk that makes Chrome/Chromium render “thin” weight of the font for Chinese/Japanese/Korean characters, whenever the page’s language is not explicitly set to cn/jp/ko. This makes the characters unreadable. For example, in a Google search page which contains both English and CJK texts, the CJK text will be rendered thin.

(Example image taken from AskUbuntu)

I’ve reported this bug. It seems the issue is caused by the fact that fonts-noto-cjk uses an all-in-one file which contains all the weights of the font. The problem probably lies with fontconfig or Chrome/Chromium’s font fallback mechanism.

In the meantime, the temporary fix is to remove the package, download the font Noto Sans CJK from Google with each weight contained in an individual file, and manually install the fonts. Then Chrome/Chromium should use the correct weight of the font to render CJK characters.

A member of the Ubuntu Development Team, Gunnar Hjalmarsson, has made a personal PPA containing the font files individually. You may also consider directly using that PPA.

解决Ubuntu 16.04 Chrome 里中文文字发虚问题

2016-04-29T00:00:00+00:00

更新: 这个 bug 的补丁已经被发布在 xenial-updates.

(English version here)

Ubuntu 16.04 LTS 的默认中日韩字体package fonts-noto-cjk 有一个 bug。如果网页没有把其页面语言指定为中文/日文/韩文，Chrome/Chromium 会采用”细”字体来显示中/日/韩字符，非常难看。下面是从 AskUbuntu 拿的一个示例图片。

我 report 了这个 bug。这似乎是由于 fonts-noto-cjk 只使用一个文件来包括所有粗细标准的字体。有可能是 fontconfig 或者是 Chrome/Chromium 选择默认字体的机制造成了问题。

暂时的解决方案是，移除 fonts-noto-cjk package, 手动从 Google 下载并安装每个 weight 都单独存于一个文件中的 Noto Sans CJK 字体。这样 Chrome/Chromium 就能够使用正确的粗细了。

Ubuntu 开发团队的 Gunnar Hjalmarsson 制作了一个个人 fonts-noto-cjk PPA，单独包含每个字体文件。你也可以考虑使用他的PPA。

A Few of my Org Mode Customizations

2015-07-13T00:00:00+00:00

Org mode is already a part of my life. It’s only natural I’d like to perform various customizations to my liking. Fortunately customizability is exactly Emacs’ core philosophy which means I get to do it with freedom. The following are just a few of them which I think might be helpful to others.

Paste rich format text from web pages into `Org mode`

When I first started to use Org mode there was a glaring deficit to me, which is the inability to paste any formatted text from web pages. If everything pasted becomes plain, much of the original information would be lost, and this would be a great hindrance for anybody wanting to use Org mode exclusively. Luckily this is Emacs where everything is possible. After I posed the question on StackExchange, user @mankoff was quick to come up with a solution using perl and pandoc. Note that this version of code is tested on OS X, but for other systems the difference should only be the utility to call in order to retrieve text from system clipboard, which in this case is osascript.

(defun kdm/html2org-clipboard ()
  "Convert clipboard contents from HTML to Org and then paste (yank)."
  (interactive)
  (kill-new (shell-command-to-string "osascript -e 'the clipboard as \"HTML\"' | perl -ne 'print chr foreach unpack(\"C*\",pack(\"H*\",substr($_,11,-3)))' | pandoc -f html -t json | pandoc -f json -t org --no-wrap"))
  (yank))

There are actually a couple of points worthy of further attention. First is that pandoc automatically hard-wraps its output at 70 characters per line. I normally just soft-wrap my notes so I added the option --no-wrap command there.

Then note that pandoc automatically uses Non-breaking space instead of normal space when converting formatted inline text(bold, italics, code etc.), which is not recognized by orgmodeby default. Non-breaking space has to be added to org-emphasis-regexp-components in order for those texts to be formatted correctly in orgmode. The following is my code for that. There seems to be two space characters each line because non-breaking space looks exactly the same as normal space as rendered here, but Emacs recognizes it with a underline.

;; This one is for the beginning char
(setcar org-emphasis-regexp-components " \t('\" {")
;; This one is for the ending char.
(setcar (nthcdr 1 org-emphasis-regexp-components) "- \t.,: !?;'\")}\\")

With this function copy-and-pasting in Org mode supports rich format text as other note-taking apps.

Set non-monospace font for normal notes

(Update Jan 2021: There is now a package mixed-pitch which does a great job on this. I recommend checking it out: https://gitlab.com/jabranham/mixed-pitch)

Nobody likes to read normal notes in monospace. But the source code snippets better be aligned. Is there a way to achieve both? Definitely. The variable-pitch-mode in Emacs is built just for that.

Remember to first set variable-pitch face in your configuration. Just M-x customize-face on variable-pitch would do. I use Helvetica Neue as I think it’s easiest on the eyes when you stare at it all day. It’s easy to change when you want. The following code was added by customize after I set the font:

'(variable-pitch ((t (:family "Helvetica Neue" :height 160))))

Add hook to turn on variable-pitch-mode when the target major modes are enabled. Note I set org-code etc. to be fixed-pitch so that they’re still monospace.

  (defun set-buffer-variable-pitch ()
    (interactive)
    (variable-pitch-mode t)
    (setq line-spacing 3)
     (set-face-attribute 'org-table nil :inherit 'fixed-pitch)
     (set-face-attribute 'org-code nil :inherit 'fixed-pitch)
     (set-face-attribute 'org-block nil :inherit 'fixed-pitch)
     (set-face-attribute 'org-block-background nil :inherit 'fixed-pitch)
    )

  (add-hook 'org-mode-hook 'set-buffer-variable-pitch)
  (add-hook 'eww-mode-hook 'set-buffer-variable-pitch)
  (add-hook 'markdown-mode-hook 'set-buffer-variable-pitch)
  (add-hook 'Info-mode-hook 'set-buffer-variable-pitch)

There is a nice post offering a more detailed configuration and claiming to preserve more font attributes such as org table coloring. However for general purposes the simpler configuration should suffice.

Monospace together with normal font:

Hiding Text-Formatting Markers

This is probably more of a matter of personal taste but I don’t really like seeing text-formatting markers all around. Why display /text/ when you can just have text. Turns out there’s a nice option in Org mode: org-hide-emphasis-markers which helps exactly in that. Just M-x customize-variable and then set org-hide-emphasis-markers to t. I didn’t remember seeing this option when I was reading org manual and only stumbled upon it later, so I thought listing it out might help.

Inserting timestamp for each note.

One potentially useful feature of normal note-taking apps is the ability to have the notes timestamped automatically. In Org mode one possible way of doing so is to utilize org-insert-heading-hook. You can define a function insert-inactive-timestamp as

(defun insert-inactive-timestamp ()
(interactive)
(org-insert-time-stamp nil t t nil nil nil))

I am fine with having the timestamp in the header directly. So I just have

(add-hook 'org-insert-heading-hook 'insert-inactive-timestamp)

There’s another solution in an Org mode mail list discussion which inserts the timestamp into the body:

(defun bh/insert-heading-inactive-timestamp ()
  (save-excursion
    (org-return)
    (org-cycle)
    ('insert-inactive-timestamp)))

(add-hook 'org-insert-heading-hook 'bh/insert-heading-inactive-timestamp)

Automatic pull/push for MobileOrg

Traditionally there’s been open source projects called MobileOrg to bring org mode’s functionality to your smartphones. Both Android and iOS versions utilize the same interface to interact with Emacs, which requires manually “pushing” and “pulling” changes from Emacs. There are a few timer-interval based auto-sync solutions out there. However, as I normally don’t use my phone to take notes with my laptop on, just simply pulling on Emacs initialization and pushing on shutting down works surprisingly well,

(add-hook 'after-init-hook 'org-mobile-pull)
(add-hook 'kill-emacs-hook 'org-mobile-push)

Note: Recently there has been an Android app called Orgzly which directly syncs your .org files instead of going through the push/pull interface defined for MobileOrg. I find it more refined and actively developed than the previous MobileOrg Android. It’s not open-source, but currently it feels a better app and I’ll be using it for now.

Automatically start the next pomodoro with `org-pomodoro`

I was used to automatically starting the next pomodoro when using other apps, as you’ll often forget to do it manually when a break finishes. There was an issue in org-pomodoro which prevented this from being configured, but now it’s fixed and you can just use org-pomodoro-break-finished-hook to achieve it:

(add-hook 'org-pomodoro-break-finished-hook
  (lambda ()
    (interactive)
    (org-pomodoro '(25)))

I’m only starting to have fun with Org mode so I might find some other interesting customizations later and list them here.

From OneNote/Evernote to Org Mode

2015-07-10T00:00:00+00:00

(Note: This post is more a kind of rambling personal recount/opinion, so if you’re already an Org mode user and are just interested in some Org mode customizations you might want to see this.)

I’m obsessed with note-taking. I’ve lost count of the number of diary books I wrote as a kid. And back when I was into reading newspapers, I would spend hours each night, carefully cutting and pasting newspaper articles into notebooks, accompanied with my comments; I easily accumulated hundred of pages of paperclips in this fashion. Thus it’s only natural that I immediately set out to find the perfect note-taking app for me after getting into the world of programming. However, the process was unexpectedly fraught with setbacks and angst, until I finally met Emacs Org mode, which is everything a programmer(or possibly any serious note-taker) could ever dream of. I’m amazed by how my daily efficiency massively improved upon using Org mode, how other note-taking apps are just kind of reinventing the wheel, and how I didn’t get to know it earlier. It’s not only for programmers either. I’ve seen many people using Emacs and Org mode for their scientific research, creative writing, personal journal, or whatever purpose out there.

My first note-taking app was OneNote. It was great - it had multiple layers of organization comprising of Notebook, Section, Page, and even Subpage. I gladly wrote a whole lot of notes on books/movies/interesting stuffs I would find on the web. It was an integral part of my life when I first learnt English and prepared for university application. However at that time Microsoft was known to be stubborn to not offer their products outside of Windows. So when I switched to Mac as many programmers would, OneNote was just out of the picture.

Evernote was all the rage at that time. However I didn’t have much fondness for it from the very beginning. It has always seemed to me that Evernote pays more attention to marketing and appealing to “business” users rather than spending serious efforts in the product itself. For a starter, its note organization was amateurish compared with OneNote’s multi-layered approach. It just doesn’t make much sense if all I can do is to throw various notes into one “Notebook” if I were reading several multi-chaptered books, and each chapter has various subtopics. The dealbreaker for me, however, was its incredibly sluggish and unresponsive synchronization. I, being an obsessive-compulsive note-taker, had easily accumulated more than ten thousand notes. Every once in a while Evernote would just automatically initiate its sync progress, freezing the UI in the process and leaving me unable to type anything for 15 seconds. Hell, any half-decent undergrad programmer would tell you that letting a background operation hang the UI is a beginner’s mistake. How could a company with hundreds of millions of dollars in funding not even get such basics straight?

After my unhappy years with Evernote, finally, as a part of its strategy shift, Microsoft began to offer an OS X version of OneNote. I thought it would mark the return of my happy time with note-taking. Wrong. What I didn’t realize was that programming has long been a part of my life. I constantly need to note down fragments of code, reason with them and write down my own observations; I may also take notes with mathematical notations for more theoretical contents (mostly in LaTeX). Besides, I sometimes need to compose and record my own code artifacts, to review and maybe reuse them later. Both Evernote and OneNote were doing a terrible job in that regard. OneNote only offered a simple “Code” formatting, without any syntax highlighting nor language support, and if you paste anything from the Internet, the format was usually broken in one way or another. Evernote didn’t even offer option for you to spontaneously type in code at all, though the pasting support was better. Simply put, it’s hard for me to imagine any programmer-notetaker being happy with the limited support provided.

I set out to search for viable solutions. Markdown-based note-taking software was the first thing that came to my mind. There were a few excellent apps for markdown note-taking, such as Mou, MacDown, and Kobito. They all do a decent job in providing syntax highlighting support and flexible markup. The problem was with structural organization - They were mostly nothing more than single-file editors. Searching, comparing and archiving contents quickly became a nightmare.

Then I came across an app called Quiver, which is actually a decent enough app considering it’s solely a one-man effort. It has an Evernote-esque interface. But what sets it apart is its support for syntax highlighting, Markdown and LaTeX, as it utilizes AceEditor in its core; It even supports Vim/Emacs keybindings. I was happy with it for a while, until I finally got to use Org mode after I picked up Emacs through the excellent Spacemacs project. I was immediately hooked and wondered why I didn’t get to use it sooner:
Its original functionality set already outclasses everything that’s out there: With org-babel you edit source code snippets directly in Emacs major modes for each language seamlessly. If you want, you can even just run the snippet on-the-fly. With org-clock and org-pomodoro coupled with TODO you have a personal project-management + time-tracking suite much better integrated and flexible than the countless (closed-sourced) solutions out there; I no longer have to endure the craziness of using one app to keep track of my tasks, another to log my time, and yet another to take notes related to those tasks. With tools like helm-ack I can search my whole notes repository in a flash with good old ack which is simply more efficient than search functionalities in shiny modern apps. But that’s not even the most important part: No matter how good a program is, there are always things that don’t suit you 100% and on which you want to tweak, which Org mode allows you to do, a lot. This has made me a much happier Org user.

Coming to think about it, I guess one big reason why it had not occurred to me to try out Org mode at all despite it being around for a long time is really that I’ve been a Vim guy. From what I read around the web, I got this notion that Vim and Emacs are just two distinct things that you both try out and then decide to stick with one of them, incompatible with anything that the other camp has to offer. It’s not that I didn’t try out Emacs before, I did and thought the text editing experience was clumsy compared with Vim, and the conceptual overhead of learning another system is huge. However, following this line of thought I just got lazy and didn’t even try out various Vim emulation tools out there such as the excellent evil, not to mention Org mode; this proved to be a huge mistake. To a large part I’m grateful to the open-source project Spacemacs, which offers an Emacs Kit focused on Vim emulation. You can immediately start to use it as if it were Vim, just with awesome Emacs features such as Org mode supported natively (with Vim-like keybindings). Later you’ll naturally feel the thirst to tweak its behaviors here and there, thus gradually becoming proficient with all the quirks and inner mechanism of Emacs. It also has another set of configuration focused on holy-mode for those who don’t use Vim. Whether you’re familiar with Vim or not, I definitely recommend you to check it out as a nice entry point to the wonderful world of Emacs and Org mode.

What a kind of makes me uneasy is the perceived tendency where Vim and Emacs are just simply being heralded less and less by programmers of the new generation, while in fact they’re still unparalleled in many aspects. I know my university is nothing compared with top-quality CS schools, but the fact that barely anybody around me uses Vim and Emacs daily, and that a large portion of students have absolutely zero idea what those things are, still staggers me. Also I had also been a SublimeText user for some time, but still, I realized it doesn’t really compare to either native Vim or Emacs. Vim offers unparalleled editing efficiency and freedom which no emulation I’ve seen so far has been able to 100% bring. As of why Emacs is important, a perhaps inappropriate analogy I’ve come up with is owning a car. Emacs embodies this philosophy where you want full ability to customize your own beloved vehicle. You may power up its engine, change its window, paint it a different color… Anything imaginable, you do it with your own hands instead of having to choose from some limited factory preset or ever resorting to any external hands, which will likely inevitably fall short of your expectations one way or another. You are the mechanic here, and I bet many a mechanic’s ultimate dream would be a fully-powered, wholly customized car of his own. Emacs is just such a car, and its whole design is transparent and open to contributions so if you think you might improve it here and there in its next generation you can just do it. Not to mention it’s also free. The only thing that I currently see might come close is Atom, though with the community and accumulated packages for Emacs I don’t see myself using Atom anytime soon.

I’ve read in quite a few places how “true hackers use Emacs”. I didn’t quite buy into it when I only used Vim. But now I’d say I kind of understand what it means, as I already can’t live without Emacs. Its general esotericism might be one of the hindrances for it and Org mode to attain deserved level of public following proportional to its power, but actually there are already a few nice starter kits out there besides the aforementioned Spacemacs which eases the learning curve. You may find them listed at Awesome Emacs, an “Awesome” style community-driven list of Emacs resources.

Xiang

Comparing the Actor Model and CSP Concurrency with Elixir and Clojure

Introduction

BEAM vs. JVM

BEAM

Per-process memory

Preemptive scheduling

JVM

Shared Heap

OS threads and OS scheduler

The Actor Model vs. CSP

The Actor Model on the BEAM

CSP with core.async in Clojure

Aside: Conventional approaches to concurrency in Clojure

core.async and CSP

“go blocks” in core.async

Inversion of control with go blocks

Conclusion

References and Further Resources

Displaying Org Mode Pomodoro Clock in the Menu Bar

Differences between Vim Regex, Emacs Regex and PCRE

Differences between Emacs Lisp regex and PCRE regex

Differences between Vim regex and PCRE regex

Elixir Process Orchestration In Kubernetes With Libcluster And Swarm

libcluster and Swarm

Clustering with libcluster

Automatic Clustering

Optional: Local cluster setup with docker-compose

Process registration with Swarm

Emacs Setup for Elixir and Vue

Language Server Protocol

Vue.js

Elixir

Tweaks and Customizations

My XMonad Configuration (with Screenshots of Common XMonad Layouts)

From i3 to XMonad

Useful modules from xmonad-contrib

Layouts

Tabbed (XMonad.Layout.Tabbed)

TwoPane (XMonad.Layout.TwoPane)

ResizableTall (XMonad.Layout.ResizableTile)

emptyBSP (XMonad.Layout.BinarySpacePartition)

Spiral, Dwindle (XMonad.Layout.Dwindle)

XMonad.Actions.PhysicalScreens

XMonad.Layout.Spacing

XMonad.Hooks.EwmhDesktops

XMonad.Layout.NoBorders

historyHook (XMonad.Actions.GroupNavigation)

xmobar and trayer

Adding an entry in /usr/share/xsessions for startup applications

My full configuration files

~/.xmonad/xmonad.hs

~/.xmobarrc

Multitenant Phoenix application deployment with SSL using HAProxy

Installing HAProxy

Phoenix-specific HAProxy configurations

Checking for errors in the HAProxy configuration file

Configuring Phoenix Apps

Working with Distillery

Adding TLS support via HAProxy

Circumventing the Great Fire Wall (GFW) in China with Shadowsocks

Update March 2020

Server

Client

Introduction

Spinning Up a VPS (Virtual Private Server)

Install the server application for Shadowsocks

Install the client

Solving CJK Font Rendering Issue in Chrome on Ubuntu 16.04

解决Ubuntu 16.04 Chrome 里中文文字发虚问题

A Few of my Org Mode Customizations

Paste rich format text from web pages into Org mode

Set non-monospace font for normal notes

Hiding Text-Formatting Markers

Inserting timestamp for each note.

Automatic pull/push for MobileOrg

Automatically start the next pomodoro with org-pomodoro

From OneNote/Evernote to Org Mode

`core.async` and CSP

“`go` blocks” in core.async

Inversion of control with `go` blocks

Tabbed (`XMonad.Layout.Tabbed`)

TwoPane (`XMonad.Layout.TwoPane`)

ResizableTall (`XMonad.Layout.ResizableTile`)

emptyBSP (`XMonad.Layout.BinarySpacePartition`)

Spiral, Dwindle (`XMonad.Layout.Dwindle`)

`XMonad.Actions.PhysicalScreens`

`XMonad.Layout.Spacing`

`XMonad.Hooks.EwmhDesktops`

`XMonad.Layout.NoBorders`

`historyHook` (`XMonad.Actions.GroupNavigation`)

`~/.xmonad/xmonad.hs`

`~/.xmobarrc`

Paste rich format text from web pages into `Org mode`

Automatically start the next pomodoro with `org-pomodoro`