Ellis Michael

Deadlock Freedom: Beyond Liveness Properties

Thu, 19 Jan 2023 00:00:00 -0800

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

Properties

In their classic paper, “Defining Liveness” [1], Alpern and Schneider show that all properties of distributed systems are the conjunction of a safety property and a liveness property. Informally, safety properties specify the “bad things” that should not happen while liveness properties specify the “good things” that must happen. More formally, safety properties are the properties which can always be falsified by finite traces and liveness properties are the properties which can only be falsified by infinite traces. If $S$ is the set of program states, $S^\omega$ is the set of infinite sequences of program states, and $S^*$ is the set of finite sequences of program states then $P$ is a safety property if

\[\forall \sigma \in S^\omega : \exists i \ge 0 : \forall \beta \in S^\omega : \sigma \not \in P \implies \sigma[..i]\beta \not \in P\]

and $P$ is a liveness property if

\[\forall \alpha \in S^* : \exists \beta \in S^\omega : \alpha \beta \in P \ .\]

Here, $\sigma[..i]$ is just the finite sequence with the first $i$ elements of $\sigma$ and $\sigma[..i] \beta$ and $\alpha \beta$ are concatenations of sequences.

Alpern and Schneider also give two different proofs that any property can be decomposed into a safety property and a liveness property, one based on topology and one based on automata theory [1, 2]. I haven’t yet defined what a “property” is, though. It’s implicit in the above definitions, but in this characterization, a property is just a set of (infinite) traces!

The Properties of Consensus

Properties are powerful and can be used to specify important things we want from our distributed systems. For example, the specification of the consensus problem can be expressed as the intersection of a safety property – all correct processes that decide a value decide the same value, and that value must have been some input value – and a liveness property – all correct (non-failing) processes eventually decide a value.

Sometimes, proving properties (especially liveness properties) of systems requires making additional assumptions. Consider the termination guarantees of a system like Paxos. We know that consensus in an asynchronous environment with the possibility of a single process failure cannot be solved (without randomization) [3]. In particular, protocols which guarantee safety cannot guarantee termination. However, people use protocols such as Paxos in practice. Are these practitioners just religious zealots in the Paxos cult, believing without any basis that Paxos will save them from the evils of impossibility results? Of course not! We can prove things about the liveness of Paxos; in particular, a Paxos implementation with a well-designed leader election mechanism can guarantee liveness during periods of sufficient synchrony.

How do we formalize what “periods of sufficient synchrony” are? One way people do this is through the use of failure detectors. If you equip each replica with a failure detector which is responsible for informing the replica when the current leader has failed, then we know that as long as the failure detectors behave at least as well as $\diamond W$, the weakest failure detector that can solve consensus [4], (and as long as the network itself satisfies basic fairness criteria¹) then the system is guaranteed to eventually make progress. The way that you could disprove that Paxos (or an implementation of Paxos) satisfies this “$\diamond W$-termination” liveness property is by describing an infinite (fair) execution of Paxos in which the failure detectors satisfy the $\diamond W$ guarantees, no more than $f$ processes fail, and some correct process never decides a value.

Deadlock Freedom

While proving liveness properties of consensus protocols like Paxos is useful, it will always involve making additional assumptions about the well-behavedness of the environment. I claim, however, that there is something much more basic that we would like to say about Paxos and many other distributed protocols and implementations of those protocols that does not rely on any additional assumptions. I’ve been calling this property deadlock freedom.

Deadlock Freedom: The system cannot reach a state from which no terminating state is reachable.

Notice that deadlock freedom is neither a safety nor liveness property – it’s not a property at all! We can easily express it in a branching temporal logic such as CTL (e.g., $\mathbf{AG}. \mathbf{EF}. termination$), but deadlock freedom cannot be falsified with a single trace. Just because a system can fail to make progress forever doesn’t mean that it ever reaches a state from which progress is impossible. However, any liveness properties that you wish to prove about a system will imply deadlock freedom, unless additional assumptions you employ to prove liveness make the dead states unreachable.

You might think that deadlock freedom is actually a safety property. After all, deadlocks in a multi-threaded locking scheme can be detected by looking for cycles in the wait-for graph. If a locking system cannot reach a cyclically waiting state, then it is free of deadlocks.

“Deadlocks” in distributed systems, however, can be much more subtle. Imagine implementing a 500,000 line system that contains a dozen different roles that nodes can play. Proving (or even knowing) all of the ways in which such a system could get permanently stuck, were they to occur, might be extremely difficult. But expressing the ways in which the system can make progress might be comparatively easy. We would like to be able to express useful attributes of our systems (such as deadlock freedom) without needing to prove complicated theorems.

Deadlock freedom is also a useful attribute of systems to look for in model checkers. In DSLabs, while we do not check for deadlock freedom in general, we do check that student implementations can make progress from specific intermediate states.

Hyperproperties

While deadlock freedom is not a property, it is a hyperproperty. Hyperproperties, as defined by Clarkson and Schneider, are sets of sets of (infinite) traces [5]. A distributed system satisfies a hyperproperty if the complete set of possible executions it can produce is in the hyperproperty. In light of hyperproperties, what we have been just calling “properties” up until now are referred to as trace properties for clarity.

It turns out that there is an analogous safety and liveness decomposition for hyperproperties! Safety hyperproperties are the hyperproperties which can always be falsified by finite sets of finite traces,² and liveness hyperproperties are hyperproperties which cannot be falsified by finite sets of finite traces.³ (See the hyperproperties paper for a more full discussion of the definitions of hyperproperties [5].) Furthermore, every hyperproperty is the conjunction of a safety hyperproperty and a liveness hyperproperty.

Deadlock freedom is a liveness hyperproperty. It cannot be falsified with finite traces at all. A counter-example to deadlock freedom would have to show that there is some state that is reachable, and in all possible executions which reach that state, termination/progress is never achieved.

Hyperproperties were introduced to specify important security policies (such as secure information flow) and complex liveness requirements (e.g., SLAs such as mean response time). However, as we have seen, they can be much more basic. Deadlock freedom is a liveness hyperproperty which is broadly applicable and extremely desirable. While trace properties are a useful narrowing of hyperproperties because they can be more amenable to certain formal methods techniques such as verification and model checking, a broader understanding of the specification of distributed systems must include hyperproperties, which I do not think are as widely understood, hence the reason for this blog post.

References

Alpern, B. and Schneider, F.B. Defining liveness. Information Processing Letters. 21(4):181–185. 1985.
Alpern, B. and Schneider, F.B. Recognizing Safety and Liveness. Distributed Computing. 2(3):117–126. Sep. 1987.
Fischer, M.J., Lynch, N.A. and Paterson, M.S. Impossibility of Distributed Consensus with One Faulty Process. J. ACM. 32(2):374–382. Apr. 1985.
Chandra, T.D., Hadzilacos, V. and Toueg, S. The Weakest Failure Detector for Solving Consensus. jacm. 43(4):685–722. Jul. 1996.
Clarkson, M.R. and Schneider, F.B. Hyperproperties. Journal of Computer Security. 18(6):1157–1210. Sep. 2010.

The fairness criteria that are assumed vary, but one basic assumption that is usually necessary is that all processes either fail or take infinitely many steps. The FLP impossibility result assumes that all messages are eventually delivered (as long as the receiver has not failed), which is a particularly strong assumption but only broadens the applicability of the impossibility result. When proving the liveness of protocols, weaker assumptions about message delivery are often assumed. For example, a fair-lossy network only guarantees that if a message is sent infinitely many times (from the same sender to the same receiver), then it is delivered infinitely many times. ↩
That is, no complete set of executions which extends a counter-example set of finite traces will satisfy a safety hyperproperty. If we have two sets of (possibly infinite) traces $A$ and $B$, then $A \le B$ if $\forall a \in A : \exists b \in B : a \le b$ where $a \le b$ means that $a$ is a prefix of $b$. A hyperproperty is a safety hyperproperty if, for every set of executions, $T$ which does not satisfy the safety hyperproperty, there exists a finite set of finite traces $C \le T$ such that for all $T' \ge C$, $T'$ also does not satisfy the hyperproperty. ↩
A hyperproperty is a liveness hyperproperty if, for all finite sets of finite traces $C$, $\exists T \ge C$ such that $T$ satisfies the hyperproperty. ↩

Linearizable, Wait-free Reads of Replicated State Machines

Mon, 13 Apr 2020 00:00:00 -0700

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

I recently read the Gryff paper [1], published this year at NSDI. As I was reading it, I noticed that the authors utilized a trick for achieving linearizable reads of replicated data that has value beyond their specific system. Many people who’ve studied fault-tolerant replication have probably derived it on their own (as I did years ago), but I don’t recall ever seeing the technique described in an earlier paper.¹ So, I thought I’d take the opportunity to describe the technique on its own as well as how it applies to any state machine replication (SMR) protocol, beyond the Gryff protocol described by Burke et al.

Any SMR protocol that guarantees linearizability [2] of updates provides the abstraction that the changes to the underlying state (called “commands”) are processed by a single, fault-tolerant service. Or equivalently, there is a logical shared log onto which clients’ commands are appended. In practice, this shared log is often explicitly constructed, and separate machines (called “replicas”) consume the log by executing the commands in log order. We know from the FLP impossibility result [3] that in an asynchronous system in which some replicas can fail, there will always be the possibility that the system ceases to make progress and will forever be prevented from appending commands to the shared log, even if one assumes that all messages that get sent are eventually delivered.

Does this then imply that reading the state of a replicated state machine is necessarily subject to the same limitations? Implementing read operations for an SMR protocol naïvely is straightforward. Reads can simple be treated as commands and can go through the normal replication protocol. Using this method, however, would subject reads to the impossibility result described above and the performance bottlenecks that come with some SMR protocols. However, it turns out that these downsides are not inherent. Instead, we can guarantee that responses to read requests can be returned in a wait-free manner [4]. That is, we can guarantee that these read operations will terminate in finite rounds of communication.

Firstly, if one only cares about serializability of reads, rather than linearizability, the solution is straightforward. Any client reads the state of any replica in a single round of communication (sending the initial request to a majority to tolerate faults). As long as replicas have access to the state of the system after applying some prefix of the current shared log (as is almost always the case with SMR protocols), then a single replica’s response to the client will suffice.² In essence, this weakening of linearizability to serializability means that the client might receive stale data. Whether or not this is acceptable depends on the application, but mere serializability can sometimes lead to unintuitive results.

To see how we will achieve wait-free, linearizable reads, we’ll take the multi-instance Paxos protocol [5] as a running example (using some of the terminology from Paxos Made Moderately Complex [6]). In this execution, we see a single client, $c_1$ sending a command to the system. The leader sends a message to the followers and waits for responses from a majority (including itself). Once this is done, the leader executes the command, replies to the client, and lets the followers know that a decision has been reached and that they can safely execute the command as well.

Suppose then, that after $c_1$ receives a response to its command but before any of the followers are notified of the decision, $c_2$ attempts to read the state of the replicated state machine. In principle, $c_2$ is allowed as many rounds of communication with a majority of replicas as is necessary. For now, let’s just consider a single round of communication. $c_2$ sends the query to all replicas and because we would like to tolerate the failure of any minority, it only waits for a response from a majority. Now, suppose that none of the responses it receives comes from the leader replica.

Because $c_2$ initiated its request after $c_1$ got a response, the state $c_2$ reads must reflect $c_1$’s command. However, as they respond to the $c_2$’s read, none of the followers yet knows that a decision has been reached about $c_1$’s command, even if they do know that there is a decision pending. It would seem that the only safe thing for to do in this scenario is to wait until a decision on the pending command is reached and to delay a response to the read until then. However, that would put us back in the realm where the FLP result applies and preclude any hopes of wait-freedom.

The solution is fairly straightforward. The problem is that the state updates are replied to before knowledge of their commitment has reached a majority. So, we will delay the response the client until that knowledge has reached a majority. The leader waits until a majority executes the command and acknowledges the decision before responding.³

We’re almost done. You might think that this completely resolves the issue, and in the setting we’ve described so far in which there are only two clients – one sending updates and the other reading the state – it does. However, there is a subtle problem that arises when there are multiple clients sending reads. In this example, $c_3$ is also attempting to read the state of the replicated state machine. Here, the result of $c_1$’s command will be visible to $c_2$ but not to $c_3$. Since $c_3$ initiated its read after $c_2$’s already finished, this would be a violation of linearizability.

We solve this issue the same way the ABD register protocol [7] does. We add a write-back phase to the read protocol, in which the client ensures that a majority of the replicas are at least as up-to-date as the state the client learned about during the first phase of the read. It distributes any decisions which have not yet reached a majority and waits until there is some up-to-date majority before considering the read completed.

Of course, if the original majority the client read from all had the same set of decisions (or at least all returned the same result if only a subset of the state was read), then this second phase can be skipped entirely.

So, we have two strategies: (1) delaying the client-side completion of updates until they’re permanently committed at a majority and (2) reading using a two-round protocol, the second round of which ensures that a majority is up-to-date with the value read. Neither of these strategies is specific to Paxos and can be applied to SMR protocols broadly. However, they do present trade-offs, as Burke et al. describe.

References

Burke, M., Cheng, A. and Lloyd, W. Gryff: Unifying Consensus and Shared Registers. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). Feb. 2020. 591–617.
Herlihy, M.P. and Wing, J.M. Linearizabiliy: A Correctness Condition for Concurrent Objects. ACM Transactions on Programming Languages and Systems. 12(3):463–492. Jul. 1990.
Fischer, M.J., Lynch, N.A. and Paterson, M.S. Impossibility of Distributed Consensus with One Faulty Process. J. ACM. 32(2):374–382. Apr. 1985.
Herlihy, M. Wait-Free Synchronization. ACM Trans. Program. Lang. Syst. 13(1):124–149. Jan. 1991.
Lamport, L. The Part-time Parliament. ACM Transactions on Computer Systems. 16(2):133–169. May 1998.
Van Renesse, R. and Altinbuken, D. Paxos Made Moderately Complex. ACM Comput. Surv. 47(3):42:1–42:36. Feb. 2015.
Attiya, H., Bar-Noy, A. and Dolev, D. Sharing Memory Robustly in Message-Passing Systems. J. ACM. 42(1):124–142. Jan. 1995.

If this technique has been previously described in the literature, I would very much appreciate someone sending me a link! ↩
Each response will need to be tagged with some sort of version number, and the client will have to cache responses locally to enforce the process order requirement of serializability. ↩
The phase added to the update protocol could instead be initiated by the client, and the client could wait for acknowledgments to the decision before considering the update complete. In the Paxos context, having the leader drive this second phase to completion is simpler. ↩

Randomized Consensus on Unknown Domains in Finite Rounds

Mon, 10 Jun 2019 00:00:00 -0700

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

The most widely known result in distributed computing is the FLP result that proves the impossibility of consensus [1]. One of the assumptions of this theorem is that processes are deterministic. When randomness is allowed, the Ben-Or algorithm gives simple, elegant way to solve binary consensus [2]. Or rather, $\text{lim}_{t \to \infty} P(\text{consensus has been reached}) = 1$. The algorithm goes something like this.

a = input({0, 1})

loop:
  send_phase1(a)
  A = receive_phase1()
  if ∃aʹ∈ A : |{a''∈ A : a'' = a'}| > n/2:
    b = a'
  else:
    b = ⊥

  send_phase2(b)
  B = receive_phase2()
  if ∃bʹ∈ B : b' != ⊥ && |{b'' ∈ B : b'' = b'}| > f:
    decide(bʹ)
  if ∃bʹ∈ B : b' != ⊥:
    a = b'
  else:
    a = choose_random({0, 1})

The algorithm proceeds in asynchronous rounds, each with two phases. In each phase, each process broadcasts its value for the phase and then waits for the values of $n - f$ other processes. The safety argument is not complicated, and the termination argument is even more straightforward. Eventually the random choices made by processes in some round will be overwhelmingly in favor of either 0 or 1, and that value will decided by every process in the next round.

This blog post is a record of a couple simple observations I made when re-reading the paper recently (in preparation for teaching it in UW’s distributed systems course this past quarter) that I couldn’t find written up anywhere else.

Unknown Domains

The first observation is that this algorithm can be modified to allow processes to have inputs values from larger domains than $\{0, 1\}$. In fact, the input domain can be infinite. Furthermore, we don’t even need the processes to know the input domain a priori. On the very last line of the protocol, when processes make a random choice, they simply choose from all values seen so far in any message.

Since there are at most n possible input values (because there are at most n processes), the termination argument still applies. The only real difference is that termination could take longer. The original paper contained a theorem that when $f$ is $O(\sqrt{n})$, the expected number of rounds is constant. That theorem no longer applies when the set of possible decision values grows with $n$.

The other interesting fact is that this protocol can be further modified to support a model where not all processes get input values. Processes without input values initialize $a = \bot$ on the first line. And on the last line when random choices are made, processes choose from all non-$\bot$ values seen. As long as at least $f + 1$ processes get input values, termination is still guaranteed. Even more interestingly, the check on first-phase messages can be refined somewhat in this case. The property that must be maintained is that no second-phase messages with different non-$\bot$ values should exist for the same round. So, assuming we get $n - f$ messages in the second round, if the number of messages for one non-$\bot$ value is more than $f$ greater than for every other non-$\bot$ value, then we can safely choose that for the second phase.

Both of these modifications are important for the way people use consensus in the real world — namely to implement state machine replication. The commands proposed to the state machine log often come from an infinite domain and are chosen by clients at runtime. The servers implementing the state machine do not know which commands will be proposed a priori. Furthermore, we would like the system to make progress even when clients don’t send their commands to all servers.

Finite Rounds

The other observation is that the original protocol had all processes taking protocol steps and sending messages forever, even after they had decided values. This is not necessary. Instead, we can have processes send messages for each phase proactively up until they’ve decided a value and the reactively thereafter. That is, once a process has reached a decision, it only sends a message for a phase after it has received at least one message for that phase (or a later phase).

Because all non-faulty processes eventually decide a value, eventually all processes stop sending messages. Note, however, that this does not mean processes can halt. They must continue listening for messages forever (or at least $f+1$ processes must). Eventually, however, they will no longer send any new messages.

When combined the model above where not all processes get input values, we could also specify that processes which have not seen any non-$\bot$ values remain in a reactive mode until they receive a non-$\bot$ value.

References

Fischer, M.J., Lynch, N.A. and Paterson, M.S. Impossibility of Distributed Consensus with One Faulty Process. J. ACM. 32(2):374–382. Apr. 1985.
Ben-Or, M. Another Advantage of Free Choice (Extended Abstract): Completely Asynchronous Agreement Protocols. In Proceedings of the Second Annual ACM Symposium on Principles of Distributed Computing. Montreal, Quebec, Canada. 1983. 27–30.

Understanding Traces

Mon, 18 Feb 2019 00:00:00 -0800

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

For DSLabs, one of the pieces I built was a post-processing phase that runs on traces generated by the model checker. It takes a trace (i.e., a list of events) and produces a semantically equivalent trace which is, hopefully, more understandable to humans.

To motivate the problem, lets look at an example. Suppose we have a distributed system with four processes — $p_1$, $p_2$, $p_3$, and $p_4$, and consider the events represented by the following space-time diagram:

Because there are concurrent events (i.e., events not ordered by the happens-before relation) in the above execution, there is more than one way to represent these events as a well-formed trace.¹ Here, a well-formed trace is one in which the happens-before relation is respected by the ordering of events. However, not all traces are created equal. Without knowing the specifics of the protocol being run in this execution, I think most people, if asked to produce a trace, would write something like this:

$p_1$ sends $m_1$ to $p_2$
$p_2$ receives $m_1$, sends $m_2$ to $p_1$
$p_1$ receives $m_2$, sends $m_3$ to $p_2$
$p_2$ receives $m_3$
$p_3$ sends $m_4$ to $p_4$
$p_4$ receives $m_4$, sends $m_5$ to $p_3$
$p_3$ receives $m_5$, sends $m_6$ to $p_4$
$p_4$ receives $m_6$

They might, instead, pair up $m_1$ and $m_4$, $m_2$ and $m_5$, and $m_3$ and $m_6$. But they certainly wouldn’t write down the following:

$p_1$ sends $m_1$ to $p_2$
$p_3$ sends $m_4$ to $p_4$
$p_2$ receives $m_1$, sends $m_2$ to $p_1$
$p_1$ receives $m_2$, sends $m_3$ to $p_2$
$p_4$ receives $m_4$, sends $m_5$ to $p_3$
$p_2$ receives $m_3$
$p_3$ receives $m_5$, sends $m_6$ to $p_4$
$p_4$ receives $m_6$

Not just that, but the latter trace is also harder to follow as a reader. It has several unnecessary “context switches.” The DSLabs model checker, however, does not distinguish between these two traces; they both have the same length. If an invariant was violated upon completion of the execution in question, the concurrent events in the trace produced by the model checker would be randomly ordered.

The post-processing phase I implemented is based on a simple idea, that events in traces should be followed by their immediate consequences. First, I took the trace produced by the model checker and built up the associated event graph. The event graph looks just like the space-time diagram above. Vertices are events; edges go between corresponding send and receive events and between subsequent events at the same process.

Then, I ran a depth-first topological sort of the event graph. By topologically sorting the graph, we get a well-formed trace. By doing it in a depth-first manner, we list events and their immediate consequences whenever possible.

To see what this looks like in practice, let’s look at an example from the primary-backup lab. First, we show an animated trace demonstrating an invariant-violation in a buggy solution.

Next, let’s see what the same invariant-violation looks with the post-processing phase for understandability.

Of course, you can’t necessarily follow what’s going on without knowing the details of the protocol. But, it should be much easier to follow the “flow” of the second video.

There are many possibilities for further improvement. You could bias the sorting of events towards message delivery or towards staying with the same process. You could even try to recognize group communication patterns (e.g., the “one→all→one” pattern) and order them in particular ways. For now, though, I think this solution works quite well.

In fact, there are 20 such traces. ↩

Producers and Consumers

Mon, 27 Aug 2018 00:00:00 -0700

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

At a recent talk of Leslie Lamport, he raised the question of how many possible executions there are for a bounded-buffer producer-consumer protocol for a given set of input values. Leslie’s point was that, from the standpoint of the values produced and consumed, there is only one execution, and the only things that we can say about the ordering of these events is that a value must be produced before it is consumed and that before a value can be produced, enough values must have been consumed so that there is room in the buffer.

I started wondering, though, just how many executions are there exactly? For a given buffer size and length of execution, how many possible (legal) sequences of produce and consume events are possible? Equivalently, how many strings of $P$s and $C$s of a given length are there such that the number of $C$s in any prefix is less than or equal to the number of $P$s and the number of $P$s minus the number of $C$s in any prefix is less than or equal to the buffer size? Writing down the recurrence for the general case is easy enough, and computers are very fast and good with numbers, so let’s see a few results.

import functools

@functools.lru_cache(maxsize=None)
def f(B, S, U=0):
    if U < 0 or U > B: return 0
    if S == 0: return 1
    return f(B, S - 1, U - 1) + f(B, S - 1, U + 1)

Here, $B$ is the size of the buffer, $S$ is the number of steps in the execution, and $U$ is the number of used slots in the buffer (initially 0).

>>> print([ f(1, i) for i in range(15) ])
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

When the buffer size is 1, there only ever one execution, no matter how many steps we take. It’s just the $S$-length prefix of the infinite $PCPCPCPCPC...$ string. What about when the buffer size is 2?

>>> print([ f(2, i) for i in range(15) ])
[1, 1, 2, 2, 4, 4, 8, 8, 16, 16, 32, 32, 64, 64, 128]

Here, we see once-repeated increasing powers of 2. To see why this is the case, imagine the state of the buffer. When it’s empty or full, there is only one action that can be taken, a produce or consume, respectively. On the other hand, when there is a single value in the buffer, both actions are possible.

Now, given both of those results, try to guess what the sequence looks like when $B = 3$.

>>> print([ f(3, i) for i in range(15) ])
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]

Did you guess the Fibonacci sequence? This one’s a little harder to see. It’s not obvious (to me at least) why the Fibonacci recurrence should apply. If $f(3, i)$ is the number of executions possible with exactly $i$ steps with a buffer of size 3, let’s let $f_{0, 3}(3, i)$ be the number of those executions where the final state of the buffer is full or empty. Then, let $f_{1, 2}(3, i)$ be the number of executions ending in the buffer having 1 or 2 values. First, we have the obvious identity.

\[f(3, i) = f_{0, 4}(3, i) + f_{1, 2}(3, i)\]

Then, we have a pair of recurrences that are easiest to see pictorially.

Namely, the following:

\[f_{0, 4}(3, i) = f_{1, 2}(3, i - 1)\] \[f_{1, 2}(3, i) = f_{0, 4}(3, i - 1) + f_{1, 2}(3, i - 1)\]

Then, the derivation is straightforward.

\[\begin{align*} f(3, i) &= f_{0, 4}(3, i) + f_{1, 2}(3, i) \\ &= f_{0, 4}(3, i - 1) + 2 f_{1, 2}(3, i - 1) \\ &= f(3, i - 1) + f_{1, 2}(3, i - 1) \\ &= f(3, i - 1) + f_{0, 4}(3, i - 2) + f_{1, 2}(3, i - 2) \\ &= f(3, i - 1) + f(3, i - 2) \end{align*}\]

Finally, let’s see what happens when the buffer is unbounded (by setting the size of the buffer to the number of steps in the execution).

>>> print([ f(i, i) for i in range(15) ])
[1, 1, 2, 3, 6, 10, 20, 35, 70, 126, 252, 462, 924, 1716, 3432]

Here, we see a different sequence. The number of possible executions with $i$ steps is ${i \choose \lfloor i / 2 \rfloor}$. Let $X$ be set of $i$-length $P/C$ sequences with exactly $\left \lfloor \frac{i}{2} \right \rfloor$ $C$s (some of which represent invalid executions) and let $Y$ be the set of valid sequences.

First, given a sequence in $X$, we can transform it into a valid sequence in $Y$ by scanning left to right, keeping track the number of values in the buffer. If a $C$ is invalid (i.e., would result in the number of values in the buffer becoming negative), we change it to a $P$ and treat it as a null operation on the buffer (i.e., neither a $C$ nor a $P$). So, $CPCCPPPP$ would become $PPCPPPPP$. Then, to invert that transformation, we count the number of $P$s that should be transformed to $C$s and then repeatedly scan from right to left, replacing a $P$ with a $C$ every time doing so would cause the sequence to become invalid, treating the new $C$ as neither a $P$ nor $C$ for the remainder of the operation. Here are these two transformations in Python.

def t(x):
    pc = 0
    r = ''
    for xi in x:
        if pc <= 0 and xi == 'C':
            r += 'P'
        else:
            r += xi
            if xi == 'P':
                pc += 1
            else:
                pc -= 1
    return r

def ti(y):
    y = list(y)
    p = y.count('P')
    c = len(y) - p

    while p > len(y)/2:
        pi, ci  = p, c
        for i in range(len(y) - 1, -1, -1):
            if y[i] == 'C':
                ci -= 1

            if y[i] == 'P':
                pi -= 1

                if ci >= pi:
                    p -= 1
                    y[i] = 'X'
                    break

    return ''.join('C' if yi == 'X' else yi for yi in y)

Since these transformations are, indeed, inverses of each other, we get $|X| = |Y|$.

Of course, none of the above observations are new, and some/all are simple enough to be put to undergraduates in a discrete math course. Typically, these problems would be put in terms of the number of sequences of 1 and -1 where the partial sums are all nonnegative and less than some bound. I just liked that you could describe interesting combinatorics problems in terms of a common synchronization primitive.

Furthermore, there are still more interesting sequences for different values of $B$. For instance, when $B = 4$, we get:

\[f(4, i) = \begin{cases} 1 & i = 0 \\ 3^{\frac{i - 1}{2}} & \text{$i$ odd} \\ 2 \cdot 3^{\frac{i}{2} - 1} & \text{$i$ even}, i \ne 0 \end{cases}\]

If anyone knows a closed-form expression for $f (B, S)$, I would be grateful.

Acknowledgments

Thanks to Helgi for pointing out a bug in an earlier version of my t(x) and ti(y) functions.

A Game of Thrones

Fri, 12 Jan 2018 00:00:00 -0800

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

Suppose you’re the ruler of a vast empire, composed of $k$ kingdoms. As emperor, you’ve decided that who sits on each throne is too important to be left up to random things like birth order; you’d like to manage the succession to each throne personally. You don’t want to be bothered each time a king or queen dies, however. You are emperor, after all.

Instead, you decide that you will handpick the line of succession for each of the $k$ thrones. These succession orders will be valid until $f$ nobles sitting on or in line to a throne die, at which point you will revisit the succession question and design a new chart. There is one caveat, however: you do not want any of the kingdoms to be consolidated. None of your vassals should sit on two thrones simultaneously.

Let $n$ be the number of your vassals you consider worthy of sitting on one of your empire’s thrones. We’d like to answer the following question: how large must $n$ be to support lines of succession for $k$ thrones such that no king or queen will ever sit on two thrones even if $f$ nobles in those succession lines die?

In order to have a suitable succession chart, what you need to come up with is a matrix:

\[%-------------------- % PREAMBLE %-------------------- \require{cancel} \require{color} \renewcommand\CancelColor{\color{blue}} \newcommand{\ullabeld}[3]{ {\scriptstyle #3} \left \{ \vphantom{#1}\smash{\overbrace{#1}^{#2}} \right. \vphantom{\overbrace{#1}^#2} } %-------------------- % END PREAMBLE %-------------------- \ullabeld{\begin{matrix} \nu_{1,1} & \nu_{1,2} & \cdots & \nu_{1,k} \\ \nu_{2,1} & \nu_{2,2} & \cdots & \nu_{2,k} \\ \vdots \\ \nu_{f+1,1} & \nu_{f+1,2} & \cdots & \nu_{f+1,k} \end{matrix}}{k}{f+1}\]

where $\nu_{i,j}$ is the $i$th in line to the $k$th throne (the 1st in line currently sits on the throne). For a given $j$, each of the $\nu_{i, j}$ must be unique. This matrix also needs to have the special property that if up to $f$ of the nobles you appoint die, none of your vassals occupies two thrones simultaneously.

For instance, a succession setup for $k=3$, $f=3$ of the following form is not admissible under the previous condition.

\[\begin{matrix} a & b & c \\ d & e & f \\ g & d & h \\ i & j & k \\ \end{matrix}\]

If the following 3 nobles died, then $d$ would sit on two thrones at the same time.

\[\begin{matrix} \xcancel{a} & \xcancel{b} & c \\ d & \xcancel{e} & f \\ g & d & h \\ i & j & k \\ \end{matrix}\]

With the groundwork laid, we’re ready to start tackling the problem of how large $n$ must be to support an admissible succession plan for any given $k$ and $f$. The first thing to notice is that none of your vassals currently sitting on a throne can be in the line of succession for any other throne; otherwise, if the $f$ nobles ahead of them in the other line of succession died, they’d sit on two thrones simultaneously.

\[\begin{matrix} v_1 & \xcancel{\cdot} \\ \cdot & \xcancel{\cdot} \\ \cdot & \xcancel{\cdot} \\ \vdots \\ \cdot & v_1 \\ \end{matrix}\]

So, the picture we have so far looks like this:

\[\begin{matrix} v_1 & v_2 & v_3 & \cdots & v_k \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \vdots \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \end{matrix}\]

The next thing to notice is that any second-in-line noble can occupy any of the last-in-line positions to any other throne but no higher position.

\[\begin{matrix} v_1 & v_2 & v_3 & \cdots & v_k \\ v_{k+1} & \cdot & \cdot & \cdots & \cdot \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \vdots \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \cdot & v_{k+1} & \cdot & \cdots & \cdot \\ \end{matrix}\]

In fact, a second-in-line noble can occupy all of the other last-in-line positions.

But, we need at least $k$ second-in-line nobles, so we might as well distribute the other positions equally.

\[\begin{matrix} v_1 & v_2 & v_3 & \cdots & v_k \\ v_{k+1} & v_{k+2} & v_{k+3} & \cdots & v_{2k} \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ \vdots \\ \cdot & \cdot & \cdot & \cdots & \cdot \\ v_{2k} & v_{k+1} & v_{k+2} & \cdots & v_{2k -1} \\ \end{matrix}\]

Similarly, a third-in-line noble can be second-to-last in another line of succession but no higher.

\[\begin{matrix} v_1 & v_2 & v_3 & \cdots & v_k \\ v_{k+1} & v_{k+2} & v_{k+3} & \cdots & v_{2k} \\ v_{2k+1} & v_{2k+2} & v_{2k+3} & \cdots & v_{3k} \\ \vdots \\ v_{3k} & v_{2k+1} & v_{2k+2} & \cdots & v_{3k -1} \\ v_{2k} & v_{k+1} & v_{k+2} & \cdots & v_{2k -1} \\ \end{matrix}\]

This trend continues until we get to the “middle” row (the $\left \lceil \frac{f}{2} \right \rceil + 1$ row). If $f$ is even, then this is the exact same situation as the above. Nobles in these positions in the lines of successions can also occupy the position directly below this one in another line of succession but cannot simultaneously occupy any positions on the same row or higher. However, if $f$ is odd, a single noble can simultaneously occupy all positions in this row, since $2 \left \lceil \frac{f}{2} \right \rceil > f$.

This brings us finally to a lower bound on a value of $n$, as well as a plan for what the lines of succession could look like when the bound is tight.

\[n \ge \begin{cases} k\left (\frac{f}{2} + 1 \right ) & \text{$f$ even} \\ k \left (\left \lfloor \frac{f}{2} \right \rfloor + 1 \right ) + 1 & \text{$f$ odd} \end{cases}\]

On Ink Shortages

Fri, 20 Oct 2017 00:00:00 -0700

In the original presentation of the Paxos algorithm, Lamport used an extended metaphor involving the legislators of a bygone civilization [1]. These legislators would communicate by messenger and pass (non-contradictory) laws. One of the key assumptions of this protocol was that the legislators had special ledgers they carried around and could write on with indelible ink.

This is my (almost certainly misguided) attempt at an allegorical explanation of our recent work on diskless recovery in distributed systems [2].

Naxos

Having recently undertaken a long archaeological study of my own, I discovered that, shortly after its development, the parliamentary protocol used on Paxos spread throughout the region and was adopted by other parliaments (curiously, always with a few tweaks and always with a letter or two prepended to the name “Paxos”).

The protocol eventually spread to the the far-away (though similarly-named) island of Naxos. The Naxons, however, were not as wealthy as the Paxons and could not afford the vast quantities of expensive indelible ink the Paxon legislators used to record their decrees. (All other available inks had a tendency to fade over time and were thus unsuitable for parliamentary business.) However, Naxon legislators—out of necessity—had prodigious memories, and when they committed to a promise during the legislative process, they did not forget it.

At first, the Naxon legislators used the multi-decree Paxos protocol without issue. This protocol worked perfectly well for a number of years, until the first time a Naxon legislator was replaced. The Naxon parliament had no term limits, and incumbents almost always won (thanks to massive campaign contributions from the olive oil industry), but when a challenger finally won election for the first time, something happened that had never happened before—the parliament passed contradictory laws.

The Incident

As per parliamentary procedure, Naxon legislators used each other’s official titles to conduct all business (e.g., the gentleman from Σεαττλε). While the aforementioned election was ongoing, the MP from Μουνταιν Ϝιεω attempted to pass a law making Λεσλιε the cheese inspector for the coming year. Μουνταιν Ϝιεω’s representative was able to secure votes for this decree from a majority of legislators—including an MP who was about to be replaced, the MP from Βοστον.

However, this process of gathering votes took quite a bit of time and lasted well past the election. After he was elected, the new legislator from Βοστον proposed a decree making Βαρβαρα the cheese inspector for the coming year, which was also able to pass. As you can imagine, this led to some controversy.

The Naxon political scientists were not complete neophytes and knew about the potential for problems when new legislators were elected; they had designed a procedure specifically to handle this process. Because this procedure had to work in the case of the death of an MP, the Naxon protocol could not rely on the ability of a newly elected legislator to learn all previous decisions from the former legislator of their district. The political scientists, therefore, decided that newly elected legislators would, upon election, contact a majority of the parliament to learn about previously agreed-to decrees and adopted ballots.

The problem arose during the passing of the decree to make Λεσλιε cheese inspector. The decree’s sponsor rightly reckoned that the old MP from Βοστον might be soon replaced, and so sent the vote request to him via express courier. While waiting for replies from the other legislators, the election happened, and the representative from Βοστον was replaced. The new Βοστον MP was able to learn about all new decrees from a majority before any in that majority received the request to make Λεσλιε cheese inspector. The representative from Μουνταιν Ϝιεω was away at his villa and did not learn about the election results but did receive the remaining votes for his decree. At this point, he had received votes from a majority of districts to make Λεσλιε cheese inspector, but it was not true at that moment that a majority of legislators in the current parliament knew about this decree. Thus, the new MP from Βοστον was able to also make Βαρβαρα cheese inspector (using a majority that did not know about the previous decision).

The Solution

At first, the Naxon political scientists blamed the inconsistency on corruption; members of the political elite in Naxos had a habit of taking vacations in Byzantium—to the Naxons, a clear sign of moral bankruptcy. However, they eventually spotted the problem and came up with a solution.

The Naxon parliament needed a way to distinguish different representatives from the same district. They decided to refer to each legislator by their district name and the year in which the legislator was first elected (e.g., the gentleman from Σεαττλε, first elected in the year 453).¹ ² Then, in all of their messages to each other, the legislators included their current belief about the composition of the parliament. (I assume that the Naxons had a shorthand way of expressing this, but that particular detail has been lost to history.)

This procedure ensured that the cheese inspector incident could never happen again. Any legislator attempting to pass a decree while an election was ongoing would learn about any new legislators elected thanks to the added information attached to each parliamentary message (or the decree would complete before the new legislator could become an active member of parliament). In the example above, the Μουνταιν Ϝιεω representative would have found out that the incumbent from Βοστον had been replaced. When this happened, under the new rules, the Μουνταιν Ϝιεω MP would have to contact the new representative from the district before considering that district’s vote as counting towards a majority.

There is archaeological evidence to suggest that the Naxons realized this process of assembling majorities and replacing legislators could be used for other civic procedures, ones with weaker requirements than their law-making process, but exactly what procedures they did invent, we cannot say for sure.

How Paxos Solved the Succession Problem

On Paxos, thanks to their permanent ledgers and indelible ink, the succession problem was solved much more simply. A new legislator would be given at the start of their term the ledger of their predecessor and would continue on with the protocol as normal.

References

Lamport, L. The Part-time Parliament. ACM Transactions on Computer Systems. 16(2):133–169. May 1998.
Michael, E., Ports, D.R.K., Sharma, N.K. and Szekeres, A. Recovering Shared Objects Without Stable Storage. In Proceedings of the 31st International Symposium on Distributed Computing (DISC ’17). Vienna, Austria. Oct. 2017.

The Naxon parliament could have, instead, referred to each member using their unique names, but they preferred to stand on more ceremony rather than less. ↩
Importantly, any legislator serving non-consecutive terms was referred to by the year of election for their current consecutive tenure. ↩

Raft is (Equivalent to) Paxos, VR

Tue, 28 Feb 2017 00:00:00 -0800

Warning: This post contains LaTeX rendered with MathJax. It may not render properly in your RSS reader.

I shouldn’t have to write this blog post, but one too many times have I received online comments, questions at conferences, and even paper reviews claiming that Raft is more efficient than Paxos. This is not the case; as Ongaro and Ousterhout note, Raft, VR, and Paxos are equivalent in efficiency.

While the original Paxos protocol was rather concise in its description [1], it can be easily modified into a full-featured state machine replication implementation. These modifications have been described in many places (e.g., “Paxos Made Live” [2]). The normal case operation of Paxos is visualized and described below:

The client sends its command, $c$, to the current leader.
The leader proposes $c$ in the latest unfilled slot in its log, sending the proposal to the other replicas.
The replicas put $c$ into the same slot in their logs and respond to the leader.
The leader waits for a majority of replicas (including itself) to respond and then executes the command and responds to the client with the result.

Viewstamped Replication (VR) was developed at the same time as Paxos [3, 4]. While VR includes some practical details left out of the original Paxos papers, the core mechanism it uses is the same as Paxos.¹ Moreover, the normal case protocol is exactly the same as described above, and its performance is identical.

Raft was published much later, in an effort to be more understandable than the existing descriptions of Paxos and VR [5], and the Raft protocol is almost identical to VR.² Its normal case operation is also exactly the same as described above. In terms of message delays, number of messages sent and received, and amount of computation needed in the normal case, Raft has no performance benefits compared to VR or a full-featured Paxos implementation.

References

Lamport, L. The Part-time Parliament. ACM Transactions on Computer Systems. 16(2):133–169. May 1998.
Chandra, T.D., Griesemer, R. and Redstone, J. Paxos Made Live: An Engineering Perspective. In Proceedings of the Twenty-sixth Annual ACM Symposium on Principles of Distributed Computing. Portland, Oregon, USA. 2007. 398–407.
Oki, B.M. and Liskov, B.H. Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems. In Proceedings of the 7th ACM Symposium on Principles of Distributed Computing (PODC ’88). Toronto, Canada. Aug. 1988.
Liskov, B. and Cowling, J. Viewstamped Replication Revisited. Technical Report MIT-CSAIL-TR-2012-021. MIT Computer Science and Artificial Intelligence Laboratory.
Ongaro, D. and Ousterhout, J. In Search of an Understandable Consensus Algorithm. In Proceedings of the 2014 USENIX Annual Technical Conference. Philadelphia, PA. 2014. 305–320.

There is a slight difference between VR and Paxos in the way that invariants are guaranteed across leader elections, but it does not affect the normal case operations of the protocols. ↩
The differences between Raft and VR are detailed in the Raft paper [5]. ↩

Quines as a Side-channel

Fri, 02 Sep 2016 00:00:00 -0700

I recently competed in IceCTF, and two of my favorite problems were about quines. A quine is a program which prints its own source code; the term was coined by Douglas Hofstadter in his (excellent) book, Göedel, Escher, Bach.

The Problem

The quine problems involved finding a vulnerability in a service (run on a remote machine) which accepted a C source file as input and repeatedly compiled the source file, ran the program, and overwrote the source file with the program’s output. After some number of iterations the server would return the resulting source code, provided some conditions held (more on that below).

In their purest form, quines should not take any input. For example, the following Python one-liner is considered “cheating”:

print(open(__file__).read(), end="")

However, pure quines are quite useless for the purposes of a CTF competition. If the quine service had only returned one bit of information, whether or not the input program was a true quine, then it would have been safe from the attack we used. However, the C programs the server ran could read from files, and the user the programs were run by had permissions to read /flag.txt, the file whose contents we were supposed to exfiltrate.

In the first version of the quine problem, which sent programs through exactly 20 iterations, I had the idea of simply wrapping a target program with layers of print statements. To illustrate, I’ll do this in Python. Consider the following target program:

print(open('/flag.txt').read())

If we wanted this program to be run during the final iteration and there were 5 iterations, we would wrap it in 4 print statements like so:

print("print(\"print(\\\"print(\\\\\\\"print(open('/flag.txt').read())\\\\\\\")\\\")\")")

This approach didn’t work. The service spit out an error, complaining about source files not matching. We took this to mean that the service would only print out the final source file if it matched the input source file. In other words, we assumed that we would only get one bit of information out: quine or no quine. This turned out not to be the case, but I like the solution we came up with better than the intended one.

The Attack

Since we’re assuming that the server only returns the quine or no quine bit, the key insight is that if the programs being run can take some input, then we can create programs which are quines if and only if some condition holds.

I started with the following C quine, which I found on Stack Overflow:

#include <stdio.h>
char*s="#include <stdio.h>%cchar*s=%c%s%c;%cint main(void){printf(s,10,34,s,34,10,10);}%c";
int main(void){printf(s,10,34,s,34,10,10);}

How does it work? The critical bit is the %s inside s. This allows the program to print s using a copy of itself. The other trick is that it uses code points to print out characters that need escaping. (Try representing newlines as \n inside s and removing all the 10s from the printf arguments, and you’ll see what goes wrong.)

Using that basic quine as a starting point, I came up with this generator in Python which creates C quines based on some function, test (passed to gen_quine as a string):

TO_REPLACE = "\n\r\"\'"

def code_point(c):
    return str(ord(c))

def strip_normal_chars(s):
    return ''.join([c for c in s if c in TO_REPLACE])

def gen_codepoints(s):
    cps = ','.join(map(code_point, strip_normal_chars(s)))
    if cps:
        cps += ','
    return cps

def gen_char_s(s):
    return ''.join(['%c' if c in TO_REPLACE else c for c in s])

def gen_quine(test_function):
    return (
"""#include <stdio.h>
#include <stdlib.h>
%s
char*s="#include <stdio.h>%%c#include <stdlib.h>%%c%s%%cchar*s=%%c%%s%%c;%%cint main(void){if(!test()){exit(1);}printf(s,10,10,%s10,34,s,34,10,10);}%%c";
int main(void){if(!test()){exit(1);}printf(s,10,10,%s10,34,s,34,10,10);}
"""
    ) % (test_function,
         gen_char_s(test_function),
         gen_codepoints(test_function), 
         gen_codepoints(test_function))

For example gen_quine("int test() {return 1;}") returns:

#include <stdio.h>
#include <stdlib.h>
int test() {return 1;}
char*s="#include <stdio.h>%c#include <stdlib.h>%cint test() {return 1;}%cchar*s=%c%s%c;%cint main(void){if(!test()){exit(1);}printf(s,10,10,10,34,s,34,10,10);}%c";
int main(void){if(!test()){exit(1);}printf(s,10,10,10,34,s,34,10,10);}

This is a pure quine.

I’ll leave you to figure out the fine points of the generator. There’s no real insight, just some fiddling with details. The point is that using it, we can generate arbitrary (pseudo-)quines! We then used that generator with the following function:

#include <unistd.h>
#include <fcntl.h>
int test() {
    int fd = open("/flag.txt", O_RDONLY);
    char x;
    pread(fd, &x, 1, 0);
    return x == 'I';
}

It worked! The server told us that the generated program was, indeed, a quine. (An important fact here is that all flags had the format IceCTF{...}.) From there, all we had to do was loop through each bit of the flag and test it. Since all of the flags were less than 80 characters long, this took less than 640 requests to the server (640 separate quines to checks). Not too bad.

Conclusion

Was arbitrary (pseudo-)quine generation necessary? Well, no. It turned out that the server printing out the final source code was important, and the problems could have been solved in a single request. But our way was more fun.

Moral of the story: if you’re ever building QCaaS (Quine Checking as a Service), be careful. You might be returning more information than you think.

Using Travis CI with Github Pages

Fri, 12 Jun 2015 00:00:00 -0700

I’ve been using GitHub Pages for my personal website for some time now. Since I’ve never needed dynamic content, GitHub has been a perfect solution. It’s simple to use and free. GitHub will either serve a completely static site or a Jekyll site, which it automatically generates.

Travis CI is a popular testing and deployment service. You can use it to generate and deploy your website to GitHub Pages instead of using GitHub’s automatic Jekyll generator.

Why use Travis?

Most of the time, using Travis is not necessary. Vanilla Jekyll works quite well and has most of the features that you’d want out of a static site generator. However, GitHub Pages only allows the use of a limited number of Jekyll plugins and runs Jekyll builds in safe mode. If you want to use Jekyll plugins other than the ones available on GitHub Pages, use your own custom plugins, or generate parts of your site by non-Jekyll means, then you should consider using Travis to automatically do the job.

My original impetus for switching to Travis was that I wanted the generation process for my website to generate my resume automatically, which gets built from a YAML file using LaTeX and a Python script. However, now that I’m free from GitHub Pages’ limitations, I’m considering switching from Jekyll to Pelican because I like Jinja templates better than Liquid.

Making the Switch

The first thing you need to do is create a new branch, as the generated site will eventually get put into the GitHub Pages branch (either gh-pages or master depending on the type of repo). Call the new branch source (or maybe pages-source if you prefer).

$ git checkout -b source

Next, you’ll want to enable Travis for the repo. You can do so here.

You should then create a file called .travis.yml in the root of your repository in the new source branch.

Configuring Travis

Your .travis.yml file tells Travis what to do. You’ll use it to install your site’s dependencies, build your site, and then push the generated site back to GitHub.

Authenticating Travis with GitHub

In order for Travis to be able to push your site to GitHub, it will need to authenticate with GitHub. Travis has built in encryption tools which allow you to encrypt some secret which only the Travis build agent for your repo will be able to decrypt. Then, you can upload the encrypted secret to a public repo without fear that you’re giving everyone access they shouldn’t have.

Some people have suggested using an encrypted GitHub Personal Access Token. This is a decent solution, but the problem is that the permissions settings for access tokens are too coarse. They only allow you to determine whether or not the token has access to all your public repos and all your private repos. Instead, a better solution is using a deploy key on the repository. If that key somehow gets compromised, the most anyone will have access to is a single repo.

First, you’ll have to generate the key.

$ ssh-keygen

Name the file .travisdeploykey and do not put a password on the key. Upload the public key (in .travisdeploykey.pub) to your repo (Settings > Deploy keys). Now, encrypt the private key. You’ll need the Travis client installed to do that.

$ gem install travis

Then, encrypt the file.

$ travis encrypt-file .travisdeploykey

This will output the .travisdeploykey.enc file and print out a command to be added to the build script. Save that print out and add .travisdeploykey.enc to your repo. Do NOT add the unencrypted private key to the repo. If you already did, remove the public key from the repo, and start over with a new key. If not, you can go ahead and delete the private key as well as the public key.

The Build Script

Copy the following to .travis.yml

install:
  - gem install jekyll
before_script:
  # Replace with the line that travis encrypt-file printed out
  - openssl aes-256-cbc ...
  - chmod go-rwx .travisdeploykey
  - eval `ssh-agent -s`
  - ssh-add .travisdeploykey
  - git config user.name "Travis-CI"
  - git config user.email "noreply@travis-ci.org"
  - COMMIT_MESSAGE="Publishing site on `date "+%Y-%m-%d %H:%M:%S"` from
    `git log -n 1 --format='commit %h - %s'`"

script:
  - set -e
  - make # Or jekyll build or whatever command you use to generate your site
  - git checkout -b master # Replace master with gh-pages if applicable
  - git add -f _site/ # Replace _site/ with the build's output directory
  - 'git commit -m "${COMMIT_MESSAGE}"'
  - git filter-branch -f --subdirectory-filter _site/ # Again, replace _site/
  # Obviously, replace with your repo's SSH URL and the appropriate branch
  - git push -f YOUR_REPOS_SSH_URL master:master

branches:
  only:
    - source

If you have any other dependencies to install (e.g. Python packages), you should add the necessary commands under install. Otherwise, once you make the appropriate adjustments to .travis.yml, you should be good to go!

Final Notes

Caveat user: debugging your install script and adding all of the dependencies can be a somewhat frustrating process, especially if the dependencies take a while to install. You might find yourself making a change, pushing that change to the repo to kick off a Travis build, and then waiting 5 minutes only to find out that something else went wrong.

Now that you’re building your site with Travis, you don’t need GitHub to build it with it’s built-in Jekyll support. By default, GitHub will only invoke Jekyll if there is a _config.yml file present, but if you want, you can go one step further and add a .nojekyll file to the site. If you’re still using Jekyll, then you’ll need to add the following to _config.yml.

include:
  - .nojekyll

NOPaxos to Appear in OSDI

Sun, 31 Jul 2016 00:00:00 -0700

Our paper on eliminating Paxos overhead with network support will appear at OSDI 2016 in Savannah, GA.

Diskless Distributed Recovery

Thu, 25 Aug 2016 00:00:00 -0700

Our paper on providing stable storage for the Diskless Crash-Recovery failure model is now available.

NOPaxos Madrona Prize Runner-Up

Wed, 19 Oct 2016 00:00:00 -0700

NOPaxos was recognized at UW CSE's annual industrial affiliates open house by Madrona Venture Group for its start-up potential. See the full post on CSE news.

NOPaxos Featured on The Morning Paper

Thu, 08 Dec 2016 00:00:00 -0800

NOPaxos was featured on The Morning Paper blog, with a good write-up of the paper's main contributions.

NOPaxos Recognized at Industry-Academia Partnership Cloud Workshop

Fri, 31 Mar 2017 00:00:00 -0700

More recognition for NOPaxos. It was awarded the Best Poster Award at the Industry-Academia Partnership Cloud Workshop at UW.

NOPaxos Source Code Available

Mon, 08 May 2017 00:00:00 -0700

The source code for NOPaxos is now available.

Diskless Recovery at DISC 2017

Sat, 01 Jul 2017 00:00:00 -0700

Our paper on diskless distributed recovery has been selected to appear at DISC 2017.

Eris at SOSP 2017

Fri, 04 Aug 2017 00:00:00 -0700

Eris, our network-integrated distributed transaction processing system, will appear at SOSP 2017 in Shanghai, China.

Eris Source Code Available

Wed, 07 Mar 2018 00:00:00 -0800

The source code for Eris, our network-integrated distributed transaction processing protocol, is now available.

PaPoC Workshop Talk

Fri, 27 Apr 2018 00:00:00 -0700

I recently attended PaPoC '18 and presented our short paper on providing efficient causal message delivery in datacenters using programmable network hardware.