Everything You Didn’t Know About the Handshake Naming System 

The project was conceived by the people who brought you Purse and Private Internet Access.

Handshake, despite popular misconception, is not a blockchain project that seeks to decentralize existing DNS infrastructure in its entirety. That’d be biting off something much more than any cryptocurrency-backed system could chew. Indeed, it is part of the reason why the predecessors of Handshake have not seen true adoption.

According to the project paper,

“The Handshake naming protocol differs from its predecessors in that it has no concept of namespacing or subdomains at the consensus layer. Its purpose is currently not to replace all of DNS, but to replace the root zone file and the root servers.”

Many fail to realize that DNS is already decentralized, with the exception of a single, critical component, of which trust is centralized: the root zone, or simply, a collection of top level domains (TLDs). And this trust anchor is kept by a small federation of authoritative bodies, where ICANN is currently the ultimate authority.

Handshake is a project that intends to disintermediate the role of ICANN and, in its place, applies a decentralized blockchain using cryptographic signatures produced by Proof-of-Work to operate this integral piece of infrastructure to maintain the very security of the Internet itself.

And that’s enough of the fun but surface level stuff. You came here for the raw, juicy stuff, distilled from the project paper and from the mouths of the project’s contributors themselves.

I interviewed Christopher Jeffrey, the prolific systems engineer who introduced the alternative Bitcoin client bcoin, whom I henceforth refer to as Brain, as in, the brainiac who architected and implemented the Handshake blockchain.

I’ve additionally had many discussions with Joseph Poon, the mad mind behind two major layer-two blockchain scaling frameworks (Lightning Network and Plasma), a contributor to Handshake who participated in designing the proposed distribution mechanisms.

Ready? ’Cause you’re about to plunge into the depths of this protocol.

Newbies beware! This article gets progressively more technical the further you read.

Highly recommended! Even though it can stand on its own, for maximum understandability, this article should be complementary and is best taken as a side dish to the project paper.

For a primer covering DNS architecture, Imran Khan’s article provides a solid foundation.

Steven McKie’s post additionally provides an investor’s perspective about the viability of Handshake.

Certificate Authorities

Certificate Authorities in the DNS network constructed the way it is today are the trusted stewards for the operation of the Internet. These stewards, as explained in the project paper, are profit-maximizing entities. Meaning, ICANN has no altruistic incentive to act honestly, yet has every incentive to maintain its natural monopoly over the riches that come with governing a critical layer of the Internet. And even if CAs intend to be good stewards, the premise of the decentralization movement posits that we should not ever need to rely on any single authority, especially one that acts as the gatekeeper to the highway of all human knowledge.

Where untold riches are exploitable by those central authorities at the very top of the hierarchy, decentralizing the trust model over the namespace can, if successful, distribute that rich resource body and allocate it across this new model of a permissionless network backed by a distributed ledger which can securely and efficiently replace the existing paradigm.

Enter Handshake

“Handshake is an ongoing project to establish a decentralized network whereby cryptoeconomic incentives are established to coordinate consensus on the association between names and certificates.”

Most of the problems we face today, contrary to popular belief, are not solved with a blockchain. And many of you skeptics understand this acutely. But for Handshake, how is their application-specific blockchain actually incredibly useful, indeed, ideal where domain names are concerned?

Zooko’s trilemma posits that you could only satisfy 2 out of the 3 properties when it comes to namespaces:

1. human-readable names
2. secure
3. distributed namespace

“Recent innovations with the blockchain has possibly maneuvered around this trilemma by creating a single point of consensus around the association between names and certificates, in a single decentralized blockchain represented by many actors verifying the network. Achieving this security property requires not only redesigning the trust anchor in certificate authorities, but also requires deep integration in the naming infrastructure itself.” — Handshake project paper

In other words, a blockchain is the ideal mechanism for enabling global consensus to be achieved about the ownership and canonical ordering of the transfers of ownership of domain names. Up until this solution, there had been no good way to subvert the role of CAs to prove that yes, you do in fact own this asset. Indeed, “the blockchain itself maintains the state of these assets” whereby “the assets in question are names.”

Handshake as a base layer blockchain won’t likely ever face any scalability issues.

When asked about the scalability of the Handshake blockchain, Brain explained, “It’s not meant for payments, so it doesn’t need to scale to VISA-levels of transactions. And the existing root zone is only about 1500 domains. So, in order to replace the existing root zone, the blockchain just needs to be able to handle 1500 domains, which is nothing. We think it can handle around 50–100 million transactions. With renewals and certain limits on-chain, it probably won’t ever scale past 50–100 million domain names. Eventually, renewals will comprise most of the blocks on the blockchain. Every name requires a bi-yearly renewal.”

A Soft Fork to Mass Adoption

The crux facing every naming protocol seeking to amass true adoption has been over solving the problem of integration and usability.

How does your inherently slower blockchain-based application even attempt to outcompete a much faster, more user-friendly, centralized solution that billions are already using?

Blockstack and ENS are two examples of naming systems that will not see real usage for their intended purposes. Let me explain why. In order to resolve a domain using either, a user is required to run a fully validating node. Now why would I want to do that when all I need to do to open a webpage is query Google’s DNS resolver through my browser?

“We intend to be transparently compatible with the protocol used by IANA and ICANN over the decentralized system, all while requiring zero intervention on the part of most users.”

Handshake was painstakingly implemented to be a soft-fork to the existing Internet infrastructure, knowing that any attempts to migrate users over to a new, untested system, is an unrealistic demand impeding widespread usage. Here, assimilation beats migration. And it makes usage of the Handshake naming system orders of magnitude simpler to use than the approaches of its predecessors.

The two methods that the project uses to foster adoption are:

• Compact proofs for local resolution of names (SPV)
• Incentivize integration from package maintainers

SPV Name Resolution

Handshake uses Proof-of-Work to allow the blockchain to be maximally permissionless, decentralized, secure, and most importantly, censorship-resistant. This design decision allows for those running full and Simplified Payment Verification (SPV) nodes to trustlessly (and rapidly) resolve names, at a speed that is actually quicker than querying Google’s DNS resolvers. It’s quicker in the case that your full/SPV node has the data locally cached. (I’ve experienced it firsthand from running the SPV client!!)

Unlike that of the aforementioned protocols preceding Handshake, the latter breaks new ground by enabling compact SPV proofs. This by definition allows the process of verification to be done (very importantly!) trustlessly, as your SPV client allows you to run your own local recursive DNS resolver. The architecture in the existing DNS just doesn’t allow for this direct resolution without a central authoritative proxy.

Handshake was formerly using a memory-hard graph function used by projects like æternity and Mimblewimble called Cuckoo Cycle, created by John Tromp. This is being scrapped in favor of SHA3, a good old fashioned hashing function that won’t be broken even with all the existing computing (and future quantum computing) power in the world. Brain very seriously suggested wanting the Handshake blockchain to persist even after mankind has departed Earth.

Integration from the FOSS Community

“…our goal is to work with DNS, not against it.”

“In order for the root zone to be replaced transparently, recursive resolvers must point at an authoritative nameserver which serves records committed to the blockchain rather than ICANN’s root zone file. This is a difficult dynamic to work with, as virtually no consumer devices currently ship with a recursive resolver running locally.” — Handshake project paper

The “airdrop”, or faucet distribution, rather, targets the FOSS community in its entirety via GitHub, Internet Relay Chat (IRC), and even going so far as to airdrop to longtime holders of PGP and SSH keys, and aims to financially incentivize, for instance, open-source developers and package maintainers to include the Handshake integration library in their software or add the Handshake daemon (hnsd) package natively. The more integration there is, the greater the adoption.

“This removes the need for an average user to install extra software, and instead places the responsibility on the developers of software.”

This just made too much sense. (Why don’t other projects do this?)

The idea is that it pays back the people who’ve contributed free software to open-source who had no expectation of financial reward. Similarly, Handshake will contribute financial reward to that same group of people with no expectation of receiving software in return.

Yet the social incentive to create software that integrates Handshake’s library integration remains.

The Coming of Urkel

From Sparse Merkle Trees to Merklix Trees and the MerkleSet (say it 10 times fast!) to a highly optimized authenticated data store

The following section is transcribed from my interview with Brain:

“A lot of the naming projects in the past didn’t have very good SPV, and that’s an issue if you actually want to resolve names securely. And you have to run a full node to do it; most people aren’t going to do that. Most people aren’t going to store hundreds of gigs of a blockchain just to make a web browser work. So SPV support was extremely important for the Handshake project. So in order to do that we needed some sort of authenticated data structure.”

— Brain

The advent of Handshake brought many innovations that I foresee having ripple effects in the cryptocurrency space. One of which, very notably, is a provable data store named the Urkel Tree. Unlike every blockchain system before it, Handshake uses its own authenticated data structure — the first of its kind implemented specifically for blockchain use — which doesn’t touch levelDB or any other existing data store.

The Urkel Tree is what is described in the section of the project paper called “Flat-File Merkle Tree”. Before coming up with the FFMT, Brain spent months of research exploring the data structures currently used in production systems, namely, Ethereum’s base-16 trie and Google’s Sparse Merkle Tree.

But because the protocol intends to replace the DNS root zone, the speed of lookup must be unencumbered and rapid, and the proofs must be compact. These uncompromising requirements led to the coming of Urkel. “The average DNS message is a few hundred bytes tops. So if you want to match the speed of the existing root zone over DNS and want to minimize overhead, you needsmall proof sizes,” Brain says.

“Our tree — since it’s in a series of append-only flat files, essentially making the tree its own database (which is why we don’t need levelDB) — has every traditional database feature you’d want: snapshotting, crash-consistency, range queries, and iteration.”

Urkel is currently programmed in JavaScript and a portable C library is planned for the near-future.

The FFMT is an adaption building upon that of Bram Cohen’s (Chia Network) and Amaury Séchet’s, aka deadal nix (Bitcoin ABC), proposed (though not implemented in production) Merkelized data structures, called MerkleSet and Merklix Tree, respectively.

After finding that neither Ethereum’s base-16 trie nor the SMT were suitable options given the requirements for Handshake, Brain explains:

“So I started working on my own Merkelized base-2 trie. I wanted to do it base-2 so that the proofs would be really small. Each internal node would only have two possible child nodes. And I also wanted to implement it in flat files, just directly on the file system — no levelDB. Simply, each internal node has pointers to the next nodes, and those are stored somewhere on a file. And for each node, you just traverse the next one by reading the pointer. This worked extremely well. The code was simpler, it was faster, the proofs were smaller.”

Exploring Ethereum’s base-16 trie

“We looked at a lot of different options; spent months researching. We first looked at Ethereum’s base-16 trie — Ethereum uses several of them, actually — it’s one of the few authenticated data structures used in production, so we took a look at that. The problem with that — there were several problems, actually — primarily was proof size. Ethereum trie proofs are extremely big because the trie is base-16. That means, each internal node in the trie has 16 child nodes. And that makes internal nodes huge, so once the trie gets relatively deep, you start to see really big proofs. We were seeing several KBs, and that was unacceptable to us. The other problem with it was that it is implemented on top of levelDB, and that slows performance down to a crawl, as most of your CPU and system load is dedicated to just compaction — levelDB’s compaction thread. It’s going crazy all the time because there’s so many updates to levelDB’s tree. So now you have a tree inside of a tree; it’s not very optimal.”

Key research findings:

• In performance tests, Urkel provided a roughly 50–100x speedup benchmarked against Ethereum’s base-16 trie.
• Urkel’s base-2 proofs were 4x smaller than that of Ethereum’s base-16 proofs.

Exploring the Sparse Merkle Tree

In testing the available data structures used in production before deciding to implement a standalone data structure tailor-made for blockchains, Brain came upon a construction that Vitalik was toying with to be the authenticated data structure of choice for Plasma designs, called the Sparse Merkle Tree(SMT). In implementing it, Brain found that it was not a viable option, as the heavy caching required for database lookups severely hindered usability. So that was nixed. (Vitalik Buterin, if you’re reading this, please consider these findings.)

Google’s certificate transparency project is the only one using a SMT in production, which is also implemented on top of an existing data store (levelDB).

It was extremely slow in terms of disk I/O and CPU cycles. To insert 5000 leaves, you would have to do a million rounds of hashing, or something insane. That’s fine if you’re storing certificate revocation lists because it doesn’t get updated that frequently. But if you have a blockchain which makes frequent updates to the tree, it’s sort of a bad solution.

It’s slow in so many different ways. In order to make it fast, it requires heavy caching. In a language like Go or JavaScript or any memory-managed language, it’s scary putting all that stuff in memory. Eventually, you’ll put too much heat on the garbage collector, garbage collection pauses, and it starts getting noticeable. Generally, it’s good practice not to put that much stuff in memory. With the SMT, you need to keep an insane amount of stuff in memory just to make the lookups fast. You *could* batch insertions. Meaning, you can insert 5000 leaves at the same time. If those leaves are anywhere near each other, you can skip a few rounds of hashing. Problem is, if the tree is 256 levels deep, there’s a 1 in 2²⁵⁶ chance that a node is going to be next to another one — basically impossible. As you go further up the tree though, you save on rounds of hashing. But it’s not even worth it to batch insertions; that’s how pathetically slow it is. Because the process for batching insertions is annoying to implement and confusing. But at the end of the day, it doesn’t even add that much just because that data structure is so inherently slow. It’s fine if you’re using it for certificate revocations like Google is, but using it on a blockchain is an extremely bad idea. So that’s why we strayed away from it.”

Key research findings:

• Benchmarked against Google’s Sparse Merkle Tree implementation, Urkel provided an astounding 500x speedup.
• Yet Urkel’s compact proofs were roughly the same size as compressed SMT proofs, as in, <1 KB with tens of millions of leaves. (Bingo!)

Out of this research, Brain concluded that “authenticated data structures implemented…on top of existing data stores to be inherently flawed in terms of scalability.”

And that’s…the actual origin story of Urkel, not the one they show you on TV.

Covenants

The Handshake blockchain does more than keep the history of domain name ownership; it implements an auction system in-protocol and on-chain, by way of prescribed “smart contracts”, called covenants, which were first described by Malte Möser, Ittay Eyal, and Emin Gün Sirer.

These covenants allow for dynamic consensus-level behavior normally absent in highly-constrained UTXO-based state machines like Bitcoin. The desire for covenant-like behavior was what spawned the birth of the application platform experiment better known as ethereum, an account-based system. Yet with Handshake, implementing smart contract behavior on top of a UTXO-based system with new script opcodes in production-ready code is another example of innovation breaking new ground for the ecosystem. This construct, due to being generically implemented albeit “prescribed with consensus code”, enables future covenant types to be soft-forked into the protocol in a way that is forwards-compatible with future DNS networking infrastructure. However, covenants on Handshake are still special-purpose and not intended for building decentralized applications like smart contracts are on Ethereum — another design decision that allows the Handshake blockchain to be highly optimized for its specific naming application.

“Using our generic consensus-level covenant system, we are able to implement almost any kind of smart contract on the blockchain layer.”

How the Auction System Works

The auction system is implemented as a consensus-level covenant, broken down into various actions described as covenant_type on the output layer (as low-level as it gets). Handshake implements a Vickrey auction, a second-price auction named after its inventor, William Vickrey. We won’t go into the decision process that led to setting up the auction structure as a Vickrey auction, but we will go into the process illustrating how a name is procured on Handshake.

To illustrate, when you want to bid on a name, you’d use the bid covenant type. You’d send money into a covenant, and the money that you bid is locked up until the auction is over. These bids, mind you, are blind bids. Meaning, you have an actual bid (e.g. 5 HNS), but you may want to produce a lockup value of 10 HNS, which obfuscates your true bid value.

What the convenant does is it takes your actual bid, concatenates it with some nonce, hashes it, and provides you with that hash. After a weeklong revealperiod, you reveal the nonce you used, which reveals your actual bid of 5 HNS. After you reveal, you withdraw 5 HNS out of the convenant as change.

(lockup value) — (actual bid)= your change

Then, your 5 HNS remains locked up for a 2 week period until the auction is closed. When the auction ends, the winner is chosen, and the losers get their money back — losers are allowed to exit the covenant.

The winner’s money, however — remember, winners pay the second highest bid — is sent to another covenant path, the register covenant. The register covenant is designed for maintaining and updating names. All money that gets sent to that path could never exit the covenant, which means that every winners’ coins get effectively burned.

Revocation

One such covenant_type I take particular interest in is that with the REVOKE functionality. Similar to the function of the exit game in Plasma, revocation on Handshake is the last-resort option that allows the rightful owner of a name to revoke, or challenge, a name transfer in case the owner ever loses his or her keys. Having this option renders the incentive to steal names completely null.

DNSSEC “Ownership Proofs” for Bootstrapping the Network

Another flaw in the design of Handshake’s predecessors that will prove to be fatal is in neglecting to consider existing nameholders. If I am Microsoft and I have invested substantial amounts of capital into my brand whose value is encapsulated in my domain name, and here comes this new proposed DNS system that tells me that I don’t own this name anymore in their system — why would I ever migrate over and use that system?

With that in mind, Handshake pre-reserves the Alexa top 100,000 names claimable — within a “Sunrise period”, or a 4-year window from the date of its mainnet launch — by the rightful owners of those names. To claim their names, the owners simply produce a very specific DNSSEC proof which the Handshake protocol has cleverly repurposed to be used as secure proofs of name ownership. But here’s the thing — they’re doing this claims distribution permissionlessly, whereby the blockchain — not anybody involved with the project — gives the names to their rightful owners.

To implement this for a permissionless distribution, Brain tells us:

“It’s tricky to do it cryptographically because you don’t have all those peoples’ keys. And not very many people actually have DNSSEC set up. We were looking at DNSSEC and we realized you could actually use it for proofs of ownership — that’s not what they’re intended for. They’re intended for ensuring you’re not being man-in-the-middled, that somebody’s not changing data while the data is in-flight. But you could use it to prove ownership using a much much stricter subset of DNSSEC proofs that we’re dubbing “DNSSEC ownership proofs”.

What we do is, since most people don’t have DNSSEC set up, is we take ICANN’s existing key signing keys (KSKs). They have two of them: one they generated in 2010, and they’re about to roll over to a new key that they generated last year (2017). Basically, having these two RSA keys hard-coded into consensus rules allows anyone in the Alexa top 100,000 in the existing root zone to prove that they own a name, even if they don’t have DNSSEC currently set up — they can set it up later. The blockchain will validate the DNSSEC trust chain, starting from ICANN’s KSKs.

This was tricky to implement because 1) DNSSEC is tricky and there’s lots of strange edge cases, which is why we’re only using a small subset of DNSSEC for ownership proofs, and 2) because most of the keys are RSA keys. RSA keys are extremely large and signatures are just as large as the keys. Every cryptographic algorithm — ECDSA included — have always had really strange edge cases that you need to be aware of. Also, for RSA in particular, there are DoS vectors because keys can be of a different size. So it required us to put a lot of effort into researching DoS limits for RSA keys and edge cases with PKCS1 v1.5 signatures. We discovered edge cases in RSA signatures that I don’t think have yet been discovered on DNSSEC. I think we discovered some new ones while we were sprucing it up for a consensus protocol. It’s not mentioned in any RFCs. Even though DNSSEC has been thoroughly specified over the course of a decade in so many different RFCs, none of them mentioned the edge cases that we found. It makes the point that specifying a consensus protocol doesn’t really add much because there’s always going to be those edge cases.”

To further incentivize existing nameholders, the Handshake blockchain gives nameholders a coin reward if they redeem their names.

Here is where the use of VCs and funds come in handy. Where the ICO model potentially obsoletes them, the abundance model assigns a very relevant function for them: give these newly created coins a valuation backed by your investment amount. With their investment backing, the coins hold inherent value, even before they’re released into the free market for price discovery. At the time of writing, the VCs valuated one HNS token to be worth $0.10.

• A total of 7.5% of the 1.36 billion HNS coins are pre-allocated to existing nameholders, provided that they produce a sufficiently secure DNSSEC proof of ownership.
• Next, a total of 2.5% of the 1.36 billion coins will go to TLD owners, provided that they claim their TLDs.

The rest, left unclaimed, get burned after the Sunrise period is over.

Name Rollout Interval

People can open an auction for any name as long as it’s not on the pre-reserved name list and is within the rollout interval. The rollout interval is a weekly rollout of names that become available for bidding. Over the course of one year, all the names will be available for bidding.

To determine whether a name is within the rollout interval, similar to how ENS does it, the Handshake protocol hashes the name you want to open, runs SHA3 over the string, modulo the hash by 52, giving you the number of a specific week, multiplies that number by the average number of blocks per week, then you have the block height that your name will roll out on. Done this way, the rollout interval is deterministically random and uniformly distributed for all the names that are available.

Concluding Remarks

I write this because I’m very bullish Handshake, knowing the level of craft and meticulousness that went into this project.

And while much of what has been discussed in this article has not been validated yet by the free market, the ideas presented are nonetheless compelling. The hope is that, with enough bright minds witnessing this social experiment for what it is and what it has the potential to become, more entrepreneurs, brainiacs, and mad scientists could walk away with these technical and mechanism designs to set off with their own, positive-sum ones playing the Game of Abundance instead of the Game of Exploitation.

“It is exceptionally costly to build a top-down organization of millions in a hierarchy to replace entrenched interests, hence a ground-up structure is proposed using gift economies, which is only possible with emerging technologies which is capable of accounting value without central authorities (the blockchain), as well as tools for wide-scale coordination across millions…. Handshake is an open performance by all parties worldwide to participate and deliver a new naming system as a gift to wider society.”

When more thought leaders and entrepreneurs structure their models around Abundance Games, the ecosystem can begin to move away from the model of ICOs and welcome, instead, a new era of gift economics *with returns on gifting*.

And that’s your big idea.