Canonical References

• Canonical what?
• Git History in the Making
• A Quick Guide to Canonical References

Canonical what?

Most people who use Git are used to referring to things called branches (refs/heads). “I pushed the branch”, “check out the feature branch to see the work”, “Ok, I’ll pull your branch”, etc. A branch is simply a specific kind of reference in a Git. You’re also likely familiar with tags (refs/tags), and these are another kind of references, too! A reference, quite simply, is a name that points to a Git object, like a commit object or a tag object. The naming convention and what objects they point to allow us to talk more specifically about branches, remote branches, tags, and notes.

So what’s a canonical reference? In the Radicle protocol, all your references live under your namespace (refs/namespaces/<nid>/refs/{head,tags,*}). We want Radicle repositories to also act like more traditional Git repositories that have the typical references refs/heads/main, refs/tags/v1.0.0, refs/heads/qa/feature-1. So we introduced a way to synthesise these references from the references under the set of refs/namespaces/<nid>/refs. These references become canon¹ and can be considered for use, e.g. fetching a specific tag object for building binaries.

Git History in the Making

Since the dawn of the heartwood implementation of the Radicle protocol, the concept of the defaultBranch was included in the identity document. Alongside this, we have had the threshold value. Even more importantly, we have had the delegates managing the identity of the identity document, as well. By combining these 3 values we were able to calculate the canonical reference of the defaultBranch. The defaultBranch becomes a canonical reference when a threshold number of delegates decide on a commit to use for its canonical form.

If you have been using Radicle, you’ll have noticed that whenever you push Git tags that they get placed under your namespace. Most people who have used git would expect these tags to be placed under refs/tags. Up until now, we have broken this expectation, because we are working in a decentralised environment using Radicle, protecting the global namespace of the repository. So, we asked ourselves, “Can the defaultBranch approach be generalised?” This way, the refs/tags namespace can be populated using the canonical form of the tag.

“It should be easy!”, I said – famous last words.

What ensued was a RIP² and two patches³⁴, over a time period that felt like forever. It turned out that evolving things in a protocol can be quite tricky when it comes to compatibility. You may notice that the merged patch and the RIP will differ slightly in the approach recommended. For now, the approach we are using bases itself on the extensible payloads in the identity document.

After all that, we are proud to announce that you can now start using canonical references, so let’s go through how to use them.

A Quick Guide to Canonical References

As alluded to above, we need three values to specify a rule for a canonical reference. We need a reference the rule applies to, a set of allowed DIDs (currently always did:keys), and a threshold. These three values form a canonical reference rule. In fact, we can generalise the reference value to use a Git reference pattern⁵.

So, where do these new rules go? The new update of Radicle will now interpret a new payload identified by the key xyz.radicle.crefs. Under this key, it expects a rules key, which in turn can hold a number of canonical reference rules that are identified by their Git reference pattern.

For each rule, there are two keys: allow and threshold – these refer to the allowed set of DIDs and the threshold value respectively.

To demonstrate, the maintainers of the heartwood repository have prepared a rule already so that we can use release tags. I (@fintohaps) used the command rad id update --edit to open up my editor with the identity document pre-populated. I added the new payload with a rule for refs/tags/releases/*, the allowed set of the current 3 maintainers of the repository, and a threshold of 2. Here is what the document looked like when saving:

{
  "payload": {
    "xyz.radicle.project": {
      "defaultBranch": "master",
      "description": "Radicle Heartwood Protocol & Stack",
      "name": "heartwood"
    },
    "xyz.radicle.crefs": {
      "rules": {
        "refs/tags/releases/*": {
          "allow": [
            "did:key:z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz",
            "did:key:z6MkgFq6z5fkF2hioLLSNu1zP2qEL1aHXHZzGH1FLFGAnBGz",
            "did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM"
          ],
          "threshold": 2
        }
      }
    }
  },
  "delegates": [
    "did:key:z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT",
    "did:key:z6MktaNvN1KVFMkSRAiN4qK5yvX1zuEEaseeX5sffhzPZRZW",
    "did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM",
    "did:key:z6MkgFq6z5fkF2hioLLSNu1zP2qEL1aHXHZzGH1FLFGAnBGz",
    "did:key:z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz"
  ],
  "threshold": 1
}

And here’s the output of the rad id command:

╭───────────────────────────────────────────────────────────────────────────────────╮
│ Title    Add Release Tag Canonical Refs Rule                                      │
│ Revision 45e43cc54284f579deb7ae64e4d162274c04fa3b                                 │
│ Blob     def6df9a842e68525b371462200b6d5454269601                                 │
│ Author   did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM                 │
│ State    active                                                                   │
│ Quorum   no                                                                       │
│                                                                                   │
│ Adding the `xyz.radicle.crefs` payload that starts with a single for the          │
│ `refs/tags/releases/*` rules that uses Erik, Fintan, and Lorenz as the allowed    │
│ delegates, and a threshold of 2.                                                  │
├───────────────────────────────────────────────────────────────────────────────────┤
│ ✓ did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM fintohaps        (you) │
│ ? did:key:z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT cloudhead              │
│ ? did:key:z6MktaNvN1KVFMkSRAiN4qK5yvX1zuEEaseeX5sffhzPZRZW cloudhead-laptop       │
│ ? did:key:z6MkgFq6z5fkF2hioLLSNu1zP2qEL1aHXHZzGH1FLFGAnBGz erikli                 │
│ ✓ did:key:z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz lorenz                 │
╰───────────────────────────────────────────────────────────────────────────────────╯

@@ -1,17 +1,29 @@
 {
   "payload": {
+    "xyz.radicle.crefs": {
+      "rules": {
+        "refs/tags/releases/*": {
+          "allow": [
+            "did:key:z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz",
+            "did:key:z6MkgFq6z5fkF2hioLLSNu1zP2qEL1aHXHZzGH1FLFGAnBGz",
+            "did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM"
+          ],
+          "threshold": 2
+        }
+      }
+    },
     "xyz.radicle.project": {
       "defaultBranch": "master",
       "description": "Radicle Heartwood Protocol & Stack",
       "name": "heartwood"
     }
   },
   "delegates": [
     "did:key:z6MksFqXN3Yhqk8pTJdUGLwATkRfQvwZXPqR2qMEhbS9wzpT",
     "did:key:z6MktaNvN1KVFMkSRAiN4qK5yvX1zuEEaseeX5sffhzPZRZW",
     "did:key:z6MkireRatUThvd3qzfKht1S44wpm4FEWSSa4PRMTSQZ3voM",
     "did:key:z6MkgFq6z5fkF2hioLLSNu1zP2qEL1aHXHZzGH1FLFGAnBGz",
     "did:key:z6MkkPvBfjP4bQmco5Dm7UGsX2ruDBieEHi8n9DVJWX5sTEz"
   ],
   "threshold": 1
 }

You can look at the evolution of the proposal by using rad id show 45e43cc54284f579deb7ae64e4d162274c04fa3b in the heartwood repository.

Note that since the rules are being added to the identity document, it is necessary to reach a majority quorum for proposing these changes.

Once the rules are in place, any time you push or fetch a reference that matches a rule, it will compute the canonical form for that reference. In the case above, if any of the 2 allowed did:keys agree on a tag that lives under their namespace as refs/tags/releases, then a corresponding refs/tags will be computed.

This means that there will be more top-level references appearing in Radicle repositories, and importantly, tooling and processes that require Git tags can now take advantage of Radicle repositories!

Now, it’s time to go try out this feature and report back to us! Release tags to your hearts content ❤️🌱