-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathannotated-c4.html
384 lines (383 loc) · 42.4 KB
/
annotated-c4.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-beta.2/css/bootstrap.min.css" integrity="sha384-PsH8R72JQ3SOdhVi3uxftmaW6Vc51MKb0q5P2rRUpPvrszuE4W1povHYgTpBfshb" crossorigin="anonymous">
<link rel="stylesheet" href="c4.css">
</head>
<body>
<div class="row">
<div class="col-2"></div>
<div class="col-8">
<h3 id="toc11"><span><a href="#The-ZeroMQ-Process-C">The Annoted C4</a></span></h3>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-141">prev</a> <a href="#header-143">next</a></td>
</tr>
</table>
<p>The C4 contribution protocol used by Shitfork was originally created by Pieter Hintjens for the ZeroMQ project. When we say ZeroMQ we sometimes mean <tt>libzmq</tt>, the core library. In early 2012, we synthesized the <tt>libzmq</tt> process into a formal protocol for collaboration that we called the <a href="http://rfc.zeromq.org/spec:22">Collective Code Construction Contract</a>, or C4. You can see this as a layer above the GPL. These are our rules, and I'll explain the reasoning behind each one.</p>
<p>C4 is an evolution of the GitHub <a href="http://help.github.com/send-pull-requests/">Fork + Pull Model</a>. You may get the feeling I'm a fan of git and GitHub. This would be accurate: these two tools have made such a positive impact on our work over the last years, especially when it comes to building community.</p>
<p><a name="Language"></a><a name="header-143"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc12"><span><a href="#Language">Language</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-142">prev</a> <a href="#header-144">next</a></td>
</tr>
</table>
<blockquote class="blockquote">
<p>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.</p>
</blockquote>
<p>By starting with the RFC 2119 language, the C4 text makes very clear its intention to act as a protocol rather than a randomly written set of recommendations. A protocol is a contract between parties that defines the rights and obligations of each party. These can be peers in a network or they can be strangers working in the same project.</p>
<p>I think C4 is the first time anyone has attempted to codify a community's rulebook as a formal and reusable protocol spec. Previously, our rules were spread out over several wiki pages, and were quite specific to <tt>libzmq</tt> in many ways. But experience teaches us that the more formal, accurate, and reusable the rules, the easier it is for strangers to collaborate up-front. And less friction means a more scalable community. At the time of C4, we also had some disagreement in the <tt>libzmq</tt> project over precisely what process we were using. Not everyone felt bound by the same rules. Let's just say some people felt they had a special status, which created friction with the rest of the community. So codification made things clear.</p>
<p>It's easy to use C4: just host your project on GitHub, get one other person to join, and open the floor to pull requests. In your README, put a link to C4 and that's it. We've done this in quite a few projects and it does seem to work. I've been pleasantly surprised a few times just applying these rules to my own work, like CZMQ. None of us are so amazing that we can work without others.</p>
<p><a name="Goals"></a><a name="header-144"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc13"><span><a href="#Goals">Goals</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-143">prev</a> <a href="#header-145">next</a></td>
</tr>
</table>
<blockquote class="blockquote">
<p>C4 is meant to provide a reusable optimal collaboration model for open source software projects.</p>
</blockquote>
<p>The short term reason for writing C4 was to end arguments over the <tt>libzmq</tt> contribution process. The dissenters went off elsewhere. <a href="https://github.com/zeromq/libzmq/graphs/contributors">The ZeroMQ community blossomed</a> smoothly and easily, as I'd predicted. Most people were surprised, but gratified. There's been no real criticisms of C4 except its branching policy, which I'll come to later as it deserves its own discussion.</p>
<p>There's a reason I'm reviewing history here: as founder of a community, you are asking people to invest in your property, trademark, and branding. In return, and this is what we do with ZeroMQ, you can use that branding to set a bar for quality. When you download a product labeled "ZeroMQ", you know that it's been produced to certain standards. It's a basic rule of quality: write down your process; otherwise you cannot improve it. Our processes aren't perfect, nor can they ever be. But any flaw in them can be fixed, and tested.</p>
<p>Making C4 reusable is therefore really important. To learn more about the best possible process, we need to get results from the widest range of projects.</p>
<blockquote class="blockquote">
<p>It has these specific goals:<br />
To maximize the scale of the community around a project, by reducing the friction for new Contributors and creating a scaled participation model with strong positive feedbacks;</p>
</blockquote>
<p>The number one goal is size and health of the community—not technical quality, not profits, not performance, not market share. The goal is simply the number of people who contribute to the project. The science here is simple: the larger the community, the more accurate the results.</p>
<blockquote class="blockquote">
<p>To relieve dependencies on key individuals by separating different skill sets so that there is a larger pool of competence in any required domain;</p>
</blockquote>
<p>Perhaps the worst problem we faced in <tt>libzmq</tt> was dependence on people who could understand the code, manage GitHub branches, and make clean releases—all at the same time. It's like looking for athletes who can run marathons and sprint, swim, and also lift weights. We humans are really good at specialization. Asking us to be really good at two contradictory things reduces the number of candidates sharply, which is a Bad Thing for any project. We had this problem severely in <tt>libzmq</tt> in 2009 or so, and fixed it by splitting the role of maintainer into two: one person makes patches and another makes releases.</p>
<blockquote class="blockquote">
<p>To allow the project to develop faster and more accurately, by increasing the diversity of the decision making process;</p>
</blockquote>
<p>This is theory—not fully proven, but not falsified. The diversity of the community and the number of people who can weigh in on discussions, without fear of being criticized or dismissed, the faster and more accurately the software develops. Speed is quite subjective here. Going very fast in the wrong direction is not just useless, it's actively damaging (and we suffered a lot of that in <tt>libzmq</tt> before we switched to C4).</p>
<blockquote class="blockquote">
<p>To support the natural life cycle of project versions from experimental through to stable, by allowing safe experimentation, rapid failure, and isolation of stable code;</p>
</blockquote>
<p>To be honest, this goal seems to be fading into irrelevance. It's quite an interesting effect of the process: <em>the git master is almost always perfectly stable</em>. This has to do with the size of changes and their <em>latency</em>, i.e., the time between someone writing the code and someone actually using it fully. However, people still expect "stable" releases, so we'll keep this goal there for a while.</p>
<blockquote class="blockquote">
<p>To reduce the internal complexity of project repositories, thus making it easier for Contributors to participate and reducing the scope for error;</p>
</blockquote>
<p>Curious observation: people who thrive in complex situations like to create complexity because it keeps their value high. It's the Cobra Effect (Google it). Git made branches easy and left us with the all too common syndrome of "git is easy once you understand that a git branch is just a folded five-dimensional lepton space that has a detached history with no intervening cache". Developers should not be made to feel stupid by their tools. I've seen too many top-class developers confused by repository structures to accept conventional wisdom on git branches. We'll come back to dispose of git branches shortly, dear reader.</p>
<blockquote class="blockquote">
<p>To enforce collective ownership of the project, which increases economic incentive to Contributors and reduces the risk of hijack by hostile entities.</p>
</blockquote>
<p>Ultimately, we're economic creatures, and the sense that "we own this, and our work can never be used against us" makes it much easier for people to invest in an open source project like ZeroMQ. And it can't be just a feeling, it has to be real. There are a number of aspects to making collective ownership work, we'll see these one-by-one as we go through C4.</p>
<p><a name="Preliminaries"></a><a name="header-145"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc14"><span><a href="#Preliminaries">Preliminaries</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-144">prev</a> <a href="#header-146">next</a></td>
</tr>
</table>
<blockquote class="blockquote">
<p>The project SHALL use the git distributed revision control system.</p>
</blockquote>
<p>Git has its faults. Its command-line API is horribly inconsistent, and it has a complex, messy internal model that it shoves in your face at the slightest provocation. But despite doing its best to make its users feel stupid, git does its job really, really well. More pragmatically, I've found that if you stay away from certain areas (branches!), people learn git rapidly and don't make many mistakes. That works for me.</p>
<blockquote class="blockquote">
<p>The project SHALL be hosted on github.com or equivalent, herein called the "Platform".</p>
</blockquote>
<p>I'm sure one day some large firm will buy GitHub and break it, and another platform will rise in its place. Until then, Github serves up a near-perfect set of minimal, fast, simple tools. I've thrown hundreds of people at it, and they all stick like flies stuck in a dish of honey.</p>
<blockquote class="blockquote">
<p>The project SHALL use the Platform issue tracker.</p>
</blockquote>
<p>We made the mistake in <tt>libzmq</tt> of switching to Jira because we hadn't learned yet how to properly use the GitHub issue tracker. Jira is a great example of how to turn something useful into a complex mess because the business depends on selling more "features". But even without criticizing Jira, keeping the issue tracker on the same platform means one less UI to learn, one less login, and smooth integration between issues and patches.</p>
<blockquote class="blockquote">
<p>The project SHOULD have clearly documented guidelines for code style.</p>
</blockquote>
<p>This is a protocol plug-in: insert code style guidelines here. If you don't document the code style you use, you have no basis except prejudice to reject patches.</p>
<blockquote class="blockquote">
<p>A "Contributor" is a person who wishes to provide a patch, being a set of commits that solve some clearly identified problem.<br />
A "Maintainer" is a person who merge patches to the project. Maintainers are not developers; their job is to enforce process.</p>
</blockquote>
<p>Now we move on to definitions of the parties, and the splitting of roles that saved us from the sin of structural dependency on rare individuals. This worked well in <tt>libzmq</tt>, but as you will see it depends on the rest of the process. C4 isn't a buffet; you will need the whole process (or something very like it), or it won't hold together.</p>
<blockquote class="blockquote">
<p>Contributors SHALL NOT have commit access to the repository unless they are also Maintainers.<br />
Maintainers SHALL have commit access to the repository.</p>
</blockquote>
<p>What we wanted to avoid was people pushing their changes directly to master. This was the biggest source of trouble in <tt>libzmq</tt> historically: large masses of raw code that took months or years to fully stabilize. We eventually followed other ZeroMQ projects like PyZMQ in using pull requests. We went further, and stipulated that <em>all</em> changes had to follow the same path. No exceptions for "special people".</p>
<blockquote class="blockquote">
<p>Everyone, without distinction or discrimination, SHALL have an equal right to become a Contributor under the terms of this contract.</p>
</blockquote>
<p>We had to state this explicitly. It used to be that the <tt>libzmq</tt> maintainers would reject patches simply because they didn't like them. Now, that may sound reasonable to the author of a library (though <tt>libzmq</tt> was not written by any one person), but let's remember our goal of creating a work that is owned by as many people as possible. Saying "I don't like your patch so I'm going to reject it" is equivalent to saying, "I claim to own this and I think I'm better than you, and I don't trust you". Those are toxic messages to give to others who are thinking of becoming your co-investors.</p>
<p>I think this fight between individual expertise and collective intelligence plays out in other areas. It defined Wikipedia, and still does, a decade after that work surpassed anything built by small groups of experts. For me, we make software by slowly synthesizing the most accurate knowledge, much as we make Wikipedia articles.</p>
<p><a name="Licensing-and-Ownership"></a><a name="header-146"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc15"><span><a href="#Licensing-and-Ownership">Licensing and Ownership</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-145">prev</a> <a href="#header-147">next</a></td>
</tr>
</table>
<blockquote class="blockquote">
<p>The project SHALL use the GPLv3 or a variant thereof (LGPL, AGPL).</p>
</blockquote>
<p>I've already explained how full remixability creates better scale and why the GPL and its variants seems the optimal contract for remixable software. If you're a large business aiming to dump code on the market, you won't want C4, but then you won't really care about community either.</p>
<blockquote class="blockquote">
<p>All contributions to the project source code ("patches") SHALL use the same license as the project.</p>
</blockquote>
<p>This removes the need for any specific license or contribution agreement for patches. You fork the GPL code, you publish your remixed version on GitHub, and you or anyone else can then submit that as a patch to the original code. BSD doesn't allow this. Any work that contains BSD code may also contain unlicensed proprietary code so you need explicit action from the author of the code before you can remix it.</p>
<blockquote class="blockquote">
<p>All patches are owned by their authors. There SHALL NOT be any copyright assignment process.</p>
</blockquote>
<p>Here we come to the key reason people trust their investments in ZeroMQ: it's logistically impossible to buy the copyrights to create a closed source competitor to ZeroMQ. iMatix can't do this either. And the more people that send patches, the harder it becomes. ZeroMQ isn't just free and open today—this specific rule means it will remain so forever. Note that it's not the case in all GPL projects, many of which still ask for copyright transfer back to the maintainers.</p>
<blockquote class="blockquote">
<p>The project SHALL be owned collectively by all its Contributors.</p>
</blockquote>
<p>This is perhaps redundant, but worth saying: if everyone owns their patches, then the resulting whole is also owned by every contributor. There's no legal concept of owning lines of code: the "work" is at least a source file.</p>
<blockquote class="blockquote">
<p>Each Contributor SHALL be responsible for identifying themselves in the project Contributor list.</p>
</blockquote>
<p>In other words, the maintainers are not karma accountants. Anyone who wants credit has to claim it themselves.</p>
<p><a name="Patch-Requirements"></a><a name="header-147"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc16"><span><a href="#Patch-Requirements">Patch Requirements</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-146">prev</a> <a href="#header-148">next</a></td>
</tr>
</table>
<p>In this section, we define the obligations of the contributor: specifically, what constitutes a "valid" patch, so that maintainers have rules they can use to accept or reject patches.</p>
<blockquote class="blockquote">
<p>Maintainers and Contributors MUST have a Platform account and SHOULD use their real names or a well-known alias.</p>
</blockquote>
<p>In the worst case scenario, where someone has submitted toxic code (patented, or owned by someone else), we need to be able to trace who and when, so we can remove the code. Asking for real names or a well-known alias is a theoretical strategy for reducing the risk of bogus patches. We don't know if this actually works because we haven't had the problem yet.</p>
<blockquote class="blockquote">
<p>A patch SHOULD be a minimal and accurate answer to exactly one identified and agreed problem.</p>
</blockquote>
<p>This implements the Simplicity Oriented Design process that I'll come to later in this chapter. One clear problem, one minimal solution, apply, test, repeat.</p>
<blockquote class="blockquote">
<p>A patch MUST adhere to the code style guidelines of the project if these are defined.</p>
</blockquote>
<p>This is just sanity. I've spent time cleaning up other peoples' patches because they insisted on putting the <tt>else</tt> beside the <tt>if</tt> instead of just below as Nature intended. Consistent code is healthier.</p>
<blockquote class="blockquote">
<p>A patch MUST adhere to the "Evolution of Public Contracts" guidelines defined below.</p>
</blockquote>
<p>Ah, the pain, the pain. I'm not speaking of the time at age eight when I stepped on a plank with a 4-inch nail protruding from it. That was relatively OK. I'm speaking of 2010-2011 when we had multiple parallel releases of ZeroMQ, each with different <em>incompatible</em> APIs or wire protocols. It was an exercise in bad rules, pointlessly enforced, that still hurts us today. The rule was, "If you change the API or protocol, you SHALL create a new major version". Give me the nail through the foot; that hurt less.</p>
<p>One of the big changes we made with C4 was simply to ban, outright, this kind of sanctioned sabotage. Amazingly, it's not even hard. We just don't allow the breaking of existing public contracts, period, unless everyone agrees, in which case no period. As Linus Torvalds famously put it on 23 December 2012, "WE DO NOT BREAK USERSPACE!"</p>
<blockquote class="blockquote">
<p>A patch SHALL NOT include nontrivial code from other projects unless the Contributor is the original author of that code.</p>
</blockquote>
<p>This rule has two effects. The first is that it forces people to make minimal solutions because they cannot simply import swathes of existing code. In the cases where I've seen this happen to projects, it's always bad unless the imported code is very cleanly separated. The second is that it avoids license arguments. You write the patch, you are allowed to publish it as LGPL, and we can merge it back in. But you find a 200-line code fragment on the web, and try to paste that, we'll refuse.</p>
<blockquote class="blockquote">
<p>A patch MUST compile cleanly and pass project self-tests on at least the principle target platform.</p>
</blockquote>
<p>For cross-platform projects, it is fair to ask that the patch works on the development box used by the contributor.</p>
<blockquote class="blockquote">
<p>A patch commit message SHOULD consist of a single short (less than 50 character) line summarizing the change, optionally followed by a blank line and then a more thorough description.</p>
</blockquote>
<p>This is a good format for commit messages that fits into email (the first line becomes the subject, and the rest becomes the email body).</p>
<blockquote class="blockquote">
<p>A "Correct Patch" is one that satisfies the above requirements.</p>
</blockquote>
<p>Just in case it wasn't clear, we're back to legalese and definitions.</p>
<p><a name="Development-Process"></a><a name="header-148"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc17"><span><a href="#Development-Process">Development Process</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-147">prev</a> <a href="#header-149">next</a></td>
</tr>
</table>
<p>In this section, we aim to describe the actual development process, step-by-step.</p>
<blockquote class="blockquote">
<p>Change on the project SHALL be governed by the pattern of accurately identifying problems and applying minimal, accurate solutions to these problems.</p>
</blockquote>
<p>This is a unapologetic ramming through of thirty years' software design experience. It's a profoundly simple approach to design: make minimal, accurate solutions to real problems, nothing more or less. In ZeroMQ, we don't have feature requests. Treating new features the same as bugs confuses some newcomers. But this process works, and not just in open source. Enunciating the problem we're trying to solve, with every single change, is key to deciding whether the change is worth making or not.</p>
<blockquote class="blockquote">
<p>To initiate changes, a user SHALL log an issue on the project Platform issue tracker.</p>
</blockquote>
<p>This is meant to stop us from going offline and working in a ghetto, either by ourselves or with others. Although we tend to accept pull requests that have clear argumentation, this rule lets us say "stop" to confused or too-large patches.</p>
<blockquote class="blockquote">
<p>The user SHOULD write the issue by describing the problem they face or observe.</p>
</blockquote>
<p>"Problem: we need feature X. Solution: make it" is not a good issue. "Problem: user cannot do common tasks A or B except by using a complex workaround. Solution: make feature X" is a decent explanation. Because everyone I've ever worked with has needed to learn this, it seems worth restating: document the real problem first, solution second.</p>
<blockquote class="blockquote">
<p>The user SHOULD seek consensus on the accuracy of their observation, and the value of solving the problem.</p>
</blockquote>
<p>And because many apparent problems are illusionary, by stating the problem explicitly we give others a chance to correct our logic. "You're only using A and B a lot because function C is unreliable. Solution: make function C work properly."</p>
<blockquote class="blockquote">
<p>Users SHALL NOT log feature requests, ideas, suggestions, or any solutions to problems that are not explicitly documented and provable.</p>
</blockquote>
<p>There are several reasons for not logging ideas, suggestions, or feature requests. In our experience, these just accumulate in the issue tracker until someone deletes them. But more profoundly, when we treat all change as problem solutions, we can prioritize trivially. Either the problem is real and someone wants to solve it now, or it's not on the table. Thus, wish lists are off the table.</p>
<blockquote class="blockquote">
<p>Thus, the release history of the project SHALL be a list of meaningful issues logged and solved.</p>
</blockquote>
<p>I'd love the GitHub issue tracker to simply list all the issues we solved in each release. Today we still have to write that by hand. If one puts the issue number in each commit, and if one uses the GitHub issue tracker, which we sadly don't yet do for ZeroMQ, this release history is easier to produce mechanically.</p>
<blockquote class="blockquote">
<p>To work on an issue, a Contributor SHALL fork the project repository and then work on their forked repository.</p>
</blockquote>
<p>Here we explain the GitHub fork + pull request model so that newcomers only have to learn one process (C4) in order to contribute.</p>
<blockquote class="blockquote">
<p>To submit a patch, a Contributor SHALL create a Platform pull request back to the project.</p>
</blockquote>
<p>GitHub has made this so simple that we don't need to learn git commands to do it, for which I'm deeply grateful. Sometimes, I'll tell people who I don't particularly like that command-line git is awesome and all they need to do is learn git's internal model in detail before trying to use it on real work. When I see them several months later they look… changed.</p>
<blockquote class="blockquote">
<p>A Contributor SHALL NOT commit changes directly to the project.</p>
</blockquote>
<p>Anyone who submits a patch is a contributor, and all contributors follow the same rules. No special privileges to the original authors, because otherwise we're not building a community, only boosting our egos.</p>
<blockquote class="blockquote">
<p>To discuss a patch, people MAY comment on the Platform pull request, on the commit, or elsewhere.</p>
</blockquote>
<p>Randomly distributed discussions may be confusing if you're walking up for the first time, but GitHub solves this for all current participants by sending emails to those who need to follow what's going on. We had the same experience and the same solution in Wikidot, and it works. There's no evidence that discussing in different places has any negative effect.</p>
<blockquote class="blockquote">
<p>To accept or reject a patch, a Maintainer SHALL use the Platform interface.</p>
</blockquote>
<p>Working via the GitHub web user interface means pull requests are logged as issues, with workflow and discussion. I'm sure there are more complex ways to work. Complexity is easy; it's simplicity that's incredibly hard.</p>
<blockquote class="blockquote">
<p>Maintainers SHALL NOT accept their own patches.</p>
</blockquote>
<p>There was a rule we defined in the FFII years ago to stop people burning out: no less than two people on any project. One-person projects tend to end in tears, or at least bitter silence. We have quite a lot of data on burnout, why it happens, and how to prevent it (even cure it). I'll explore this later in the chapter, because if you work with or on open source you need to be aware of the risks. The "no merging your own patch" rule has two goals. First, if you want your project to be C4-certified, you have to get at least one other person to help. If no one wants to help you, perhaps you need to rethink your project. Second, having a control for every patch makes it much more satisfying, keeps us more focused, and stops us breaking the rules because we're in a hurry, or just feeling lazy.</p>
<blockquote class="blockquote">
<p>Maintainers SHALL NOT make value judgments on correct patches.</p>
</blockquote>
<p>We already said this but it's worth repeating: the role of Maintainer is not to judge a patch's substance, only its technical quality. The substantive worth of a patch only emerges over time: people use it, and like it, or they do not. And if no one is using a patch, eventually it'll annoy someone else who will remove it, and no one will complain.</p>
<blockquote class="blockquote">
<p>Maintainers SHALL merge correct patches rapidly.</p>
</blockquote>
<p>There is a criteria I call <em>change latency</em>, which is the round-trip time from identifying a problem to testing a solution. The faster the better. If maintainers cannot respond to pull requests as rapidly as people expect, they're not doing their job (or they need more hands).</p>
<blockquote class="blockquote">
<p>The Contributor MAY tag an issue as "Ready" after making a pull request for the issue.</p>
</blockquote>
<p>By default, GitHub offers the usual variety of issues, but with C4 we don't use them. Instead, we need just two labels, "Urgent" and "Ready". A contributor who wants another user to test an issue can then label it as "Ready".</p>
<blockquote class="blockquote">
<p>The user who created an issue SHOULD close the issue after checking the patch is successful.</p>
</blockquote>
<p>When one person opens an issue, and another works on it, it's best to allow the original person to close the issue. That acts as a double-check that the issue was properly resolved.</p>
<blockquote class="blockquote">
<p>Maintainers SHOULD ask for improvements to incorrect patches and SHOULD reject incorrect patches if the Contributor does not respond constructively.</p>
</blockquote>
<p>Initially, I felt it was worth merging all patches, no matter how poor. There's an element of trolling involved. Accepting even obviously bogus patches could, I felt, pull in more contributors. But people were uncomfortable with this so we defined the "correct patch" rules, and the Maintainer's role in checking for quality. On the negative side, I think we didn't take some interesting risks, which could have paid off with more participants. On the positive side, this has led to <tt>libzmq</tt> master (and in all projects that use C4) being practically production quality, practically all the time.</p>
<blockquote class="blockquote">
<p>Any Contributor who has value judgments on a correct patch SHOULD express these via their own patches.</p>
</blockquote>
<p>In essence, the goal here is to allow users to try patches rather than to spend time arguing pros and cons. As easy as it is to make a patch, it's as easy to revert it with another patch. You might think this would lead to "patch wars", but that hasn't happened. We've had a handful of cases in <tt>libzmq</tt> where patches by one contributor were killed by another person who felt the experimentation wasn't going in the right direction. It is easier than seeking up-front consensus.</p>
<blockquote class="blockquote">
<p>Maintainers MAY commit changes to non-source documentation directly to the project.</p>
</blockquote>
<p>This exit allows maintainers who are making release notes to push those without having to create an issue which would then affect the release notes, leading to stress on the space time fabric and possibly involuntary rerouting backwards in the fourth dimension to before the invention of cold beer. Shudder. It is simpler to agree that release notes aren't technically software.</p>
<p><a name="Creating-Stable-Releases"></a><a name="header-149"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc18"><span><a href="#Creating-Stable-Releases">Creating Stable Releases</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-148">prev</a> <a href="#header-150">next</a></td>
</tr>
</table>
<p>We want some guarantee of stability for a production system. In the past, this meant taking unstable code and then over months hammering out the bugs and faults until it was safe to trust. iMatix's job, for years, has been to do this to <tt>libzmq</tt>, turning raw code into packages by allowing only bug fixes and no new code into a "stabilization branch". It's surprisingly not as thankless as it sounds.</p>
<p>Now, since we went full speed with C4, we've found that git master of <tt>libzmq</tt> is mostly perfect, most of the time. This frees our time to do more interesting things, such as building new open source layers on top of <tt>libzmq</tt>. However, people still want that guarantee: many users will simply not install except from an "official" release. So a stable release today means two things. First, a snapshot of the master taken at a time when there were no new changes for a while, and no dramatic open bugs. Second, a way to fine tune that snapshot to fix the critical issues remaining in it.</p>
<p>This is the process we explain in this section.</p>
<blockquote class="blockquote">
<p>The project SHALL have one branch ("master") that always holds the latest in-progress version and SHOULD always build.</p>
</blockquote>
<p>This is redundant because every patch always builds but it's worth restating. If the master doesn't build (and pass its tests), someone needs waking up.</p>
<blockquote class="blockquote">
<p>The project SHALL NOT use topic branches for any reason. Personal forks MAY use topic branches.</p>
</blockquote>
<p>I'll come to branches soon. In short (or "tl;dr", as they say on the webs), branches make the repository too complex and fragile, and require up-front agreement, all of which are expensive and avoidable.</p>
<blockquote class="blockquote">
<p>To make a stable release someone SHALL fork the repository by copying it and thus become maintainer of this repository.<br />
Forking a project for stabilization MAY be done unilaterally and without agreement of project maintainers.</p>
</blockquote>
<p>It's free software. No one has a monopoly on it. If you think the maintainers aren't producing stable releases right, fork the repository and do it yourself. Forking isn't a failure, it's an essential tool for competition. You can't do this with branches, which means a branch-based release policy gives the project maintainers a monopoly. And that's bad because they'll become lazier and more arrogant than if real competition is chasing their heels.</p>
<blockquote class="blockquote">
<p>A stabilization project SHOULD be maintained by the same process as the main project.</p>
</blockquote>
<p>Stabilization projects have maintainers and contributors like any project. In practice we usually cherry pick patches from the main project to the stabilization project, but that's just a convenience.</p>
<blockquote class="blockquote">
<p>A patch to a repository declared "stable" SHALL be accompanied by a reproducible test case.</p>
</blockquote>
<p>Beware of a one-size-fits-all process. New code does not need the same paranoia as code that people are trusting for production use. In the normal development process, we did not mention test cases. There's a reason for this. While I love testable patches, many changes aren't easily or at all testable. However, to stabilize a code base you want to fix only serious bugs, and you want to be 100% sure every change is accurate. This means before and after tests for every change.</p>
<p><a name="Evolution-of-Public-Contracts"></a><a name="header-150"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc19"><span><a href="#Evolution-of-Public-Contracts">Evolution of Public Contracts</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-149">prev</a> <a href="#header-151">next</a></td>
</tr>
</table>
<p>By "public contracts", I mean APIs and protocols. Up until the end of 2011, <tt>libzmq</tt>'s naturally happy state was marred by broken promises and broken contracts. We stopped making promises (aka "road maps") for <tt>libzmq</tt> completely, and our dominant theory of change is now that it emerges carefully and accurately over time. At a 2012 Chicago meetup, Garrett Smith and Chuck Remes called this the "drunken stumble to greatness", which is how I think of it now.</p>
<p>We stopped breaking public contracts simply by banning the practice. Before then it had been "OK" (as in we did it and everyone complained bitterly, and we ignored them) to break the API or protocol so long as we changed the major version number. Sounds fine, until you get ZeroMQ v2.0, v3.0, and v4.0 all in development at the same time, and not speaking to each other.</p>
<blockquote class="blockquote">
<p>All Public Contracts (APIs or protocols) SHOULD be documented.</p>
</blockquote>
<p>You'd think this was a given for professional software engineers but no, it's not. So, it's a rule. You want C4 certification for your project, you make sure your public contracts are documented. No "It's specified in the code" excuses. Code is not a contract. (Yes, I intend at some point to create a C4 certification process to act as a quality indicator for open source projects.)</p>
<blockquote class="blockquote">
<p>All Public Contracts SHALL use Semantic Versioning.</p>
</blockquote>
<p>This rule is mainly here because people asked for it. I've no real love for it, as Semantic Versioning is what led to the so-called "Why does ZeroMQ not speak to itself?!" debacle. I've never seen the problem that this solved. Something about runtime validation of library versions, or some-such.</p>
<blockquote class="blockquote">
<p>All Public Contracts SHOULD have space for extensibility and experimentation.</p>
</blockquote>
<p>Now, the real thing is that public contracts <em>do change</em>. It's not about not changing them. It's about changing them safely. This means educating (especially protocol) designers to create that space up-front.</p>
<blockquote class="blockquote">
<p>A patch that modifies a stable Public Contract SHOULD not break existing applications unless there is overriding consensus on the value of doing this.</p>
</blockquote>
<p>Sometimes the patch is fixing a bad API that no one is using. It's a freedom we need, but it should be based on consensus, not one person's dogma. However, making random changes "just because" is not good. In ZeroMQ v3.x, did we benefit from renaming <tt>ZMQ_NOBLOCK</tt> to <tt>ZMQ_DONTWAIT</tt>? Sure, it's closer to the POSIX socket <tt>recv()</tt> call, but is that worth breaking thousands of applications? No one ever reported it as an issue. To misquote Stallman: "your freedom to create an ideal world stops one inch from my application."</p>
<blockquote class="blockquote">
<p>A patch that introduces new features to a Public Contract SHOULD do so using new names.</p>
</blockquote>
<p>We had the experience in ZeroMQ once or twice of new features using old names (or worse, using names that were <em>still in use</em> elsewhere). ZeroMQ v3.0 had a newly introduced "ROUTER" socket that was totally different from the existing ROUTER socket in 2.x. Dear lord, you should be face-palming, why? The reason: apparently, even smart people sometimes need regulation to stop them doing silly things.</p>
<blockquote class="blockquote">
<p>Old names SHOULD be deprecated in a systematic fashion by marking new names as "experimental" until they are stable, then marking the old names as "deprecated".</p>
</blockquote>
<p>This life cycle notation has the great benefit of actually telling users what is going on with a consistent direction. "Experimental" means "we have introduced this and intend to make it stable if it works". It does not mean, "we have introduced this and will remove it at any time if we feel like it". One assumes that code that survives more than one patch cycle is meant to be there. "Deprecated" means "we have replaced this and intend to remove it".</p>
<blockquote class="blockquote">
<p>When sufficient time has passed, old deprecated names SHOULD be marked "legacy" and eventually removed.</p>
</blockquote>
<p>In theory this gives applications time to move onto stable new contracts without risk. You can upgrade first, make sure things work, and then, over time, fix things up to remove dependencies on deprecated and legacy APIs and protocols.</p>
<blockquote class="blockquote">
<p>Old names SHALL NOT be reused by new features.</p>
</blockquote>
<p>Ah, yes, the joy when ZeroMQ v3.x renamed the top-used API functions (<tt><a href="http://api.zeromq.org/3-2:zmq_send">zmq_send()</a></tt> and <tt><a href="http://api.zeromq.org/3-2:zmq_recv">zmq_recv()</a></tt>) and then recycled the old names for new methods that were utterly incompatible (and which I suspect few people actually use). You should be slapping yourself in confusion again, but really, this is what happened and I was as guilty as anyone. After all, we did change the version number! The only benefit of that experience was to get this rule.</p>
<blockquote class="blockquote">
<p>When old names are removed, their implementations MUST provoke an exception (assertion) if used by applications.</p>
</blockquote>
<p>I've not tested this rule to be certain it makes sense. Perhaps what it means is "if you can't provoke a compile error because the API is dynamic, provoke an assertion".</p>
<p><a name="Project-Administration"></a><a name="header-151"></a></p>
<table style="width:100%">
<tr>
<td>
<h4 id="toc20"><span><a href="#Project-Administration">Project Administration</a></span></h4>
</td>
<td style="text-align:right; font-size:80%;"><a href="#top">top</a> <a href="#header-150">prev</a> <a href="#header-152">next</a></td>
</tr>
</table>
<blockquote class="blockquote">
<p>The project founders SHALL act as Administrators to manage the set of project Maintainers.</p>
</blockquote>
<p>Someone needs to administer the project, and it makes sense that the original founders start this ball rolling.</p>
<blockquote class="blockquote">
<p>The Administrators SHALL ensure their own succession over time by promoting the most effective Maintainers.</p>
</blockquote>
<p>At the same time, as founder of a project you really want to get out of the way before you become over-attached to it. Promoting the most active and consistent maintainers is good for everyone.</p>
<blockquote class="blockquote">
<p>A new Contributor who makes a correct patch SHALL be invited to become a Maintainer.</p>
</blockquote>
<p>I met Felix Geisendörfer in Lyons in 2012 at the <a href="http://www.mix-it.fr">Mix-IT conference</a> where I presented Social Architecture and one thing that came out of this was Felix's now famous <a href="http://felixge.de/2013/03/11/the-pull-request-hack.html">Pull Request Hack</a>. It fits elegantly into C4 and solves the problem of maintainers dropping out over time.</p>
<blockquote class="blockquote">
<p>Administrators MAY remove Maintainers who are inactive for an extended period of time, or who repeatedly fail to apply this process accurately.</p>
</blockquote>
<p>This was Ian Barber's suggestion: we need a way to crop inactive maintainers. Originally maintainers were self-elected but that makes it hard to drop troublemakers (who are rare, but not unknown).</p>
<p>C4 is not perfect. Few things are. The process for changing it (Digistan's COSS) is a little outdated now: it relies on a single-editor workflow with the ability to fork, but not merge. This seems to work but it could be better to use C4 for protocols like C4.</p>
<p><a name="A-Real-Life-Example"></a><a name="header-152"></a></p>
<table style="width:100%">
<tr>
<td>
</div>
</div>
</body>