Subject: [Gendispatch] Grounding the exclusionary language discussion

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 10:54:14 +0200

Dear all,

draft-moore-exclusionary-language and draft-gondwana-effective-terminology both seem very concerned with the potential for disruption of IETF processes if one were to provide measures for the discussion of the issue of exclusionary language in new I-Ds and RFCs. Both drafts also seem to find it very important that if there is a (perceived) issue, this should be extensively documented with evidence in the form of citations and numbers.

However, both drafts don't provide any citations or quantification to back up their own assertions and recommendations. I think it would be very useful for a constructive discussion if the draft authors could perhaps provide a bit more grounding to their arguments.

This could for example be in scientific literature, or perhaps by showing observed disruptions in similar or adjacent communities to the IETF that have already instated such measures (such as Python, Django, Linux Project, Bluetooth SIG, Drupal, Postgres, NIST, RedHat, etc).

Best,

Niels

-- 
Niels ten Oever
Researcher and PhD Candidate - DATACTIVE Research Group - University of Amsterdam
Postdoctoral Scholar (abd) - Communications Department - Texas A&M University
Research Fellow - Centre for Internet and Human Rights - European University Viadrina
Associated Scholar - Centro de Tecnologia e Sociedade - Fundação Getúlio Vargas

W: https://nielstenoever.net
E: [email-masked]
T: @nielstenoever
P/S/WA: +31629051853
PGP: 2458 0B70 5C4A FD8A 9488 643A 0ED8 3F3A 468A C8B3

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 Aug 2020 05:10:19 -0400

> draft-moore-exclusionary-language and draft-gondwana-effective-terminology both seem very concerned with the potential for disruption of IETF processes if one were to provide measures for the discu…

To clarify somewhat, my draft says that when an aggrieved party 
perceives an issue and complains about it, that should be treated as 
sincere.

My intent is that when some third party perceives an issue and complains 
about that issue on behalf of some aggrieved party, substantiation 
should be required before such a claim is regarded as credible.

> However, both drafts don't provide any citations or quantification to back up their own assertions and recommendations. I think it would be very useful for a constructive discussion if the draft au…

My draft was written assuming an audience of experienced IETF 
participants who likely have personal experience of such disruption.   
Much like an aggrieved party who has personal experience of being 
excluded should not have that personal experience questioned, 
experienced IETF participants who have experienced how discussions can 
get out of hand, should not have to prove that their personal 
experiences are valid.

More broadly, the burden of proof should be on those demanding 
potentially disruptive changes, not on those who have long experience 
with consensus-based decision making in IETF.

The recent discussions on the ietf@ list are but one example of how 
these discussions can get out of hand.

Keith

---

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 11:21:44 +0200

Dear Keith,

On 8/27/20 11:10 AM, Keith Moore wrote:
> To clarify somewhat, my draft says that when an aggrieved party perceives an issue and complains about it, that should be treated as sincere.

I think that exclusionary language aggrieves (or should aggrieve) everyone, because it leads to less diverse communities. It has been proven time and again that less diverse communities produce lesser quality work than diverse communities. See also RFC7704 and a wide range of academic literature on this topic. 

> 
> My intent is that when some third party perceives an issue and complains about that issue on behalf of some aggrieved party, substantiation should be required before such a claim is regarded as credible.
> 

I am not sure what makes an aggrieved party. As discussed above, exclusionary language is a problem for everyone. 

> My draft was written assuming an audience of experienced IETF participants who likely have personal experience of such disruption.   Much like an aggrieved party who has personal experience of being excluded should not have that personal experience questioned, experienced IETF participants who have experienced how discussions can get out of hand, should not have to prove that their personal experiences are valid.
> 
> More broadly, the burden of proof should be on those demanding potentially disruptive changes, not on those who have long experience with consensus-based decision making in IETF.

I don't think draft-knodel-terminology asks for disruptive changes, it asks for voluntary removal of exclusionary language of new RFCs and I-Ds. I do not see how that is disruptive. If that is disruptive, we have another kind of problem. 

As mentioned, other communities have made this commitment and I have not seen any case of disruption. But feel free to point us to it. 

Best,

Niels

> 
> The recent discussions on the ietf@ list are but one example of how these discussions can get out of hand.
>

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 Aug 2020 06:11:23 -0400

On 8/27/20 5:21 AM, Niels ten Oever wrote:

> I think that exclusionary language aggrieves (or should aggrieve) everyone, because it leads to less diverse communities.

Perhaps, but who gets to decide what is exclusionary?   Surely the 
aggrieved parties are in a better position to make such claims, than 
third parties purporting to speak on their behalf without supporting 
evidence.

> I don't think draft-knodel-terminology asks for disruptive changes, it asks for voluntary removal of exclusionary language of new RFCs and I-Ds. I do not see how that is disruptive. If that is disr…

I haven't read the latest version so this comment will be based on the 
previous version.   I mostly agree that the specific changes mentioned 
in the previous draft are uncontroversial unlikely to be disruptive.

The problem as I see it is that bring up the subject invites some people 
to try to find as many words as possible to exclude from the language, 
and some of those suggestions are controversial.

> As mentioned, other communities have made this commitment and I have not seen any case of disruption. But feel free to point us to it.

Other communities have different purposes and different rules. I don't 
want to dissect each community and try to analyze how each one manages 
its internal conflicts, or whether the narrow scope of the community 
afforded them some insulation from potential controversy not afforded to 
IETF, and gauge how predictive each community's experience would be for 
IETF.  We don't need to resort to dubious analogies, we have already 
witnessed firsthand a small taste of how this specific discussion can be 
disruptive to IETF.

You claim you're asking for "voluntary" removal.  But what does 
"voluntary" mean?   In IETF some people have the power to compel authors 
to edit their language, else have their work discarded or lose influence 
over their work (depending on IPR terms chosen by the author).   So in 
the context of IETF the question of censorship inherently comes up, and 
it's certainly not a surprise that censorship is a controversial topic.

This is a Pandora's box, or a can of worms - pick whatever analogy you 
wish.   It's something that has a large potential for undesirable 
unintended consequences.

But anticipating and managing potential unintended consequences is a big 
part of what engineers do.   So please don't fault us for trying to do 
that.   Some of us are just trying to figure out how to manage the 
occasional disruptive corner cases that we believe, based on our 
experiences, are likely to crop up if we move in the direction you want 
us to move.   My personal sense is that most of what you want is 
uncontroversial and that there's a likelihood of earning consensus IFF 
measures to limit the potential for disruption can be agreed on.

Keith

---

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 12:31:27 +0200

Dear Keith,

On 8/27/20 12:11 PM, Keith Moore wrote:
> Perhaps, but who gets to decide what is exclusionary?   Surely the aggrieved parties are in a better position to make such claims, than third parties purporting to speak on their behalf without supporting evidence.
> 
> I haven't read the latest version so this comment will be based on the previous version.   I mostly agree that the specific changes mentioned in the previous draft are uncontroversial unlikely to be disruptive.
> 
> The problem as I see it is that bring up the subject invites some people to try to find as many words as possible to exclude from the language, and some of those suggestions are controversial.
> 
> Other communities have different purposes and different rules. I don't want to dissect each community and try to analyze how each one manages its internal conflicts, or whether the narrow scope of the community afforded them some insulation from potential controversy not afforded to IETF, and gauge how predictive each community's experience would be for IETF.  We don't need to resort to dubious analogies, we have already witnessed firsthand a small taste of how this specific discussion can be disruptive to IETF.

You are comparing the discussion about such measures with how they would be implemented. I do not think that's fair. The assumptions you make about the potential disruptive impact, which is the main reason for _not_ changing anything and rather erring on the side of the status quo and exclusionary language, are not very much substantiated. Neither in theory, nor in practice.

> 
> You claim you're asking for "voluntary" removal.  But what does "voluntary" mean?   In IETF some people have the power to compel authors to edit their language, else have their work discarded or lose influence over their work (depending on IPR terms chosen by the author).   So in the context of IETF the question of censorship inherently comes up, and it's certainly not a surprise that censorship is a controversial topic.

Again, I don't think there is any reason to believe this would be the case. But any evidence for this would be very much appreciated.

What we are currently doing is excluding people from the conversation - we only have to look at the composition of our community to confirm that. 

> 
> This is a Pandora's box, or a can of worms - pick whatever analogy you wish.   It's something that has a large potential for undesirable unintended consequences.
> 
> But anticipating and managing potential unintended consequences is a big part of what engineers do.   So please don't fault us for trying to do that.   

Sure. It just seems you're just optimizing a system based on assumptions rather than actual data and observations, which does not seem very scientific or efficient. 

> Some of us are just trying to figure out how to manage the occasional disruptive corner cases that we believe, based on our experiences, are likely to crop up if we move in the direction you want us to move.  

Could you please document those experiences (cases) where you (or others) have been 'censored' citing the usage of exclusionary language?

> My personal sense is that most of what you want is uncontroversial and that there's a likelihood of earning consensus IFF measures to limit the potential for disruption can be agreed on.

I do hope so.

Best,

Niels

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 Aug 2020 06:34:53 -0400

> You are comparing the discussion about such measures with how they would be implemented. I do not think that's fair. The assumptions you make about the potential disruptive impact, which is the mai…

Well, I'm not arguing for not changing anything.  I'm arguing for 
changing things but with due care.    And I trust my 30-year experience 
in IETF over the mere assertions of someone else.

Keith

---

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 12:45:01 +0200

Dear Keith,

On 8/27/20 12:34 PM, Keith Moore wrote:
> Well, I'm not arguing for not changing anything.  I'm arguing for changing things but with due care.    And I trust my 30-year experience in IETF over the mere assertions of someone else.
> 

The problem is, I am not making assertions. The whole of draft-knodel-terminology has been based on research. Whereas draft-moore-exclusionary-language and draft-gondwana-effective-terminology do not back up their claims with literature or data. That does not seem a very scientific or engineering way to go about it to me. But of course, happy to discuss. 

Best,

Niels

---

From: Eliot Lear <[email-masked]>
Date: Thu, 27 Aug 2020 13:58:04 +0200

Hi

Let me prefix my comments that I am not a linguist, an anthropologist, a sociologist, a neurologist, a philosopher (except on the net ;-), or a psychologist.  My ability to survey the literature in this space is limited.  This having been said...

> On 27 Aug 2020, at 12:45, Niels ten Oever <[email-masked]> wrote:
> 
> The problem is, I am not making assertions. The whole of draft-knodel-terminology has been based on research.

This requires some teasing out.    I enjoyed reading about Lakoff and Wyatt in your draft, and I will take as axiomatic your statements from Burgest.  

However, I take exception to how you and others have used Eglash.  What he published was an essay (an enjoyable one at that), and if one were to call it research, it would be of the most informal, unscientific kind, which he himself noted (which I thoroughly appreciate).  It is particularly important that you cite his work in this light because his name is being bantered about across the Internet, as is your draft, as if what he wrote was in some way dispository, which cannot in any way be claimed.

I am not claiming his conclusions are wrong; I am not qualified to do so; I merely claim that you and others need to place them into a proper context.

And so...

> Whereas draft-moore-exclusionary-language and draft-gondwana-effective-terminology do not back up their claims with literature or data. That does not seem a very scientific or engineering way to go about it to me. But of course, happy to discuss. 

Before you pursue that line, I think it’s important that you address the point I made.

Eliot

---

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 14:04:16 +0200

On 8/27/20 1:58 PM, Eliot Lear wrote:
>> On 27 Aug 2020, at 12:45, Niels ten Oever <[email-masked] <mailto:[email-masked]>> wrote:
>>
> However, I take exception to how you and others have used Eglash.  What he published was an *essay* (an enjoyable one at that), and if one were to call it research, it would be of the most informal, unscientific kind, which he himself noted (which I thoroughly appreciate).  It is particularly important that you cite his work in this light because his name is being bantered about across the Internet, as is your draft, as if what he wrote was in some way dispository, which cannot in any way be claimed.
> 

Thank you very much Eliot, could you also review the other drafts through the same critical lens? Much appreciated.

Best,

Niels

---

From: Eliot Lear <[email-masked]>
Date: Thu, 27 Aug 2020 14:09:40 +0200

> On 27 Aug 2020, at 14:04, Niels ten Oever <[email-masked]> wrote:
>> 

Of course.  Nobody gets a free ride ;-)

Eliot

---

From: "Bron Gondwana" <[email-masked]>
Date: Thu, 27 Aug 2020 23:24:04 +1000

On Thu, Aug 27, 2020, at 18:54, Niels ten Oever wrote:
> Both drafts also seem to find it very important that if there is a (perceived) issue, this should be extensively documented with evidence in the form of citations and numbers.

That's a very interesting reading of my draft. That was not my intent, so I'd be interested in feedback about how my language failed to communicate my intent to you.

My entire intent was to remove the need to provide citations and numbers, because neither "party" as such has been able to do so, instead stating assumptions as fact.

I shall try a couple of of citations now from this very thread:

*"It has been proven time and again that less diverse communities produce lesser quality work than diverse communities. See also RFC7704 and a wide range of academic literature on this topic"* -- <[email-masked]>

*"What we are currently doing is excluding people from the conversation - we only have to look at the composition of our community to confirm that."* -- <[email-masked]>

I believe there's a logical point you a trying to make here with these two messages, and I'll paraphrase it - please let me know if you disagree.

*"If we adopt draft-knodel-terminology that will lead to increased diversity in the IETF, and hence produce higher quality work".* -- me, paraphrasing the above

I don't believe there is disagreement on that sentiment, merely on the terminology used to state it, the effectiveness of any particular policy prescription, and whether we should be centering the mission of the IETF, or centering current political hot-button issues.

Having read Keith's draft, both he and I were careful to say that outside consensus already comes in to the IETF quite effectively and any failure there is a bug which should be fixed.  Indeed, I went further and said that we should be mindful of emerging consensus and try to avoid using new terms which are likely to cause issues in the future due to the long lifetime of our documents.

I think you and I are coming to the same place that increasing diversity is good, but I'm putting the priority on the IETF's needs, not the needs of currently non-contributers to the IETF who we believe we will be attracted if we only remove this one impediment.

Regarding the specific issue of the request to cite work supporting the underlying position - I cite the 50 year history of the IETF and the high esteem in which our work is held - high enough that we're clearly meeting our mission effectively, even if not perfectly.

...

I think we're almost entirely quibbling over semantics and approach here, but they do matter.  I'm going to quote from my slides and riff on what I'm planning to say next week:

*There are three ways to reach consensus: Persuasion, Fatigue and Fiat.  *

This topic could do a lot of damage before we reach Fatigue of all participants!

A lot of the response to the IESG statement was a fear that I would paraphrase as:* consensus will be declared by fiat, because "this topic is too important, and <position> is clearly morally right, so objection to policy provisions that claim to further its goals is inherently racist and can be dismissed". * I don't know how well founded that fear is - I hope not at all - but that fear is what I saw in the reactivity of that email thread.

So... my goal was to try to document the bit where existing consensus is, and find a way to persuade, using wording which - as John Levine said - speaks to its readers as peers.  I hope to discuss my draft next week with all comers and see how good my persuasive skills are!

Regards,

Bron.

--
  Bron Gondwana, CEO, Fastmail Pty Ltd
  [email-masked]

---

From: "Livingood, Jason" <[email-masked]>
Date: Thu, 27 Aug 2020 14:56:15 +0000

(personal views - no hats)

>    More broadly, the burden of proof should be on those demanding
    potentially disruptive changes

[JL] I'm not sure how a change from the terminology of master-slave to primary-secondary can be considered disruptive - it seems a little silly to suggest so.

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 Aug 2020 11:13:15 -0400

On 8/27/20 10:56 AM, Livingood, Jason wrote:

>>     More broadly, the burden of proof should be on those demanding potentially disruptive changes
>      

Without going into detail, I don't think I have as much confidence as you do about that.

But to clarify, the potential disruption I anticipate and want to avoid would probably be due to any of:

(a) interminable or repeated arguments about whether it's ok to ban or even discourage particular terms (like "master") for which there's little etomologic justification and a large potential loss in terms of readability if the term were banned;
(b) the possibility of documents being arbitrarily blocked by IESG over the documents' use of certain words, without the backing of community consensus; or
(c) documents held up interminably (in a WG or anywhere else) over disputes about terminology.

Keith

---

From: "John Levine" <[email-masked]>
Date: 27 Aug 2020 12:14:56 -0400

In article <[email-masked]> you write:
>However, both drafts don't provide any citations or quantification to back up their own assertions and recommendations. I think it would be
>very useful for a constructive discussion if the draft authors could perhaps provide a bit more grounding to their arguments.

In my experience, appeals to external authority in the IETF are deeply
counterproductive. I have also found that it is a poor idea to assume
that none of the people in the IETF are involved in or familiar with
other software and standards organizations and their processes. 

R's,
John

---

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 18:33:41 +0200

> other software and standards organizations and their processes. 

I actually _do_ assume that people are involved with other software and standards organizations, or that we could learn lessons from them. Else we would suffer from extreme 'not invented here'. We wouldn't want to say that any knowledge or experience outside of the IETF should not be deemed valid inside the IETF now, do we ? Especially not even if these are people who studied this topic extensively or have implemented exactly what is proposed here? 

Best,

Niels

---

From: Niels ten Oever <[email-masked]>
Date: Thu, 27 Aug 2020 18:34:53 +0200

> (c) documents held up interminably (in a WG or anywhere else) over disputes about terminology.

My question would be: what is this anticipation concretely based on? This this actually happen?

Best,

Niels

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 Aug 2020 12:47:44 -0400

> My question would be: what is this anticipation concretely based on? This this actually happen?

Yes.

---

From: "Livingood, Jason" <[email-masked]>
Date: Thu, 27 Aug 2020 17:21:07 +0000

(personal views - no hats - see inline)

>   (a) interminable or repeated arguments about whether it's ok to ban or even discourage particular terms (like "master") for which there's little etomologic justification and a large potential los…

[JL] This seems diversionary - the discussion has focused on replacing master-slave with something like primary-secondary. There are simple terms for which I suspect rough consensus exists. Are you arguing that you favor retaining master-slave?

> (c) documents held up interminably (in a WG or anywhere else) over disputes about terminology.

[JL] Those both seem like a stretch but in any case the IETF has well-worn processes to sort out publishing documents. I don't think you have made the case for significant disruption.

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 Aug 2020 13:49:54 -0400

> [JL] This seems diversionary - the discussion has focused on replacing master-slave with something like primary-secondary. There are simple terms for which I suspect rough consensus exists. Are you…

I am somewhat doubtful that rough consensus exists on that, particularly 
if we extend the discussion across all of IETF and not bury it within 
gendispatch.   I actually think the best choice might differ on a 
case-by-case basis.

For instance, if we're talking about database replicas in single-master 
databases, I'd be fine with primary-replica as IMO it's actually more 
descriptive and the substitution can be made with little or no loss in 
clarity.   But I don't feel comfortable making that judgment about all 
potential uses of master-slave, because "master" and "slave" actually 
mean different things in different cases and it's not always clear that 
there's a good substitute.

Also, these are not always simple terms, and there are potential ripple 
effects from such changes.  What if we're talking about what are 
currently called "multi-master" databases that do conflict resolution 
without reference to a single primary copy? "primary-replica" is an easy 
substitution for the single master database, but "multi-primary" doesn't 
sound right at all to me. I'm not saying this problem can't be sorted 
out, but it's not something that should be imposed from on high.

What we're discussing here is whether we should cripple the language 
that we use to communicate subtle technical concepts to others.   There 
are real costs to changing our language that should not be summarily 
dismissed.   And the idea that somehow we should just make a blanket 
decision for how to change things, or even which words to avoid, that 
makes sense across every subject domain in IETF seems like a huge stretch.

> [JL] Those both seem like a stretch but in any case the IETF has well-worn processes to sort out publishing documents. I don't think you have made the case for significant disruption.

IMO, disruption is exactly what bringing up this whole issue has caused, 
and the leadership's response has been to try to sweep the whole issue 
under a rug.

Keith

---

From: Dan Harkins <[email-masked]>
Date: Thu, 27 Aug 2020 10:55:58 -0700

> The problem is, I am not making assertions. The whole of draft-knodel-terminology has been based on research. Whereas draft-moore-exclusionary-language and draft-gondwana-effective-terminology do n…

   draft-knodel-terminology refers to papers behind paywalls which makes 
evaluation
of the claims made pretty much impossible. Not all "research" is 
scientific and much
of it cannot be reproduced [1]. This is especially true of research into 
social
sciences, which is what you are referencing, that is plagued by small 
sample size,
high variability, and selection bias.

   And you are making assertions. You assert that a metaphor is 
exclusionary but
don't say who is excluded and why. Happy to discuss? Start there.

   You talk about the "scientific or engineering way" but then in [2] 
you say,
"What we are currently doing is excluding people from the conversation - 
we only
have to look at the composition of our community to confirm that." There is
absolutely nothing scientific about that claim. There may be multiple 
factors
involved in creating the composition of a community that is constantly 
in flux.
To take a snapshot and attribute language to be the cause of it's 
current make-up
is decidedly un-scientific. Would you attribute the make-up of the National
Basketball Association (NBA) in the USA-- upwards of 75% of the NBA is black
while blacks make up only 13% of the population of the USA-- to some 
exclusionary
factor? No, of course not. So don't try and use the make-up of the IETF as
proof that some language you don't like is exclusionary. That's not 
scientific.

   Dan.

[1] 
https://www.westburg.eu/blog/having-hard-times-reproducing-your-experiments
[2] message-id <[email-masked]>

---

From: "Pete Resnick" <[email-masked]>
Date: Thu, 27 Aug 2020 14:35:23 -0500

Niels (and all),

Due to other work constraints, I have not had a chance to go through the 
other 20-or-so messages that were sent today (and I have only had a 
chance to skim the new documents), and I probably will not have a chance 
to get to them until later tomorrow. However, just as I replied 
yesterday with regard to the comments on draft-knodel, I must repeat 
that focusing on the particulars in a draft (in this case, concerns that 
some of the claims lack citations) is not particularly helpful in 
evaluating the primary dispatch questions. Like draft-knodel, neither 
draft-moore or draft-gondwana have been adopted by any WG, and one or 
more of the three may or may not be used as a starting place to any WG 
we might propose. Commenting on the drafts themselves, as against 
identifying things that should or shouldn't be done if this work is to 
move forward, does not progress the discussion.

pr

On 27 Aug 2020, at 3:54, Niels ten Oever wrote:

> Dear all,
>
> draft-moore-exclusionary-language and 
> draft-gondwana-effective-terminology both seem very concerned with the 
> potential for disruption of IETF processes if one were to provide 
> measures for the discussion of the issue of exclusionary language in 
> new I-Ds and RFCs. Both drafts also seem to find it very important 
> that if there is a (perceived) issue, this should be extensively 
> documented with evidence in the form of citations and numbers.
>
> However, both drafts don't provide any citations or quantification to 
> back up their own assertions and recommendations. I think it would be 
> very useful for a constructive discussion if the draft authors could 
> perhaps provide a bit more grounding to their arguments.
>
> This could for example be in scientific literature, or perhaps by 
> showing observed disruptions in similar or adjacent communities to the 
> IETF that have already instated such measures (such as Python, Django, 
> Linux Project, Bluetooth SIG, Drupal, Postgres, NIST, RedHat, etc).
>
> Best,
>
> Niels
>
> -- 
> Niels ten Oever

-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best