DRAFT: Subjective applicability #539
Conversation
| @@ -405,11 +405,20 @@ The applicability describes what parts of the [=test subject=] are tested. | |||
|
|
|||
| ### Applicability for Atomic Rules ### {#applicability-atomic} | |||
|
|
|||
| The applicability section is a required part of an [=atomic rule=]. It <em class="rfc2119">must</em> contain a precise description of the parts of the [=test subject=] to which the rule applies. For example, specific nodes in the DOM [[DOM]] tree, or tags that are incorrectly closed in an HTML [[HTML]] document. These are known as the [=test targets=]. The applicability <em class="rfc2119">must</em> only use information made available through the listed [input aspects](#input-aspects) in the rule. No other information can be used in the applicability. Applicability <em class="rfc2119">must</em> be described objectively, unambiguously and in plain language. | |||
| The applicability section is a required part of an [=atomic rule=]. It <em class="rfc2119">must</em> contain a precise description of the parts of the [=test subject=] to which the rule applies. For example, specific nodes in the DOM [[DOM]] tree, or tags that are incorrectly closed in an HTML [[HTML]] document. These are known as the [=test targets=]. The applicability <em class="rfc2119">must</em> only use information made available through the listed [input aspects](#input-aspects) in the rule. Definitions used in the description can be put in the rule [glossary](#glossary), or they can be defined in the section where they are used. No other information can be used in the applicability. | |||
There was a problem hiding this comment.
As an aside, I think we should remove the
tags that are incorrectly closed in an HTML document
wording, since this refers to 4.1.1 which is getting deprecated in 2.2.
act-rules-format/act-rules-format.bs
Outdated
|
|
||
| An objective description is one that can be resolved without uncertainty, in a given technology. Examples of objective properties in HTML are tag names, their computed role, the distance between two elements, etc. Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. | ||
| Applicability <em class="rfc2119">must</em> either be described objectively or use an **allowed subjective form**, and <em class="rfc2119">must</em> be written unambiguously and in plain language. An objective description is one that can be resolved without uncertainty, in a given technology. Examples of objective properties in HTML are tag names, their computed role, the distance between two elements, etc. When possible, objective descriptions <em class="rfc2119">must</em> be used since they reduce uncertainty. |
There was a problem hiding this comment.
The last sentence here
When possible, objective descriptions must be used since they reduce uncertainty.
And the second sentence of the paragraph below
When possible, subjectivity must be avoided since it can easily be misunderstood.
say the same thing, but I couldn't figure out in which location I liked it better, so I kept both and wanted to hear opinions. Is it better to be affirmative (i.e., "Do use objectivity) or negative (i.e., "Do not use subjectivity")?
There was a problem hiding this comment.
In last sentence, must -> should and explain when/why
act-rules-format/act-rules-format.bs
Outdated
| Even concepts like headings and images can be misunderstood. These terms could refer to the tag name, the semantic role, or the element's purpose on the web page because they are ambiguous. The latter of which is almost impossible to define objectively. When used in applicability, potentially ambiguous concepts <em class="rfc2119">must</em> be defined objectively. Definitions can be put in the rule [glossary](#glossary), or they can be defined in the section where they are used. | ||
| Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. When possible, subjectivity <em class="rfc2119">must</em> be avoided since it can easily be misunderstood. For example, concepts like headings and images could refer to the tag name, the semantic role, or the element’s purpose on the web page because they are ambiguous. The latter of which is almost impossible to define objectively. In some cases, where meeting the objectivity requirement is impossible, the use of a subjective description in the applicability is acceptable. | ||
|
|
||
| To deal with the inherent problems of subjectivity, any applicability including subjective descriptions must meet the following criteria: (1) use one of the allowed subjective forms, (2) place all subjectivity inside of a glossary definition, and (3) the applicability section must be clearly marked as subjective. |
There was a problem hiding this comment.
TODO: We need to determine how we want to mark the applicability as either objective or subjective.
Possible easy solutions could be changing the heading per rule to be "Objective Applicability" or "Subjective Applicability", or having a small sentence/note that says "This applicability is objective/subjective". Open to opinions.
There was a problem hiding this comment.
I have a slight preference to not modify the Applicability heading. Indicating subjective in the text would be fine.
Would it be acceptable to only require indicating subjective? Then, "unless indicated otherwise, the applicability is objective" could be used.
There was a problem hiding this comment.
I do like Kathy's call out, in that you are stating the baseline is objective, unless you are calling out the exception and then go in to what the exception is, (subjectiveness)
There was a problem hiding this comment.
I'm not sure we even need to track subjectivity in the rule. That's an open question for me still. If we do, it feels more like metadata, like the rule ID or the publication data. It's not that important for most readers I expect
There was a problem hiding this comment.
Put in the metadata but have displayed on the published page for implementer/user situational awareness.
|
|
||
| To deal with the inherent problems of subjectivity, any applicability including subjective descriptions must meet the following criteria: (1) use one of the allowed subjective forms, (2) place all subjectivity inside of a glossary definition, and (3) the applicability section must be clearly marked as subjective. | ||
|
|
||
| **Allowed Subjective Forms**: |
There was a problem hiding this comment.
A couple of thoughts here, is it clear that the things in brackets are the "fill in the blank" portion of the form? I should maybe add more verbiage to this.
Is "Objective target description" the best verbiage we can use to state what the subjectivity is applying to?
Is "role/common design pattern/subjective attribute" good wording for what kinds of subjectivity go with each form? I think not, this to me is the weakest part of this addition and the one that needs the most fleshing out and thought.
act-rules-format/act-rules-format.bs
Outdated
| 2. [Objective target description] functions like a [role/common design pattern] | ||
| 3. [Objective target description] is/has/expresses [subjective attribute] | ||
|
|
||
| The subjectivity allowed in each of these forms must be explicity included in glossary definitions to ensure common understanding of the scope of the subjectivity. For example, a rule with a subjective applicability “HTML element is styled as a heading” would require the creation of an agreed upon definition for what constitutes “styled as a heading”. |
There was a problem hiding this comment.
Do we like this? Or is this just additional overhead? What other ways could we form it?
There was a problem hiding this comment.
Should it be stated to minimize subjectivity or is that obvious? Meaning [role] and [role/common design pattern] are objective. Use these whenever possible.
Examples with explanation under each would be helpful. Copying from act-rules/act-rules.github.io#2061 (comment):
[Thing] is styled/looks like a [role] - Example: Element is styled as a heading
[Thing] operates like a [role/widget] - Example: Element operates like a checkbox
[Thing] is/has/expresses [subjective attribute] - Examples: Text node is purely decorative, Text node expresses anything in human language, Element is non-text content
Another thought -
I've encountered [Thing] functions as a heading (but not styled as a heading). For ex:
Applicability. The applicability describes what parts of the [=test subject=] are tested.
With this, I wonder if [functions as] could be used instead of [styled] in other [styled] examples.
There was a problem hiding this comment.
On Kathy's note on functions as vs. styled, are we staying away as markup and coded as terms to use? If we are evaluating Web content, yes the element is going to styled , however the element could be a paragraph tag or it could be a heading level tag, so the first thing would be to confirm the facts of what elements are used ('p') or (heading level) then determine applicability / subjectivity .
| **Allowed Subjective Forms**: | ||
| 1. [Objective target description] is styled as a [role] | ||
| 2. [Objective target description] functions like a [role/common design pattern] | ||
| 3. [Objective target description] is/has/expresses [subjective attribute] |
There was a problem hiding this comment.
This one feels too vague to me. One thing I think I'd like to do is for us to go over the subjective definitions we have today and see if we can categorise them somehow. Maybe see which ones we think would be okay for use in an applicability and which ones aren't.
There was a problem hiding this comment.
It will be good to revisit it now with more context, but my initial stab at finding some of the subjectivity types we use are in act-rules/act-rules.github.io#2061 (comment). That is where these 3 forms originally came from, they seemed to deal with the examples I found as well as some I had seen people want to use.
Co-authored-by: Wilco Fiers <[email protected]>
| <li>The element text broadly describes the content that follows it in the reading order of the page.</li> | ||
| </ul> | ||
| </blockquote> | ||
| </aside> |
There was a problem hiding this comment.
I tried to create an example of what form these types of definitions might take. In the format above, I left it up to the definition author to determine the logic behind the classification (i.e., it doesn't always have to be meet 3 of X criteria to apply). For example, I considered making the logic here that a heading should be at least one of the first 2 bullet points and then must meet the 3rd.
There was a problem hiding this comment.
I haven't made any changes to the format yet, but here are some of my initial responses to the questions we raised at the last meeting:
What if we have a test target that doesn't meet the definition, but that we would still like to have as a test target? The example given was a heading that only fulfills one or none of the conditions above?
I think in all cases this would mean that the definition requires further expansion/editing. We already have processes in place to handle this, and I think as long as we stick to them we will be fine. I am a bit concerned that we will open the door to "cascading" complexity if we try to make our definitions cover too much ground, but I don't have any methods for preventing that at the moment.
What if we want to specify what something isn't?
With our current conversation, we have discussed (and I believe at least initially agreed) to allow more complex logic inside of the definitions. In my opinion, this means usages of the standard and, or, and not operators will all be allowed when writing these definitions.
| <ul> | ||
| <li>Element text is visually distinct from nearby text, such as through the use of a larger font-size or bolding.</li> | ||
| <li>Element is clearly delineated from surrounding text, either through the use of spacing or other visual effects.</li> | ||
| <li>The element text broadly describes the content that follows it in the reading order of the page.</li> |
There was a problem hiding this comment.
To avoid including unintended targets, how about making the third item and instead of or? So, (1 OR 2) AND 3.
| Even concepts like headings and images can be misunderstood. These terms could refer to the tag name, the semantic role, or the element's purpose on the web page because they are ambiguous. The latter of which is almost impossible to define objectively. When used in applicability, potentially ambiguous concepts <em class="rfc2119">must</em> be defined objectively. Definitions can be put in the rule [glossary](#glossary), or they can be defined in the section where they are used. | ||
| Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. When possible, subjectivity <em class="rfc2119">should</em> be avoided since it can easily be misunderstood. For example, concepts like headings and images could refer to the tag name, the semantic role, or the element’s purpose on the web page because they are ambiguous. The latter of which is almost impossible to define objectively. In some cases, where meeting the objectivity requirement is impossible, the use of a subjective description in the applicability is acceptable. | ||
|
|
||
| Any applicability including subjective descriptions <em class="rfc2119">must</em> use one of the **Allowed Subjective Forms** shown below and place all subjectivity within a glossary definition. Glossary definitions used to identify subjective test targets <em class="rfc2119">must</em> provide a list of features or characteristics that are common to the intended test targets and the logic for determining if an element is applicable. |
There was a problem hiding this comment.
I have had this in my own comments for awhile, but am finally putting it here. From our discussion at TPAC, we settled on basically having two types of definitions. The would include logical definitions (i.e., allowed to use AND, OR, NOT) and the "Getting warmer/colder" approach that generally defines the applicable test targets.
For a rule with an applicability of "The rule applies to any HTML element that is styled as a heading." we would need to create the definition for "Styled as a heading".
Example "getting warmer" definition:
Styled as a heading definition - The more of the following conditions that an element matches, the more likely it is to be a heading.
- Element text is visually distinct from nearby text, such as through the use of a larger font-size or bolding.
- Element is clearly delineated from surrounding text, either through the use of spacing or other visual effects.
- The element text broadly describes the content that follows it in the reading order of the page.
NOTE: The last element in this list may need to be removed, its description mentions nothing about styling and may be the definitive definition of what it means for something to be a heading.
Some of my current thoughts as of 12/5/2023 -
- The allowed subjective forms do not seem to have gained much traction, they also may just complicate the rules format.
- We have identified several best practices for creating subjective definitions - creating multiples types of ways we may formulate the definitions and potentially also a method for creating rules for definitions to ensure more global consistency (see Turn "visible" examples into a rule act-rules/act-rules.github.io#2087)
- Are the glossary definitions really a good way to quality check that an applicability is acceptable? If the definition is only used in one rule it doesn't give us much and the rule is still going to be reviewed anyways.
So what - It seems that we continue to identify best practices for writing subjective rules, but not changes that need to be made to the rules format in order to allow subjective rules. Given this we could simply include only the changes above that allow subjective rules and then have community rule writing guidelines that ensure the quality of the rules. This would make the rules format less prescriptive but much more adaptable. Lastly, I still think it would be beneficial to clearly label rules as objective or subjective (perhaps for both Applicability and Expectation) to ensure the expectations of readers/implementers are clear.
Building on comments from this discussion (act-rules/act-rules.github.io#2061) on subjective applicability, I am drafting up some possible changes to the format itself.
I have left inline comments in places I know need additional conversation/editing.