Navigation findings and recommendations
- Move forward with the following:
- Three ways to get to the TOC, which is great for multiple navigation styles
- TOC in middle area of reg part homepage
- Default TOC view in nav sidebar TOC
- Expanded TOC option in nav sidebar
- Labeled icons in the nav sidebar
- Generally default to labeled icons going forward)
- Three ways to get to the TOC, which is great for multiple navigation styles
- Iterate on the following:
- "Jump-to" and enabling the feature to jump to another part
- TOC icon in the nav sidebar
- Could try moving it to right above the XYZ.1, possibly with some additional UI copy ("Expand Table of Contents")
- Keep an eye on/revisit at a later date
- Type size for sidebar (readability issue may be an artifact of Figma)
- Location and treatment of the other reg tool links
- Topic box treatment as part of the reg part homepage (either as collapsible or in the supplemental sidebar). It may also be applicable as part of the "learning to use eRegs" work.
- Process decision
- Keep existing format for findings but add screenshots
- Participants preferred having the TOC in the middle vs just the whole part
- When displayed prominently in the supplemental sidebar, additional regulatory tools may detract from the utility of eRegs
- The navigation sidebar is visually prominent, yet poses some readability issues for some participants
- Most participants were drawn to—and wanted to use—the topics box
- Participants assumed the nav sidebar and the middle TOC content were different (even though they are the same content, just different presentations)
- Participants were ready and willing to “jump” to other sections and parts
- Participants emphatically preferred labeled icons for the supplemental sidebar
We wanted to know how helpful (or extraneous) the table of contents in the middle content area was for participants on the Reg Part home page. All 5 participants (here described as P1, P2, P3, P4* and P5) preferred having the Table of Contents (TOC) in the middle area in addition to the sidebar, regardless of order shown. A few participants anticipated using one over another for different reasons:
- P3 saw the left as “a way to search for something when you don’t have the precise number [has a ‘summary’]. The middle is for when I know what I am looking for.”
- P5 thought the TOC was easier to read than the sidebar: “In the previous one, I would have to scroll through. [The TOC] gives me an overall view of how deep the regulation goes: A, B, C.”
Just one participant wished they could also have access to the full reg text as well, but most preferred not to have to scroll.
- P5 liked that “you can collapse the blue. I'd like to be able to scroll the whole thing [the whole part)].”
Recommendation
- Move forward with including the table of contents in the main content area
- While implementing a subpart-by-subpart structure seems to fit most use cases, there is still a need for working with a whole part. Consider having a way for users to view the whole part in an easy and accessible way.
Two participants (the ones with more experience using regulations as part of their day-to-day work and navigate to specific regulations on a regular basis ) thought that the information provided bordered on overwhelming.
- Most participants were drawn to the topic box (regardless of prototype shown).
- P4 said there was too much info, but liked “recent changes” if topic-specific.
- P5 said “There's a lot going on in this page. I'm not sure what's here, I'm sure it's powerful, I think it would need some training to take advantage of it”
At the end of the session, P5 shared concerns around trust and accuracy (something other participants in previous studies have shared with us):
“I would want to know - is it possible for there to be errors in the text? What's the source of truth for this page? How would I know how current the page is? When was it updated?”
Additionally, P5 found the inclusion of additional regulatory tools in the supplementary sidebar distracting. They recommended that we “make it easier to use what's already in the system, rather than going out to another resource. I might make these [tools] less apparent, or give a warning that you're leaving and going somewhere else.”
Recommendation
- Reconsider the context, location, and emphasis of additional regulatory tools. A few options could be to keep the list in place but collapsed by default, or move the links to another part of the site (another page, the footer, etc.)
- Look into additional ways to communicate the “source of truth” and currency of the content.
Most participants were drawn to the blue navigation sidebar due to its visual prominence, but two participants shared that the combination of dark background with small text was difficult for them to read:
- P4 said the text was too small (hard to read). Liked that you could collapse the blue (would rather scroll the whole part). “My eye still gravitates to the blue rather than the white. But this is easier, in the white, more similar to the book.”
- P5 said that the dark background with small text is challenging to read and that "I find it easier to read white background sections."
Recommendation
- Consider improving the readability of the sidebar by making some visual adjustments (color, type size, etc.).
The topics section was popular with 4 out of 5 participants. They were all immediately drawn to the topics and a few wanted to use it for finding a specific phrase.
- P3 said “I like that idea of being able to get a snippet of information to help narrow my focus. eCFR doesn't have that.”
However, P4, (the participant group who had the most experience out of all the participants as a whole) didn’t like it, and thought it took up too much room. They weren’t sure how different it was from the Table of Contents. They wanted it either gone or moved to the Supplementary Content sidebar: “not sure what the "topics in this section" includes. Are those just excerpts from the reg? Not sure how that's different from the TOC or what its purpose is.”
Recommendation
- Consider relabeling “Topics” to something more descriptive, i.e., if it’s geared more towards less experienced reg users, make it more obvious who that is for.
- Consider either making the topics in the middle of the Reg Part Homepage collapsible, or move it to the Supplementary content sidebar, so the main focus of the Reg Part Homepage is on the TOC itself.
The facilitator asked each participant how they would find a phrase and then a citation. Since search was specifically excluded as an option, participants were forced to use what was in front of them.
We observed some clear patterns around navigation. People were mostly drawn to either the navigation sidebar or the topics when looking for a specific phrase.
- P1: Drawn to topics first, then the nav sidebar. Would use the TOC in the middle last if still looking.
- P2: Drawn to the nav sidebar and then topics, but also wanted ctrl+f.
- P3: Drawn to the nav sidebar first, and then topics.
- P4: Drawn to the nav sidebar first (assume “it’s the TOC of the subpart”).
- P5: Drawn to topics first, and then the “Federal Register” in the right sidebar.
When looking for a specific phrase, two participants specifically cited wanting to use ctrl+f. P2 said they “would try ctrl-f and if I got 500 entries I would change my query to be more specific.” P4 wanted to use it as well.
Interestingly, most participants thought the content in the nav sidebar and the middle TOC were different, and treated them differently as a result, despite the copy being identical.
- P3 saw the nav sidebar as a way to search for something when you don’t have the precise number (has a “summary”): "The middle is for when I know what I am looking for." They preferred having the TOC in the middle for this reason.
- P5 felt similarly: “I might use the left if I was looking for a specific phrase, and I might use the center for looking for a specific citation number.”
Participants really liked the “Jump to” function and scrolling for exact citations. Most wanted to jump, but two participants said they would also rely on scrolling.
"I like the ‘jump to reg section’. A lot of the time in writing, I know what it is but need to double-check what it says. Nice to hop without scrolling through an entire thing." - P1
Notably, participants also wanted to jump to other places.
- P3 thought clicking the ‘go’ button without filling it in would take them to the beginning of the part.
- P4 said they thought it’d be helpful to jump between 438 and 447 so they could “go all in one place without having to back out of things like in the eCFR.”
We also asked a few questions around how participants would get back to the Reg Part Homepage. Besides P3 using jump to get back to the beginning (see above desired usage), a handful gravitated towards the first item in the nav sidebar:
- P3 also thought clicking on 433.1 might open the whole reg part: “I'm still wondering though what's the intent behind the 433.1? Is that the beginning of the reg? Does that just bring you to the top of the section, or both?”
- P5 thought 433.1 would get back to the TOC in the middle.
No participants noted the part header link next to the site logo as a way of getting back to the Reg Part Homepage. Nor did any participant think they could use the nav sidebar’s TOC to do so.
Recommendation
- Make it possible to leave the Jump field empty and have it “go” to the beginning of the reg part.
- Consider making it possible for users to change the part number as well (from 433 to 435, for instance).
- Since there are some user expectations around the top-most item in the nav sidebar taking them back to the beginning, consider adding an additional way to do just that.
All 5 participants were communicative about wanting to use search, and considered that feature to be crucial to their research workflows. They didn’t know what to expect, and we asked questions around their needs.
- P1 wanted to be able to search for a term like “signatures” across the reg parts.
- P2 expected that the search bar would search the part (based on noticing the part header near the site logo) and then have an option to expand out to other parts. They specifically said they’d want to broaden search to Title 42.
- P4 wasn’t clear if search would just search that part or everything (interested in both kinds of search).
- P5 relies on search more than anything, and would like to be involved in testing that feature.
All 5 participants preferred the labeled icons. Most participants were wary of the ‘X’ icon and were concerned they would lose their place. The ‘>>’ icon was confusing to 4 of the participants.
Only 1 participant correctly guessed that ‘>>’ would expand the sidebar. 4 participants had interesting thoughts around what ‘>>’ might do.
- P1 said X didn’t bother them and the >> reminded them of fast forward, and wondered if the arrows would skip to Subpart D.
- P2 said the X makes sense, but made them nervous. Not sure how you would get the TOC back. Guessed that the >> would widen it but also wondered if it would zoom out to Title 42 TOC.
- P3 liked the X icon, but puzzled by the >> “Might it broaden the contents a little? Would it include more regs somehow than 433?”
- P4 said “I would probably never click the X or maybe not the arrows because I'd be concerned I wouldn't know how to get it back, especially with the X”
Recommendation
- Move forward with the labeled icons.
A few participants had interesting ideas around the nav sidebar’s double arrow (‘>>’) icon. When we asked participants what they thought it would do if they clicked on it, we heard some similar expectations.
P2 thought the icon would collapse categories even more so they could see all of Title 42 and not just 433: “Like if I wanted to look at something in 437 and see parts instead of subparts.”
When we asked P3 what the double arrow was for: “I have no idea. Might it broaden the contents a little? Would it include more regs somehow than 433?
*P4 was not a single participant, but a group of 3 individuals who were generous to give us their time and feedback.
To observe and learn from users in order to
- Identify which kind of structure participants prefer when finding and navigating content (whole part, subpart, or section)
- Assess how users navigate content (using table of contents; scrolling; viewing by whole part, subpart, or section)
- Determine which design approach is more clear and helpful
- Sidebar with TOC or Sidebar with tabs
- Reg Part Homepage with TOC or with just the full text
-
How users prefer to view/navigate content
- What are participants’ general impressions of the layout of the whole part, subpart, and section live code prototypes?
-
Using the table of contents for navigation
- Is it helpful to have the TOC in the main content area or is it sufficient to just have the TOC just in the sidebar?
- When users are researching regs, does having the TOC on the reg part homepage more efficient?
- How would you navigate sections in Part 433 Subpart C?
-
Sidebar preference
- What are the general impressions of the layout of both Sidebar TOC or the Sidebar tab prototypes?
- How discoverable is it to adjust the width of the content to see more of it at once?
- Which sidebar approach participants prefer
- Is the additional TOC view helpful or extraneous? Or is the sidebar without it sufficient as is?
- Which expand/collapse approach is more understandable?
- Which approach is easier to use when navigating to a particular section?
-
General questions (ask before the usability portion)
- How do you use regulations in your day to day work?
- How do you generally look up regs? Which online reg tools do you generally use?
- Do you tend to scan content by whole part, subpart, or section?
- On what devices do you generally use to look at regs? (e.g. desktop, laptop, mobile, etc.)
User group(s): CMCS Staff, policy owners, both those with a lot of regulatory/policy research experience and those that are occasional users
We will conduct a moderated formative evaluation using the think-aloud protocol. Since four prototypes, as well as live code are being tested, the order will be flipped for each participant so as to not bias users towards one over the other.