Decision: Content review workflow

[DRAFT]

Thing	Info
Relevant features	Content authoring
Date started	2021-09-01
Date finished
Decision status
Summary of outcome

Background/context

Our policy SME team members and our CMS partner policy SMEs need to be able to review our supplemental content in systematic ways, for both internal checking of work on entering content and for CMS review and approval of the content.

Core questions

Primary question: How should we enable them to do this work?

Supporting questions: What do we need to know more about before we can decide? How can we learn more, including prototypes people can try and questions we can ask?

What we know

Capabilities and tradeoffs

It is possible to modify our admin panel and website to provide various functionality, including providing views of the data or downloadable CSVs/spreadsheets.

It is sometimes more straightforward to have data review directly within an admin panel - instead of on our custom website or in a custom exported format - to reduce the risk of website display or logic issues interfering with viewing the information.

User expertise and tool access

Some policy SMEs may be comfortable with complex spreadsheets of content, but would need to validate overall competency and comfort with format.

CMS policy SMEs usually don’t have access to Google Sheets for collaboration, but some of them use Box.net for a similar purpose - you can upload a spreadsheet and work on it in the browser together. Could also potentially use SharePoint but requires access permissions.

What do people need to do?

Both internal and partner SME review work may include:

• Make sure content is entered correctly (no typos, no accidental mixups, etc)
• See if anything is missing on accident (such as something not getting saved properly due to an issue with the content authoring workflow)
• Make sure the database already has all the relevant information that I know about (such as all the relevant content for a particular part or section, or all the relevant section tags for a piece of content)
• Check whether an obsolete or withdrawn piece of content is included
• Check whether a brand-new piece of content is included
• Review what’s been added or changed since the last time they checked
• Check whether supplemental content is tagged to the appropriate subpart/section, including if a subpart/section is missing or needs to be deleted
• Determine whether a document or resource link should not be included on the site
• Check whether supplemental content is tagged to the category

Policy SME suggested that these views would be helpful for review work:

• Choose a regulation subpart or section and see a readable list of the documents attached to that subpart or section.
• Choose a document and see a readable list of the regulation sections that are attached to it.
• See a list of documents that don't have any regulation sections attached to them.
• See a list of regulation subparts and sections that don't have any documents attached to them.
• Choose a category and see what documents are are attached to that category

Things we need to decide + options for them

So far, we’ve discussed potential ways to support reviewing content:

• Within the admin panel, enable the policy SME to pull up various “report” views that give them insight into the data in the system.
• Within the admin panel, enable exporting a CSV/spreadsheet that a policy SME could use for analysis, insight, and review.

We’ve also discussed ways to enable managing/approving content:

• Within the admin panel, enable policy SMEs to approve, delete, hide, or comment on items
• A spreadsheet of content that has columns for approving, deleting, hiding, and commenting on items (team policy SME then inputs any necessary changes into the admin panel)

And ways to support oversight and transparency in general:

• Provide a detailed “recent actions” view in the admin panel
• Provide a log on the website of recent actions by user
• Enable exporting supplemental content information from the website as a CSV/spreadsheet

What do we need to know before we can decide?

How can we learn more, including prototypes people can try and questions we can ask?

We're already trying out a spreadsheet-based prototype for policy SME review! We should see how that goes.

Conduct usability testing on the admin tool and/or spreadsheets with policy SMEs. (We'd need to describe our usability testing goals.)

Consequences