Skip to content

Commit

Permalink
Break apart support page
Browse files Browse the repository at this point in the history
  • Loading branch information
consideRatio committed Apr 19, 2024
1 parent 4defbf5 commit f479b4e
Show file tree
Hide file tree
Showing 10 changed files with 200 additions and 205 deletions.
2 changes: 1 addition & 1 deletion engineering/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Engineering is responsible for all of our technical infrastructure, and for the
- Perform daily operation of ongoing cloud services, addressing technical problems as-needed.
- Resolve operational issues related to our cloud infrastructure.
- Oversee our [Incident Response process](../projects/managed-hubs/incidents.md).
- Design the system of roles and practices that defines our [Support process](../projects/managed-hubs/support.md).
- Design the system of roles and practices that defines our [Support process](support:index).

## Develop our cloud infrastructure

Expand Down
5 changes: 4 additions & 1 deletion index.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,10 @@ We document some major aspects of this service in the sections below.
:maxdepth: 2
:glob:
projects/managed-hubs/index
projects/managed-hubs/*
projects/managed-hubs/support/index
projects/managed-hubs/incidents
projects/managed-hubs/showcase-hub
projects/managed-hubs/sales
List of running hubs <https://infrastructure.2i2c.org/reference/hubs/>
```

Expand Down
9 changes: 0 additions & 9 deletions projects/managed-hubs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,3 @@ The Managed JupyterHub Service is an ongoing service to **sustain and scale** a
**[`docs.2i2c.org`](https://docs.2i2c.org) has most of the information about this service**.

The sections here contain information that is more relevant to 2i2c team members (like support process documentation).

```{toctree}
:maxdepth: 2
showcase-hub
sales
support
timeboxed-initial-ticket-evaluation
incidents
```
34 changes: 34 additions & 0 deletions projects/managed-hubs/support/communication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Communication channels

## External communication

We use these channels for communicating with external stakeholders like Community Representatives:

- **[[email protected]](mailto:[email protected])** is our point-of-contact for all support-related external communication.
- **[The 2i2c FreshDesk account](support:freshdesk)** is where we track all support requests and communication.
- **{doc}`the "Get Support" page <docs:support>`** provides guidance that communities may follow to get support.

(support:freshdesk)=
### FreshDesk

We use FreshDesk to manage all of our hub support questions and change requests.
It is at the following URL:

[**`2i2c.freshdesk.com`**](https://2i2c.freshdesk.com/)

Any emails sent to `[email protected]` will be routed to this FreshDesk account, and all team members should have access to it.

[**Canned responses**](https://support.freshdesk.com/en/support/solutions/articles/37577-creating-common-reply-templates-with-canned-responses) allow us to pre-populate common responses for many kinds of support.

```{button-link} https://2i2c.freshdesk.com/a/admin/canned_responses/folders
:color: primary
Our canned responses.
```

## Internal communication

We have a few channels for communicating around support requests:

- Our [FreshDesk account](https://2i2c.freshdesk.com/a/) allows for internal team communication via the {guilabel}`Add Note` button. This can be useful for sharing quick internal updates.
- The [Eng & Prod board that collects support-related issues](https://github.com/orgs/2i2c-org/projects/22/views/47) identified via metadata `type==support`.
- (deprecated) [Issues with the {guilabel}`support` label](https://github.com/2i2c-org/infrastructure/issues?q=is%3Aopen+label%3Asupport+sort%3Aupdated-desc) were where we tracked support requests related to {term}`Change Requests` and {term}`Guidance Requests`.
14 changes: 14 additions & 0 deletions projects/managed-hubs/support/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
(support:index)=
# Support

Product at 2i2c is responsible for guiding the evolution of our services and products, and for representing the needs and potential impact of the communities we serve.
They work alongside engineering teams to define ways in which we aim to improve our services, and to prioritize deliverables according to their impact.

```{toctree}
terminology
roles-and-team
process
timeboxed-initial-ticket-evaluation
communication
time-off
```
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
(support:guide)=
(support:process)=
# Support process

:::{admonition} In Beta!
Expand All @@ -11,7 +11,6 @@ This section contains information that our team uses to triage, communicate, and

Currently, Engineering has the authority to design the system of roles and processes that defines "support at 2i2c", and to advocate for the resources needed to fill those roles. This will naturally be done in collaboration with others in 2i2c, but Engineering leads the effort and has an outsized role in the decision-making.

(support:process)=
## Overview of the support process

Here's a brief overview of our support process[^github-support-issues]:
Expand All @@ -34,123 +33,6 @@ Here's a brief overview of our support process[^github-support-issues]:
- For these types of support, each community has a {term}`Support Budget` that defines the hours we can sustainably spend on non-incident support requests.
- When any support issue is resolved, we will communicate with the {term}`Community Representative` to confirm with them.

(support:types)=
## Types of support requests

Below are a few key terms that describe how we categorize Support Requests.

```{glossary}
Support Request
Support Requests
Any request that a community sends to us via `[email protected]`.
Support requests are generally un-planned and happen in response to a community needing assistance with something unexpected.
There are a few main categories of support that we consider, each is described below.
Incident
Incidents
An event that significantly degrades the JupyterHub service. Support requests that are related to incidents should be prioritized over all other work items.
[](incidents:what) defines the kind of incidents we respond to via PagerDuty and consider immediate issues to be resolved.
We do not have a limit on the support time we provide related to incidents (as opposed to Change and Guidance requests, which have a {term}`Support Budget`).
:::{seealso}
See [](incidents.md) for more information.
:::
Change Request
Change Requests
A request for a desired change to a hub's infrastructure that is not related to an incident. For example:
- Changing the user's software environment.
- Changing the resources available to users.
- Updating and deploying changes from upstream tools for a community.
- Making an improvement to open source tools to benefit a community.
Change Requests are generally non-urgent and should not be associated with significant diminished service. They are often things that communities _could_ carry out themselves with the proper guidance and infrastructure setup. We aim to make our hubs as configurable as possible _by the community_ so that we are not on the critical path for things like environment updates.
Guidance Request
Guidance Requests
A Support Request that is not tied directly to a change in infrastructure. Sometimes support requests are not tied to a specific change, but a desire to discuss and request guidance. In this case we may set up a meeting to discuss as a group, or have some back-and-forth via support channels.
```

## Roles and team structure

Supporting a 2i2c hub is a collaborative process between 2i2c and the community we serve.

The **Support Team** is one of the main teams in our **Managed JupyterHub Service Team**.

This consists of three main roles: {term}`Support Triagers`, {term}`Community Representatives`, and {term}`Hub Administrators`.

```{glossary}
Support Triager
Support Triagers
A **two-person team** of {term}`Support Triagers` work together to triage and communicate with all external support requests.
Tenure on the support team is **for four weeks**.
Every **two weeks** (generally at the sprint meeting), a Support Triager cycles off the support team, and a new team member joins the team.
The support team rotates through [the “Open Infrastructure Engineering Team”](https://compass.2i2c.org/reference/team/), in alphabetical order.
The primary responsibilities of the Support Triagers are:
- Ensure that we meet {external+docs:ref}`our Support Service Level Objectives <objectives:support>`.
- Carry out [our support process](support:process).
- Act as primary points of contact with {term}`Community Representative`s.
- Trigger an [Incident Response](support:incident-response) if need be.
Common alternate terms: **Customer Liason**, **External Liason**, or **Customer Support**.
Community Representative
Community Representatives
See [the service documentation](https://docs.2i2c.org/about/service/shared-responsibility.html#std-role-Community-Representative).
Hub Administrator
Hub Administrators
See [the service documentation](https://docs.2i2c.org/about/service/shared-responsibility.html#std-role-Hub-Administrator).
```

### Who are our Support Team?

The Support Triager role rotates through the below Support Team members.

```{include} ../../_data/tmp/support-triagers.txt
```

## Communication channels

### External communication

We use these channels for communicating with external stakeholders like Community Representatives:

- **[[email protected]](mailto:[email protected])** is our point-of-contact for all support-related external communication.
- **[The 2i2c FreshDesk account](support:freshdesk)** is where we track all support requests and communication.
- **{doc}`the "Get Support" page <docs:support>`** provides guidance that communities may follow to get support.

(support:freshdesk)=
#### FreshDesk

We use FreshDesk to manage all of our hub support questions and change requests.
It is at the following URL:

[**`2i2c.freshdesk.com`**](https://2i2c.freshdesk.com/)

Any emails sent to `[email protected]` will be routed to this FreshDesk account, and all team members should have access to it.

[**Canned responses**](https://support.freshdesk.com/en/support/solutions/articles/37577-creating-common-reply-templates-with-canned-responses) allow us to pre-populate common responses for many kinds of support.

```{button-link} https://2i2c.freshdesk.com/a/admin/canned_responses/folders
:color: primary
Our canned responses.
```

### Internal communication

We have a few channels for communicating around support requests:

- Our [FreshDesk account](https://2i2c.freshdesk.com/a/) allows for internal team communication via the {guilabel}`Add Note` button. This can be useful for sharing quick internal updates.
- The [Eng & Prod board that collects support-related issues](https://github.com/orgs/2i2c-org/projects/22/views/47) identified via metadata `type==support`.
- (deprecated) [Issues with the {guilabel}`support` label](https://github.com/2i2c-org/infrastructure/issues?q=is%3Aopen+label%3Asupport+sort%3Aupdated-desc) were where we tracked support requests related to {term}`Change Requests` and {term}`Guidance Requests`.

## Process for support triage and resolution

Here is the process that we follow when triaging and resolving support requests.
Expand Down Expand Up @@ -305,78 +187,3 @@ We have a few roles that are particularly useful for triaging and prioritizing o
When deciding how to prioritize a Change Request, ask for guidance from these two team members.

:::

### Support request key terms

The following are other important terms and ideas in the support process.

```{glossary}
Support Budget
A fixed amount of time that we can spend providing support for each community that we serve. This helps us ensure that we can sustainably serve many communities. Any support request that is **not tied to an {term}`Incident`** will draw from the support budget for that community. If a community requests support beyond their support budget, we may request extra funds to help cover our costs.
:::{note}
We currently keep this term intentionally vague, and ask that communities are respectful of our time when making change requests.
We are investigating the support budget that we should give to each community, and will update here when we have specific numbers in mind.
Here is a rough idea of the rationale we follow as we identify more specific numbers for support budget:
- Assume 173 working hours a month per engineer.
- Assume any given engineer should spend no more than 20% of their time (one day a week) on average dealing with support requests.
- That amounts to 34 hours a month on support requests.
- Then choose a support budget for each community based on the number of communities we think one engineer can sustainably serve.
Tie this number back to our question "how many hubs should be able to sustain a single engineer?".
That ranges between 20 and 8 (depending on edu/research, and dedicated cluster).
:::
```

(support:expected-time-off)=
## Expected time off and downtime

There are some cases when we intentionally reduce our operations and support capacity.
See [](time-off:annual-expected) for our broader policies and support commitments during this time.

(support:expected-time-off:policy)=
### Expected time off support policy

Below is our policy for support during expected time off:

- We monitor our support e-mail for major incidents, but do not guarantee a response for non-essential requests or questions.
% NOTE: In the future we may define a policy for support if a community knows and
% tells us they are doing essential work during expected time off.
% We can update this policy when we discuss and make a decision on this.
%
% Some conversation in: https://github.com/2i2c-org/meta/issues/435
- We will take steps to minimize harm and avoid catastrophic problems (e.g. incidents that drastically increase costs), but will not perform non-essential actions. We will assume there is _no essential work happening on any of our hubs for all communities_.
- If there is a catastrophic incident, we will take the minimal number of actions to reduce risk and damage to a reasonable level.
- For non-catastrophic incidents and general change requests, we will wait until _after this period_ to resolve them.

### Support process during expected time off

Here are the steps we take to set expectations for our team and for other organizations during expected time off:

- **One month before the start of time off**. Add footer content to our `[email protected]` and `[email protected]` responses that communicates our intent.
See [](freshdesk:footer) for instructions on doing this.

Example text:

> **Note:** the 2i2c team will have **expected reduced capacity** from `STARTDATE` to `ENDDATE`.
> During this time, we will provide operational support for significant cloud incidents and outages, but not for non-essential questions or change requests.
> We will be less responsive than normal, and will return to answer your questions and resolve issues after the time off.
- **During the time off**. Add an auto-responder to our FreshDesk accounts with a message like the following:

> **Note:** the 2i2c team is operating at an **expected reduced capacity** until `ENDDATE`.
> We will provide operational support for significant cloud incidents and outages, but not for non-essential questions or change requests.
> We will be less responsive than normal, and will return to answer your questions and resolve issues after the time off.
% TODO: In the future, we want to add two extra steps:
%
% 1. Send an e-mail to our mailing list for Community Representatives communicating our intent to be on expected reduced capacity.
% ref: https://github.com/2i2c-org/team-compass/issues/579
%
% 2. Add a banner announcement to each of our community hubs.
% ref: https://github.com/2i2c-org/infrastructure/issues/1501

### Rotating team members during expected time off

Because team members continue to serve as support triagers during this time, we should take care to avoid the same person serving in this role across multiple periods of expected time off.
40 changes: 40 additions & 0 deletions projects/managed-hubs/support/roles-and-team.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Roles and team structure

Supporting a 2i2c hub is a collaborative process between 2i2c and the community we serve.

The **Support Team** is one of the main teams in our **Managed JupyterHub Service Team**.

This consists of three main roles: {term}`Support Triagers`, {term}`Community Representatives`, and {term}`Hub Administrators`.

```{glossary}
Support Triager
Support Triagers
A **two-person team** of {term}`Support Triagers` work together to triage and communicate with all external support requests.
Tenure on the support team is **for four weeks**.
Every **two weeks** (generally at the sprint meeting), a Support Triager cycles off the support team, and a new team member joins the team.
The support team rotates through [the “Open Infrastructure Engineering Team”](https://compass.2i2c.org/reference/team/), in alphabetical order.
The primary responsibilities of the Support Triagers are:
- Ensure that we meet {external+docs:ref}`our Support Service Level Objectives <objectives:support>`.
- Carry out [our support process](support:process).
- Act as primary points of contact with {term}`Community Representative`s.
- Trigger an [Incident Response](support:incident-response) if need be.
Common alternate terms: **Customer Liason**, **External Liason**, or **Customer Support**.
Community Representative
Community Representatives
See [the service documentation](https://docs.2i2c.org/about/service/shared-responsibility.html#std-role-Community-Representative).
Hub Administrator
Hub Administrators
See [the service documentation](https://docs.2i2c.org/about/service/shared-responsibility.html#std-role-Hub-Administrator).
```

## Who are our Support Team?

The Support Triager role rotates through the below Support Team members.

```{include} ../../../_data/tmp/support-triagers.txt
```
Loading

0 comments on commit f479b4e

Please sign in to comment.