Skip to content

Commit

Permalink
First check of links after Evelyn's port to quarto.
Browse files Browse the repository at this point in the history
  • Loading branch information
JorgeBCh committed Sep 5, 2024
1 parent 9f463c5 commit 80b2f33
Show file tree
Hide file tree
Showing 7 changed files with 25 additions and 39 deletions.
2 changes: 1 addition & 1 deletion _quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ website:
href: https://twitter.com/SpanCloud
aria-label: Quarto Twitter
- icon: github
href: https://github.com/Cloud-SPAN/nerc-metagenomics-v2q
href: https://github.com/Cloud-SPAN/cloud-admin-guide-v2q
aria-label: Quarto GitHub
# - icon: rss
# href: https://quarto.org/docs/blog/index.xml
Expand Down
13 changes: 0 additions & 13 deletions cloud-admin-guide-v2q.Rproj

This file was deleted.

2 changes: 1 addition & 1 deletion docs/lesson01-setting-work-envs/index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Setting Up Your Cloud and Terminal Environments
---

Please read [**Workshops Organisation**](/docs/miscellanea/precourse-instructions.qmd) if you haven't done so.
Please read [**Workshops Organisation**](/index.qmd#workshops-organisation) if you haven't done so.

The Bash shell scripts that automatically manage multiple AWS instances will be referred to as the "Scripts" from now on.

Expand Down
12 changes: 5 additions & 7 deletions docs/lesson02-managing-aws-instances/02-instances-management.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -43,19 +43,17 @@ This section describes the typical use-case scenario of instances management for
7. **Unforeseen Instance Management**\
This section describes our approach to handle unforeseen instance management requests such as creating additional instances for a workshop due to late registrations, or deleting some (not all) instances before the end of a workshop due to cancellations.

8. **Troubleshooting**\
8. **[Troubleshooting](#troubleshooting)**\
This section presents some problems we have come across in managing instances and how to solve them.

# 1. The Scripts Running Environment
These are Scripts that create and manage AWS instances

``` {.bash filename="Output"}
aws_domainNames_create.sh aws_instances_configure.sh csinstances_create.sh
aws_domainNames_delete.sh aws_instances_launch.sh csinstances_delete.sh
aws_elasticIPs_allocate.sh aws_instances_terminate.sh csinstances_start.sh
aws_elasticIPs_associate2ins.sh aws_loginKeyPair_create.sh csinstances_stop.sh
aws_elasticIPs_deallocate.sh aws_loginKeyPair_delete.sh
aws_elasticIPs_disassociate.sh colour_utils_functions.sh
aws_domainNames_create.sh aws_instances_terminate.sh csinstances_create.sh
aws_domainNames_delete.sh aws_loginKeyPair_create.sh csinstances_delete.sh
aws_instances_configure.sh aws_loginKeyPair_delete.sh csinstances_start.sh
aws_instances_launch.sh colour_utils_functions.sh csinstances_stop.sh
```

The four scripts `csinstances_*.sh` (on the right) are to be run by the **user of the Scripts**, the person in charge of creating, configuring, stopping, starting and deleting AWS instances for a workshop /course. The scripts `aws_*.sh` are invoked by the scripts `csinstances_create.sh` or `csinstances_delete.sh` to either create or delete instances and the corresponding domain names, IP addresses, and login keys. The file `colours_utils_functions.sh` provides (is "sourced" by) the other scripts with text colouring functions (for the results of the other scripts to be easier to read) and other utility functions.
Expand Down
4 changes: 2 additions & 2 deletions docs/lesson02-managing-aws-instances/03-ami-management.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -473,9 +473,9 @@ By the way, you can change the instance type of an instance in the AWS Console a
- change the instance type: click on **Actions** at the top, then on **Instance settings**, and then on **Change instance type**. A new page will appear where you will be able to change the instance type --- save the changes
- restart the instance.
# 6. Configure an 'instance-to-become-AMI'
# 6. Configure an instance to become AMI
This section presents our approach to configure an 'instance-to-become-AMI' including:
THIS section presents our approach to configure an 'instance-to-become-AMI' including:
6.1. Updating system software\
6.2. Changing the end-user username\
Expand Down
7 changes: 3 additions & 4 deletions docs/miscellanea/precourse-instructions.qmd
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
---
title: Precourse Instructions
---

To take the course **Automated Management of AWS Instances** you are expected to:

- install Git Bash --- Windows users
- install or update Bash --- Mac users
- use a laptop or desktop computer with access to the Internet and a browser (such as Chrome) installed

This *Setup* section provides instructions to install Git Bash on Windows computers and to install or update Bash in Mac computers.
This section provides instructions to install Git Bash on Windows computers and to install or update Bash in Mac computers.

### **Introduction**

Expand All @@ -20,7 +19,7 @@ To run the scripts you need to use a **terminal** that runs the Bash shell. The

**Windows users** will need to install **Git for Windows** on their computer as described below **prior to the course**. Git for Windows includes `Git Bash`, a Windows version of the Unix Bash shell, `ssh` (secure shell) and many other useful programs. `ssh` is used to login to the AWS instances created with the scripts.

**Mac users** will need to install or update Bash on their computer as described below. Mac computers usually have installed both the Zsh shell and the Bash shell; and the **terminal** runs Zsh by default. **Zsh** and the **Bash version** usually installed (3.2.57 or so) **cannot** run the scripts that manage AWS instances. Hence you need to update Bash (the version we updated to in early 2023 was 5.2.15). Note that you can "run" the scripts from a terminal that runs Zsh, but as the first line of each script is this `#!/usr/bin/env bash`, a Zsh terminal will run Bash to **actually run** each script.
**Mac users** will need to install or update Bash on their computer as described below. Mac computers usually have installed both the Bash shell and the Zsh shell. The **terminal** runs Zsh by default. **Zsh** and the **Bash version** usually installed (3.2.57 or so) **cannot** run the scripts that manage AWS instances. Hence you need to update Bash (the version we updated to in early 2023 was 5.2.15). Note that you can "run" the scripts from a terminal that runs Zsh, but as the first line of each script is this `#!/usr/bin/env bash`, a Zsh terminal will run Bash to **actually run** each script.

::: {.callout-note collapse="true"}
## Install Git Bash (Git for Windows) --- Windows users
Expand Down Expand Up @@ -116,7 +115,7 @@ You will see an output like the following:
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
```

Check the Bash version installed with the command:
Check the Bash version installed with the command below:

``` {.bash filename="Code"}
$ bash --version
Expand Down
24 changes: 13 additions & 11 deletions index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ This course teaches how to automatically manage multiple Amazon Web Services (AW

We use the scripts to manage multiple instances for training. When running a workshop, instances are created with 'omics data and software required for the workshop. Each student is granted exclusive access to one instance through the use of an encrypted login key.

To create, configure or delete instances, the scripts require only the names of the instances. The login keys, IP addresses, and domain names used by the instances are created or deleted automatically. Creating over 30 instances takes 10-15 minutes.
To create, configure, stop, start or delete instances, the scripts require only the names of the instances. The login keys, IP addresses, and domain names used by the instances are created or deleted automatically. Creating over 30 instances takes 10-15 minutes.

The target audience of the course is anyone interested in deploying and managing cloud resources for training. While the course is focused on AWS, and particularly Elastic Compute Cloud (EC2) instances, the scripts can be adapted for use with other cloud providers and other types of cloud services.

Expand All @@ -20,7 +20,7 @@ The course is designed for 4-6 hours of self-study, depending on the number of r

The course assumes that learners have **no** prior experience with the AWS concepts and tools covered in the course.

However, learners are expected to have experience with both the Linux/Unix **terminal** and **Bash shell programming** --- the terminal is also known as the **shell** and the **command line interface** or CLI. **Windows users** need to install and configure the Git Bash terminal and **Mac users** need to install or update the Bash shell as instructed in the [Setup](docs/miscellanea/precourse-instructions.qmd) section; see also the [Workshops Organisation](#workshops-organisation) below.
However, learners are expected to have experience with both the Linux/Unix **terminal** and **Bash shell programming** --- the terminal is also known as the **shell** and the **command line interface** or CLI. **Windows users** need to install and configure the Git Bash terminal and **Mac users** need to install or update the Bash shell as instructed in the [Precourse Instructions](docs/miscellanea/precourse-instructions.qmd) section; see also the [Workshops Organisation](#workshops-organisation) below.

Learners are also expected to use a laptop or desktop computer. Tablets and mobile phones are not suitable for taking the course, as you will be using the keyboard to type commands to the terminal or to edit text files.
:::
Expand All @@ -29,16 +29,18 @@ Learners are also expected to use a laptop or desktop computer. Tablets and mobi

"*Cloud computing is the on-demand availability \[through the Internet\] of computer system resources \[such as\] data storage and computing power ... \[that\] relies on a "pay-as-you-go" model ..."\[[Wikipedia](https://en.wikipedia.org/wiki/Cloud_computing)\].* That means that we can **rent** as many computing resources as we need, whenever we need them, and pay only for the time we use them.

The main advantage of Cloud computing is that we don't have to commit too much time and money in managing the IT resources needed to try out a new idea or experiment. Or as is the case of the Cloud-SPAN and similar projects, running hands-on training workshops by providing a properly configured instance (virtual machine) to each participant without having to handle nor invest in hardware resources nor physical space. Instead, an instance in the cloud is first configured with all the data and software tools required by a workshop. This instance is then configured as a template, or Amazon Machine Image (AMI) in AWS terminology. Finally, a number of instances is created from the AMI and configured individually as to domain name, IP address and access login key. Once the course is over, the instances are deleted to stop incurring costs. The AMI is typically preserved to serve as the starting point either (1) to create new instances for a new run of the workshop, or (2) to create a new AMI with updated data or software or both, through creating an instance, updating the data or software, and creating an AMI out of the instance.
The main advantage of Cloud computing is that we don't have to commit too much time and money in managing the IT resources needed to try out a new idea or experiment. Or, as is the case of the Cloud-SPAN and similar projects, running hands-on training workshops by providing a properly configured instance (virtual machine) to each participant without having to handle nor invest in hardware resources nor physical space. Instead, an instance in the cloud is first configured with all the data and software tools required by a workshop. This instance is then configured as a template, or Amazon Machine Image (AMI) in AWS terminology. Finally, a number of instances is created from the AMI and configured individually as to domain name, IP address and access login key. Once the course is over, the instances are deleted to stop incurring costs. The AMI is typically preserved to serve as the starting point either (1) to create new instances for a new run of the workshop, or (2) to create a new AMI with updated data or software or both, through creating an instance, updating the data or software, and creating an AMI out of the instance.

Despite such convenience, managing multiple instances through a Graphical User Interface (GUI), such as the AWS Console, is really cumbersome and error-prone. Hence we developed the scripts, and through their use we have noted a few best practices to manage multiple instances that are covered in the course.
Despite such convenience, managing multiple instances through a Graphical User Interface (GUI), such as the AWS Console, is really cumbersome and error-prone. As the number of workshop participants increases, the problem is magnified. The nature of running workshops means participants may drop out, join the course late or not turn up, resulting in further manual management being required.

To address this problem, we developed the scripts, and through their use we have noted a few best practices to manage multiple instances that are covered in the course.

# Course Overview

| Lesson | Overview |
|----------------------------------------------------|--------------------|
| 1\. [Setting Up Your Cloud and Terminal Environments](https://cloud-span.github.io/cloud-admin-guide-1-setting-work-environments/) | Learn how to create and configure your AWS account for daily work, and how to install and configure the scripts in your Terminal environment to use your AWS account. |
| 2\. [Managing AWS Instances](https://cloud-span.github.io/cloud-admin-guide-2-managing-aws-instances/) | Learn how to configure your AWS account to provide AWS instances with access based on domain names and login keys, how to run the scripts to deploy and manage AWS instances for a workshop, how to create and manage AMIs, and the organisation and workings of the scripts. |
| Lesson | Overview |
|-------------------------------------------|----------|
| [Setting Up Your Cloud and Terminal Environments](docs/lesson01-setting-work-envs/index.qmd) | Learn how to create and configure your AWS account for daily work, and how to install and configure the scripts in your Terminal environment to use your AWS account. |
| [Managing AWS Instances](docs/lesson02-managing-aws-instances/index.qmd) | Learn how to configure your AWS account to provide AWS instances with access based on login keys and **optional** domain names, how to run the scripts to deploy and manage AWS instances for a workshop, how to create and manage AMIs, and the organisation and workings of the scripts. |

# Workshops Organisation {#workshops-organisation}

Expand All @@ -58,16 +60,16 @@ Lesson 2: Managing AWS Instances:\

### Attending a workshop with no AWS account --- using a Cloud-SPAN AWS account

You can attend a workshop without having to create and configure an AWS account (Episodes 1.1 and 1.2). However, as an AWS account is needed to create and manage AWS instances with the scripts, as a workshop attendee, an AWS Linux instance will be made available to you at no cost by the Cloud-SPAN team. This instance will have already **configured** the terminal environment (Episode 1.3) and instances internet access (Episode 2.1) **using** a **Cloud-SPAN AWS account**. To login to your Linux instance you will use the (secure shell) `ssh` program, and hence, prior to the workshop you need to have installed `ssh` on your (Linux or Mac) computer -- **Windows users** can install Git Bash as that will also install `ssh` on your computer, see the [Setup](./setup) section. You will receive instructions to login to your Linux instance at the worshop.
You can attend a workshop without having to create and configure an AWS account (Episodes 1.1 and 1.2). However, as an AWS account is needed to create and manage AWS instances with the scripts, as a workshop attendee, an AWS Linux instance will be made available to you at no cost by the Cloud-SPAN team. This instance will have already **configured** the terminal environment (Episode 1.3) and will use a **Cloud-SPAN AWS account** already configured as to instances internet access (Episode 2.1). To login to your Linux instance you will use the (secure shell) `ssh` program, and hence, prior to the workshop you need to have installed `ssh` on your (Linux or Mac) computer --- **Windows users** should install Git Bash as that will also install `ssh` on your computer, see the [Precourse Instructions](docs/miscellanea/precourse-instructions.qmd) section. You will receive instructions to login to your Linux instance at the worshop.

### Attending a workshop using your AWS account

You can attend a workshop and use **your AWS account** (or the AWS account of your institution) to use (run) the scripts. You need to **complete** the following episodes **before** the workshop (contact the [Cloud-SPAN team](https://cloud-span.york.ac.uk/contact) if you need assistance):
You can attend a workshop and use **your AWS account** (or the AWS account of your institution) to run the scripts. You need to **complete** the following episodes **before** the workshop (contact the [Cloud-SPAN team](https://cloud-span.york.ac.uk/contact-us) if you need assistance):

1.1. Create Your AWS Account\
1.2. Configure Your AWS Account\
1.3. Configure Your Terminal Environment **or**\
1.4. Configure Your AWS CloudShell Environment **and**\
2.1. Configure Instances Internet Access

You can configure the scripts to use your AWS account either **on your computer** (Episode 1.3: using either a Git Bash terminal, a Linux terminal or a Mac terminal) or **on your AWS CloudShell Environment** (Episode 1.4), which is a browser-based Linux terminal. If you opt for configuring the scripts on your computer: **Windows users** need to install Git Bash and **Mac users** need to install or update the Bash shell on their computer prior to the workshop, see details in the [Setup](./docs/miscellanea/precourse-instructions.qmd) section and in the introduction to Lesson 1, [Setting Up Your Cloud and Terminal Environments](./docs/lesson01-setting-work-envs/index.qmd).
You can configure the scripts to use your AWS account either **on your computer** (Episode 1.3: using either a Git Bash terminal, a Linux terminal or a Mac terminal) or **on your AWS CloudShell Environment** (Episode 1.4), which is a browser-based Linux terminal. If you opt for configuring the scripts on your computer: **Windows users** need to install Git Bash and **Mac users** need to install or update the Bash shell on their computer prior to the workshop, see details in the [Precourse Instructions](./docs/miscellanea/precourse-instructions.qmd) section and in the introduction to Lesson 1, [Setting Up Your Cloud and Terminal Environments](./docs/lesson01-setting-work-envs/index.qmd).

0 comments on commit 80b2f33

Please sign in to comment.