Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP fix: workshop updates for 2025 #17

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
10 changes: 2 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@
- Optionally [Yarn](https://yarnpkg.com/) or [Pnpm](https://pnpm.io/)
- [GitHub](https://github.com/) account
- IDE of your choice
- A [Fly.io](https://fly.io/) account with CLI installed
- Check with: `fly help`
- `Note` No need to set up the Credit card, we will use free tier

### Getting the repo

Expand Down Expand Up @@ -64,16 +61,13 @@ Join the [Discord channel](https://discord.gg/xr95Aap5)
| 📖 [Automate your workspace with local plugins and custom generators](./exercises/advanced/custom-plugins.md) |
| `☕ Break` |
| 📖 [Learn how to write and test your complex generators](./exercises/advanced/complex-generators.md) |
| 📖 [Write advanced deployment targets using a custom executor](./exercises/advanced/deploy-target-and-custom-executor.md) |
| `☕ Break` |
| 📖 [Set up CI for your pull requests, connect to Nx Cloud, enable remote caching and the GitHub integration](./exercises/advanced/setup-ci-and-connect-nx-cloud.md) |
| 📖 [Nx Caching deep dive: Strategies for debugging cache misses, optimization strategies, and fine-tuning cache inputs and outputs](./exercises/advanced/caching-deep-dive.md) |
| 📖 [Set up continuous deployment pipeline for affected applications](./exercises/advanced/continuous-deployment.md) |
| `☕ Break` |
| 📖 [Nx Caching deep dive: Strategies for debugging cache misses, optimization strategies, and fine-tuning cache inputs and outputs](./exercises/advanced/caching-deep-dive.md) |
| 📖 [Configure task distribution on CI with Nx Agents, including exploring custom launch templates and dynamic agent scaling.](./exercises/advanced/nx-agents.md) |
| `☕ Break` |
| 📖 [Leverage Nx Crystal plugins and Nx Atomizer to configure task splitting for improving CI distribution and speed](./exercises/advanced/atomizer.md) |
| 📖 [Explore flaky task detection](./exercises/advanced/flaky-tasks.md) |
| `Bonus:` [Infer Fly.io deploy target](./exercises/advanced/infer-target.md) |
| `Bonus:` [Nx import, CodeOwners, and Conformance](./exercises/advanced/bonus.md) |

---
Expand Down
16 changes: 9 additions & 7 deletions exercises/advanced/bonus.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,26 @@

## 🏋️‍♀️  Steps:

### 1. Use Nx import to import an existing Angular project
### 1. Use Nx import to import an existing Vite project

As a first step, create a new Angular project in some folder outside of the current workspace. Note use Angular 18 for now as Nx support is for v19 is about to drop but might not be out yet by the time of this workshop.
As a first step, create a new project in some folder outside of the current workspace. This project does not need to use the same technology stack as what is in the existing monorepo. For this example, you could use [Vite to create a React project](https://vite.dev/guide/#scaffolding-your-first-vite-project).

```bash
npx @angular/cli@18 new my-app
npx create vite@latest my-vite-app --template react
```

Then run the following command to import it into the current workspace:

```bash
npx nx import <path-to-ng-project>
npx nx import <path-to-vite-project>
```

Confirm the installation of the `@nx/angular` and `@nx/jest` plugin.
Once installed, try to serve the app. Nx is directly running the script of the `package.json` from the imported Angular app.
Confirm the installation of the `@nx/jest` plugin. The `@nx/angular` plugin will not be needed for the Vite project.
Once installed, try to serve the app. Nx is directly running the script of the `package.json` from the imported app.

You can even completely remove all `package.json > scripts` from the imported Angular project as the `@nx/angular` Crystal plugin will automatically detect the `angular.json` and know how to run the various targets.
You can even completely remove all `package.json > scripts` from the imported Vite project as the `@nx/vite` Crystal plugin will automatically detect the `vite.config.ts` and know how to run the various targets.

Now, Nx is aware of the Vite project. It will appear in the `nx graph`, and commands that run tasks like `nx run-many -t build` will automatically run that target for the Vite project as well as the other projects in the workspace.

### 2. Activate Powerpack

Expand Down
29 changes: 15 additions & 14 deletions exercises/advanced/caching-deep-dive.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ The hash signature is based on following variables:

First parameter is obvious. If you run `nx build movies-app`, Nx will store the command signature in the hash details and use it to calculate the final hash. Every time the command changes ever so slightly, for example when running it with `--prod` or `--verbose`, will change the final hash stamp.

Each tasks is defined by list of inputs that define the context upon which we run the task command. Inputs can be defined in `nx.json` as `targetDefaults`, in `project.json` and `package.json` target's configuration or any combination of the above. The best way to get the full list of inputs is by using `Nx Console` IDE extension or by running `nx graph`.
Each task is defined by list of inputs that define the context upon which we run the task command. Inputs can be defined in `nx.json` as `targetDefaults`, in `project.json` and `package.json` target's configuration or any combination of the above. The best way to get the full list of inputs is by using `Nx Console` IDE extension or by running `nx graph`. To see a list of inputs for targets of a specific project such as `movies-api`, you can run `nx show project movies-api`.

![Lint task inputs](../assets/lint-inputs.png)

Some files are explicitly specified, while others are represented by `namedInputs`. Named inputs are variables that encapsulate common file groups. The two typical ones are:
- `default` - includes all the files in the project
- `production` - exludes all files that are not production relevant like `spec.ts` files and lint and test configurations.
- `production` - excludes all files that are not production relevant like `spec.ts` files and lint and test configurations.

Any change to files included in the inputs will result in the new hash signature.

Expand All @@ -34,12 +34,15 @@ Try running the `build` for `movies-api`. Running the build for the second time

Besides the file contents inputs, our tasks can also depend on runtime and environment variables and even outputs of their dependencies. As we will see in the next lab, this will become crucial to limit the cases when `deploy` target should run.

Let's try playing with those inputs. Add to build target in `movies-api` `project.json` following runtime input:
Let's try playing with those inputs. Add the following runtime input to the `production` named input in `nx.json`:

```jsonc
"build": {
"namedInputs": {
// ...
"inputs": [{ "runtime": "date" }]
"production" [
// ...
{ "runtime": "date" }
]
// ...
}
```
Expand All @@ -49,9 +52,12 @@ Try running `nx build movies-api` several times in a row. Notice how we are no l
Replace that input now with a more controlled `environment` input:

```jsonc
"build": {
"namedInputs": {
// ...
"inputs": [{ "env": "MY_SECRET" }]
"production" [
// ...
{ "env": "MY_SECRET" }
]
// ...
}
```
Expand All @@ -68,7 +74,7 @@ Whenever our task depends on environment where it's being invoked or upon some e

Sometimes we get cache miss despite expecting a cache hit. Investigating why certain task was not retrieved from the cache can seem a difficult task. Let's learn some steps that will help us investigate cache misses.

By prefixing your command with `NX_NATIVE_LOGGING=trace NX_DAEMON=false ...` your task will be printed out with the full Rust trace which will include all the details about hashing. Look for the following section - `hashes=NapiDashMap({ ... })`. It containst the large object with HashDetails of all the tasks involved in your tast run.
By prefixing your command with `NX_NATIVE_LOGGING=trace NX_DAEMON=false ...` your task will be printed out with the full Rust trace which will include all the details about hashing. Look for the following section - `hashes=NapiDashMap({ ... })`. It contains the large object with HashDetails of all the tasks involved in your test run.

Finally, in case of cache hit, you will see the block starting with:
```bash
Expand Down Expand Up @@ -102,14 +108,9 @@ Each executor is represented by the NPM package that contains it, as well as its
As you will learn in the next lab, there is one input configuration that can limit this.

For the time being, it's important to be aware of this behavior.
To test it, try to install some package that is not used by any task e.g. `npm i -D jquery`.

If you run your `lint` now the results should still be retrieved from the cache. But if you run `deploy` notice how the command is running in full.

> ⚠️&nbsp;&nbsp;Don't forget to revert the changes to package.json and the lock file. We don't need jQuery in there 😃

### 5. Final thoughts

In this lab we learned how to optimize our task inputs to get the best caching results. In the next lab we will use some of those learnings to implement the continuous deployment in the most optimal way.

## [➡️ Next lab ➡️](./continuous-deployment.md)
## [➡️ Next lab ➡️](./nx-agents.md)
34 changes: 22 additions & 12 deletions exercises/advanced/complex-generators.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ function getScopes(projectMap: Map<string, ProjectConfiguration>) {
const allScopes: string[] = Array.from(projectMap.values())
.map((project) => {
if (project.tags) {
const scopes = project.tags.filter((tag: string) => tag.startsWith('scope:'));
const scopes = project.tags?.filter((tag: string) => tag.startsWith('scope:')) ?? [];
return scopes;
}
return [];
Expand Down Expand Up @@ -130,7 +130,7 @@ import {
getProjects,
} from '@nx/devkit';

export default async function (tree: Tree) {
export async function generator(tree: Tree) {
const scopes = getScopes(getProjects(tree));
updateSchemaJson(tree, scopes);
updateSchemaInterface(tree, scopes);
Expand All @@ -141,7 +141,7 @@ function getScopes(projectMap: Map<string, ProjectConfiguration>) {
const projects: any[] = Array.from(projectMap.values());
const allScopes: string[] = projects
.map((project) =>
project.tags.filter((tag: string) => tag.startsWith('scope:'))
project.tags?.filter((tag: string) => tag.startsWith('scope:')) ?? []
)
.reduce((acc, tags) => [...acc, ...tags], [])
.map((scope: string) => scope.slice(6));
Expand Down Expand Up @@ -220,6 +220,8 @@ their scope files.
}
```

You may need to install husky with `npm install -D husky`.

</details>

### 6. `✨ BONUS` Create a unit test to verify functionality
Expand All @@ -234,22 +236,30 @@ Create a test to automate verification of this generator in `libs/internal-plugi
```typescript
import { readJson, Tree } from '@nx/devkit';
import { createTreeWithEmptyWorkspace } from '@nx/devkit/testing';
import { libraryGenerator } from '@nx/js/generators';
import { libraryGenerator } from '@nx/js';
import { generatorGenerator, pluginGenerator } from '@nx/plugin/generators';
import { readFileSync } from 'fs';
import { join } from 'path';

import { Linter } from '@nx/eslint';
import generator from './generator';
import { generator } from './generator';

describe('update-scope-schema generator', () => {
let appTree: Tree;

beforeEach(async () => {
appTree = createTreeWithEmptyWorkspace();
await addUtilLibProject(appTree);
await libraryGenerator(appTree, { name: 'foo', tags: 'scope:foo' });
await libraryGenerator(appTree, { name: 'bar', tags: 'scope:bar' });
await libraryGenerator(appTree, {
name: 'foo',
tags: 'scope:foo',
directory: 'foo',
});
await libraryGenerator(appTree, {
name: 'bar',
tags: 'scope:bar',
directory: 'bar',
});
});

it('should adjust the util-lib generator based on existing projects', async () => {
Expand All @@ -272,7 +282,7 @@ describe('update-scope-schema generator', () => {
'libs/internal-plugin/src/generators/util-lib/schema.d.ts',
'utf-8'
);
expect(schemaInterface).toContain(`export interface Schema {
expect(schemaInterface).toContain(`export interface UtilLibGeneratorSchema {
name: string;
directory: 'foo' | 'bar';
}`);
Expand All @@ -282,18 +292,17 @@ describe('update-scope-schema generator', () => {
async function addUtilLibProject(tree: Tree) {
await pluginGenerator(tree, {
name: 'internal-plugin',
directory: 'libs/internal-plugin'
directory: 'libs/internal-plugin',
skipTsConfig: false,
unitTestRunner: 'jest',
linter: Linter.EsLint,
compiler: 'tsc',
skipFormat: false,
skipLintChecks: false,
minimal: true,
});
await generatorGenerator(tree, {
name: 'util-lib',
directory: 'libs/internal-plugin/src/generators/util-lib',
path: 'libs/internal-plugin/src/generators/util-lib',
unitTestRunner: 'jest',
});
const filesToCopy = [
Expand All @@ -308,6 +317,7 @@ async function addUtilLibProject(tree: Tree) {
);
}
}

```

</details>
Expand All @@ -318,4 +328,4 @@ You learned how to generate and test complex generators. In the next step we wil

⚠️&nbsp;&nbsp;Don't forget to commit everything before you move on.

## [➡️ Next lab ➡️](./deploy-target-and-custom-executor.md)
## [➡️ Next lab ➡️](./setup-ci-and-connect-nx-cloud.md)
4 changes: 3 additions & 1 deletion exercises/advanced/custom-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Try to run your generator to see what changes are being made (you can append `--

### 5. Extend other generators

We can call other generators inside of our custom generator. Import the `@nx/js:library` generator and call it inside of the default exported function of `libs/internal-plugin/src/generators/util-lib/generator.ts`
We can call other generators inside of our custom generator. Add `@nx/js` as a dependency of `internal-plugin`. Then, import the `@nx/js:library` generator and call it inside of the default exported function of `libs/internal-plugin/src/generators/util-lib/generator.ts`

<details>
<summary>🐳&nbsp;&nbsp;Hint</summary>
Expand Down Expand Up @@ -108,6 +108,8 @@ asked about which bundler to use when creating libs). The generator should gener

- `movies`-> `libs/movies/{lib name}`

The generator should also add the directory as a prefix to the lib name.

<details>
<summary>🐳&nbsp;&nbsp;Hint</summary>

Expand Down
6 changes: 3 additions & 3 deletions exercises/advanced/flaky-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ Make a couple of arbitrary changes and push them to trigger to a new CI run.

If you succeed you should see a note at the top of the dashboard indicating the presence of some flaky tasks.

![flaky tasks](images/flaky-tasks.png)
![flaky tasks](../images/flaky-tasks.png)

Also if you open the the project containing the flaky task you should see how it got retried:

![flaky task retry](images/flaky-task-retry.png)
![flaky task retry](../images/flaky-task-retry.png)

## That's it! 🎉

You made it all the way through! Are you up for some [✨ bonus tasks ✨](./infer-target.md)?
You made it all the way through! Are you up for some [✨ bonus tasks ✨](./bonus.md)?
49 changes: 9 additions & 40 deletions exercises/advanced/nx-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@ To enable Nx Agents, let's adjust the CI config. There should be already a comme
```

</details>

Also, for now, uncomment the `deploy` target. We'll come back to it later in this lab.

Now commit the changes and push them to trigger a new CI run.

Expand Down Expand Up @@ -56,32 +54,9 @@ distribute-on:

</details>

### 3. Create a custom template to install Fly.io
### 3. `✨ BONUS` Create a custom template to install other dependencies

Re-enable the `deploy` target in your CI config and check the output log. You should probably see an error about not being able to find `fly` in the PATH.

```bash
/bin/sh: 1: flyctl: not found
```

This is because the [default Nx Agents templates](https://github.com/nrwl/nx-cloud-workflows/tree/main/launch-templates) do not include the `fly` CLI.

To use the Fly CLI on Nx Agents we need to create a custom launch template. Use the [docs](https://nx.dev/ci/reference/launch-templates) as a reference to create a new template in `.nx/workflows/agents.yaml`.

<details>
<summary>🐳&nbsp;&nbsp;Hint: Installing Fly.io</summary>

```
curl -L https://fly.io/install.sh | sh
```

To add it to the PATH, you can use the following command:

```
echo PATH="$HOME/.fly/bin:$PATH" >> $NX_CLOUD_ENV
```

</details>
To customize the setup steps of Nx Agents, you can create a custom Launch Template. This is useful when you need to install additional dependencies or tools. Use the [docs](https://nx.dev/ci/reference/launch-templates) as a reference to create a new template in `.nx/workflows/agents.yaml`.

<details>
<summary>🐳&nbsp;&nbsp;Solution (custom launch template)</summary>
Expand All @@ -107,14 +82,15 @@ launch-templates:
paths: |
'../.cache/Cypress'
base-branch: 'main'
- name: Install Pnpm 9
script: |
npm install -g pnpm@9 --force
- name: Install Node Modules
uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/install-node-modules/main.yaml'
- name: Install Browsers (if needed)
uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/install-browsers/main.yaml'
- name: Install Fly.io
script: |
curl -L https://fly.io/install.sh | sh
echo PATH="$HOME/.fly/bin:$PATH" >> $NX_CLOUD_ENV
# Install any other dependencies here
# After each Nx Agent reaches this point, it will be ready to execute tasks
```

</details>
Expand All @@ -132,17 +108,10 @@ Inspect the Nx Cloud dashboard. You should see your new launch template being us

![custom-launch-template](../images/nx-cloud-custom-launch-template.png)

> ⚠️&nbsp;&nbsp;Your Fly.io deployment might still not work. Check the logs on Nx CLoud to see why.

Use the following [docs page](https://nx.dev/ci/reference/launch-templates#pass-environment-variables-to-agents) to fix the issue.

<details>
<summary>🐳&nbsp;&nbsp;Solution</summary>

You need to forward the environment variables to the Nx Agents by using the `--with-env-vars` flag:
If needed, you can forward environment variables to the Nx Agents by using the `--with-env-vars` flag on the `start-ci-run` command:

```bash
--with-env-vars="SURGE_DOMAIN_STORE,SURGE_TOKEN,FLY_API_TOKEN"
--with-env-vars="MY_SECRET_TOKEN,MY_OTHER_TOKEN"
```

</details>
Expand Down
2 changes: 1 addition & 1 deletion exercises/advanced/setup-ci-and-connect-nx-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,6 @@ git pull

Now run any test, lint or build command. Notice how it's being retreived from the remote cache.

![nx-cloud-cache](images/nx-cloud-cache.png)
![nx-cloud-cache](../images/nx-cloud-cache.png)

## [➡️ Next lab ➡️](./caching-deep-dive.md)