Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

production update #36

Merged
merged 17 commits into from
Dec 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Reference: https://github.com/frappe/frappe_docker/blob/main/docs/environment-variables.md

ERPNEXT_VERSION=v15.44.0
ERPNEXT_VERSION=v15.45.5

DB_PASSWORD=123

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/build_bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
run: echo "LATEST_BENCH_RELEASE=$(curl -s 'https://api.github.com/repos/frappe/bench/releases/latest' | jq -r '.tag_name')" >> "$GITHUB_ENV"

- name: Build and test
uses: docker/bake-action@v5.10.0
uses: docker/bake-action@v5.11.0
with:
targets: bench-test

Expand All @@ -47,7 +47,7 @@ jobs:

- name: Push
if: ${{ github.repository == 'frappe/frappe_docker' && github.event_name != 'pull_request' }}
uses: docker/bake-action@v5.10.0
uses: docker/bake-action@v5.11.0
with:
targets: bench
push: true
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docker-build-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
echo "NODE_VERSION=${{ inputs.node_version }}" >> "$GITHUB_ENV"

- name: Build
uses: docker/bake-action@v5.10.0
uses: docker/bake-action@v5.11.0
with:
push: true
env:
Expand Down Expand Up @@ -91,6 +91,6 @@ jobs:

- name: Push
if: ${{ inputs.push }}
uses: docker/bake-action@v5.10.0
uses: docker/bake-action@v5.11.0
with:
push: true
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Then run: `docker compose -f pwd.yml up -d`

After cloning the repo run this command to build multi-architecture images specifically for ARM64.

`docker buildx bake --no-cache --set *.platform=linux/arm64`
`docker buildx bake --no-cache --set "*.platform=linux/arm64"`

and then

Expand Down
2 changes: 1 addition & 1 deletion development/installer.py
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ def create_site_in_bench(args):
"new-site",
f"--db-host=mariadb", # Should match the compose service name
f"--db-type={args.db_type}", # Add the selected database type
f"--no-mariadb-socket",
f"--mariadb-user-host-login-scope=%",
f"--db-root-password=123", # Replace with your MariaDB password
f"--admin-password={args.admin_password}",
]
Expand Down
8 changes: 4 additions & 4 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,26 +144,26 @@ sed -i '/redis/d' ./Procfile
You can create a new site with the following command:

```shell
bench new-site --no-mariadb-socket sitename
bench new-site --mariadb-user-host-login-scope=% sitename
```

sitename MUST end with .localhost for trying deployments locally.

for example:

```shell
bench new-site --no-mariadb-socket development.localhost
bench new-site --mariadb-user-host-login-scope=% development.localhost
```

The same command can be run non-interactively as well:

```shell
bench new-site --mariadb-root-password 123 --admin-password admin --no-mariadb-socket development.localhost
bench new-site --db-root-password 123 --admin-password admin --mariadb-user-host-login-scope=% development.localhost
```

The command will ask the MariaDB root password. The default root password is `123`.
This will create a new site and a `development.localhost` directory under `frappe-bench/sites`.
The option `--no-mariadb-socket` will configure site's database credentials to work with docker.
The option `--mariadb-user-host-login-scope=%` will configure site's database credentials to work with docker.
You may need to configure your system /etc/hosts if you're on Linux, Mac, or its Windows equivalent.

To setup site with PostgreSQL as database use option `--db-type postgres` and `--db-host postgresql`. (Available only v12 onwards, currently NOT available for ERPNext).
Expand Down
112 changes: 112 additions & 0 deletions docs/migrate-from-multi-image-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
## Migrate from multi-image setup

All the containers now use same image. Use `frappe/erpnext` instead of `frappe/frappe-worker`, `frappe/frappe-nginx` , `frappe/frappe-socketio` , `frappe/erpnext-worker` and `frappe/erpnext-nginx`.

Now you need to specify command and environment variables for following containers:

### Frontend

For `frontend` service to act as static assets frontend and reverse proxy, you need to pass `nginx-entrypoint.sh` as container `command` and `BACKEND` and `SOCKETIO` environment variables pointing `{host}:{port}` for gunicorn and websocket services. Check [environment variables](environment-variables.md)

Now you only need to mount the `sites` volume at location `/home/frappe/frappe-bench/sites`. No need for `assets` volume and asset population script or steps.

Example change:

```yaml
# ... removed for brevity
frontend:
image: frappe/erpnext:${ERPNEXT_VERSION:?ERPNext version not set}
command:
- nginx-entrypoint.sh
environment:
BACKEND: backend:8000
SOCKETIO: websocket:9000
volumes:
- sites:/home/frappe/frappe-bench/sites
# ... removed for brevity
```

### Websocket

For `websocket` service to act as socketio backend, you need to pass `["node", "/home/frappe/frappe-bench/apps/frappe/socketio.js"]` as container `command`

Example change:

```yaml
# ... removed for brevity
websocket:
image: frappe/erpnext:${ERPNEXT_VERSION:?ERPNext version not set}
command:
- node
- /home/frappe/frappe-bench/apps/frappe/socketio.js
# ... removed for brevity
```

### Configurator

For `configurator` service to act as run once configuration job, you need to pass `["bash", "-c"]` as container `entrypoint` and bash script inline to yaml. There is no `configure.py` in the container now.

Example change:

```yaml
# ... removed for brevity
configurator:
image: frappe/erpnext:${ERPNEXT_VERSION:?ERPNext version not set}
restart: "no"
entrypoint:
- bash
- -c
command:
- >
bench set-config -g db_host $$DB_HOST;
bench set-config -gp db_port $$DB_PORT;
bench set-config -g redis_cache "redis://$$REDIS_CACHE";
bench set-config -g redis_queue "redis://$$REDIS_QUEUE";
bench set-config -gp socketio_port $$SOCKETIO_PORT;
environment:
DB_HOST: db
DB_PORT: "3306"
REDIS_CACHE: redis-cache:6379
REDIS_QUEUE: redis-queue:6379
SOCKETIO_PORT: "9000"
# ... removed for brevity
```

### Site Creation

For `create-site` service to act as run once site creation job, you need to pass `["bash", "-c"]` as container `entrypoint` and bash script inline to yaml. Make sure to use `--mariadb-user-host-login-scope=%` as upstream bench is installed in container.

The `WORKDIR` has changed to `/home/frappe/frappe-bench` like `bench` setup we are used to. So the path to find `common_site_config.json` has changed to `sites/common_site_config.json`.

Example change:

```yaml
# ... removed for brevity
create-site:
image: frappe/erpnext:${ERPNEXT_VERSION:?ERPNext version not set}
restart: "no"
entrypoint:
- bash
- -c
command:
- >
wait-for-it -t 120 db:3306;
wait-for-it -t 120 redis-cache:6379;
wait-for-it -t 120 redis-queue:6379;
export start=`date +%s`;
until [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".db_host // empty"` ]] && \
[[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_cache // empty"` ]] && \
[[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_queue // empty"` ]];
do
echo "Waiting for sites/common_site_config.json to be created";
sleep 5;
if (( `date +%s`-start > 120 )); then
echo "could not find sites/common_site_config.json with required keys";
exit 1
fi
done;
echo "sites/common_site_config.json found";
bench new-site --mariadb-user-host-login-scope=% --admin-password=admin --db-root-password=admin --install-app erpnext --set-default frontend;

# ... removed for brevity
```
2 changes: 1 addition & 1 deletion docs/setup_for_linux_mac.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ services:
fi
done;
echo "sites/common_site_config.json found";
bench new-site --no-mariadb-socket --admin-password=admin --db-root-password=admin --install-app erpnext --set-default frontend;
bench new-site --mariadb-user-host-login-scope=% --admin-password=admin --db-root-password=admin --install-app erpnext --set-default frontend;

db:
image: mariadb:10.6
Expand Down
8 changes: 4 additions & 4 deletions docs/single-server-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,15 +177,15 @@ Create sites `one.example.com` and `two.example.com`:
```shell
# one.example.com
docker compose --project-name erpnext-one exec backend \
bench new-site --no-mariadb-socket --mariadb-root-password changeit --install-app erpnext --admin-password changeit one.example.com
bench new-site --mariadb-user-host-login-scope=% --db-root-password changeit --install-app erpnext --admin-password changeit one.example.com
```

You can stop here and have a single bench single site setup complete. Continue to add one more site to the current bench.

```shell
# two.example.com
docker compose --project-name erpnext-one exec backend \
bench new-site --no-mariadb-socket --mariadb-root-password changeit --install-app erpnext --admin-password changeit two.example.com
bench new-site --mariadb-user-host-login-scope=% --db-root-password changeit --install-app erpnext --admin-password changeit two.example.com
```

#### Create second bench
Expand Down Expand Up @@ -236,10 +236,10 @@ Create sites `three.example.com` and `four.example.com`:
```shell
# three.example.com
docker compose --project-name erpnext-two exec backend \
bench new-site --no-mariadb-socket --mariadb-root-password changeit --install-app erpnext --admin-password changeit three.example.com
bench new-site --mariadb-user-host-login-scope=% --db-root-password changeit --install-app erpnext --admin-password changeit three.example.com
# four.example.com
docker compose --project-name erpnext-two exec backend \
bench new-site --no-mariadb-socket --mariadb-root-password changeit --install-app erpnext --admin-password changeit four.example.com
bench new-site --mariadb-user-host-login-scope=% --db-root-password changeit --install-app erpnext --admin-password changeit four.example.com
```

#### Create custom domain to existing site
Expand Down
85 changes: 85 additions & 0 deletions docs/site-operations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Site operations

> 💡 You should setup `--project-name` option in `docker-compose` commands if you have non-standard project name.

## Setup new site

Note:

- Wait for the `db` service to start and `configurator` to exit before trying to create a new site. Usually this takes up to 10 seconds.

```sh
docker-compose exec backend bench new-site --mariadb-user-host-login-scope=% --db-root-password <db-password> --admin-password <admin-password> <site-name>
```

If you need to install some app, specify `--install-app`. To see all options, just run `bench new-site --help`.

To create Postgres site (assuming you already use [Postgres compose override](images-and-compose-files.md#overrides)) you need have to do set `root_login` and `root_password` in common config before that:

```sh
docker-compose exec backend bench set-config -g root_login <root-login>
docker-compose exec backend bench set-config -g root_password <root-password>
```

Also command is slightly different:

```sh
docker-compose exec backend bench new-site --mariadb-user-host-login-scope=% --db-type postgres --admin-password <admin-password> <site-name>
```

## Push backup to S3 storage

We have the script that helps to push latest backup to S3.

```sh
docker-compose exec backend push_backup.py --site-name <site-name> --bucket <bucket> --region-name <region> --endpoint-url <endpoint-url> --aws-access-key-id <access-key> --aws-secret-access-key <secret-key>
```

Note that you can restore backup only manually.

## Edit configs

Editing config manually might be required in some cases,
one such case is to use Amazon RDS (or any other DBaaS).
For full instructions, refer to the [wiki](<https://github.com/frappe/frappe/wiki/Using-Frappe-with-Amazon-RDS-(or-any-other-DBaaS)>). Common question can be found in Issues and on forum.

`common_site_config.json` or `site_config.json` from `sites` volume has to be edited using following command:

```sh
docker run --rm -it \
-v <project-name>_sites:/sites \
alpine vi /sites/common_site_config.json
```

Instead of `alpine` use any image of your choice.

## Health check

For socketio and gunicorn service ping the hostname:port and that will be sufficient. For workers and scheduler, there is a command that needs to be executed.

```shell
docker-compose exec backend healthcheck.sh --ping-service mongodb:27017
```

Additional services can be pinged as part of health check with option `-p` or `--ping-service`.

This check ensures that given service should be connected along with services in common_site_config.json.
If connection to service(s) fails, the command fails with exit code 1.

---

For reference of commands like `backup`, `drop-site` or `migrate` check [official guide](https://frappeframework.com/docs/v13/user/en/bench/frappe-commands) or run:

```sh
docker-compose exec backend bench --help
```

## Migrate site

Note:

- Wait for the `db` service to start and `configurator` to exit before trying to migrate a site. Usually this takes up to 10 seconds.

```sh
docker-compose exec backend bench --site <site-name> migrate
```
Loading
Loading