From 0ee0a0c6ae13588e82edf1cf9e8cb9857c94c31b Mon Sep 17 00:00:00 2001 From: Sebastiaan Huber Date: Fri, 28 Jun 2024 23:01:36 +0200 Subject: [PATCH] Docs: Rework the installation section (#6455) The `verdi setup` and `verdi quicksetup` commands have been deprecated and replaced by `verdi profile setup` and `verdi presto`. The installation docs were heavily outdated and the flow was scattered. The biggest change is that there now is a "quick install guide" that relies on `verdi presto` to provide an install route that is fool proof and will work on almost any system in a minimal amount of commands. Then there is the "complete installation guide" that provides all the details necessary to fully customize an installation. --- docs/source/conf.py | 6 - docs/source/howto/daemon.rst | 13 + docs/source/howto/index.rst | 1 + docs/source/howto/installation.rst | 10 +- docs/source/howto/interact.rst | 22 +- docs/source/howto/ssh.rst | 2 +- docs/source/index.rst | 45 +- docs/source/installation/docker.rst | 181 ++++++++ docs/source/installation/guide_complete.rst | 408 ++++++++++++++++++ docs/source/installation/guide_quick.rst | 85 ++++ docs/source/installation/index.rst | 97 +++++ .../troubleshooting.rst | 14 +- docs/source/intro/cheatsheet.rst | 16 - docs/source/intro/get_started.rst | 98 ----- docs/source/intro/index.rst | 23 +- docs/source/intro/install_conda.rst | 163 ------- docs/source/intro/install_system.rst | 298 ------------- docs/source/intro/installation.rst | 295 ------------- docs/source/intro/run_docker.rst | 236 ---------- docs/source/redirects.txt | 10 +- docs/source/reference/cheatsheet.rst | 16 + .../cheatsheet}/cheatsheet.png | Bin .../cheatsheet}/cheatsheet.svg | 0 .../cheatsheet}/cheatsheet_v.pdf | Bin docs/source/reference/index.rst | 1 + docs/source/topics/storage.rst | 4 +- docs/source/tutorials/basic.md | 2 +- 27 files changed, 879 insertions(+), 1167 deletions(-) create mode 100644 docs/source/howto/daemon.rst create mode 100644 docs/source/installation/docker.rst create mode 100644 docs/source/installation/guide_complete.rst create mode 100644 docs/source/installation/guide_quick.rst create mode 100644 docs/source/installation/index.rst rename docs/source/{intro => installation}/troubleshooting.rst (97%) delete mode 100644 docs/source/intro/cheatsheet.rst delete mode 100644 docs/source/intro/get_started.rst delete mode 100644 docs/source/intro/install_conda.rst delete mode 100644 docs/source/intro/install_system.rst delete mode 100644 docs/source/intro/installation.rst delete mode 100644 docs/source/intro/run_docker.rst create mode 100644 docs/source/reference/cheatsheet.rst rename docs/source/{intro/_cheatsheet => reference/cheatsheet}/cheatsheet.png (100%) rename docs/source/{intro/_cheatsheet => reference/cheatsheet}/cheatsheet.svg (100%) rename docs/source/{intro/_cheatsheet => reference/cheatsheet}/cheatsheet_v.pdf (100%) diff --git a/docs/source/conf.py b/docs/source/conf.py index 9cfaae4847..745ed79c30 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -49,15 +49,9 @@ # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [ - 'datatypes/**', 'developer_guide/**', - 'get_started/**', - 'howto/installation_more/index.rst', 'import_export/**', 'internals/global_design.rst', - 'internals/orm.rst', - 'scheduler/index.rst', - 'working_with_aiida/**', ] # The name of the Pygments (syntax highlighting) style to use. diff --git a/docs/source/howto/daemon.rst b/docs/source/howto/daemon.rst new file mode 100644 index 0000000000..3076547206 --- /dev/null +++ b/docs/source/howto/daemon.rst @@ -0,0 +1,13 @@ + +.. _how-to:manage-daemon: + +How to manage the daemon +------------------------ + +The AiiDA daemon process runs in the background and takes care of processing your submitted calculations and workflows, checking their status, retrieving their results once they are finished and storing them in the AiiDA database. + +The AiiDA daemon is controlled using three simple commands: + +* ``verdi daemon start``: start the daemon +* ``verdi daemon status``: check the status of the daemon +* ``verdi daemon stop``: stop the daemon diff --git a/docs/source/howto/index.rst b/docs/source/howto/index.rst index 28fd41d2df..b405bc3f8a 100644 --- a/docs/source/howto/index.rst +++ b/docs/source/howto/index.rst @@ -6,6 +6,7 @@ How-To Guides :maxdepth: 1 interact + daemon plugins_install run_codes run_workflows diff --git a/docs/source/howto/installation.rst b/docs/source/howto/installation.rst index a5dc6c00a1..50547b8f21 100644 --- a/docs/source/howto/installation.rst +++ b/docs/source/howto/installation.rst @@ -14,7 +14,7 @@ Creating profiles ----------------- Each AiiDA installation can have multiple profiles, each of which can have its own individual database and file repository to store the contents of the :ref:`provenance graph`. Profiles allow you to run multiple projects completely independently from one another with just a single AiiDA installation and at least one profile is required to run AiiDA. -A new profile can be created using :ref:`verdi quicksetup` or :ref:`verdi setup`, which works similar to the former but gives more control to the user. +A new profile can be created using :ref:`verdi presto` or :ref:`verdi profile setup`, which works similar to the former but gives more control to the user. Listing profiles ---------------- @@ -296,7 +296,7 @@ Isolating multiple instances An AiiDA instance is defined as the installed source code plus the configuration folder that stores the configuration files with all the configured profiles. It is possible to run multiple AiiDA instances on a single machine, simply by isolating the code and configuration in a virtual environment. -To isolate the code, make sure to install AiiDA into a virtual environment, e.g., with conda or venv, as described :ref:`here `. +To isolate the code, make sure to install AiiDA into a virtual environment, e.g., with conda or venv. Whenever you activate this particular environment, you will be running the particular version of AiiDA (and all the plugins) that you installed specifically for it. This is separate from the configuration of AiiDA, which is stored in the configuration directory which is always named ``.aiida`` and by default is stored in the home directory. @@ -619,12 +619,12 @@ Alternatively to the CLI command, one can also manually create a backup. This re .. _how-to:installation:backup:restore: Restoring data from a backup -================================== +============================ Restoring a backed up AiiDA profile requires: * restoring the profile information in the AiiDA ``config.json`` file. Simply copy the`profiles` entry from - the backed up `config.json`to the one of the running AiiDA instance (see `verdi status` for exact location). + the backed up ``config.json`` to the one of the running AiiDA instance (see ``verdi status`` for exact location). Some information (e.g. the database parameters) might need to be updated. * restoring the data of of the backed up profile according to the ``config.json`` entry. @@ -642,7 +642,7 @@ To test if the restoration worked, run ``verdi -p status`` to ver **PostgreSQL database** - To restore the PostgreSQL database from the ``db.psql`` file that was backed up, first you should create an empty database following the instructions described in :ref:`database ` skipping the ``verdi setup`` phase. + To restore the PostgreSQL database from the ``db.psql`` file that was backed up, first you should create an empty database following the instructions described in :ref:`the installation guide `. The backed up data can then be imported by calling: .. code-block:: console diff --git a/docs/source/howto/interact.rst b/docs/source/howto/interact.rst index 530bcef915..ab6c8c6d85 100644 --- a/docs/source/howto/interact.rst +++ b/docs/source/howto/interact.rst @@ -125,6 +125,18 @@ Interactive notebooks ===================== Similar to :ref:`interactive shells `, AiiDA is also directly compatbile with interactive Python notebooks, such as `Jupyter `_. +To install the required Python packages, install ``aiida-core`` with the ``notebook`` extra, e.g. run: + +.. code-block:: console + + pip install aiida-core[notebook] + +You should now be able to start a Jupyter notebook server: + +.. code-block:: console + + jupyter notebook + To use AiiDA's Python API in a notebook, first a profile has to be loaded: .. code-block:: ipython @@ -142,8 +154,14 @@ The same can be accomplished using the following magic statement: %load_ext aiida %aiida -This magic line will replicate the same environment as :ref:`the interactive shell ` provided by ``verdi shell``. -However, it does require some one-time installation, as detailed in the section on how to :ref:`intro:install:jupyter`. +This magic line replicates the same environment as :ref:`the interactive shell ` provided by ``verdi shell``. + +It is also possible to run ``verdi`` commands inside the notebook, for example: + +.. code-block:: ipython + + %verdi status + .. _how-to:interact-restapi: diff --git a/docs/source/howto/ssh.rst b/docs/source/howto/ssh.rst index 9a2e24a366..59fe9093b6 100644 --- a/docs/source/howto/ssh.rst +++ b/docs/source/howto/ssh.rst @@ -269,6 +269,6 @@ Using kerberos tokens If the remote machine requires authentication through a Kerberos token (that you need to obtain before using ssh), you typically need to * install ``libffi`` (``sudo apt-get install libffi-dev`` under Ubuntu) -* install the ``ssh_kerberos`` extra during the installation of aiida-core (see :ref:`intro:install:setup`). +* install the ``ssh_kerberos`` extra during the installation of aiida-core (see :ref:`installation:guide-complete:python-package:optional-requirements`). If you provide all necessary ``GSSAPI`` options in your ``~/.ssh/config`` file, ``verdi computer configure`` should already pick up the appropriate values for all the gss-related options. diff --git a/docs/source/index.rst b/docs/source/index.rst index b9a218b023..e4c9d7b94d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,9 +1,5 @@ :sd_hide_title: -################################# -Welcome to AiiDA's documentation! -################################# - .. grid:: :reverse: :gutter: 2 3 3 3 @@ -23,7 +19,8 @@ Welcome to AiiDA's documentation! .. rubric:: AiiDA - An open-source Python infrastructure to help researchers with automating, managing, persisting, sharing and reproducing the complex workflows associated with modern computational science and all associated data (see :ref:`features`). + An open-source Python infrastructure to help researchers with automating, managing, persisting, sharing and + reproducing the complex workflows associated with modern computational science and all associated data (see :ref:`features`). **aiida-core version:** |release| @@ -32,15 +29,32 @@ Welcome to AiiDA's documentation! .. grid:: 1 2 2 2 :gutter: 3 - .. grid-item-card:: :fa:`rocket;mr-1` Getting Started + .. grid-item-card:: :fa:`circle-play;mr-1` Introduction + :text-align: center + :shadow: md + + Overview of what AiiDA is and what it can do. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-ref:: intro/index + :ref-type: doc + :click-parent: + :expand: + :color: primary + :outline: + + To the introduction + + .. grid-item-card:: :fa:`rocket;mr-1` Installation :text-align: center :shadow: md - AiiDA installation, configuration and troubleshooting. + Installation guides and troubleshooting. +++++++++++++++++++++++++++++++++++++++++++++ - .. button-ref:: intro/get_started + .. button-ref:: installation/index :ref-type: doc :click-parent: :expand: @@ -87,7 +101,7 @@ Welcome to AiiDA's documentation! :text-align: center :shadow: md - Background information on AiiDA's underlying concepts. + Background information on AiiDA concepts. +++++++++++++++++++++++++++++++++++++++++++++ @@ -104,7 +118,7 @@ Welcome to AiiDA's documentation! :text-align: center :shadow: md - Comprehensive documentation of AiiDA components: command-line interface, Python interface, and RESTful API. + Comprehensive documentation of CLI, Python API and REST API. +++++++++++++++++++++++++++++++++++++++++++++ @@ -121,7 +135,7 @@ Welcome to AiiDA's documentation! :text-align: center :shadow: md - Notes on AiiDA's design and architecture aimed at core developers. + Detailed information on AiiDA's design and architecture. +++++++++++++++++++++++++++++++++++++++++++++ @@ -154,15 +168,16 @@ Welcome to AiiDA's documentation! :hidden: intro/index + installation/index tutorials/index howto/index topics/index reference/index internals/index -*********** +=========== How to cite -*********** +=========== If you use AiiDA for your research, please cite the following work: @@ -175,9 +190,9 @@ If the ADES concepts are referenced, please also cite: .. highlights:: Pizzi, Giovanni, Andrea Cepellotti, Riccardo Sabatini, Nicola Marzari, and Boris Kozinsky. *AiiDA: automated interactive infrastructure and database for computational science*, Computational Materials Science **111**, 218-230 (2016); DOI: `10.1016/j.commatsci.2015.09.013 `_ -**************** +================ Acknowledgements -**************** +================ AiiDA is supported by the `MARVEL National Centre of Competence in Research`_, the `MaX European Centre of Excellence`_ and by a number of other supporting projects, partners and institutions, whose complete list is available on the `AiiDA website acknowledgements page`_. diff --git a/docs/source/installation/docker.rst b/docs/source/installation/docker.rst new file mode 100644 index 0000000000..d1a2a2f22e --- /dev/null +++ b/docs/source/installation/docker.rst @@ -0,0 +1,181 @@ +.. _installation:docker: + +====== +Docker +====== + +The AiiDA team maintains a number of `Docker `_ images on `Docker Hub `_. +These images contain a fully pre-configured AiiDA environment which make it easy to get started using AiiDA if you are familiar with Docker. + +Currently, there are three image variants: + +.. grid:: auto + :gutter: 3 + + .. grid-item-card:: :fa:`bullseye;mr-1` aiida-core-base + :text-align: center + :shadow: md + + This is the base image. + It comes just with the ``aiida-core`` package installed. + It expects that the RabbitMQ and PostgreSQL services are provided. + + + .. grid-item-card:: :fa:`puzzle-piece;mr-1` aiida-core-with-services + :text-align: center + :shadow: md + + This images builds on top of ``aiida-core-base`` but also installs RabbitMQ and PostgreSQL as services inside the image. + This image is therefore complete and ready to be used. + + + .. grid-item-card:: :fa:`code;mr-1` aiida-core-dev + :text-align: center + :shadow: md + + This image builds on top of ``aiida-core-with-services`` with the only difference that the ``aiida-core`` package is installed from source in editable mode. + This makes this image suitable for development of the ``aiida-core`` package. + + +Start a container +================= + +To start a container from an image, run: + +.. code-block:: console + + docker run -it --name aiida aiidateam/aiida-core-with-services:latest bash + +In this example, the ``aiida-core-with-services`` image is started where ``latest`` refers to the latest tag. +The ``--name`` option is optional but is recommended as it makes it easier to restart the same container at a later point in time. +The ``-it`` option is used to run the container in interactive mode and to allocate a pseudo-TTY. +After the container start up has finished, a bash shell inside the container is opened. + +An AiiDA profile is automatically created when the container is started. +By default the profile is created using the ``core.psql_dos`` storage plugin and a default user is created. +See section :ref:`container configuration ` how to customize certain parts of this setup. + +To confirm that everything is up and running as required, run: + +.. code-block:: console + + verdi status + +which should show something like:: + + ✔ version: AiiDA v2.5.1 + ✔ config: /home/aiida/.aiida + ✔ profile: default + ✔ storage: Storage for 'default' [open] @ postgresql://aiida:***@localhost:5432 + ✔ rabbitmq: Connected to RabbitMQ v3.10.18 as amqp://guest:guest@127.0.0.1:5672 + ✔ daemon: Daemon is running with PID 324 + +If all checks show green check marks, the container is ready to go. +The container can be shut down by typing ``exit`` or pressing ``CTRL+d``. +The container can be restarted at a later time, see :ref:`restarting a container ` for details. +Any data that was created in a previous session is still available. + +.. caution:: + + When the container is not just stopped but *deleted*, any data stored in the container, including the data stored in the profile's storage, is permanently localhost + To ensure the data is not lost, it should be persisted on a volume that is mounted to the container. + Refer to the section on :ref:`persisting data ` for more details. + + +.. _installation:docker:restarting-container: + +Restarting a container +====================== + +After shutting down a container, it can be restarted with: + +.. code-block:: console + + docker start -i aiida + +The name ``aiida`` here is the reference given with the ``--name`` option when the container was originally created. +To open an interactive bash shell inside the container, run: + +.. code-block:: console + + docker exec -it aiida bash + + +.. _installation:docker:persisting-data: + +Persisting data +=============== + +The preferred way to persistently store data across Docker containers is to `create a volume `__. +To create a simple volume, run: + +.. code-block:: console + + docker volume create container-home-data + +In this case, one needs to specifically mount the volume the very first time that the container is created: + +.. code-block:: console + + docker run -it --name aiida -v container-home-data:/home/aiida aiidateam/aiida-core-with-services:latest bash + +By mounting the volume, any data that gets stored in the ``/home/aiida`` path within the container is stored in the ``container-home-data`` volume and therefore persists even if the container is deleted. + +When installing packages with pip, use the ``--user`` flag to store the Python packages installed in the mounted volume (if you mount the home specifically to a volume as mentioned above) permanently. +The packages will be installed in the ``/home/aiida/.local`` directory of the container, which is mounted on the ``container-home-data`` volume. + +You can also mount a folder in the container to a local directory, please refer to the `Docker documentation `__ for more information. + + +.. _installation:docker:container-configuration: + +Container configuration +======================= + +Upon container creation, the following environment variables can be set to configure the default profile that is created: + +* ``AIIDA_PROFILE_NAME``: the name of the profile to be created (default: ``default``) +* ``AIIDA_USER_EMAIL``: the email of the default user to be created (default: ``aiida@localhost``) +* ``AIIDA_USER_FIRST_NAME``: the first name of the default user to be created (default: ``Giuseppe``) +* ``AIIDA_USER_LAST_NAME``: the last name of the default user to be created (default: ``Verdi``) +* ``AIIDA_USER_INSTITUTION``: the institution of the default user to be created (default: ``Khedivial``) + +These environment variables can be set when starting the container with the ``-e`` option. + +.. note:: + + The ``AIIDA_CONFIG_FILE`` variable points to a path inside the container. + Therefore, if you want to use a custom configuration file, it needs to be mounted from the host path to the container path. + +.. _installation:docker:container-backup: + +Container backup +================ + +To backup the data of AiiDA, you can follow the instructions in the `Backup and restore `__ section. +However, Docker provides a convenient way to backup the container data by taking a snapshot of the entire container or the mounted volume(s). + +The following is adapted from the `Docker documentation `__. +If you don't have a volume mounted to the container, you can backup the whole container by committing the container to an image: + +.. code-block:: console + + docker container commit aiida aiida-container-backup + +The above command will create from the container ``aiida`` a new image named ``aiida-container-backup``, containing all the data and modifications made in the container. +The container can then be exported to a tarball and for it to be stored permanently: + +.. code-block:: console + + docker save -o aiida-container-backup.tar aiida-container-backup + +To restore the container, pull the image, or load from the tarball: + +.. code-block:: console + + docker load -i aiida-container-backup.tar + +This creates a container that can then be started with ``docker start``. + +Any `named volumes `__, can be backed up independently. +Refer to `Backup, restore, or migrate data volumes `__ for more information. diff --git a/docs/source/installation/guide_complete.rst b/docs/source/installation/guide_complete.rst new file mode 100644 index 0000000000..49136bbba7 --- /dev/null +++ b/docs/source/installation/guide_complete.rst @@ -0,0 +1,408 @@ +.. _installation:guide-complete: + +=========================== +Complete installation guide +=========================== + +The :ref:`quick installation guide ` is designed to make the setup as simple and portable as possible. +However, the resulting setup has some :ref:`limitations ` concerning the available functionality and performance. +This guide provides detailed information and instructions to set up a feature-complete and performant installation. + +Setting up a working installation of AiiDA, involves the following steps: + +#. :ref:`Install the Python Package ` +#. :ref:`(Optional) RabbitMQ ` +#. :ref:`Create a profile ` + + +.. _installation:guide-complete:python-package: + +Install Python Package +====================== + +.. important:: + AiiDA requires a recent version of Python. + Please refer to the `Python Package Index (PyPI) `_ for the minimum required version. + +To install AiiDA, the ``aiida-core`` Python package needs to be installed which can be done in a number of ways: + + +.. tab-set:: + + .. tab-item:: pip + + Installing ``aiida-core`` from PyPI. + + #. Install `pip `_. + #. Install ``aiida-core``: + + .. code-block:: console + + pip install aiida-core + + .. tab-item:: conda + + Installing ``aiida-core`` using Conda. + + #. Install `conda `_. + + #. Create an environment and install ``aiida-core``: + + .. code-block:: console + + conda create -n aiida -c conda-forge aiida-core + + .. tip:: + As of conda v23.10, the `dependency solver `_ has been significantly improved. + If you are experiencing long installation times, you may want to consider updating conda. + + .. tab-item:: source + + Installing ``aiida-core`` directory from source. + + #. Install `git `_ + #. Clone the repository from Github + + .. code-block:: console + + git clone https://github.com/aiidateam/aiida-core + + #. Install `pip `_. + #. Install ``aiida-core``: + + .. code-block:: console + + cd aiida-core + pip install -e . + + The ``-e`` flag installs the package in editable mode which is recommended for development. + Any changes made to the source files are automatically picked up by the installation. + + +.. _installation:guide-complete:python-package:optional-requirements: + +Optional requirements +--------------------- + +The ``aiida-core`` Python package defines a number of optional requirements, subdivided in the following categories: + +* ``atomic_tools`` : Requirements to deal with atomic data and structures +* ``docs`` : Requirements to build the documentation +* ``notebook`` : Requirements to run AiiDA in Jupyter notebooks +* ``pre-commit`` : Requirements to automatically format and lint source code for development +* ``rest`` : Requirements to run the REST API +* ``ssh_kerberos`` : Requirements for enabling SSH authentication through Kerberos +* ``tests`` : Requirements to run the test suite +* ``tui`` : Requirements to provide a textual user interface (TUI) + +These optional requirements can be installed using pip by adding them as comma separated list, for example: + +.. code-block:: console + + pip install aiida-core[atomic_tools,docs] + + +.. _installation:guide-complete:rabbitmq: + +RabbitMQ +======== + +`RabbitMQ `_ is an optional but recommended service for AiiDA. +It is a messsage broker that is required to run AiiDA's daemon. +The daemon is a system process that runs in the background that manages one or multiple daemon workers that can run AiiDA processes. +This way, the daemon helps AiiDA to scale as it is possible to run many processes in parallel on the daemon workers instead of blockingly in a single Python interpreter. +To facilitate communication with the daemon workers, RabbitMQ is required. + +Although it is possible to run AiiDA without a daemon it does provide significant benefits and therefore it is recommended to install RabbitMQ. + +.. tab-set:: + + .. tab-item:: conda + + #. Install `conda `_. + + #. Create an environment and install ``aiida-core.services``: + + .. code-block:: console + + conda create -n aiida -c conda-forge aiida-core.services + + .. important:: + + The ``aiida-core.services`` package ensures that RabbitMQ is installed in the conda environment. + However, it is not a _service_, in the sense that it is not automatically started, but has to be started manually. + + .. code-block:: console + + rabbitmq-server -detached + + Note that this has to be done each time after the machine has been rebooted. + The server can be stopped with: + + .. code-block:: console + + rabbitmqctl stop + + + .. tab-item:: Ubuntu + + #. Install RabbitMQ through the ``apt`` package manager: + + .. code-block:: console + + sudo apt install rabbitmq-server + + This should automatically install startup scripts such that the server is automatically started when the machine boots. + + + .. tab-item:: MacOS X + + #. Install `Homebrew `. + + #. Install RabbitMQ: + + .. code-block:: console + + brew install rabbitmq + brew services start rabbitmq + + .. important:: + + The service has to manually be started each time the machine reboots. + + .. tab-item:: Other + + For all other cases, please refer to the `official documentation `_ of RabbitMQ. + + + +.. _installation:guide-complete:create-profile: + +Create a profile +================ + +After the ``aiida-core`` package is installed, a profile needs to be created. +A profile defines where the data generated by AiiDA is to be stored. +The data storage can be customized through plugins and so the required configuration changes based on the selected storage plugin. + +To create a new profile, run: + +.. code-block:: console + + verdi profile setup + +where ```` is the entry point name of the storage plugin selected for the profile. +To list the available storage plugins, run: + +.. code-block:: console + + verdi plugin list aiida.storage + +AiiDA ships with a number of storage plugins and it is recommended to select one of the following: + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item-card:: :fa:`feather;mr-1` ``core.sqlite_dos`` + :text-align: center + :shadow: md + + Use this for use-cases to explore AiiDA where performance is not critical. + + This storage plugin does not require any services, making it easy to install and use. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-ref:: installation:guide-complete:create-profile:core-sqlite-dos + :click-parent: + :expand: + :color: primary + :outline: + + Create a ``core.sqlite_dos`` profile + + .. grid-item-card:: :fa:`bolt;mr-1` ``core.psql_dos`` + :text-align: center + :shadow: md + + Use this for production work where database performance is important. + + This storage plugin uses PostgreSQL for the database and provides the greatest performance. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-ref:: installation:guide-complete:create-profile:core-psql-dos + :click-parent: + :expand: + :color: primary + :outline: + + Create a ``core.psql_dos`` profile + + +.. seealso:: + + See the :ref:`topic on storage ` to see a more detailed overview of the storage plugins provided by ``aiida-core`` with their strengths and weaknesses. + +Other packages may provide additional storage plugins, which are also installable through ``verdi profile setup``. + + +.. _installation:guide-complete:create-profile:common-options: + +Common options +-------------- + +The exact options available for the ``verdi profile setup`` command depend on the selected storage plugin, but there are a number of common options and functionality: + +* ``--profile``: The name of the profile. +* ``--set-as-default``: Whether the new profile should be defined as the new default. +* ``--email``: Email for the default user that is created. +* ``--first-name``: First name for the default user that is created. +* ``--last-name``: Last name for the default user that is created. +* ``--institution``: Institution for the default user that is created. +* ``--use-rabbitmq/--no-use-rabbitmq``: Whether to configure the RabbitMQ broker. + Required to enable the daemon and submitting processes to it. + The default is ``--use-rabbitmq``, in which case the command tries to connect to RabbitMQ running on the localhost with default connection parameters. + If this fails, a warning is issued and the profile is configured without a broker. + Once the profile is created, RabbitMQ can still be enabled through ``verdi profile configure-rabbitmq`` which allows to customize the connection parameters. +* ``--non-interactive``: By default, the command prompts to specify a value for all options. + Alternatively, the ``--non-interactive`` flag can be specified, in which case the command never prompts and the options need to be specified directly on the command line. + This is useful when using ``verdi profile setup`` is used in non-interactive environments, such as scripts. +* ``--config``: Instead of passing all options through command line options, the value can be defined in a YAML file and pass its filepath through this option. + + +.. _installation:guide-complete:create-profile:core-sqlite-dos: + +``core.sqlite_dos`` +------------------- + +This storage plugin uses `SQLite `_ and the `disk-objectstore `_ to store data. +The ``disk-objectstore`` is a Python package that is automatically installed as a dependency when installing ``aiida-core``, which was covered in the :ref:`Python package installation section `. +The installation instructions for SQLite depend on your system; please visit the `SQLite website `_ for details. + +Once the prerequisistes are met, create a profile with: + +.. code-block:: console + + verdi profile setup core.sqlite_dos + +The options specific to the ``core.sqlite_dos`` storage plugin are: + +* ``--filepath``: Filepath of the directory in which to store data for this backend. + + +.. _installation:guide-complete:create-profile:core-psql-dos: + +``core.psql_dos`` +----------------- + +This storage plugin uses `PostgreSQL `_ and the `disk-objectstore `_ to store data. +The ``disk-objectstore`` is a Python package that is automatically installed as a dependency when installing ``aiida-core``, which was covered in the :ref:`Python package installation section `. +The storage plugin can connect to a PostgreSQL instance running on the localhost or on a server that can be reached over the internet. +Instructions for installing PostgreSQL is beyond the scope of this guide. + +.. tip:: + + The creation of the PostgreSQL user and database as explained below is implemented in an automated way in the ``verdi presto`` command. + Instead of performing the steps below manually and running ``verdi profile setup core.psql_dos`` manually, it is possible to run: + + .. code-block:: + + verdi presto --use-postgres + +Before creating a profile, a database (and optionally a custom database user) has to be created. +First, connect to PostgreSQL using ``psql``, the `native command line client for PostgreSQL `_: + +.. code-block:: console + + psql -h -U -W + +If PostgreSQL is installed on the localhost, ```` can be replaced with ``localhost``, and the default ```` is ``postgres``. +While possible to use the ``postgres`` default user for the AiiDA profile, it is recommended to create a custom user: + +.. code-block:: sql + + CREATE USER aiida-user WITH PASSWORD ''; + +replacing ```` with a secure password. +The name ``aiida-user`` is just an example name and can be customized. +Note the selected username and password as they are needed when creating the profile later on. + +After the user has been created, create a database: + +.. code-block:: sql + + CREATE DATABASE aiida-database OWNER aiida-user ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8'; + +Again, the selected database name ``aiida-database`` is purely an example and can be customized. +Make sure that the ``OWNER`` is set to the user that was created in the previous step. +Next, grant all privileges on this database to the user: + +.. code-block:: sql + + GRANT ALL PRIVILEGES ON DATABASE aiida-database to aiida-user; + +After the database has been created, the interactive ``psql`` shell can be closed. +To test if the database was created successfully, run the following command: + +.. code-block:: console + + psql -h -d -U -W + +replacing ```` and ```` with the chosen names for the database and user in the previous steps, and providing the chosen password when prompted. + +Once the database has been created, create a profile with: + +.. code-block:: console + + verdi profile setup core.psql_dos + +The options specific to the ``core.psql_dos`` storage plugin are: + +* ``--database-engine`` The engine to use to connect to the database. +* ``--database-hostname`` The hostname of the PostgreSQL server. +* ``--database-port`` The port of the PostgreSQL server. +* ``--database-username`` The username with which to connect to the PostgreSQL server. +* ``--database-password`` The password with which to connect to the PostgreSQL server. +* ``--database-name`` The name of the database in the PostgreSQL server. +* ``--repository-uri`` URI to the file repository. + +.. _installation:guide-complete:validate-installation: + + +Validate installation +===================== + +Once a profile has been created, validate that everything is correctly set up with: + +.. code-block:: console + + verdi status + +The output should look something like the following:: + + ✔ version: AiiDA v2.5.1 + ✔ config: /path/.aiida + ✔ profile: profile-name + ✔ storage: SqliteDosStorage[/path/.aiida/repository/profile-name]: open, + ✔ broker: RabbitMQ v3.8.2 @ amqp://guest:guest@127.0.0.1:5672?heartbeat=600 + ⏺ daemon: The daemon is not running. + +If no lines show red crosses, AiiDA has been correctly installed and is ready to go. +When a new profile is created, the daemon will not yet be running, but it can be started using: + +.. code-block:: console + + verdi daemon start + +.. note:: + + The storage information depends on the storage plugin that was selected. + The broker may be shown as not having been configured which occurs for profiles created with the :ref:`quick installation method `. + This is fine, however, :ref:`some functionality is not supported ` for broker-less profiles. + + +.. admonition:: Not all green? + :class: warning + + If the status reports any problems, please refer to the :ref:`troubleshooting section `. diff --git a/docs/source/installation/guide_quick.rst b/docs/source/installation/guide_quick.rst new file mode 100644 index 0000000000..af9aaa8dc0 --- /dev/null +++ b/docs/source/installation/guide_quick.rst @@ -0,0 +1,85 @@ +.. _installation:guide-quick: + +======================== +Quick installation guide +======================== + +First, install the ``aiida-core`` Python package: + +.. code-block:: console + + pip install aiida-core + +.. attention:: + + AiiDA requires a recent version of Python. + Please refer to the `Python Package Index `_ for the minimum required version. + +Next, set up a profile where all data is stored: + +.. code-block:: console + + verdi presto + +Verify that the installation was successful: + +.. code-block:: console + + verdi status + +If none of the lines show a red cross, indicating a problem, the installation was successful and you are good to go. + +.. admonition:: What next? + :class: hint + + If you are a new user, we recommend to start with the :ref:`basic tutorial `. + Alternatively, check out the :ref:`next steps guide `. + +.. admonition:: Problems during installation? + :class: warning + + If you encountered any issues, please refer to the :ref:`troubleshooting section `. + +.. warning:: + + Not all AiiDA functionality is supported by the quick installation. + Please refer to the :ref:`section below ` for more information. + + +.. _installation:guide-quick:limitations: + +Quick install limitations +========================= + +Functionality +------------- + +Part of AiiDA's functionality requires a `message broker `_, with the default implementation using `RabbitMQ `_. +The message broker is used to allow communication with the :ref:`daemon `. +Since RabbitMQ is a separate service and is not always trivial to install, the quick installation guide sets up a profile that does not require it. +As a result, the daemon cannot be started and processes cannot be submitted to it but can only be run locally. + +.. note:: + The ``verdi presto`` command automatically checks if RabbitMQ is running on the localhost. + If it can successfully connect, it configures the profile with the message broker and therefore the daemon functionality will be available. + +.. tip:: + The connection parameters of RabbitMQ can be (re)configured after the profile is set up with ``verdi profile configure-rabbitmq``. + This can be useful when the RabbitMQ setup is different from the default that AiiDA checks for and the automatic configuration of ``verdi presto`` failed. + + +Performance +----------- + +The quick installation guide by default creates a profile that uses `SQLite `_ for the database. +Since SQLite does not require running a service, it is easy to install and use on essentially any system. +However, for certain use cases it is not going to be the most performant solution. +AiiDA also supports `PostgreSQL `_ which is often going to be more performant compared to SQLite. + +.. tip:: + If a PostgreSQL service is available, run ``verdi presto --use-postgres`` to set up a profile that uses PostgreSQL instead of SQLite. + The command tries to connect to the service and automatically create a user account and database to use for the new profile. + AiiDA provides defaults that work for most setups where PostgreSQL is installed on the localhost. + Should this fail, the connection parameters can be customized using the ``--postgres-hostname``, ``--postgres-port``, ``--postgres-username``, ``--postgres-password`` options. + +Please refer to the :ref:`complete installation guide ` for instructions to set up a feature-complete and performant installation. diff --git a/docs/source/installation/index.rst b/docs/source/installation/index.rst new file mode 100644 index 0000000000..38e350ab7a --- /dev/null +++ b/docs/source/installation/index.rst @@ -0,0 +1,97 @@ +.. _installation: + +.. toctree:: + :maxdepth: 2 + :hidden: + + guide_quick + guide_complete + docker + troubleshooting + +============ +Installation +============ + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item-card:: :fa:`rocket;mr-1` Quick install + :text-align: center + :shadow: md + + Install AiiDA in the most simple way that should work on most systems. + Choose this method if you are new to AiiDA and simply want to try it out. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-ref:: guide_quick + :ref-type: doc + :click-parent: + :expand: + :color: primary + :outline: + + To the quick installation guide + + .. grid-item-card:: :fa:`info-circle;mr-1` Complete installation guide + :text-align: center + :shadow: md + + Install AiiDA with full control over the configuration. + Choose this method if you are an advanced user or you want to optimize the setup for your system. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-ref:: guide_complete + :ref-type: doc + :click-parent: + :expand: + :color: primary + :outline: + + To the complete installation guide + + +Preinstalled environments +========================= + +Instead of installing AiiDA manually, there are also solutions that provide an environment with AiiDA and its requirements installed and pre-configured: + +.. grid:: 1 2 2 2 + :gutter: 3 + + .. grid-item-card:: :fa:`cube;mr-1` Docker + :text-align: center + :shadow: md + + AiiDA provides a number of Docker containers that come with the batteries included. + This is a great option if you are already familiar with Docker. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-ref:: docker + :ref-type: doc + :click-parent: + :expand: + :color: primary + :outline: + + To the Docker installation guide + + .. grid-item-card:: :fa:`cloud;mr-1` Virtual machine + :text-align: center + :shadow: md + + Quantum Mobile is a Virtual Machine for computational materials science. + It comes with AiiDA installed as well as several materials simulation codes. + + +++++++++++++++++++++++++++++++++++++++++++++ + + .. button-link:: https://quantum-mobile.readthedocs.io/en/latest/ + :click-parent: + :expand: + :color: primary + :outline: + + To the Quantum Mobile website diff --git a/docs/source/intro/troubleshooting.rst b/docs/source/installation/troubleshooting.rst similarity index 97% rename from docs/source/intro/troubleshooting.rst rename to docs/source/installation/troubleshooting.rst index b90232fc2a..e70b82a17b 100644 --- a/docs/source/intro/troubleshooting.rst +++ b/docs/source/installation/troubleshooting.rst @@ -1,4 +1,4 @@ -.. _intro:troubleshooting: +.. _installation:troubleshooting: *************** Troubleshooting @@ -21,12 +21,12 @@ In the example output, all service have a green check mark and so should be runn If all services are up and running and you are still experiencing problems or if you have trouble with the installation of aiida-core and related services, consider the commonly encountered problems below. In case you are still experiencing problems, you can request support by opening a post on the `Discourse server `_. -.. _intro:troubleshooting:installation: +.. _installation:troubleshooting:installation: Installation issues ------------------- -.. _intro:troubleshooting:installation:rabbitmq: +.. _installation:troubleshooting:installation:rabbitmq: RabbitMQ incompatibility ........................ @@ -136,16 +136,16 @@ A way to do it is to add a line similar to the following to the ``~/.bashrc`` an .. _Stackoverflow link: http://stackoverflow.com/questions/21079820/how-to-find-pg-config-pathlink -.. _intro:troubleshooting:installation:postgresql-autodetect-issues: +.. _installation:troubleshooting:installation:postgresql-autodetect-issues: Autodetection of the PostgreSQL setup ..................................... -Sometimes AiiDA fails to autodetect the local configuration of PostgreSQL when running ``verdi quicksetup``. +Sometimes AiiDA fails to autodetect the local configuration of PostgreSQL when running ``verdi presto --use-postgres``. In that case try to: - 1. Create the database manually in PostgreSQL (see :ref:`here`). - 2. Then run the full ``verdi setup`` command (see :ref:`here`). + 1. Create the database manually in PostgreSQL (see :ref:`here`). + 2. Then run the full ``verdi profile setup core.psql_dos``. RabbitMQ Installation (Unix) diff --git a/docs/source/intro/cheatsheet.rst b/docs/source/intro/cheatsheet.rst deleted file mode 100644 index f91c7cc698..0000000000 --- a/docs/source/intro/cheatsheet.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. _intro:cheatsheet: - -===================== -The AiiDA cheat sheet -===================== - -The AiiDA cheat sheet gives a broad overview of the most commonly used `verdi` commands, the inheritance hierarchy of the main AiiDA classes, their attributes and methods, as well as a showcase of the `QueryBuilder`. - -When clicking on the embedded image, the pdf version will be opened in the browser. Where applicable, text elements contain hyperlinks to the relevant sections of the documentation. - -The file can also be :download:`downloaded <_cheatsheet/cheatsheet_v.pdf>` in two-page layout for printing. - -Happy exploring! - -.. image:: ./_cheatsheet/cheatsheet.png - :target: ../_static/cheatsheet_h.pdf diff --git a/docs/source/intro/get_started.rst b/docs/source/intro/get_started.rst deleted file mode 100644 index 52d1bb1e89..0000000000 --- a/docs/source/intro/get_started.rst +++ /dev/null @@ -1,98 +0,0 @@ -.. _intro:get_started: - -**************** -Getting started -**************** - -An AiiDA installation consists of three core components (plus any external codes you wish to run): - -* |aiida-core|: The main Python package and the associated ``verdi`` command line interface -* |PostgreSQL|: The service that manages the database that AiiDA uses to store data. -* |RabbitMQ|: The message broker used for communication within AiiDA. - -.. toctree:: - :maxdepth: 1 - :hidden: - - install_system - install_conda - run_docker - installation - -.. _intro:install:setup: -.. _intro:get_started:setup: - -Setup -===== - -There are multiple routes to setting up a working AiiDA environment. -Which of those is optimal depends on your environment and use case. -If you are unsure, use the :ref:`system-wide installation ` method. - -.. grid:: 1 2 2 2 - :gutter: 3 - - .. grid-item-card:: :fa:`desktop;mr-1` System-wide installation - - .. button-ref:: intro:get_started:system-wide-install - :ref-type: ref - :click-parent: - :class: btn-link - - Install all software directly on your workstation or laptop. - - Install the prerequisite services using standard package managers (apt, homebrew, etc.) with administrative privileges. - - .. grid-item-card:: :fa:`folder;mr-1` Installation into Conda environment - - .. button-ref:: intro:get_started:conda-install - :ref-type: ref - :click-parent: - :class: btn-link - - Install all software into an isolated conda environment. - - This method does not require administrative privileges, but involves manual management of start-up and shut-down of services. - - .. grid-item-card:: :fa:`cube;mr-1` Run via docker container - - .. button-ref:: intro:get_started:docker - :ref-type: ref - :click-parent: - :class: btn-link - - Run AiiDA and prerequisite services as a single docker container. - - Does not require the separate installation of prerequisite services. - Especially well-suited to get directly started on the **tutorials**. - - .. grid-item-card:: :fa:`cloud;mr-1` Run via virtual machine - - .. button-link:: https://quantum-mobile.readthedocs.io/ - :click-parent: - :class: btn-link - - Use a virtual machine with all the required software pre-installed. - - `Materials Cloud `__ provides both downloadable and web based VMs, - also incorporating pre-installed Materials Science codes. - -.. _intro:get_started:next: - -What's next? -============ - -After successfully completing one of the above outlined setup routes, if you are new to AiiDA, we recommed you go through the :ref:`Basic Tutorial `, -or see our :ref:`Next steps guide `. - -If however, you encountered some issues, proceed to the :ref:`troubleshooting section `. - -.. admonition:: In-depth instructions - :class: seealso title-icon-read-more - - For more detailed instructions on configuring AiiDA, :ref:`see the configuration how-to `. - -.. |aiida-core| replace:: `aiida-core `__ -.. |PostgreSQL| replace:: `PostgreSQL `__ -.. |RabbitMQ| replace:: `RabbitMQ `__ -.. |Homebrew| replace:: `Homebrew `__ diff --git a/docs/source/intro/index.rst b/docs/source/intro/index.rst index 779def1ae1..eb62171c21 100644 --- a/docs/source/intro/index.rst +++ b/docs/source/intro/index.rst @@ -1,15 +1,13 @@ +.. _intro: + +.. toctree:: + :maxdepth: 1 + :hidden: + ============ Introduction ============ -.. _intro:about: - - -************* -What is AiiDA -************* - - AiiDA is an open-source Python infrastructure to help researchers with automating, managing, persisting, sharing and reproducing the complex workflows associated with modern computational science and all associated data. AiiDA is built to support and streamline the four core pillars of the ADES model: Automation, Data, Environment, and Sharing (described `here `__). Some of the key features of AiiDA include: @@ -24,12 +22,3 @@ AiiDA is built to support and streamline the four core pillars of the ADES model * **Open source:** AiiDA is released under the `MIT open-source license `__. See also the `list of AiiDA-powered scientific publications `__ and `testimonials from AiiDA users `__. - - -.. toctree:: - :maxdepth: 1 - - get_started - ../tutorials/index - cheatsheet - troubleshooting diff --git a/docs/source/intro/install_conda.rst b/docs/source/intro/install_conda.rst deleted file mode 100644 index 1f8a3bfd82..0000000000 --- a/docs/source/intro/install_conda.rst +++ /dev/null @@ -1,163 +0,0 @@ -.. _intro:get_started:conda-install: - -*********************************** -Installation into Conda environment -*********************************** - -This installation route installs all necessary software -- including the prerequisite services PostgreSQL and RabbitMQ -- into a Conda environment. -This is the recommended method for users on shared systems and systems where the user has no administrative privileges. -If you want to install AiiDA onto you own personal workstation/laptop, it is recommanded to use the :ref:`system-wide installation `. - -.. important:: - - This installation method installs **all** software into a conda environment, including PostgreSQL and RabbitMQ. - See the :ref:`system-wide installation ` to use Conda only to install the AiiDA (core) Python package. - -.. grid:: 1 - :gutter: 3 - - .. grid-item-card:: Install prerequisite services + AiiDA (core) - - *Install the aiida-core package and all required services in a Conda environment.* - - #. We strongly recommend using ``mamba`` instead of the default ``conda`` (or environment resolution may time out). - Consider using `Mambaforge `_ when starting from scratch, or ``conda install -c conda-forge mamba``. - - #. Open a terminal and execute: - - .. code-block:: console - - $ mamba create -n aiida -c conda-forge aiida-core aiida-core.services - $ mamba activate aiida - - .. grid-item-card:: Start-up services and initialize data storage - - Before working with AiiDA, you must first initialize a database storage area on disk. - - .. code-block:: console - - (aiida) $ initdb -D mylocal_db - - This *database cluster* (located inside a folder named ``mylocal_db``) may contain a collection of databases (one per profile) that is managed by a single running server process. - We start this process with: - - .. code-block:: console - - (aiida) $ pg_ctl -D mylocal_db -l logfile start - - .. tip:: - - The default port ``5432`` may already be in use by another process. - In this case, you can pass the ``-o "-F -p "`` option to the ``pg_ctl`` command, ```` being the desired port number. - Then for the ``psql`` command, you can pass the ``-p `` option. - - .. admonition:: Further Reading - :class: seealso title-icon-read-more - - - `Creating a Database Cluster `__. - - `Starting the Database Server `__. - - - - Then, start the RabbitMQ server: - - .. code-block:: console - - (aiida) $ rabbitmq-server -detached - - .. important:: - - The services started this way will use the default ports on the machine. - Conflicts may happen if there are more than one user running AiiDA this way on the same machine, or you already have the server running in a system-wide installation. - To get around this issue, you can explicitly define the ports to be used. - - .. grid-item-card:: Setup profile - - Next, set up an AiiDA configuration profile and related data storage, with the ``verdi quicksetup`` command. - - .. code-block:: console - - (aiida) $ verdi quicksetup - Info: enter "?" for help - Info: enter "!" to ignore the default and set no value - Profile name: me - Email Address (for sharing data): me@user.com - First name: my - Last name: name - Institution: where-i-work - - .. tip:: - - In case of non-default ports are used for the *database cluster* and the RabbitMQ server, you can pass them using ``--db-port`` and ``--broker-port`` options respectively. - - - .. admonition:: Is AiiDA unable to auto-detect the PostgreSQL setup? - :class: attention title-icon-troubleshoot - - If you get an error saying that AiiDA has trouble autodetecting the PostgreSQL setup, you will need to do the manual setup explained in the :ref:`troubleshooting section`. - - Once the profile is up and running, you can start the AiiDA daemon(s): - - .. code-block:: console - - (aiida) $ verdi daemon start 2 - - .. important:: - - The verdi daemon(s) must be restarted after a system reboot. - - .. tip:: - - Do not start more daemons then there are physical processors on your system. - - .. grid-item-card:: Check setup - - To check that everything is set up correctly, execute: - - .. code-block:: console - - (aiida) $ verdi status - ✓ version: AiiDA v2.0.0 - ✓ config: /path/to/.aiida - ✓ profile: default - ✓ storage: Storage for 'default' @ postgresql://username:***@localhost:5432/db_name / file:///path/to/repository - ✓ rabbitmq: Connected as amqp://127.0.0.1?heartbeat=600 - ✓ daemon: Daemon is running as PID 2809 since 2019-03-15 16:27:52 - - .. admonition:: Missing a checkmark or ecountered some other issue? - :class: attention title-icon-troubleshoot - - :ref:`See the troubleshooting section `. - - .. button-ref:: intro:get_started:next - :ref-type: ref - :expand: - :color: primary - :outline: - :class: sd-font-weight-bold - - What's next? - - .. grid-item-card:: Shut-down services - - After finishing with your aiida session, particularly if switching between profiles, you may wish to power down the daemon and the services: - - .. code-block:: console - - (aiida) $ verdi daemon stop - (aiida) $ pg_ctl -D mylocal_db stop - (aiida) $ rabbitmqctl stop - - .. grid-item-card:: Restart the services - - If you want to restart the services and the daemon: - - .. code-block:: console - - (aiida) $ pg_ctl -D mylocal_db start - (aiida) $ rabbitmq-server -detached - (aiida) $ verdi daemon start - - .. tip:: - - If different ports are used, you have to pass them here as well. diff --git a/docs/source/intro/install_system.rst b/docs/source/intro/install_system.rst deleted file mode 100644 index 48464cc981..0000000000 --- a/docs/source/intro/install_system.rst +++ /dev/null @@ -1,298 +0,0 @@ -.. _intro:get_started:system-wide-install: - -************************ -System-wide installation -************************ - -The system-wide installation will install the prerequisite services (PostgreSQL and RabbitMQ) via standard package managers such that their startup and shut-down is largely managed by the operating system. -The AiiDA (core) Python package is then installed either with Conda or pip. - -.. warning:: RabbitMQ v3.5 and below are EOL and not supported at all. For versions RabbitMQ v3.8.15 and up, AiiDA is not compatible with default server configurations. For details refer to the :ref:`dedicated troubleshooting section`. - -This is the *recommended* installation method to setup AiiDA on a personal laptop or workstation for the majority of users. - -.. grid:: 1 - :gutter: 3 - - .. grid-item-card:: Install prerequisite services - - AiiDA is designed to run on `Unix `_ operating systems and requires a `bash `_ or `zsh `_ shell, and Python >= 3.7. - - .. tab-set:: - - .. tab-item:: Ubuntu - - *AiiDA is tested on Ubuntu versions 16.04, 18.04, and 20.04.* - - Open a terminal and execute: - - .. code-block:: console - - $ sudo apt install git python3-dev python3-pip postgresql postgresql-server-dev-all postgresql-client rabbitmq-server - - .. tab-item:: MacOS X (Homebrew) - - The recommended installation method for Mac OS X is to use `Homebrew `__. - - #. Follow `this guide `__ to install Homebrew on your system if not installed yet. - - #. Open a terminal and execute: - - .. code-block:: console - - $ brew install postgresql rabbitmq git python - $ brew services start postgresql - $ brew services start rabbitmq - - .. tab-item:: Windows Subsystem for Linux - - *The following instructions are for setting up AiiDA on WSL 1/2 in combination with Ubuntu.* - - #. Installing RabbitMQ: - - * (WSL 1) Install and start the `Windows native RabbitMQ `_. - - * (WSL 2) Install RabbitMQ inside the the WSL: - - .. code-block:: console - - $ sudo apt install rabbitmq-server - - then start the ``rabbitmq`` server: - - .. code-block:: console - - $ sudo service rabbitmq-server start - - #. Install Python and PostgreSQL: - - .. code-block:: console - - $ sudo apt install postgresql postgresql-server-dev-all postgresql-client git python3-dev python-pip - - then start the PostgreSQL server: - - .. code-block:: console - - $ sudo service postgresql start - - .. dropdown:: How to setup WSL to automatically start services after system boot. - - Create a file ``start_aiida_services.sh`` containing the following lines: - - .. code-block:: console - - $ service postgresql start - $ service rabbitmq-server start # Only for WSL 2! - - and store it in your preferred location, e.g., the home directory. - Then make the file executable, and editable only by root users with: - - .. code-block:: console - - $ chmod a+x,go-w /path/to/start_aiida_services.sh - $ sudo chown root:root /path/to/start_aiida_services.sh - - Next, run - - .. code-block:: console - - $ sudo visudo - - and add the line - - .. code-block:: sh - - ALL=(root) NOPASSWD: /path/to/start_aiida_services.sh - - replacing ```` with your Ubuntu username. - This will allow you to run *only* this specific ``.sh`` file with ``root`` access (without password), without lowering security on the rest of your system. - - Now you can use the Windows Task Scheduler to automatically execute this file on startup: - - #. Open Task Scheduler. - - #. In the "Actions" menu, click "Create Task". - - #. In "General/Security options", select "Run whether user is logged on or not". - - #. In the "Triggers" tab, click "New...". - - #. In the "Begin the task:" dropdown, select "At startup". - - #. Click "OK" to confirm. - - #. In the "Actions" tab, click "New...". - - #. In the "Action" dropdown, select "Start a program". - - #. In the "Program/script" text field, add ``C:\Windows\System32\bash.exe``. - - #. In the "Add arguments (optional)" text field, add ``-c "sudo /path/to/start_aiida_services.sh"``. - - #. Click "OK" to confirm. - - #. Click "OK" to confirm the task. - - You can tweak other details of this task to fit your needs. - - .. tab-item:: Other - - #. Install RabbitMQ following the `instructions applicable to your system `__. - #. Install PostgreSQL following the `instructions applicable to your system `__. - - .. tip:: - - Alternatively use the :ref:`pure conda installation method `. - - .. grid-item-card:: Install AiiDA (core) - - .. tab-set:: - - .. tab-item:: pip + venv - - *Install the aiida-core package from PyPI into a virtual environment.* - - Open a terminal and execute: - - .. code-block:: console - - $ python -m venv ~/envs/aiida - $ source ~/envs/aiida/bin/activate - (aiida) $ pip install aiida-core - - .. important:: - - Make sure the ``python`` executable is for a Python version that is supported by AiiDA. - You can see the version using: - - .. code-block:: console - - $ python --version - - You can find the supported Python versions for the latest version of AiiDA `on the PyPI page `__. - - .. tip:: - - See the `venv documentation `__ if the activation command fails. - The exact command for activating a virtual environment differs slightly based on the used shell. - - .. dropdown:: :fa:`plus-circle` Installation extras - - There are additional optional packages that you may want to install, which are grouped in the following categories: - - * ``atomic_tools``: packages that allow importing and manipulating crystal structure from various formats - * ``ssh_kerberos``: adds support for ssh transport authentication through Kerberos - * ``REST``: allows a REST server to be ran locally to serve AiiDA data - * ``docs``: tools to build the documentation - * ``notebook``: jupyter notebook - to allow it to import AiiDA modules - * ``tests``: python modules required to run the automatic unit tests - * ``pre-commit``: pre-commit tools required for developers to enable automatic code linting and formatting - - In order to install any of these package groups, simply append them as a comma separated list in the ``pip`` install command, for example: - - .. code-block:: console - - (aiida) $ pip install aiida-core[atomic_tools,docs] - - .. dropdown:: :fa:`wrench` Kerberos on Ubuntu - - If you are installing the optional ``ssh_kerberos`` and you are on Ubuntu you might encounter an error related to the ``gss`` package. - To fix this you need to install the ``libffi-dev`` and ``libkrb5-dev`` packages: - - .. code-block:: console - - $ sudo apt-get install libffi-dev libkrb5-dev - - .. tab-item:: Conda - - *Install the aiida-core package in a Conda environment.* - - #. Make sure that conda is installed, e.g., by following `the instructions on installing Miniconda `__. - - #. Open a terminal and execute: - - .. code-block:: console - - $ conda create -yn aiida -c conda-forge aiida-core - $ conda activate aiida - - .. tab-item:: From source - - *Install the aiida-core package directly from the cloned repository.* - - Open a terminal and execute: - - .. code-block:: console - - $ git clone https://github.com/aiidateam/aiida-core.git - $ cd aiida-core/ - $ python -m venv ~/envs/aiida - $ source ~/envs/aiida/bin/activate - (aiida) $ pip install . - - .. grid-item-card:: Setup profile - - Next, set up an AiiDA configuration profile and related data storage, with the ``verdi quicksetup`` command. - - .. code-block:: console - - (aiida) $ verdi quicksetup - Info: enter "?" for help - Info: enter "!" to ignore the default and set no value - Profile name: me - Email Address (for sharing data): me@user.com - First name: my - Last name: name - Institution: where-i-work - - .. admonition:: Is AiiDA unable to auto-detect the PostgreSQL setup? - :class: attention title-icon-troubleshoot - - If you get an error saying that AiiDA has trouble autodetecting the PostgreSQL setup, you will need to do the manual setup explained in the :ref:`troubleshooting section`. - - .. grid-item-card:: Start verdi daemons - - Start the verdi daemon(s) that are used to run AiiDA workflows. - - .. code-block:: console - - (aiida) $ verdi daemon start 2 - - .. important:: - - The verdi daemon(s) must be restarted after a system reboot. - - .. tip:: - - Do not start more daemons then there are physical processors on your system. - - .. grid-item-card:: Check setup - - To check that everything is set up correctly, execute: - - .. code-block:: console - - (aiida) $ verdi status - ✓ version: AiiDA v2.0.0 - ✓ config: /path/to/.aiida - ✓ profile: default - ✓ storage: Storage for 'default' @ postgresql://username:***@localhost:5432/db_name / file:///path/to/repository - ✓ rabbitmq: Connected as amqp://127.0.0.1?heartbeat=600 - ✓ daemon: Daemon is running as PID 2809 since 2019-03-15 16:27:52 - - At this point you should now have a working AiiDA environment, from which you can add and retrieve data. - - .. admonition:: Missing a checkmark or encountered some other issue? - :class: attention title-icon-troubleshoot - - :ref:`See the troubleshooting section `. - - .. button-ref:: intro:get_started:next - :ref-type: ref - :expand: - :color: primary - :outline: - :class: sd-font-weight-bold - - What's next? diff --git a/docs/source/intro/installation.rst b/docs/source/intro/installation.rst deleted file mode 100644 index 479ba53666..0000000000 --- a/docs/source/intro/installation.rst +++ /dev/null @@ -1,295 +0,0 @@ -.. _intro:install: -.. _intro:advanced-config: - -********************** -Advanced configuration -********************** - -This chapter covers topics that go beyond the :ref:`standard setup of AiiDA `. -If you are new to AiiDA, we recommend you first go through the :ref:`Basic Tutorial `, -or see our :ref:`Next steps guide `. - -.. _intro:install:database: - -Creating the database ---------------------- - -AiiDA uses a database to store the nodes, node attributes and other information, allowing the end user to perform fast queries of the results. -Currently, the highly performant `PostgreSQL`_ database is supported as a database backend. - -.. _PostgreSQL: https://www.postgresql.org/downloads - -.. admonition:: Find out more about the database - :class: seealso title-icon-read-more - - - `Creating a Database Cluster `__. - - `Starting the Database Server `__. - - :ref:`The database topic `. - -To manually create the database for AiiDA, you need to run the program ``psql`` to interact with postgres. -On most operating systems, you need to do so as the ``postgres`` user that was created upon installing the software. -To assume the role of ``postgres`` run as root: - -.. code-block:: console - - $ su - postgres - -(or, equivalently, type ``sudo su - postgres``, depending on your distribution) and launch the postgres program: - -.. code-block:: console - - $ psql - -.. tip:: - - If you have installed PostgreSQL through Conda and you see an error like ``psql: FATAL: role "" does not exist`` or ``psql: FATAL: database "" does not exist``, the default role and database apparently do no exist. - The command ``psql -l`` prints the list of existing databases and the associated roles. - You can try connecting to one of those by using the ``-d`` and ``-U`` option to specify the database and role, respectively, for example, ``psql -d template0 -U some-role``. - -Create a new database user account for AiiDA by running: - -.. code-block:: sql - - CREATE USER aiida WITH PASSWORD ''; - -replacing ```` with a password of your choice. - -You will need to provide the password again when you configure AiiDA to use this database through ``verdi setup``. -If you want to change the password you just created use the command: - -.. code-block:: sql - - ALTER USER aiida PASSWORD ''; - -Next, we create the database itself. We enforce the UTF-8 encoding and specific locales: - -.. code-block:: sql - - CREATE DATABASE aiidadb OWNER aiida ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE=template0; - -and grant all privileges on this DB to the previously-created ``aiida`` user: - -.. code-block:: sql - - GRANT ALL PRIVILEGES ON DATABASE aiidadb to aiida; - -You have now created a database for AiiDA and you can close the postgres shell by typing ``\q``. -To test if the database was created successfully, you can run the following command as a regular user in a bash terminal: - -.. code-block:: console - - $ psql -h localhost -d aiidadb -U aiida -W - -and type the password you inserted before, when prompted. -If everything worked well, you should get no error and see the prompt of the ``psql`` shell. - -If you use the same names as in the example commands above, then during the ``verdi setup`` phase the following parameters will apply to the newly created database: - -.. code-block:: console - - $ Database engine: postgresql_psycopg2 - $ Database host: localhost - $ Database port: 5432 - $ AiiDA Database name: aiidadb - $ AiiDA Database user: aiida - $ AiiDA Database password: - -.. admonition:: Don't forget to backup your database! - :class: tip title-icon-tip - - See the :ref:`Database backup how-to `), and :ref:`how to move your database `. - -Database setup using 'peer' authentication ------------------------------------------- - -On Ubuntu Linux, the default PostgreSQL setup is configured to use ``peer`` authentication, which allows password-less login via local Unix sockets. -In this mode, PostgreSQL compares the Unix user connecting to the socket with its own database of users and allows a connection if a matching user exists. - -.. note:: - - This is an alternative route to set up your database - the standard approach will work on Ubuntu just as well. - -Below we are going to take advantage of the command-line utilities shipped on Ubuntu to simplify creating users and databases compared to issuing the SQL commands directly. - -Assume the role of ``postgres``: - -.. code-block:: console - - $ sudo su postgres - -Create a database user with the **same name** as the UNIX user who will be running AiiDA (usually your login name): - -.. code-block:: console - - $ createuser - -replacing ```` with your username. - -Next, create the database itself with your user as the owner: - -.. code-block:: console - - $ createdb -O aiidadb - -Exit the shell to go back to your login user. -To test if the database was created successfully, try: - -.. code-block:: console - - $ psql aiidadb - -During the ``verdi setup`` phase, use ``!`` to leave host empty and specify your Unix user name as the *AiiDA Database user*.: - -.. code-block:: console - - $ Database engine: postgresql_psycopg2 - $ Database host: ! - $ Database port: 5432 - $ AiiDA Database name: aiidadb - $ AiiDA Database user: - $ AiiDA Database password: "" - - -RabbitMQ configuration ----------------------- - -In most normal setups, RabbitMQ will be installed and run as a service on the same machine that hosts AiiDA itself. -In that case, using the default configuration proposed during a profile setup will work just fine. -However, when the installation of RabbitMQ is not standard, for example it runs on a different port, or even runs on a completely different machine, all relevant connection details can be configured with ``verdi setup``. - -The following parameters can be configured: - -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Parameter | Option | Default | Explanation | -+==============+===========================+===============+=========================================================================================================================+ -| Protocol | ``--broker-protocol`` | ``amqp`` | The protocol to use, can be either ``amqp`` or ``amqps`` for SSL enabled connections. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Username | ``--broker-username`` | ``guest`` | The username with which to connect. The ``guest`` account is available and usable with a default RabbitMQ installation. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Password | ``--broker-password`` | ``guest`` | The password with which to connect. The ``guest`` account is available and usable with a default RabbitMQ installation. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Host | ``--broker-host`` | ``127.0.0.1`` | The hostname of the RabbitMQ server. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Port | ``--broker-port`` | ``5672`` | The port to which the server listens. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Virtual host | ``--broker-virtual-host`` | ``''`` | Optional virtual host. Should not contain the leading forward slash, this will be added automatically by AiiDA. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ -| Parameters | not available | n.a. | These are additional broker parameters that are typically encoded as URL parameters, for example, to specify SSL | -| | | | parameters such as the filepath to the certificate that is to be used. The parameters are currently not definable | -| | | | through the CLI but have to be added manually in the ``config.json``. A key ``broker_parameters`` should be added that | -| | | | is a dictionary, which can contain fields: ``cafile``, ``capath``, ``cadata``, ``certfile``, ``keyfile`` and | -| | | | ``no_verify_ssl``. | -+--------------+---------------------------+---------------+-------------------------------------------------------------------------------------------------------------------------+ - - -.. _intro:install:verdi_setup: - -verdi setup ------------ - -After the database has been created, do: - -.. code-block:: console - - $ verdi setup --profile - -where `` is a profile name of your choosing. -The ``verdi setup`` command will guide you through the setup process through a series of prompts. - -The first information asked is your email, which will be used to associate the calculations to you. -In AiiDA, the email is your username, and acts as a unique identifier when importing/exporting data from AiiDA. - -.. note:: - - The password, in the current version of AiiDA, is not used (it will be used only in the REST API and in the web interface). - If you leave the field empty, no password will be set and no access will be granted to the user via the REST API and the web interface. - -Then, the following prompts will help you configure the database. Typical settings are: - -.. code-block:: console - - $ Default user email: richard.wagner@leipzig.de - $ Database engine: postgresql_psycopg2 - $ PostgreSQL host: localhost - $ PostgreSQL port: 5432 - $ AiiDA Database name: aiida_dev - $ AiiDA Database user: aiida - $ AiiDA Database password: - $ AiiDA repository directory: /home/wagner/.aiida/repository/ - [...] - Configuring a new user with email 'richard.wagner@leipzig.de' - $ First name: Richard - $ Last name: Wagner - $ Institution: BRUHL, LEIPZIG - $ The user has no password, do you want to set one? [y/N] y - $ Insert the new password: - $ Insert the new password (again): - -.. admonition:: Don't forget to backup your data! - :class: tip title-icon-tip - - See the :ref:`installation backup how-to `. - -.. _intro:install:start_daemon: - -Managing the daemon -------------------- - -The AiiDA daemon process runs in the background and takes care of processing your submitted calculations and workflows, checking their status, retrieving their results once they are finished and storing them in the AiiDA database. - -The AiiDA daemon is controlled using three simple commands: - -* ``verdi daemon start``: start the daemon -* ``verdi daemon status``: check the status of the daemon -* ``verdi daemon stop``: stop the daemon - -.. note:: - - While operational, the daemon logs its activity to a file in ``~/.aiida/daemon/log/`` (or, more generally, ``$AIIDA_PATH/.aiida/daemon/log``). - Get the latest log messages via ``verdi daemon logshow``. - -.. _intro:install:jupyter: - -Using AiiDA in Jupyter ----------------------- - - 1. Install the AiiDA ``notebook`` extra **inside** the AiiDA python environment, e.g. by running ``pip install aiida-core[notebook]``. - - -With this setup, you're ready to use AiiDA in Jupyter notebooks. - -Start a Jupyter notebook server: - -.. code-block:: console - - $ jupyter notebook - -This will open a tab in your browser. Click on ``New -> Python``. - -To load the `aiida` magics extension, simply run: - -.. code-block:: ipython - - %load_ext aiida - -Now you can load a profile (the default unless specified) by: - -.. code-block:: ipython - - %aiida - -After executing the cell by ``Shift-Enter``, you should receive the message "Loaded AiiDA DB environment." -Otherwise, you can load the profile manually as you would in a Python script: - -.. code-block:: python - - from aiida import load_profile, orm - load_profile() - qb = orm.QueryBuilder() - # ... - -You can also run `verdi` CLI commands, using the currently loaded profile, by: - -.. code-block:: ipython - - %verdi status diff --git a/docs/source/intro/run_docker.rst b/docs/source/intro/run_docker.rst deleted file mode 100644 index 227dd4b263..0000000000 --- a/docs/source/intro/run_docker.rst +++ /dev/null @@ -1,236 +0,0 @@ -.. _intro:get_started:docker: -.. _intro:install:docker: - -**************************** -Run AiiDA via a Docker image -**************************** - -The AiiDA team maintains a `Docker `__ image on `Docker Hub `__. -This image contains a fully pre-configured AiiDA environment which makes it particularly useful for learning and developing purposes. - -.. caution:: - - All data stored in a container will persist only over the lifetime of that particular container (i.e., removing the container will also purge the data) unless you use volumes to persist the data, see :ref:`Advanced usage ` for more details. - -.. grid:: 1 - :gutter: 3 - - .. grid-item-card:: Install Docker on your PC - - Docker is available for Windows, Mac and Linux and can be installed in different ways. - - .. tab-set:: - - .. tab-item:: Colima on MacOS - - `Colima `_ is a new open-source project that makes it easy to run Docker on MacOS. - It is a lightweight alternative to Docker Engine with a focus on simplicity and performance. - - Colima is the recommended way. - With colima, you can have multiple Docker environments running at the same time, each with its own Docker daemon and resource allocation thus avoiding conflicts. - - To install the colima, on MacOS run: - - .. code-block:: console - - $ brew install colima - - Or check Check `here `__ for other installation options. - - After installation, start the docker daemon by: - - .. code-block:: console - - $ colima start - - .. tab-item:: Docker CE on Linux - - The bare minimum to run Docker on Linux is to install the `Docker Engine `_. - If you don't need a graphical user interface, this is the recommended way to install Docker. - - .. note:: - - You will need `root` privileges to perform the `post-installation steps `_. - Otherwise, you will need to use `sudo` for every Docker command. - - - - .. grid-item-card:: Start/stop container and use AiiDA interactively - - Start the image with the `docker command line interface (docker CLI) `_. - - There are differnt tags available for the AiiDA image, the ``latest`` tag is the image with the most recent stable version of ``aiida-core`` installed in the container. - You can replace the ``latest`` tag with the ``aiida-core`` or services version you want to use, check the `Docker Hub `_ for available tags. - - .. tab-set:: - - .. tab-item:: Docker CLI - - Use the Docker CLI to run the AiiDA container. - - .. code-block:: console - - $ docker run -it --name aiida-container-demo aiidateam/aiida-core-with-services:latest bash - - The ``-it`` option is used to run the container in interactive mode and to allocate a pseudo-TTY. - You will be dropped into a bash shell inside the container. - - You can specify a name for the container with the ``--name`` option for easier reference later on. - For the quick test, you can also use the ``--rm`` option to remove the container when it exits. - In the following examples, we will use the name ``aiida-container-demo`` for the container. - - To exit and stop the container, type ``exit`` or press ``Ctrl+D``. - - Please note the ``run`` sub-command is used to both create and start a container. - In order to start a container which is already created, you should use ``start``, by running: - - .. code-block:: console - - $ docker start -i aiida-container-demo - - If you need another shell inside the container, run: - - .. code-block:: console - - $ docker exec -it aiida-container-demo bash - - By default, an AiiDA profile is automatically set up inside the container. - To disable this default profile being created, set the ``SETUP_DEFAULT_AIIDA_PROFILE`` environment variable to ``false``. - - The following environment variables can be set to configure the default AiiDA profile: - - * ``AIIDA_PROFILE_NAME``: the name of the profile to be created (default: ``default``) - * ``AIIDA_USER_EMAIL``: the email of the default user to be created (default: ``aiida@localhost``) - * ``AIIDA_USER_FIRST_NAME``: the first name of the default user to be created (default: ``Giuseppe``) - * ``AIIDA_USER_LAST_NAME``: the last name of the default user to be created (default: ``Verdi``) - * ``AIIDA_USER_INSTITUTION``: the institution of the default user to be created (default: ``Khedivial``) - * ``AIIDA_CONFIG_FILE``: the path to the AiiDA configuration file used for other profile configuration parameters (default: ``/aiida/assets/config-quick-setup.yaml``). - - These environment variables can be set when starting the container with the ``-e`` option. - - Please note that the ``AIIDA_CONFIG_FILE`` variable points to a path inside the container. - Therefore, if you want to use a custom configuration file, it needs to be mounted from the host path to the container path. - - .. grid-item-card:: Check setup - - The profile named ``default`` is created under the ``aiida`` user. - - To check the status of AiiDA environment setup, execute the following command inside the container shell: - - .. code-block:: console - - $ verdi status - ✓ config dir: /home/aiida/.aiida - ✓ profile: On profile default - ✓ repository: /home/aiida/.aiida/repository/default - ✓ postgres: Connected as aiida_qs_aiida_477d3dfc78a2042156110cb00ae3618f@localhost:5432 - ✓ rabbitmq: Connected as amqp://127.0.0.1?heartbeat=600 - ✓ daemon: Daemon is running as PID 1795 since 2020-05-20 02:54:00 - - -Advanced usage -============== - -.. _intro:install:docker:advanced_usage: - -Congratulations! You have a working AiiDA environment, and can start using it. - -If you use the Docker image for development or production, you will likely need additional settings such as clone the repository and install `aiida-core` in the editable mode to make it work as expected. -See `development wiki `_ for more details. - -.. dropdown:: Copy files from your computer to the container - - .. tab-set:: - - .. tab-item:: Docker CLI - - Use the ``docker cp`` command if you need to copy files from your computer to the container or vice versa. - - For example, to copy a file named ``test.txt`` from your current working directory to the ``/home/aiida`` path in the container, run: - - .. code-block:: console - - $ docker cp test.txt aiida-container-demo:/home/aiida - - -.. dropdown:: Persist data across different containers - - The lifetime of the data stored in a container is limited to the lifetime of that particular container. - - If you stop the container (``docker stop`` or simply ``Ctrl+D`` from the container) and start it again, any data you created will persist. - However, if you remove the container, **all data will be removed as well**. - - .. code-block:: console - - $ docker rm aiida-container-demo - - The preferred way to persistently store data across Docker containers is to `create a volume `__. - - .. tab-set:: - - .. tab-item:: Docker CLI - - To create a simple volume, run: - - .. code-block:: console - - $ docker volume create container-home-data - - In this case, one needs to specifically mount the volume very first time that the container is being created: - - .. code-block:: console - - $ docker run -it --name aiida-container-demo -v container-home-data:/home/aiida aiidateam/aiida-core-with-services:latest bash - - Starting the container with the above command ensures that any data stored in the ``/home/aiida`` path within the container is stored in the ``container-home-data`` volume and therefore persists even if the container is removed. - - When installing packages with pip, use the ``--user`` flag to store the Python packages installed in the mounted volume (if you mount the home specifically to a volume as mentioned above) permanently. - The packages will be installed in the ``/home/aiida/.local`` directory of the container, which is mounted on the ``container-home-data`` volume. - - You can also mount a folder in container to a local directory, please refer to the `Docker documentation `__ for more information. - -.. dropdown:: Backup the container - - To backup the data of AiiDA, you can follow the instructions in the `Backup and restore `__ section. - However, Docker provides a convenient way to backup the container data by taking a snapshot of the entire container or the mounted volume(s). - - The following is adapted from the `Docker documentation `__. - - If you don't have a volume mounted to the container, you can backup the whole container by committing the container to an image: - - .. code-block:: console - - $ docker container commit aiida-container-demo aiida-container-backup - - The above command will create a new image named ``aiida-container-backup`` containing all the data and modifications you made in the container. - - Then, you can export the container to a local tarball and store it permanently: - - .. code-block:: console - - $ docker save -o aiida-container-backup.tar aiida-container-backup - - To restore the container, pull the image, or load from the tarball, run: - - .. code-block:: console - - $ docker load -i aiida-container-backup.tar - - You'll find a container in the list and you can then start it with ``docker start``. - - If you used a `named volume `__, you can backup the volume independently. - - .. tab-set:: - - .. tab-item:: Docker CLI - - Please check `Backup, restore, or migrate data volumes `__ for more information. - -.. button-ref:: intro:get_started:next - :ref-type: ref - :expand: - :color: primary - :outline: - :class: sd-font-weight-bold - - What's next? diff --git a/docs/source/redirects.txt b/docs/source/redirects.txt index 855d62ec73..50f8ed2029 100644 --- a/docs/source/redirects.txt +++ b/docs/source/redirects.txt @@ -1,18 +1,18 @@ developer_guide/core/transport.rst topics/transport.rst developer_guide/core/extend_restapi.rst internals/rest_api.rst -get_started/index.rst intro/get_started.rst +get_started/index.rst installation/index.rst get_started/computers.rst howto/run_codes.rst get_started/codes.rst howto/run_codes.rst howto/plugins.rst howto/plugins_develop.rst howto/exploring.rst howto/query.rst import_export/main.rst internals/storage/sqlite_zip.rst internals/data_storage.rst internals/storage/sqlite_zip.rst -install/quick_installation.rst intro/get_started.rst -install/prerequisites.rst intro/get_started.rst -install/installation.rst intro/get_started.rst +install/quick_installation.rst installation/index.rst +install/prerequisites.rst installation/index.rst +install/installation.rst installation/index.rst install/configuration.rst howto/installation.rst install/updating_installation.rst howto/installation.rst -install/troubleshooting.rst intro/troubleshooting.rst +install/troubleshooting.rst installation/troubleshooting.rst restapi/index.rst reference/rest_api.rst verdi/verdi_user_guide.rst topics/cli.rst working_with_aiida/index.rst howto/index.rst diff --git a/docs/source/reference/cheatsheet.rst b/docs/source/reference/cheatsheet.rst new file mode 100644 index 0000000000..7f636c882e --- /dev/null +++ b/docs/source/reference/cheatsheet.rst @@ -0,0 +1,16 @@ +.. _reference:cheatsheet: + +================= +AiiDA cheat sheet +================= + +The AiiDA cheat sheet gives a broad overview of the most commonly used ``verdi`` commands, the inheritance hierarchy of the main AiiDA classes, their attributes and methods, as well as a showcase of the ``QueryBuilder``. + +When clicking on the embedded image, the pdf version will be opened in the browser. Where applicable, text elements contain hyperlinks to the relevant sections of the documentation. + +The file can also be :download:`downloaded ` in two-page layout for printing. + +Happy exploring! + +.. image:: ./cheatsheet/cheatsheet.png + :target: ../_static/cheatsheet_h.pdf diff --git a/docs/source/intro/_cheatsheet/cheatsheet.png b/docs/source/reference/cheatsheet/cheatsheet.png similarity index 100% rename from docs/source/intro/_cheatsheet/cheatsheet.png rename to docs/source/reference/cheatsheet/cheatsheet.png diff --git a/docs/source/intro/_cheatsheet/cheatsheet.svg b/docs/source/reference/cheatsheet/cheatsheet.svg similarity index 100% rename from docs/source/intro/_cheatsheet/cheatsheet.svg rename to docs/source/reference/cheatsheet/cheatsheet.svg diff --git a/docs/source/intro/_cheatsheet/cheatsheet_v.pdf b/docs/source/reference/cheatsheet/cheatsheet_v.pdf similarity index 100% rename from docs/source/intro/_cheatsheet/cheatsheet_v.pdf rename to docs/source/reference/cheatsheet/cheatsheet_v.pdf diff --git a/docs/source/reference/index.rst b/docs/source/reference/index.rst index f0814e41bc..8553b84363 100644 --- a/docs/source/reference/index.rst +++ b/docs/source/reference/index.rst @@ -8,4 +8,5 @@ Reference command_line api/index rest_api + cheatsheet _changelog.md diff --git a/docs/source/topics/storage.rst b/docs/source/topics/storage.rst index 59d6761360..0d3b68d656 100644 --- a/docs/source/topics/storage.rst +++ b/docs/source/topics/storage.rst @@ -122,9 +122,9 @@ The command requires the PostgreSQL database to already exist and to be able to .. tip:: - Try the ``verdi quicksetup`` command to have the PostgreSQL database automatically created. + Try the ``verdi presto --use-postgres`` command to have the PostgreSQL database automatically created. Certain systems require root access to do so, causing the command to fail if it cannot obtain root access. - In this case, the database should be created manually (see :ref:`intro:install:database` for details). + In this case, the database should be created manually (see :ref:`installation:guide-complete:create-profile:core-psql-dos` for details). Once created, a profile can be created using the database with the command ``verdi profile setup core.psql_dos``. diff --git a/docs/source/tutorials/basic.md b/docs/source/tutorials/basic.md index 05bede4d50..f446cf2169 100644 --- a/docs/source/tutorials/basic.md +++ b/docs/source/tutorials/basic.md @@ -31,7 +31,7 @@ At the end of this tutorial, you will know how to: :::{important} If you are working on your own machine, note that the tutorial assumes that you have a working AiiDA installation and have set up your AiiDA profile in the current Python environment. -If this is not the case, consult the {ref}`getting started page`. +If this is not the case, consult the {ref}`getting started page`. ::: :::{tip}