From f4d0adf8ca3aceab813f0bab0fdaeb9b55bf6ed6 Mon Sep 17 00:00:00 2001 From: Jinzhe Zeng Date: Sat, 31 Aug 2024 08:45:45 -0400 Subject: [PATCH] docs: reformat README and documentation (#707) Move most of the contents from `README.md` to `docs`. ## Summary by CodeRabbit - **New Features** - Introduced `BondOrderSystem` class for managing chemical bonds and formal charges. - Added `MultiSystems` class for handling multiple system data from files or directories. - Implemented new `deepmd/npy/mixed` format for combining systems with different formulas. - Launched plugin documentation to enhance extensibility for user-created plugins. - **Documentation** - Updated README for clearer installation instructions and streamlined content. - Created dedicated installation guide for user-friendly setup. - Enhanced documentation structure with new entries and improved navigation. --------- Signed-off-by: Jinzhe Zeng --- README.md | 323 +++--------------------------- docs/index.rst | 9 +- docs/installation.md | 13 ++ docs/plugin.md | 9 + docs/systems/bond_order_system.md | 51 +++++ docs/systems/index.rst | 11 + docs/systems/mixed.md | 24 +++ docs/systems/multi.md | 62 ++++++ docs/systems/system.md | 112 +++++++++++ 9 files changed, 312 insertions(+), 302 deletions(-) create mode 100644 docs/installation.md create mode 100644 docs/plugin.md create mode 100644 docs/systems/bond_order_system.md create mode 100644 docs/systems/index.rst create mode 100644 docs/systems/mixed.md create mode 100644 docs/systems/multi.md create mode 100644 docs/systems/system.md diff --git a/README.md b/README.md index ab489a75e..6f9246b1c 100644 --- a/README.md +++ b/README.md @@ -4,320 +4,47 @@ [![pip install](https://img.shields.io/pypi/dm/dpdata?label=pip%20install&logo=pypi)](https://pypi.org/project/dpdata) [![Documentation Status](https://readthedocs.org/projects/dpdata/badge/)](https://dpdata.readthedocs.io/) -**dpdata** is a python package for manipulating data formats of software in computational science, including DeePMD-kit, VASP, LAMMPS, GROMACS, Gaussian. -dpdata only works with python 3.7 or above. - +**dpdata** is a Python package for manipulating atomistic data of software in computational science. ## Installation -One can download the source code of dpdata by -```bash -git clone https://github.com/deepmodeling/dpdata.git dpdata -``` -then use `pip` to install the module from source -```bash -cd dpdata -pip install . -``` - -`dpdata` can also by install via pip without source -```bash -pip install dpdata -``` - -## Quick start - -This section gives some examples on how dpdata works. Firstly one needs to import the module in a python 3.x compatible code. -```python -import dpdata -``` -The typicall workflow of `dpdata` is - -1. Load data from vasp or lammps or deepmd-kit data files. -2. Manipulate data -3. Dump data to in a desired format - - -### Load data -```python -d_poscar = dpdata.System("POSCAR", fmt="vasp/poscar") -``` -or let dpdata infer the format (`vasp/poscar`) of the file from the file name extension -```python -d_poscar = dpdata.System("my.POSCAR") -``` -The number of atoms, atom types, coordinates are loaded from the `POSCAR` and stored to a data `System` called `d_poscar`. -A data `System` (a concept used by [deepmd-kit](https://github.com/deepmodeling/deepmd-kit)) contains frames that has the same number of atoms of the same type. The order of the atoms should be consistent among the frames in one `System`. -It is noted that `POSCAR` only contains one frame. -If the multiple frames stored in, for example, a `OUTCAR` is wanted, -```python -d_outcar = dpdata.LabeledSystem("OUTCAR") -``` -The labels provided in the `OUTCAR`, i.e. energies, forces and virials (if any), are loaded by `LabeledSystem`. It is noted that the forces of atoms are always assumed to exist. `LabeledSystem` is a derived class of `System`. +DP-GEN only supports Python 3.7 and above. You can [setup a conda/pip environment](https://docs.deepmodeling.com/faq/conda.html), and then use one of the following methods to install DP-GEN: -The `System` or `LabeledSystem` can be constructed from the following file formats with the `format key` in the table passed to argument `fmt`: +- Install via pip: `pip install dpdata` +- Install via conda: `conda install -c conda-forge dpdata` +- Install from source code: `git clone https://github.com/deepmodeling/dpdata && pip install ./dpdata` -| Software| format | multi frames | labeled | class | format key | -| ------- | :--- | :---: | :---: | :--- | :--- | -| vasp | poscar | False | False | System | 'vasp/poscar' | -| vasp | outcar | True | True | LabeledSystem | 'vasp/outcar' | -| vasp | xml | True | True | LabeledSystem | 'vasp/xml' | -| lammps | lmp | False | False | System | 'lammps/lmp' | -| lammps | dump | True | False | System | 'lammps/dump' | -| deepmd | raw | True | False | System | 'deepmd/raw' | -| deepmd | npy | True | False | System | 'deepmd/npy' | -| deepmd | raw | True | True | LabeledSystem | 'deepmd/raw' | -| deepmd | npy | True | True | LabeledSystem | 'deepmd/npy' | -| deepmd | npy | True | True | MultiSystems | 'deepmd/npy/mixed' | -| deepmd | npy | True | False | MultiSystems | 'deepmd/npy/mixed' | -| gaussian| log | False | True | LabeledSystem | 'gaussian/log'| -| gaussian| log | True | True | LabeledSystem | 'gaussian/md' | -| siesta | output | False | True | LabeledSystem | 'siesta/output'| -| siesta | aimd_output | True | True | LabeledSystem | 'siesta/aimd_output' | -| cp2k(deprecated in future) | output | False | True | LabeledSystem | 'cp2k/output' | -| cp2k(deprecated in future) | aimd_output | True | True | LabeledSystem | 'cp2k/aimd_output' | -| cp2k([plug-in](https://github.com/robinzyb/cp2kdata#plug-in-for-dpdata)) | stdout | False | True | LabeledSystem | 'cp2kdata/e_f' | -| cp2k([plug-in](https://github.com/robinzyb/cp2kdata#plug-in-for-dpdata)) | stdout | True | True | LabeledSystem | 'cp2kdata/md' | -| QE | log | False | True | LabeledSystem | 'qe/pw/scf' | -| QE | log | True | False | System | 'qe/cp/traj' | -| QE | log | True | True | LabeledSystem | 'qe/cp/traj' | -| Fhi-aims| output | True | True | LabeledSystem | 'fhi_aims/md' | -| Fhi-aims| output | False | True | LabeledSystem | 'fhi_aims/scf' | -|quip/gap|xyz|True|True|MultiSystems|'quip/gap/xyz'| -| PWmat | atom.config | False | False | System | 'pwmat/atom.config' | -| PWmat | movement | True | True | LabeledSystem | 'pwmat/movement' | -| PWmat | OUT.MLMD | True | True | LabeledSystem | 'pwmat/out.mlmd' | -| Amber | multi | True | True | LabeledSystem | 'amber/md' | -| Amber/sqm | sqm.out | False | False | System | 'sqm/out' | -| Gromacs | gro | True | False | System | 'gromacs/gro' | -| ABACUS | STRU | False | False | System | 'abacus/stru' | -| ABACUS | STRU | False | True | LabeledSystem | 'abacus/scf' | -| ABACUS | cif | True | True | LabeledSystem | 'abacus/md' | -| ABACUS | STRU | True | True | LabeledSystem | 'abacus/relax' | -| ase | structure | True | True | MultiSystems | 'ase/structure' | -| DFTB+ | dftbplus | False | True | LabeledSystem | 'dftbplus' | -| n2p2 | n2p2 | True | True | LabeledSystem | 'n2p2' | - - -The Class `dpdata.MultiSystems` can read data from a dir which may contains many files of different systems, or from single xyz file which contains different systems. - -Use `dpdata.MultiSystems.from_dir` to read from a directory, `dpdata.MultiSystems` will walk in the directory -Recursively and find all file with specific file_name. Supports all the file formats that `dpdata.LabeledSystem` supports. - -Use `dpdata.MultiSystems.from_file` to read from single file. Single-file support is available for the `quip/gap/xyz` and `ase/structure` formats. - -For example, for `quip/gap xyz` files, single .xyz file may contain many different configurations with different atom numbers and atom type. - -The following commands relating to `Class dpdata.MultiSystems` may be useful. -```python -# load data - -xyz_multi_systems = dpdata.MultiSystems.from_file( - file_name="tests/xyz/xyz_unittest.xyz", fmt="quip/gap/xyz" -) -vasp_multi_systems = dpdata.MultiSystems.from_dir( - dir_name="./mgal_outcar", file_name="OUTCAR", fmt="vasp/outcar" -) - -# use wildcard -vasp_multi_systems = dpdata.MultiSystems.from_dir( - dir_name="./mgal_outcar", file_name="*OUTCAR", fmt="vasp/outcar" -) - -# print the multi_system infomation -print(xyz_multi_systems) -print(xyz_multi_systems.systems) # return a dictionaries - -# print the system infomation -print(xyz_multi_systems.systems["B1C9"].data) - -# dump a system's data to ./my_work_dir/B1C9_raw folder -xyz_multi_systems.systems["B1C9"].to_deepmd_raw("./my_work_dir/B1C9_raw") - -# dump all systems -xyz_multi_systems.to_deepmd_raw("./my_deepmd_data/") -``` - -You may also use the following code to parse muti-system: -```python -from dpdata import LabeledSystem, MultiSystems -from glob import glob - -""" -process multi systems -""" -fs = glob("./*/OUTCAR") # remeber to change here !!! -ms = MultiSystems() -for f in fs: - try: - ls = LabeledSystem(f) - except: - print(f) - if len(ls) > 0: - ms.append(ls) - -ms.to_deepmd_raw("deepmd") -ms.to_deepmd_npy("deepmd") -``` +To test if the installation is successful, you may execute -### Access data -These properties stored in `System` and `LabeledSystem` can be accessed by operator `[]` with the key of the property supplied, for example -```python -coords = d_outcar["coords"] -``` -Available properties are (nframe: number of frames in the system, natoms: total number of atoms in the system) - -| key | type | dimension | are labels | description -| --- | --- | --- | --- | --- -| 'atom_names' | list of str | ntypes | False | The name of each atom type -| 'atom_numbs' | list of int | ntypes | False | The number of atoms of each atom type -| 'atom_types' | np.ndarray | natoms | False | Array assigning type to each atom -| 'cells' | np.ndarray | nframes x 3 x 3 | False | The cell tensor of each frame -| 'coords' | np.ndarray | nframes x natoms x 3 | False | The atom coordinates -| 'energies' | np.ndarray | nframes | True | The frame energies -| 'forces' | np.ndarray | nframes x natoms x 3 | True | The atom forces -| 'virials' | np.ndarray | nframes x 3 x 3 | True | The virial tensor of each frame - - -### Dump data -The data stored in `System` or `LabeledSystem` can be dumped in 'lammps/lmp' or 'vasp/poscar' format, for example: -```python -d_outcar.to("lammps/lmp", "conf.lmp", frame_idx=0) -``` -The first frames of `d_outcar` will be dumped to 'conf.lmp' -```python -d_outcar.to("vasp/poscar", "POSCAR", frame_idx=-1) -``` -The last frames of `d_outcar` will be dumped to 'POSCAR'. - -The data stored in `LabeledSystem` can be dumped to deepmd-kit raw format, for example -```python -d_outcar.to("deepmd/raw", "dpmd_raw") -``` -Or a simpler command: -```python -dpdata.LabeledSystem("OUTCAR").to("deepmd/raw", "dpmd_raw") -``` -Frame selection can be implemented by -```python -dpdata.LabeledSystem("OUTCAR").sub_system([0, -1]).to("deepmd/raw", "dpmd_raw") -``` -by which only the first and last frames are dumped to `dpmd_raw`. - - -### replicate -dpdata will create a super cell of the current atom configuration. -```python -dpdata.System("./POSCAR").replicate( - ( - 1, - 2, - 3, - ) -) -``` -tuple(1,2,3) means don't copy atom configuration in x direction, make 2 copys in y direction, make 3 copys in z direction. - - -### perturb -By the following example, each frame of the original system (`dpdata.System('./POSCAR')`) is perturbed to generate three new frames. For each frame, the cell is perturbed by 5% and the atom positions are perturbed by 0.6 Angstrom. `atom_pert_style` indicates that the perturbation to the atom positions is subject to normal distribution. Other available options to `atom_pert_style` are`uniform` (uniform in a ball), and `const` (uniform on a sphere). -```python -perturbed_system = dpdata.System("./POSCAR").perturb( - pert_num=3, - cell_pert_fraction=0.05, - atom_pert_distance=0.6, - atom_pert_style="normal", -) -print(perturbed_system.data) -``` - -### replace -By the following example, Random 8 Hf atoms in the system will be replaced by Zr atoms with the atom postion unchanged. -```python -s = dpdata.System("tests/poscars/POSCAR.P42nmc", fmt="vasp/poscar") -s.replace("Hf", "Zr", 8) -s.to_vasp_poscar("POSCAR.P42nmc.replace") -``` - -## BondOrderSystem -A new class `BondOrderSystem` which inherits from class `System` is introduced in dpdata. This new class contains information of chemical bonds and formal charges (stored in `BondOrderSystem.data['bonds']`, `BondOrderSystem.data['formal_charges']`). Now BondOrderSystem can only read from .mol/.sdf formats, because of its dependency on rdkit (which means rdkit must be installed if you want to use this function). Other formats, such as pdb, must be converted to .mol/.sdf format (maybe with software like open babel). -```python -import dpdata - -system_1 = dpdata.BondOrderSystem( - "tests/bond_order/CH3OH.mol", fmt="mol" -) # read from .mol file -system_2 = dpdata.BondOrderSystem( - "tests/bond_order/methane.sdf", fmt="sdf" -) # read from .sdf file -``` -In sdf file, all molecules must be of the same topology (i.e. conformers of the same molecular configuration). -`BondOrderSystem` also supports initialize from a `rdkit.Chem.rdchem.Mol` object directly. -```python -from rdkit import Chem -from rdkit.Chem import AllChem -import dpdata - -mol = Chem.MolFromSmiles("CC") -mol = Chem.AddHs(mol) -AllChem.EmbedMultipleConfs(mol, 10) -system = dpdata.BondOrderSystem(rdkit_mol=mol) -``` - -### Bond Order Assignment -The `BondOrderSystem` implements a more robust sanitize procedure for rdkit Mol, as defined in `dpdata.rdkit.santizie.Sanitizer`. This class defines 3 level of sanitization process by: low, medium and high. (default is medium). -+ low: use `rdkit.Chem.SanitizeMol()` function to sanitize molecule. -+ medium: before using rdkit, the programm will first assign formal charge of each atom to avoid inappropriate valence exceptions. However, this mode requires the rightness of the bond order information in the given molecule. -+ high: the program will try to fix inappropriate bond orders in aromatic hetreocycles, phosphate, sulfate, carboxyl, nitro, nitrine, guanidine groups. If this procedure fails to sanitize the given molecule, the program will then try to call `obabel` to pre-process the mol and repeat the sanitization procedure. **That is to say, if you wan't to use this level of sanitization, please ensure `obabel` is installed in the environment.** -According to our test, our sanitization procedure can successfully read 4852 small molecules in the PDBBind-refined-set. It is necessary to point out that the in the molecule file (mol/sdf), the number of explicit hydrogens has to be correct. Thus, we recommend to use - `obabel xxx -O xxx -h` to pre-process the file. The reason why we do not implement this hydrogen-adding procedure in dpdata is that we can not ensure its correctness. - -```python -import dpdata - -for sdf_file in glob.glob("bond_order/refined-set-ligands/obabel/*sdf"): - syst = dpdata.BondOrderSystem(sdf_file, sanitize_level="high", verbose=False) -``` -### Formal Charge Assignment -BondOrderSystem implement a method to assign formal charge for each atom based on the 8-electron rule (see below). Note that it only supports common elements in bio-system: B,C,N,O,P,S,As -```python -import dpdata - -syst = dpdata.BondOrderSystem("tests/bond_order/CH3NH3+.mol", fmt="mol") -print(syst.get_formal_charges()) # return the formal charge on each atom -print(syst.get_charge()) # return the total charge of the system +```bash +dpdata --version ``` -If a valence of 3 is detected on carbon, the formal charge will be assigned to -1. Because for most cases (in alkynyl anion, isonitrile, cyclopentadienyl anion), the formal charge on 3-valence carbon is -1, and this is also consisent with the 8-electron rule. +## Supported packages -## Mixed Type Format -The format `deepmd/npy/mixed` is the mixed type numpy format for DeePMD-kit, and can be loaded or dumped through class `dpdata.MultiSystems`. +`dpdata` is aimmed to support different kinds of atomistic packages: -Under this format, systems with the same number of atoms but different formula can be put together -for a larger system, especially when the frame numbers in systems are sparse. +- Atomistic machine learning packages, such as [DeePMD-kit](https://github.com/deepmodeling/deepmd-kit); +- Molecular dynamics packages, such as [LAMMPS](https://github.com/lammps/lammps) and [GROMACS](https://gitlab.com/gromacs/gromacs); +- Quantum chemistry packages, such as [VASP](https://www.vasp.at/), [Gaussian](https://gaussian.com), and [ABACUS](https://github.com/deepmodeling/abacus-develop); +- Atomistic visualization packages, such as [3Dmol.js](https://3dmol.csb.pitt.edu/). +- Other atomistic tools, such as [ASE](https://gitlab.com/ase/ase). +- Common formats such as `xyz`. -This also helps to mixture the type information together for model training with type embedding network. +All supported formats are listed [here](https://docs.deepmodeling.com/projects/dpdata/en/master/formats.html). -Here are examples using `deepmd/npy/mixed` format: +## Quick start -- Dump a MultiSystems into a mixed type numpy directory: -```python -import dpdata +The quickest way to convert a simple file from one format to another one is to use the [command line](https://docs.deepmodeling.com/projects/dpdata/en/master/cli.html). -dpdata.MultiSystems(*systems).to_deepmd_npy_mixed("mixed_dir") +```sh +dpdata OUTCAR -i vasp/outcar -o deepmd/npy -O deepmd_data ``` -- Load a mixed type data into a MultiSystems: -```python -import dpdata - -dpdata.MultiSystems().load_systems_from_file("mixed_dir", fmt="deepmd/npy/mixed") -``` +For advanced usage with Python APIs, [read dpdata documentation](https://docs.deepmodeling.com/projects/dpdata/). ## Plugins -One can follow [a simple example](plugin_example/) to add their own format by creating and installing plugins. It's critical to add the [Format](dpdata/format.py) class to `entry_points['dpdata.plugins']` in [`pyproject.toml`](plugin_example/pyproject.toml): -```toml -[project.entry-points.'dpdata.plugins'] -random = "dpdata_random:RandomFormat" -``` +- [cp2kdata](https://github.com/robinzyb/cp2kdata) adds the latest CP2K support for dpdata. + +For how to create your own plugin packages, [read dpdata documentation](https://docs.deepmodeling.com/projects/dpdata/). diff --git a/docs/index.rst b/docs/index.rst index 07e381de7..3d98bd566 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,22 +6,23 @@ Welcome to dpdata's documentation! ================================== +dpdata is a Python package for manipulating atomistic data of software in computational science. + .. toctree:: :maxdepth: 2 :caption: Contents: - Overview + installation + systems/index try_dpdata cli formats drivers minimizers + plugin api/api credits -.. mdinclude:: ../README.md - - Indices and tables ================== diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 000000000..064f91331 --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,13 @@ +# Installation + +DP-GEN only supports Python 3.7 and above. You can [setup a conda/pip environment](https://docs.deepmodeling.com/faq/conda.html), and then use one of the following methods to install DP-GEN: + +- Install via pip: `pip install dpdata` +- Install via conda: `conda install -c conda-forge dpdata` +- Install from source code: `git clone https://github.com/deepmodeling/dpdata && pip install ./dpdata` + +To test if the installation is successful, you may execute + +```bash +dpdata --version +``` diff --git a/docs/plugin.md b/docs/plugin.md new file mode 100644 index 000000000..44e8eabf7 --- /dev/null +++ b/docs/plugin.md @@ -0,0 +1,9 @@ +# Plugins + +One can follow a simple example under `plugin_example/` directory to add their own format by creating and installing plugins. +It's critical to add the :class:`Format` class to `entry_points['dpdata.plugins']` in `pyproject.toml`: + +```toml +[project.entry-points.'dpdata.plugins'] +random = "dpdata_random:RandomFormat" +``` diff --git a/docs/systems/bond_order_system.md b/docs/systems/bond_order_system.md new file mode 100644 index 000000000..443120a76 --- /dev/null +++ b/docs/systems/bond_order_system.md @@ -0,0 +1,51 @@ + +## BondOrderSystem +A new class :class:`BondOrderSystem` which inherits from class :class:`System` is introduced in dpdata. This new class contains information of chemical bonds and formal charges (stored in `BondOrderSystem.data['bonds']`, `BondOrderSystem.data['formal_charges']`). Now BondOrderSystem can only read from .mol/.sdf formats, because of its dependency on rdkit (which means rdkit must be installed if you want to use this function). Other formats, such as pdb, must be converted to .mol/.sdf format (maybe with software like open babel). +```python +import dpdata + +system_1 = dpdata.BondOrderSystem( + "tests/bond_order/CH3OH.mol", fmt="mol" +) # read from .mol file +system_2 = dpdata.BondOrderSystem( + "tests/bond_order/methane.sdf", fmt="sdf" +) # read from .sdf file +``` +In sdf file, all molecules must be of the same topology (i.e. conformers of the same molecular configuration). +`BondOrderSystem` also supports initialize from a :class:`rdkit.Chem.rdchem.Mol` object directly. +```python +from rdkit import Chem +from rdkit.Chem import AllChem +import dpdata + +mol = Chem.MolFromSmiles("CC") +mol = Chem.AddHs(mol) +AllChem.EmbedMultipleConfs(mol, 10) +system = dpdata.BondOrderSystem(rdkit_mol=mol) +``` + +### Bond Order Assignment +The :class:`BondOrderSystem` implements a more robust sanitize procedure for rdkit Mol, as defined in :class:`dpdata.rdkit.santizie.Sanitizer`. This class defines 3 level of sanitization process by: low, medium and high. (default is medium). ++ low: use `rdkit.Chem.SanitizeMol()` function to sanitize molecule. ++ medium: before using rdkit, the programm will first assign formal charge of each atom to avoid inappropriate valence exceptions. However, this mode requires the rightness of the bond order information in the given molecule. ++ high: the program will try to fix inappropriate bond orders in aromatic hetreocycles, phosphate, sulfate, carboxyl, nitro, nitrine, guanidine groups. If this procedure fails to sanitize the given molecule, the program will then try to call `obabel` to pre-process the mol and repeat the sanitization procedure. **That is to say, if you wan't to use this level of sanitization, please ensure `obabel` is installed in the environment.** +According to our test, our sanitization procedure can successfully read 4852 small molecules in the PDBBind-refined-set. It is necessary to point out that the in the molecule file (mol/sdf), the number of explicit hydrogens has to be correct. Thus, we recommend to use + `obabel xxx -O xxx -h` to pre-process the file. The reason why we do not implement this hydrogen-adding procedure in dpdata is that we can not ensure its correctness. + +```python +import dpdata + +for sdf_file in glob.glob("bond_order/refined-set-ligands/obabel/*sdf"): + syst = dpdata.BondOrderSystem(sdf_file, sanitize_level="high", verbose=False) +``` +### Formal Charge Assignment +BondOrderSystem implement a method to assign formal charge for each atom based on the 8-electron rule (see below). Note that it only supports common elements in bio-system: B,C,N,O,P,S,As +```python +import dpdata + +syst = dpdata.BondOrderSystem("tests/bond_order/CH3NH3+.mol", fmt="mol") +print(syst.get_formal_charges()) # return the formal charge on each atom +print(syst.get_charge()) # return the total charge of the system +``` + +If a valence of 3 is detected on carbon, the formal charge will be assigned to -1. Because for most cases (in alkynyl anion, isonitrile, cyclopentadienyl anion), the formal charge on 3-valence carbon is -1, and this is also consisent with the 8-electron rule. diff --git a/docs/systems/index.rst b/docs/systems/index.rst new file mode 100644 index 000000000..dfcbe698e --- /dev/null +++ b/docs/systems/index.rst @@ -0,0 +1,11 @@ +Systems +======= + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + system + multi + bond_order_system + mixed diff --git a/docs/systems/mixed.md b/docs/systems/mixed.md new file mode 100644 index 000000000..69bf80ea8 --- /dev/null +++ b/docs/systems/mixed.md @@ -0,0 +1,24 @@ +# Mixed Type Format + +The format `deepmd/npy/mixed` is the mixed type numpy format for DeePMD-kit, and can be loaded or dumped through class :class:`dpdata.MultiSystems`. + +Under this format, systems with the same number of atoms but different formula can be put together +for a larger system, especially when the frame numbers in systems are sparse. + +This also helps to mixture the type information together for model training with type embedding network. + +Here are examples using `deepmd/npy/mixed` format: + +- Dump a MultiSystems into a mixed type numpy directory: +```python +import dpdata + +dpdata.MultiSystems(*systems).to_deepmd_npy_mixed("mixed_dir") +``` + +- Load a mixed type data into a MultiSystems: +```python +import dpdata + +dpdata.MultiSystems().load_systems_from_file("mixed_dir", fmt="deepmd/npy/mixed") +``` diff --git a/docs/systems/multi.md b/docs/systems/multi.md new file mode 100644 index 000000000..8e47acf70 --- /dev/null +++ b/docs/systems/multi.md @@ -0,0 +1,62 @@ +# `MultiSystems` + +The Class :class:`dpdata.MultiSystems` can read data from a dir which may contains many files of different systems, or from single xyz file which contains different systems. + +Use :meth:`dpdata.MultiSystems.from_dir` to read from a directory, :class:`dpdata.MultiSystems` will walk in the directory +Recursively and find all file with specific file_name. Supports all the file formats that :class:`dpdata.LabeledSystem` supports. + +Use :meth:`dpdata.MultiSystems.from_file` to read from single file. Single-file support is available for the `quip/gap/xyz` and `ase/structure` formats. + +For example, for `quip/gap xyz` files, single .xyz file may contain many different configurations with different atom numbers and atom type. + +The following commands relating to :class:`dpdata.MultiSystems` may be useful. +```python +# load data + +xyz_multi_systems = dpdata.MultiSystems.from_file( + file_name="tests/xyz/xyz_unittest.xyz", fmt="quip/gap/xyz" +) +vasp_multi_systems = dpdata.MultiSystems.from_dir( + dir_name="./mgal_outcar", file_name="OUTCAR", fmt="vasp/outcar" +) + +# use wildcard +vasp_multi_systems = dpdata.MultiSystems.from_dir( + dir_name="./mgal_outcar", file_name="*OUTCAR", fmt="vasp/outcar" +) + +# print the multi_system infomation +print(xyz_multi_systems) +print(xyz_multi_systems.systems) # return a dictionaries + +# print the system infomation +print(xyz_multi_systems.systems["B1C9"].data) + +# dump a system's data to ./my_work_dir/B1C9_raw folder +xyz_multi_systems.systems["B1C9"].to_deepmd_raw("./my_work_dir/B1C9_raw") + +# dump all systems +xyz_multi_systems.to_deepmd_raw("./my_deepmd_data/") +``` + +You may also use the following code to parse muti-system: +```python +from dpdata import LabeledSystem, MultiSystems +from glob import glob + +""" +process multi systems +""" +fs = glob("./*/OUTCAR") # remeber to change here !!! +ms = MultiSystems() +for f in fs: + try: + ls = LabeledSystem(f) + except: + print(f) + if len(ls) > 0: + ms.append(ls) + +ms.to_deepmd_raw("deepmd") +ms.to_deepmd_npy("deepmd") +``` diff --git a/docs/systems/system.md b/docs/systems/system.md new file mode 100644 index 000000000..0401a3110 --- /dev/null +++ b/docs/systems/system.md @@ -0,0 +1,112 @@ +# `System` and `LabeledSystem` + +This section gives some examples on how dpdata works. Firstly one needs to import the module in a python 3.x compatible code. +```python +import dpdata +``` +The typicall workflow of `dpdata` is + +1. Load data from vasp or lammps or deepmd-kit data files. +2. Manipulate data +3. Dump data to in a desired format + + +### Load data +```python +d_poscar = dpdata.System("POSCAR", fmt="vasp/poscar") +``` +or let dpdata infer the format (`vasp/poscar`) of the file from the file name extension +```python +d_poscar = dpdata.System("my.POSCAR") +``` +The number of atoms, atom types, coordinates are loaded from the `POSCAR` and stored to a data :class:`System` called `d_poscar`. +A data :class:`System` (a concept used by [deepmd-kit](https://github.com/deepmodeling/deepmd-kit)) contains frames that has the same number of atoms of the same type. The order of the atoms should be consistent among the frames in one :class:`System`. +It is noted that `POSCAR` only contains one frame. +If the multiple frames stored in, for example, a `OUTCAR` is wanted, +```python +d_outcar = dpdata.LabeledSystem("OUTCAR") +``` +The labels provided in the `OUTCAR`, i.e. energies, forces and virials (if any), are loaded by :class:`LabeledSystem`. It is noted that the forces of atoms are always assumed to exist. :class:`LabeledSystem` is a derived class of :class:`System`. + +The :class:`System` or :class:`LabeledSystem` can be constructed from the following file formats with the `format key` in the table passed to argument `fmt`: + + + +### Access data +These properties stored in :class:`System` and :class:`LabeledSystem` can be accessed by operator `[]` with the key of the property supplied, for example +```python +coords = d_outcar["coords"] +``` +Available properties are (nframe: number of frames in the system, natoms: total number of atoms in the system) + +| key | type | dimension | are labels | description +| --- | --- | --- | --- | --- +| 'atom_names' | list of str | ntypes | False | The name of each atom type +| 'atom_numbs' | list of int | ntypes | False | The number of atoms of each atom type +| 'atom_types' | np.ndarray | natoms | False | Array assigning type to each atom +| 'cells' | np.ndarray | nframes x 3 x 3 | False | The cell tensor of each frame +| 'coords' | np.ndarray | nframes x natoms x 3 | False | The atom coordinates +| 'energies' | np.ndarray | nframes | True | The frame energies +| 'forces' | np.ndarray | nframes x natoms x 3 | True | The atom forces +| 'virials' | np.ndarray | nframes x 3 x 3 | True | The virial tensor of each frame + + +### Dump data +The data stored in :class:`System` or :class:`LabeledSystem` can be dumped in 'lammps/lmp' or 'vasp/poscar' format, for example: +```python +d_outcar.to("lammps/lmp", "conf.lmp", frame_idx=0) +``` +The first frames of `d_outcar` will be dumped to 'conf.lmp' +```python +d_outcar.to("vasp/poscar", "POSCAR", frame_idx=-1) +``` +The last frames of `d_outcar` will be dumped to 'POSCAR'. + +The data stored in `LabeledSystem` can be dumped to deepmd-kit raw format, for example +```python +d_outcar.to("deepmd/raw", "dpmd_raw") +``` +Or a simpler command: +```python +dpdata.LabeledSystem("OUTCAR").to("deepmd/raw", "dpmd_raw") +``` +Frame selection can be implemented by +```python +dpdata.LabeledSystem("OUTCAR").sub_system([0, -1]).to("deepmd/raw", "dpmd_raw") +``` +by which only the first and last frames are dumped to `dpmd_raw`. + + +### replicate +dpdata will create a super cell of the current atom configuration. +```python +dpdata.System("./POSCAR").replicate( + ( + 1, + 2, + 3, + ) +) +``` +tuple(1,2,3) means don't copy atom configuration in x direction, make 2 copys in y direction, make 3 copys in z direction. + + +### perturb +By the following example, each frame of the original system (`dpdata.System('./POSCAR')`) is perturbed to generate three new frames. For each frame, the cell is perturbed by 5% and the atom positions are perturbed by 0.6 Angstrom. `atom_pert_style` indicates that the perturbation to the atom positions is subject to normal distribution. Other available options to `atom_pert_style` are`uniform` (uniform in a ball), and `const` (uniform on a sphere). +```python +perturbed_system = dpdata.System("./POSCAR").perturb( + pert_num=3, + cell_pert_fraction=0.05, + atom_pert_distance=0.6, + atom_pert_style="normal", +) +print(perturbed_system.data) +``` + +### replace +By the following example, Random 8 Hf atoms in the system will be replaced by Zr atoms with the atom postion unchanged. +```python +s = dpdata.System("tests/poscars/POSCAR.P42nmc", fmt="vasp/poscar") +s.replace("Hf", "Zr", 8) +s.to_vasp_poscar("POSCAR.P42nmc.replace") +```