Skip to content

Commit

Permalink
pyft renamed into pyfortool
Browse files Browse the repository at this point in the history
  • Loading branch information
SebastienRietteMTO committed Nov 6, 2024
1 parent c765b3e commit 7bfd075
Show file tree
Hide file tree
Showing 9 changed files with 77 additions and 77 deletions.
8 changes: 4 additions & 4 deletions build/with_fcm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The make\_fcm.sh script will:
- populate the fcm and fiat directories on first call
- create the arch\_$ARCH directory, poulate it with arch specific files and a compilation script
- populate the compilation directory with source code (using the prep\_code.sh script). Source code
transformation can occur during this step (in particular using the PYFT_OPTS variable set
transformation can occur during this step (in particular using the PYFT\_OPTS variable set
in the architecture env file).
- execute the newly created compilation script

Expand All @@ -37,7 +37,7 @@ Such a file can be build using a gmkfile with the command:
make\_fcm.sh --gmkfile \<GMKFILE\> --arch \<new arch name\>
The env file is sourced several times:
- before populating the build directory with the source code.
Indeed, during this second step, the source code is copied or cloned and transformed by the pyft\_tool.py script.
Indeed, during this second step, the source code is copied or cloned and transformed by the pyfortool script.
The active transformations are controlled by the --noexpand/--expand options given to the
different check\_commit\_\* scripts and by the PYFT\_OPTS that can be set in the env file (only for testprogs).
The syntax is given below.
Expand All @@ -57,10 +57,10 @@ A line can take one of these two forms:
If the line doesn't contain the FILE\_DESCRIPTOR part, it applies to all source code.

For example, to transform all source code in lower case:
> export OPTS='--lowerCase'; pyft\_tool.py --pyft\_opts\_env OPTS ...
> export OPTS='--lowerCase'; pyfortool --pyfortool\_opts\_env OPTS ...
To transform all source code in lower case, except routines in turb directory which must be
in upper case but keeping the turb.F90 in lower case:
> export OPTS='--lowerCase
> ^turb/:=:--upperCase
> ^turb/turb\..90:=:--lowerCase'; pyft\_tool.py --pyft\_opts\_env OPTS ...
> ^turb/turb\..90:=:--lowerCase'; pyfortool --pyfortool\_opts\_env OPTS ...
4 changes: 2 additions & 2 deletions build/with_fcm/arch/arch-pgi_CPU_preGPU_MFflavour.env
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ descTree=${TMPDIR:-/tmp}/descTree_$$
trap "\rm -f $descTree" EXIT

#We cannot suppress non column calls for Méso-NH, stack transformation is replaced by allocatables
#source this file, then: "export PYFT_OPTS_MNH; check_commit_mesonh.sh --prep_code-opts '--pyft_opts_env PYFT_OPTS_MNH' ...."
#source this file, then: "export PYFT_OPTS_MNH; check_commit_mesonh.sh --prep_code-opts '--pyfortool_opts_env PYFT_OPTS_MNH' ...."
#The --wrapH is automatically set by prep_code
#The --mnhExpand is controlled by the different check_commit_* scripts
#--addIncludes must be before --descTree because --addIncludes removes a wrong "file" node
Expand All @@ -66,7 +66,7 @@ $specificTransfos
^(./)?micro/rain_ice.F90:=:--removeCall ELEC_TENDENCIES $PYFT_OPTS_MNH"

#For IAL, stack transformation is replaced by allocatables
#source this file, then: "export PYFT_OPTS_IAL; check_commit_ial.sh --prep_code-opts '--pyft_opts_env PYFT_OPTS_IAL' ...."
#source this file, then: "export PYFT_OPTS_IAL; check_commit_ial.sh --prep_code-opts '--pyfortool_opts_env PYFT_OPTS_IAL' ...."
#The --wrapH is automatically set by prep_code
#The --mnhExpand is controlled by the different check_commit_* scripts
#--addIncludes must be before --descTree because --addIncludes removes a wrong "file" node
Expand Down
10 changes: 5 additions & 5 deletions build/with_fcm/make_fcm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ $0 [options]
-c performs compilation
--inplace-install install or update, if needed, fiat and fcm in the directory where the current script is
--inplace-clean remove the fiat and fcm installation present in the directory where the current script is
--ssh use the ssh protocol to clone the pyft and fxtran repositories instead of https"
--ssh use the ssh protocol to clone the pyfortool and fxtran repositories instead of https"
Unrecognized options are passed to the fcm build command. Useful options include :
--new clean build tree before building
Expand Down Expand Up @@ -425,15 +425,15 @@ if [ $packupdate -eq 1 -o $packcreation -eq 1 ]; then
#This commit is ready for inclusion
PATH=$UPDATEDPATH prep_code.sh -c $commit src
else
PATH=$UPDATEDPATH prep_code.sh --pyft_opts_env PYFT_OPTS -c $commit $expand_options $subs \
-m offline src --useParallelPyft -- --tree . --descTree $descTree
PATH=$UPDATEDPATH prep_code.sh --pyfortool_opts_env PYFT_OPTS -c $commit $expand_options $subs \
-m offline src --useParallelPyForTool -- --tree . --descTree $descTree
fi
else
echo "Copy $fromdir"
mkdir src
scp -q -r $fromdir/src src/
PATH=$UPDATEDPATH prep_code.sh --pyft_opts_env PYFT_OPTS $expand_options $subs \
-m offline src --useParallelPyft -- --tree . --descTree $descTree
PATH=$UPDATEDPATH prep_code.sh --pyfortool_opts_env PYFT_OPTS $expand_options $subs \
-m offline src --useParallelPyForTool -- --tree . --descTree $descTree
fi

# Add some code
Expand Down
2 changes: 1 addition & 1 deletion src/common/micro/modi_minpack.F90
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
MODULE MODI_MINPACK
!Automatically generated by pyft
!Automatically generated by PyForTool
IMPLICIT NONE
INTERFACE
subroutine chkder ( m, n, x, fvec, fjac, ldfjac, xp, fvecp, mode, err )
Expand Down
2 changes: 1 addition & 1 deletion tools/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ This document is written using the markdown language. With pandoc, it can be con
3. AUTOMATIC INSTALLATION
The script INSTALL.sh (INSTALL.sh -h for help) handles the installation of:

- the pyft tool needed by the different check\_commit\_\*.sh scripts, depending on options used at execution.
- the pyfortool tool needed by the different check\_commit\_\*.sh scripts, depending on options used at execution.
- the testprogs reference data which are needed by the script check\_commit\_testprogs.sh. The manual
generation of these data is explained in [INSTALL\_testprogs](./INSTALL_testprogs.md)
38 changes: 19 additions & 19 deletions tools/INSTALL.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
set -e
set -o pipefail #abort if left command on a pipe fails

pyft_version=tags/0.1.1
pyfortool_version=tags/0.2.0

#This script installs PHYEX
#Call the script with the -h option to get more information.
Expand All @@ -14,14 +14,14 @@ pyft_version=tags/0.1.1
################################

function usage {
echo "Usage: $0 [-h] [--ALL] [--dataset] [--pyft] [--clean]"
echo "Usage: $0 [-h] [--ALL] [--dataset] [--pyfortool] [--clean]"
echo " --ALL Install or clean everything"
echo " --dataset Install or clean a reduced dataset for the test programs"
echo " --pyft Install or clean the pyft tool"
echo " --pyfortool Install or clean the pyfortool tool"
echo " --fiatfcm Install or clean the fiat and fcm tools"
echo " --clean Clean instead of installing"
echo " --test Perform a test"
echo " --ssh Use the ssh protocol to clone the pyft, fxtran, fiat and fcm"
echo " --ssh Use the ssh protocol to clone the pyfortool, fxtran, fiat and fcm"
echo " repositories instead of https"
echo ""
echo "If the installation has already been done, calling again this script will update"
Expand All @@ -30,7 +30,7 @@ function usage {

ALL=0
dataset=0
pyft=0
pyfortool=0
clean=0
dotest=0
ssh=0
Expand All @@ -40,7 +40,7 @@ while [ -n "$1" ]; do
case "$1" in
'--ALL') ALL=1;;
'--dataset') dataset=1;;
'--pyft') pyft=1;;
'--pyfortool') pyfortool=1;;
'--fiatfcm') fiatfcm=1;;
'--clean') clean=1;;
'--test') dotest=1;;
Expand All @@ -53,7 +53,7 @@ done

if [ $ALL == 1 ]; then
dataset=1
pyft=1
pyfortool=1
fiatfcm=1
fi

Expand Down Expand Up @@ -83,35 +83,35 @@ if [ $dataset -eq 1 ]; then
done
fi

if [ $pyft -eq 1 ]; then
if [ $pyfortool -eq 1 ]; then
cd $PHYEXTOOLSDIR/site
if [ $clean -eq 1 ]; then
rm -rf pyft
rm -rf pyfortool
else
if [ ! -d pyft ]; then
if [ ! -d pyfortool ]; then
#Install
if [ $ssh -eq 1 ]; then
git clone [email protected]:UMR-CNRM/pyft.git
git clone [email protected]:UMR-CNRM/pyfortool.git
else
git clone https://github.com/UMR-CNRM/pyft.git
git clone https://github.com/UMR-CNRM/pyfortool.git
fi
cd pyft
cd pyfortool
else
#Update
cd pyft
cd pyfortool
if [ $ssh -eq 1 ]; then
git fetch --tags [email protected]:UMR-CNRM/pyft.git
git fetch --tags [email protected]:UMR-CNRM/pyfortool.git
else
git fetch --tags https://github.com/UMR-CNRM/pyft.git
git fetch --tags https://github.com/UMR-CNRM/pyfortool.git
fi
fi
git checkout ${pyft_version}
git checkout ${pyfortool_version}
cd ..
#Install/update
if [ $ssh -eq 1 ]; then
./pyft/bin/INSTALL.sh --ssh
./pyfortool/bin/INSTALL.sh --ssh
else
./pyft/bin/INSTALL.sh
./pyfortool/bin/INSTALL.sh
fi
echo ""
echo "To use PHYEX, you only need to source the file $PHYEXTOOLSDIR/env.sh"
Expand Down
3 changes: 1 addition & 2 deletions tools/env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,4 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
export PATH=$DIR:$PATH
export PYTHONPATH=$PYTHONPATH:$DIR/../build/with_fcm/arch_gnu/build/bin

PYFT_DIR=$DIR/site/pyft
. $PYFT_DIR/bin/env.sh
. $DIR/site/pyfortool/bin/env.sh
82 changes: 41 additions & 41 deletions tools/prep_code.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ set -e
#This script can:
# - extract a tag or a commit from the PHYEX repository
# - merge code from common and model specific directories
# - apply the pyft tool
# - apply the pyfortool script
# - push the result in a new branch of the repository


Expand All @@ -16,39 +16,39 @@ PHYEXTOOLSDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
###### COMMAND LINE ARGUMENTS
function usage {
echo "Usage: $0 [-h] [-c CHECKOUT_POINT] [-m MODEL] [-D OPTION [-D OPTION [...]]]] \\"
echo " [-s SUBDIR [-s SUBDIR [...]]] [--pyft_opts_env VAR] [-v [-v [-v]]] \\"
echo " DIRECTORY -- PYFT_OPTIONS"
echo " [-s SUBDIR [-s SUBDIR [...]]] [--pyfortool_opts_env VAR] [-v [-v [-v]]] \\"
echo " DIRECTORY -- PYFORTOOL_OPTIONS"
echo "DIRECTORY directory containing the script result"
echo "-c CHECKOUT_POINT git object to checkout, can be a specific commit"
echo " or a tag with the following syntax: tags/TAG where TAG is the tag name"
echo "-m MODEL merge the code under the common directory with the code specific to MODEL model"
echo "--mnhExpand option passed to the pyft tool"
echo "--removeACC option passed to the pyft tool"
echo "--mnhExpand option passed to the pyfortool script"
echo "--removeACC option passed to the pyfortool script"
echo "-p push the result as a new branch"
echo "-s SUB subdiretory or file (under src) to consider when merging and applying pyft"
echo "-s SUB subdiretory or file (under src) to consider when merging and applying pyfortool"
echo "--renameFf rename .F90 into .f90"
echo "--ilooprm replace indexes in do loop (and mnh_expand) by :"
echo "--repo use this repository instead of the one derived (if any) from the env variables"
echo " PHYEXREPOuser (=$PHYEXREPOuser) and PHYEXREPOprotocol (=$PHYEXREPOprotocol)"
echo "-v add verbosity (up to 3 -v)"
echo "--pyft_opts_env VAR name of an environment variable containing options to use to call"
echo " the pyft_tool.py script"
echo "--useParallelPyft use the parallel version of the pyft tool (pyft_parallel_tool.py)"
echo "-- PYFT_OPTIONS everything after '--' are used as options for pyft_tool.py"
echo "--pyfortool_opts_env VAR name of an environment variable containing options to use to call"
echo " the pyfortool script"
echo "--useParallelPyForTool use the parallel version of the pyfortool script (pyfortool_parallel)"
echo "-- PYFORTOOL_OPTIONS everything after '--' are used as options for pyfortool"
echo " These options are used for all the files."
echo ""
echo "* If the -c option is not provided, DIRECTORY must already contain files and directory as if"
echo " it was the result of a git checkout"
echo "* If the -m option is used, directory tree is modified, only relevant code is kept"
echo "* If none of --mnhExpand, --removeACC, --pyft_opts_env or PYFT_OPTIONS is not used, pyft is not called at all"
echo "* If none of --mnhExpand, --removeACC, --pyfortool_opts_env or PYFORTOOL_OPTIONS is not used, pyfortool is not called at all"
echo "* -s options are mandatory for -m, -D and -p options"
echo "* -p option is allowed only if -c and -m options are provided"
echo ""
echo "Everything after the '--' is passed to pyft for source-to-source transformation"
echo "Everything after the '--' is passed to pyfortool for source-to-source transformation"
echo ""
echo "To use the pyft tool... it must be installed"
echo "To use the pyfortool script... it must be installed"
echo ""
echo "The variable name sent with --pyft_opts_env must correspond to an exported environement"
echo "The variable name sent with --pyfortool_opts_env must correspond to an exported environement"
echo "variable. The variable can contain a multi-lines string."
echo "The variable is read line by line and the last applicable line is used."
echo "A line can take one of these two forms:"
Expand All @@ -60,13 +60,13 @@ function usage {
echo " If the line doesn't contain the FILE_DESCRIPTOR part, it applies to all source code."
echo ""
echo "For example, to transform all source code in lower case:"
echo "> export OPTS='--lowerCase'; $0 --pyft\_opts\_env OPTS ..."
echo "> export OPTS='--lowerCase'; $0 --pyfortool_opts_env OPTS ..."
echo ""
echo "To transform all source code in lower case, except routines in turb directory which must be"
echo "in upper case but keeping the turb.F90 in lower case:"
echo "> export OPTS='--lowerCase "
echo "> ^turb/:=:--upperCase "
echo "> ^turb/turb\..90:=:--lowerCase'; $0 --pyft\_opts\_env OPTS ..."
echo "> ^turb/turb\..90:=:--lowerCase'; $0 --pyfortool_opts_env OPTS ..."
}

full_command="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}") $@"
Expand All @@ -75,16 +75,16 @@ separator='_' #- be carrefull, gmkpack (at least on belenos) has multiple allerg

directory=""
checkout_point=""
pyft_options=""
pyfortool_options=""
model=""
push=0
subs=""
renameFf=0
verbose=0
ilooprm=0
useParallelPyft=0
forpyft=0
pyft_opts_env=""
useParallelPyForTool=0
forpyfortool=0
pyfortool_opts_env=""

if [ -z "${PHYEXREPOprotocol-}" ]; then
repository=""
Expand All @@ -103,21 +103,21 @@ while [ -n "$1" ]; do
'-h') usage; exit;;
'-c') checkout_point="$2"; shift;;
'-m') model="$2"; shift;;
'--mnhExpand') pyft_options="$pyft_options $1";;
'--removeACC') pyft_options="$pyft_options $1";;
'--mnhExpand') pyfortool_options="$pyfortool_options $1";;
'--removeACC') pyfortool_options="$pyfortool_options $1";;
'-s') subs="$subs $2"; shift;;
'-p') push=1;;
'--renameFf') renameFf=1;;
'--ilooprm') ilooprm=1;;
'--repo') repository=$2; shift;;
'-v') verbose=$(($verbose+1));;
'--useParallelPyft') useParallelPyft=1;;
'--pyft_opts_env') pyft_opts_env=$2; shift;;
'--') forpyft=1;;
*) if [ $forpyft -eq 0 ]; then
'--useParallelPyForTool') useParallelPyForTool=1;;
'--pyfortool_opts_env') pyfortool_opts_env=$2; shift;;
'--') forpyfortool=1;;
*) if [ $forpyfortool -eq 0 ]; then
directory="$1"
else
pyft_options="$pyft_options $1"
pyfortool_options="$pyfortool_options $1"
fi;;
esac
shift
Expand Down Expand Up @@ -174,7 +174,7 @@ else
fi

###### RENAME .F90 into .f90
#This step could also be achieved by pyft_tool.py but it must be done *before* the call to pyft_tool
#This step could also be achieved by pyfortool but it must be done *before* the call to pyfortool
if [ $renameFf -eq 1 ]; then
#we use find/while/read in case the number of files is too big to be hold on a single shell line
find . -type f -name \*.F90 -print0 | \
Expand Down Expand Up @@ -290,15 +290,15 @@ if [ $ilooprm -eq 1 ]; then
done
fi

###### PYFT
if [ "$pyft_opts_env" != "" -o -n "${pyft_options-}" ]; then
[ $verbose -gt 0 ] && echo "Applying pyft_tool"
###### PyForTool
if [ "$pyfortool_opts_env" != "" -o -n "${pyfortool_options-}" ]; then
[ $verbose -gt 0 ] && echo "Applying pyfortool"

#Update PATH and PYTHONPATH if needed
if [ $useParallelPyft -eq 1 ]; then
which pyft_parallel_tool.py > /dev/null || . $PHYEXTOOLSDIR/site/pyft/bin/env.sh
if [ $useParallelPyForTool -eq 1 ]; then
which pyfortool_parallel > /dev/null || . $PHYEXTOOLSDIR/site/pyfortool/bin/env.sh
else
which pyft_tool.py > /dev/null || . $PHYEXTOOLSDIR/site/pyft/bin/env.sh
which pyfortool > /dev/null || . $PHYEXTOOLSDIR/site/pyfortool/bin/env.sh
fi

if [ -n "${model-}" ]; then
Expand All @@ -311,13 +311,13 @@ if [ "$pyft_opts_env" != "" -o -n "${pyft_options-}" ]; then
fi

extra_opts=""
if [ "$pyft_opts_env" != "" ]; then
extra_opts="--optsByEnv $pyft_opts_env"
if [ "$pyfortool_opts_env" != "" ]; then
extra_opts="--optsByEnv $pyfortool_opts_env"
fi

if [ "$extra_opts" != "" -o -n "${pyft_options-}" ]; then
if [ $useParallelPyft -eq 1 ]; then
cmd="pyft_parallel_tool.py --wrapH $pyft_options $extra_opts --nbPar 8" #--wrapH allows to deal with h files
if [ "$extra_opts" != "" -o -n "${pyfortool_options-}" ]; then
if [ $useParallelPyForTool -eq 1 ]; then
cmd="pyfortool_parallel --wrapH $pyfortool_options $extra_opts --nbPar 8" #--wrapH allows to deal with h files
[ $verbose -gt 1 ] && echo $cmd
$cmd
else
Expand All @@ -326,8 +326,8 @@ if [ "$pyft_opts_env" != "" -o -n "${pyft_options-}" ]; then
find $rep -type f -not -name '.*.swp' -not -name '.*.swo' | while read file; do
if [ "$(echo $file | grep '\.')" != '' -a $(echo $file | rev | cut -d. -f1 | rev) != 'fypp' ]; then
#Files without extension are certainly not source code files
#.fypp files cannot be read by pyft_tool.py
cmd="pyft_tool.py --wrapH $pyft_options $extra_opts" #--wrapH allows to deal with h files
#.fypp files cannot be read by pyfortool
cmd="pyfortool --wrapH $pyfortool_options $extra_opts" #--wrapH allows to deal with h files
[ $verbose -gt 1 ] && echo $cmd "$file"
$cmd "$file"
fi
Expand Down
Loading

0 comments on commit 7bfd075

Please sign in to comment.