Home

High Performance Linpack for GPUs (HPL-GPU)

This version of Linpack differs from the HPL as published by netlib.org in multiple aspects, this includes build requirements and configuration, run configuration and license. All of these are covered in this wiki. A lot of technical details regarding the modifications can be found in the references section.

Requirements

The following software is required by HPL-GPU:

• An MPI library, tested are
• OpenMPI (http://www.open-mpi.org/)
• MVPICH (http://mvapich.cse.ohio-state.edu/)
• MVAPICH2 (http://mvapich.cse.ohio-state.edu/overview/mvapich2/)
• A "BLAS" library (see http://netlib.org/blas/)
• Intel MKL (https://software.intel.com/en-us/intel-mkl)
• AMD ACML (http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/)
• GotoBLAS2 (https://www.tacc.utexas.edu/research-development/tacc-software/gotoblas2)
• Only the above libraries are tested. Some patches must be applied for GotoBLAS2 and ACML before they can be used, in order to reserve CPU cores for the GPU. Other BLAS libraries might work as well, but similar patches are likely required.
• The CALDGEMM library (https://github.com/davidrohr/caldgemm)
• CALDGEMM requires a BLAS library itself.
• CALDGEMM provides several backends for the DGEMM operation:
  • CPU Backend: No additional requirements
  • OpenCL Backend: Requires an OpenCL 1.2 capable SDK like the AMD APP SDK (http://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-app-sdk/)
  • CUDA Backend: Requires the NVIDIA CUDA SDK (https://developer.nvidia.com/cuda-downloads)
  • CAL Backend: Requires the deprecated AMD Stream SDK (the newer SDKs do no longer contain the CAL headers).
• C++ compiler
• pthreads support
• "Intel(R) Threading Building Blocks":http://www.threadingbuildingblocks.org/
• Optional (The build process will try to download and install TBB if it is not available in the source tree. As long as the computer that builds HPL has internet access you do not have to worry about installing TBB.)
• Required for improved swaps for which you ironically currently need an AMD CPU.

Building

You can either download the source from the "files section":/projects/hpl/files or pull it from the git repository.

• You need to install all the prerequisites and set certain environment variables correctly.
• You will have to build the CALDGEMM library first, after settings its build configuration file config_options.mak.
• As the original HPL, HPL-GPU requires a build configuration file called Make.ARCHNAME. The suggested method is to adapt the Make.Generic file, which is preconfigured properly for most cases.

Running

As with the original HPL the build will put a binary and a sample configuration into @bin/ARCHNAME@.

As with the original HPL the configuration file must be called @HPL.dat@. Note, however, that the options in the configuration file have changed. Therefore you cannot copy a configuration file from the original HPL, but have to create a configuration file anew. The sample file will give you a working configuration for a single nodes. Note that the HPL-GPU process will by default use all cores of a node, therefore you should only run one process per node.

License

HPL-GPU is made up of parts that are licensed under the "GNU General Public License Version 3":http://www.gnu.org/licenses/gpl-3.0.html and parts that are licensed under the . The license of each source file is noted in the header of the file. The parts licensed under the GNU General Public License Version 3 grant the following special exception:

"Use with the Original BSD License."

Notwithstanding any other provision of the GNU General Public License Version 3, you have permission to link or combine any covered work witha work licensed under the 4-clause BSD license into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the 4-clause BSD license, clause 3, concerning the requirement of acknowledgement in advertising materials will apply to the combination as such.