CP2K GPU运行参数:MPI、OMP与后端日志
给出P100/V100原生提交、每卡一MPI两线程、完整PBE/HF输入和判断实际GPU路径的日志操作。
阅读本篇目录
先固定科学输入,再调整并行参数,才能区分“计算方法变了”和“同一任务运行更快了”。本篇展示GPU专题真正使用的输入与命令。
1. 下载#
下载CP2K GPU输入、实际输出、CSV与绘图资料包,保存为example.zip:
python3 -m zipfile -e example.zip .
cd cp2k-gpu-practical-data
bash results.sh
bash read.sh
bash plot.sh
资料来自2026年10月22日实际Slurm计算;本次复用结果,不重新跑性能任务。data/是逐次数据,reference-results/保存原生输出,figures/为可重画图片。
2. 看完整PBE32输入#
cat inputs/pbe32.inp
展开96原子PBE32完整输入
&GLOBAL
EPS_CHECK_DIAG 1.0E-12
PRINT_LEVEL MEDIUM
PROJECT pbe32
RUN_TYPE ENERGY_FORCE
&END GLOBAL
&MOTION
&MD
ENSEMBLE NVE
STEPS 10
TEMPERATURE 300.0
TIMESTEP 0.5
&END MD
&END MOTION
&FORCE_EVAL
METHOD QS
&DFT
BASIS_SET_FILE_NAME GTH_BASIS_SETS
POTENTIAL_FILE_NAME POTENTIAL
&MGRID
CUTOFF 280
REL_CUTOFF 30
&END MGRID
&QS
EPS_DEFAULT 1.0E-12
EXTRAPOLATION_ORDER 3
WF_INTERPOLATION PS
&END QS
&SCF
EPS_SCF 1.0E-7
MAX_SCF 300
SCF_GUESS ATOMIC
&OT ON
PRECONDITIONER FULL_SINGLE_INVERSE
MINIMIZER DIIS
&END OT
# SCF_GUESS RESTART
# EPS_SCF 1.0E-7
&PRINT
&RESTART OFF
&END RESTART
&END PRINT
&END SCF
&XC
&XC_FUNCTIONAL PBE
&END XC_FUNCTIONAL
&END XC
&END DFT
&SUBSYS
&CELL
ABC 9.8528 9.8528 9.8528
&END CELL
# 32 H2O (TIP5P,1bar,300K) a = 9.8528
&COORD
O 2.280398 9.146539 5.088696
O 1.251703 2.406261 7.769908
O 1.596302 6.920128 0.656695
O 2.957518 3.771868 1.877387
O 0.228972 5.884026 6.532308
O 9.023431 6.119654 0.092451
O 7.256289 8.493641 5.772041
O 5.090422 9.467016 0.743177
O 6.330888 7.363471 3.747750
O 7.763819 8.349367 9.279457
O 8.280798 3.837153 5.799282
O 8.878250 2.025797 1.664102
O 9.160372 0.285100 6.871004
O 4.962043 4.134437 0.173376
O 2.802896 8.690383 2.435952
O 9.123223 3.549232 8.876721
O 1.453702 1.402538 2.358278
O 6.536550 1.146790 7.609732
O 2.766709 0.881503 9.544263
O 0.856426 2.075964 5.010625
O 6.386036 1.918950 0.242690
O 2.733023 4.452756 5.850203
O 4.600039 9.254314 6.575944
O 3.665373 6.210561 3.158420
O 3.371648 6.925594 7.476036
O 5.287920 3.270653 6.155080
O 5.225237 6.959594 9.582991
O 0.846293 5.595877 3.820630
O 9.785620 8.164617 3.657879
O 8.509982 4.430362 2.679946
O 1.337625 8.580920 8.272484
O 8.054437 9.221335 1.991376
H 1.762019 9.820429 5.528454
H 3.095987 9.107088 5.588186
H 0.554129 2.982634 8.082024
H 1.771257 2.954779 7.182181
H 2.112148 6.126321 0.798136
H 1.776389 7.463264 1.424030
H 3.754249 3.824017 1.349436
H 3.010580 4.524142 2.466878
H 0.939475 5.243834 6.571945
H 0.515723 6.520548 5.877445
H 9.852960 6.490366 0.393593
H 8.556008 6.860063 -0.294256
H 7.886607 7.941321 6.234506
H 7.793855 9.141028 5.315813
H 4.467366 9.971162 0.219851
H 5.758685 10.102795 0.998994
H 6.652693 7.917443 3.036562
H 6.711966 7.743594 4.539279
H 7.751955 8.745180 10.150905
H 7.829208 9.092212 8.679343
H 8.312540 3.218330 6.528858
H 8.508855 4.680699 6.189990
H 9.742249 1.704975 1.922581
H 8.799060 2.876412 2.095861
H 9.505360 1.161677 6.701213
H 9.920117 -0.219794 7.161006
H 4.749903 4.186003 -0.758595
H 5.248010 5.018415 0.403676
H 3.576065 9.078451 2.026264
H 2.720238 9.146974 3.273164
H 9.085561 4.493058 9.031660
H 9.215391 3.166305 9.749133
H 1.999705 2.060411 1.927796
H 1.824184 0.564565 2.081195
H 7.430334 0.849764 7.438978
H 6.576029 1.537017 8.482885
H 2.415851 1.576460 8.987338
H 2.276957 0.099537 9.289499
H 1.160987 1.818023 4.140602
H 0.350256 2.874437 4.860741
H 5.768804 2.638450 0.375264
H 7.221823 2.257514 0.563730
H 3.260797 5.243390 5.962382
H 3.347848 3.732214 5.988196
H 5.328688 9.073059 5.982269
H 5.007063 9.672150 7.334875
H 4.566850 6.413356 3.408312
H 3.273115 7.061666 2.963521
H 3.878372 7.435003 6.843607
H 3.884673 6.966316 8.283117
H 5.918240 3.116802 5.451335
H 5.355924 2.495093 6.711958
H 5.071858 7.687254 10.185667
H 6.106394 7.112302 9.241707
H 1.637363 5.184910 4.169264
H 0.427645 4.908936 3.301903
H 9.971698 7.227076 3.709104
H 10.647901 8.579244 3.629806
H 8.046808 5.126383 2.213838
H 7.995317 4.290074 3.474723
H 1.872601 7.864672 7.930401
H 0.837635 8.186808 8.987268
H 8.314696 10.115534 2.212519
H 8.687134 8.667252 2.448452
&END COORD
&KIND H
BASIS_SET TZV2P-GTH
POTENTIAL GTH-PBE-q1
&END KIND
&KIND O
BASIS_SET TZV2P-GTH
POTENTIAL GTH-PBE-q6
&END KIND
&END SUBSYS
&PRINT
&FORCES ON
&END FORCES
&END PRINT
&END FORCE_EVAL
本例使用OT求解与固定PBE输入。安装了ELPA不表示OT任务就在进行ELPA对角化;不要为启用某个GPU库随意切换电子求解方法。
3. MPI与线程对应关系#
本例每张GPU对应1个MPI rank,每rank使用2个OpenMP线程,MPICH用-bind-to core:2绑定。单卡脚本关键段为:
cat p100-pbe32-1g.slurm
#!/bin/bash
#SBATCH --job-name=cp2k-pbe32
#SBATCH --partition=gpu-8p100-hs
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=2
#SBATCH --gres=gpu:p100-16g:1
#SBATCH --time=00:20:00
#SBATCH --output=cp2k-%j.out
set -euo pipefail
source /etc/profile.d/lmod.sh
module purge
module load cp2k/2026.2-p100
export PYTHONDONTWRITEBYTECODE=1
python3 run.py p100 pbe32 1
多卡时ntasks、gres与程序传入卡数一起变化,cpus-per-task=2保持。本例只是测试起点,并不证明所有CP2K任务的最优线程数都是2。
先在新解压目录选择与账号资源相符的提交脚本,例如P100单卡或V100 16GB单卡:
sbatch p100-pbe32-1g.slurm
或:
sbatch v100-pbe32-1g.slurm
这是核时GPU分区的示例脚本。套餐账号应使用yskit中对应账号、平台的CP2K模板,并保持本例MPI/OMP与GPU申请匹配。每个作业只跑一个配置,先预热一次,再正式运行三次;不会自动提交全部配置。新结果可以这样读取:
python3 read-one.py work/p100-pbe32-1g/r1
完整历史比较仍用bash read.sh。不要把Slurm脚本直接改成在登录节点执行,也不要把P100模块用于V100任务。
4. 程序实际怎样启动#
加载正确模块并获得Slurm资源后,P100双卡的核心命令相当于:
export OMP_NUM_THREADS=2
export OMP_PLACES=cores
export OMP_PROC_BIND=close
/public/software/cp2k/deps/2026.2/p100/mpich-5.0.1/bin/mpiexec \
-bind-to core:2 -np 2 \
/public/software/cp2k/2026.2/p100/bin/cp2k.psmp -i input.inp -o cp2k.out
这段用于解释提交脚本,正式运行仍通过Slurm。不要额外覆盖Slurm分配的CUDA_VISIBLE_DEVICES去占用其他GPU。
5. 从日志确认路径#
grep -E 'DBCSR.*ACC|DBCSR.*GPU|CUDA|libGint' reference-results/p100/pbe32-1g-r1/cp2k.out
grep 'SCF run converged' reference-results/p100/pbe32-1g-r1/cp2k.out
grep 'ENERGY|' reference-results/p100/pbe32-1g-r1/cp2k.out
DBCSR| ACC: GPU backend is enabled T证明该后端可用;再结合GPU设备利用、具体GRID/积分调用和程序阶段判断实际负载。仅分配到显卡、版本字符串带CUDA或设备占用非零,都不足以说明每个环节在GPU计算。
DBCSR/MM_DRIVER控制主机端矩阵乘法,看到XSMM不能直接认定整个程序关闭GPU。详细选项参见DBCSR输入。
6. 精确交换输入#
cat inputs/hfx3.inp
展开三水HF与libGint完整输入
&GLOBAL
EPS_CHECK_DIAG 1.0E-8
PRINT_LEVEL Medium
PROJECT hfx3
RUN_TYPE ENERGY
&END GLOBAL
&FORCE_EVAL
METHOD QS
&DFT
BASIS_SET_FILE_NAME GTH_BASIS_SETS
POTENTIAL_FILE_NAME POTENTIAL
&MGRID
CUTOFF 80
REL_CUTOFF 40
&END MGRID
&QS
EPS_DEFAULT 1.0E-12
EXTRAPOLATION_ORDER 3
WF_INTERPOLATION PS
&END QS
&SCF
EPS_SCF 1.0E-9
MAX_SCF 200
SCF_GUESS ATOMIC
&OT ON
PRECONDITIONER FULL_SINGLE_INVERSE
MINIMIZER DIIS
&END OT
&PRINT
&RESTART OFF
&END RESTART
&END PRINT
&END SCF
&XC
&HF
HFX_LIBRARY libgint
&INTERACTION_POTENTIAL
CUTOFF_RADIUS 4.5
POTENTIAL_TYPE TRUNCATED
T_C_G_DATA t_c_g.dat
&END INTERACTION_POTENTIAL
&MEMORY
EPS_STORAGE_SCALING 0.1
MAX_MEMORY 1000
&END MEMORY
&SCREENING
EPS_SCHWARZ 1.0E-10
&END SCREENING
&END HF
&XC_FUNCTIONAL NONE
&END XC_FUNCTIONAL
&END XC
&END DFT
&SUBSYS
&CELL
ABC 9.8528 9.8528 9.8528
&END CELL
# 3 H2O cut from 32 H2O (TIP5P,1bar,300K) a = 9.8528
&COORD
O 2.280398 9.146539 5.088696
H 1.762019 9.820429 5.528454
H 3.095987 9.107088 5.588186
O 1.251703 2.406261 7.769908
H 0.554129 2.982634 8.082024
H 1.771257 2.954779 7.182181
O 1.596302 6.920128 0.656695
H 2.112148 6.126321 0.798136
H 1.776389 7.463264 1.424030
&END COORD
&KIND H
BASIS_SET DZVP-GTH
POTENTIAL GTH-PADE-q1
&END KIND
&KIND O
BASIS_SET DZVP-GTH
POTENTIAL GTH-PADE-q6
&END KIND
&END SUBSYS
&END FORCE_EVAL
保留HFX_LIBRARY、TRUNCATED势及截断半径、SCF与Schwarz阈值的一致性。改变这些条件后,能量差不再只是GPU后端差异,应重新定义数值比较。