# Low-scaling post Hartree-Fock

## Brief theory recap

The low-scaling post-HF methods implemented in CP2K are SOS-MP2 and (direct) RPA. To achieve the
desired low-scaling (sub-cubic), all equations are formulated in the atomic orbital (AO) basis,
using the resolution-of-the-identity technique with a short range metric, and a numerical Laplace
transform. A central quantity of post-HF is the collection of 4-center 2-electron electron repulsion
integrals (ERIs). In the AO basis and within the RI approximation:

$$
(\mu\sigma|\nu\lambda) = (\mu\sigma\lfloor P)\ (P\lfloor Q)^{-1}\ (Q|R)\ (R\lfloor S)^{-1}\ (S\lfloor \nu\lambda)
$$

where $\lfloor$ denotes the short range RI metric, and $P,Q,R,S$ are RI basis elements. A short
range metric increases the sparsity of the 3-center RI integrals $(\mu\sigma\lfloor P)$, resulting
in more efficient sparse tensor contractions. The shortest possible ranged RI metric is the overlap.
Using the truncated Coulomb with a short truncation radius (1.5-2.0 Angstroms) is recommended in
sparser systems (e.g. water), while the overlap is typically sufficient for dense solids. The 3- and
2-center RI ERIs are calculated analytically with the libint library. The 2-center potential ERIs,
$(Q|R)$, are calculated numerically with the GPW scheme, because it uses the (very) long range 1/r
Coulomb interaction.

During post-HF calculations, the ERIs discussed above are contracted with various other quantities,
with a specific order of operations for maximum performance. The details can be found in
[](#Wilhelm2016b) for the initial RPA energy implementation. [](#Bussy2023) covers the detailed
implementation of SOS-MP2 and RPA nuclear gradients. Calculations for the energy, forces, MD, cell
and gemoetry optimization are possible. While these low-scaling methods have more favorable scaling
than their standard implementations (sub-cubic vs quartic), they suffer from a heavy prefactor.
Therefore, they are only interesting for large to very large systems (hundreds of atoms), and will
still consume a lot of resources. For smaller system, the standard MP2 and RPA implementations are
recomended (they also have gradients implemented).

```{youtube} Mi0hm54x9dA
---
url_parameters: ?start=300
align: center
privacy_mode:
---
```

## Simple examples

We cover two simple examples of low-scaling post-HF calculations: SOS-MP2 water MD, and titania
RPA@PBE0 cell optimization. These examples are meant to be representative of the available options,
and the parameters to pay attention to. They are also relatively afforable (although the titania
input might require a few nodes on an HPC). To maintain a resonable cost, the systems are small and
with low quality basis sets. In practice, such small systems should be simulated with the standard
quartic scaling implementation. Moreover, it is never recommended to run post-HF calculations with
double-zeta quality basis sets. All files required to run these examples can be found
[here](https://github.com/cp2k/cp2k-examples/tree/master/post_hartree_fock).

### SOS-MP2 liquid water

This example is a short 3 steps MD liquid water calculation (32 molecules). The RI-MP2 optimized
cc-TZ and RI_TZ basis sets are used. Whenever available, one should use pre-optimized RI basis sets
for better performance. By default, the RI basis is generated on the fly. While it yields accurate
results, the automatically generated RI basis sets are larger, and therefore less efficient. Using a
short range RI metric improves efficiency by introducing sparsity. However, in somewhat sparse
systems like water, the overlap metric lacks accuracy ([](#Bussy2023)). Using the truncated Coulomb
operator with a cutoff radius of 1.5-2.0 Angstroms as a RI metric is safer.

All post-HF calculations need an accurate and well converged SCF as a starting point. In this case,
a pure Hartree-Fock calculation is performed. Note the use of the truncated Coulomb operator as HFX
potential, with cutoff radius < L/2, as always required in periodic HF. Here, the value of 4.5
Angstroms is somewhat small, and simulating a larger water box would be advised.

A numerical Laplace transform is used to reduce the scaling of the method. The integral is replaced
by a weighted sum over a few grid points, placed according to the MINIMAX algortihm. Typically,
using 6-8 points is enough. CP2K can go up to 20 points, although more points do not necessarily
increase accuracy (and sometime bring numerical instability). The MINIMAX quadrature depends on the
band gap of the system and the total spread of the SCF eigenvalues. If the seleceted number of grid
point is innapropriate, a warning is issued.

```none
&GLOBAL
  PROJECT water32
  RUN_TYPE MD
  PRINT_LEVEL MEDIUM
&END GLOBAL
&MOTION
  &MD
    STEPS 3
  &END MD
&END MOTION
&FORCE_EVAL
  &DFT
    !cc-TZ: RI-MP2 optimized basis sets
    BASIS_SET_FILE_NAME BASIS_RI_cc-TZ
    POTENTIAL_FILE_NAME POTENTIAL
    SORT_BASIS EXP

    &MGRID
      CUTOFF 600
      REL_CUTOFF 50
      NGRIDS 5
    &END MGRID

    &SCF
      SCF_GUESS RESTART
      EPS_SCF 1.0E-6
      MAX_SCF 40
    &END SCF

    &XC
      &XC_FUNCTIONAL NONE
      &END XC_FUNCTIONAL
      &HF
        FRACTION 1.0
        &INTERACTION_POTENTIAL
          !TC potential with cutoff < L/2 for periodic HFX
          POTENTIAL_TYPE TRUNCATED
          CUTOFF_RADIUS 4.5
        &END INTERACTION_POTENTIAL
        &MEMORY
          !maximum memory allocated to HFX ERI storage, per MPI rank
          !the optimal number depends on the specifics of the computer
          MAX_MEMORY 4000
        &END MEMORY
      &END HF
      &WF_CORRELATION
        !explicit opposite-spin scaling
        SCALE_S 1.3
        &RI_SOS_MP2
          !MINIMAX quadrature by default
          QUADRATURE_POINTS 6
        &END RI_SOS_MP2
        !Enabling low-scaling
        &LOW_SCALING
          MEMORY_CUT 3
        &END LOW_SCALING
        &RI
          &RI_METRIC
            !Short range RI metric for SOS-MP2
            POTENTIAL_TYPE TRUNCATED
            CUTOFF_RADIUS 1.5
          &END RI_METRIC
        &END RI
        &INTEGRALS
          ERI_METHOD GPW
          &WFC_GPW
            !Safe yet faster than default values
            CUTOFF 200
            REL_CUTOFF 40
          &END  WFC_GPW
        &END INTEGRALS
      &END WF_CORRELATION
    &END XC
  &END DFT
  &SUBSYS
    &CELL
      ABC 9.8528 9.8528 9.8528
    &END CELL
    &TOPOLOGY
      COORD_FILE_FORMAT XYZ
      COORD_FILE_NAME ./H2O-32.xyz
    &END TOPOLOGY
    &KIND H
      BASIS_SET cc-DZ
      BASIS_SET RI_AUX RI_DZ
      POTENTIAL GTH-HF
    &END KIND
    &KIND O
      BASIS_SET cc-DZ
      BASIS_SET RI_AUX RI_DZ
      POTENTIAL GTH-HF
    &END KIND
  &END SUBSYS
&END FORCE_EVAL
```

### RPA bulk titania

This second example is 2 steps of a cell optimization of titania at the RPA@PBE0 level (72 atoms).
All low-scaling post-HF methods implemented in CP2K can be used with the ADMM (ADMM2 flavor)
approximation to speed up the Hartree-Fock calculations for the SCF, the response forces, and EXX
(in case of RPA). Here, the admm-dzp basis is used as an auxiliary ADMM basis set, and ccGRB-D as
the primary basis. The RI basis is generated automatically.

The overlap RI metric is used in this example. For dense systems, this choice of metric offers a
good tradeoff between accuracy and efficiency. Note that the `&HF` input section is repeated; it
appears once for the SCF specification, and once for the RPA. In RPA@PBE0, the SCF is first
converged at the PBE0 level of theory. Then, the RPA correlation energy is computed using the PBE0
orbitals and eigenvalues, and finally, the PBE0 exchange-correlation energy is replaced by the exact
exchange energy (EXX) calculated with the same orbitals. The `&HF` section in RPA refers to that
last step. If taken to be the same as the SCF (except for `FRACTION` and `MEMORY`), the stored ERIs
can be reused and rescaled instead of recomputed, thus saving precious computational time. The
`ADMM` keyword in RPA specifies that the EXX should be calculated within the ADMM approximation.
Note the truncated Coulomb potential used for HFX, and its cutoff radius of 4.5 Angstroms (< L/2),
necessary for periodic HFX calculations. This cutoff radius is on the shorter size, and using a
larger simulation cell would be advised.

The `MINIMAX_QUADRATURE` is explicitely requested in the input. This is a necessary step for
efficiency, as the default grid follows a Clenshaw-Curtis scheme, which requires many more
integration points for the same accuracy.

```none
&GLOBAL
  PROJECT TiO2
  RUN_TYPE CELL_OPT
  PRINT_LEVEL MEDIUM
&END GLOBAL
&MOTION
  &CELL_OPT
    MAX_ITER 2
  &END CELL_OPT
&END MOTION
&FORCE_EVAL
  STRESS_TENSOR ANALYTICAL
  &DFT
    BASIS_SET_FILE_NAME BASIS_ccGRB_UZH
    BASIS_SET_FILE_NAME BASIS_ADMM_UZH
    POTENTIAL_FILE_NAME POTENTIAL_UZH
    SORT_BASIS EXP
    !automatically generated RI basis set
    AUTO_BASIS RI_AUX SMALL

    !enabling the ADMM2 approximation
    &AUXILIARY_DENSITY_MATRIX_METHOD
      METHOD BASIS_PROJECTION
      ADMM_PURIFICATION_METHOD NONE
      EXCH_CORRECTION_FUNC PBEX
    &END AUXILIARY_DENSITY_MATRIX_METHOD

    &MGRID
      CUTOFF 600
      REL_CUTOFF 50
      NGRIDS 5
    &END MGRID

    &SCF
      EPS_SCF 1.0E-6
      MAX_SCF 20
      &OT
        PRECONDITIONER FULL_ALL
        MINIMIZER DIIS
      &END OT
      &OUTER_SCF
        MAX_SCF 5
        EPS_SCF 1.0E-6
      &END OUTER_SCF
    &END SCF

    &XC
      &XC_FUNCTIONAL
        &PBE
          SCALE_X 0.75
        &END PBE
      &END XC_FUNCTIONAL
      &HF
        FRACTION 0.25
        !always use a short range potential < L/2 in periodic HFX
        &INTERACTION_POTENTIAL
          POTENTIAL_TYPE TRUNCATED
          CUTOFF_RADIUS 4.5
        &END INTERACTION_POTENTIAL
        &MEMORY
          !maximum memory allocated to HFX ERI storage, per MPI rank
          !the optimal number depends on the specifics of the computer
          MAX_MEMORY 4000
        &END MEMORY
      &END HF
      &WF_CORRELATION
        &RI_RPA
          MINIMAX_QUADRATURE
          QUADRATURE_POINTS 6
          !calculate EXX with ADMM, using the same HF section as in SCF
          !so that the integrals can be resued without recomputing
          ADMM
          &HF
            FRACTION 1.0
            &INTERACTION_POTENTIAL
              POTENTIAL_TYPE TRUNCATED
              CUTOFF_RADIUS 4.5
            &END INTERACTION_POTENTIAL
          &END HF
        &END RI_RPA
        !enabling low-scaling
        &LOW_SCALING
          MEMORY_CUT 3
        &END LOW_SCALING
        &RI
          !overlap RI metric is appropriate for dense systems
          &RI_METRIC
            POTENTIAL_TYPE IDENTITY
          &END RI_METRIC
        &END RI
        &INTEGRALS
          ERI_METHOD GPW
          !Safe yet faster than default values
          &WFC_GPW
            CUTOFF 200
            REL_CUTOFF 40
          &END WFC_GPW
        &END INTEGRALS
      &END WF_CORRELATION
    &END XC
  &END DFT
  &SUBSYS
    &CELL
      ABC 9.330 9.330 9.107
    &END CELL
    &TOPOLOGY
      COORD_FILE_FORMAT XYZ
      COORD_FILE_NAME ./TiO2.xyz
    &END TOPOLOGY
    &KIND Ti
      BASIS_SET ccGRB-D-q12
      BASIS_SET AUX_FIT admm-dz-q12
      POTENTIAL GTH-PBE0-q12
    &END KIND
    &KIND O
      BASIS_SET ccGRB-D-q6
      BASIS_SET AUX_FIT admm-dz-q6
      POTENTIAL GTH-PBE0-q6
    &END KIND
  &END SUBSYS
&END FORCE_EVAL
```

## Important input parameters

There are a few importnat input parameters for low-scaling post-HF calculations:

- `SORT_BASIS`, in the DFT input section, should be set to EXP. This way, basis elements are sorted
  according to their exponents, leading to increased sparsity and performance.
- `EPS_FILTER`, in the LOW_SCALING input section. The bottleneck of the low-scaling post-HF
  calculations consists of sparse tensor contractions. This parameter is the threshold for block
  sparsity during calculations, with a safe default of $1.0\times10^{-9}$. A looser threshold might
  significantly speedup calculations (not recommended to go higher than $1.0\times10^{-8}$).
- `MEMORY_CUT`, in the LOW_SCALING input section. This keyword influences the batching strategy for
  large tensor contractions. In post-HF, some tensor contractions are done in multiple steps, with
  large intermediate results. Storing these intermediates can lead to memory shortage. The value of
  `MEMORY_CUT` (default of 5) indicates how large tensors are split into batches, such that smaller
  intermediates can be stored. Note that MEMORY_CUT 3 does not mean that the total memory
  consumption of the program is divided by three (initial and final tensors are not affected, as
  well as the data concerning the rest of the calculation). A higher value reduces the memory
  footprint, but leads to performance overheads.
- `RI METRIC`: the choice of RI metric is crucial for performance in periodic calculations. A
  shorter ranged RI metric will generally be more efficient, while a longer ranged metric (e.g. TC
  with a cutoff radius of 1.5 Angstrom) can be more accurate. See [](#Bussy2023) for a full
  discussion.
