Chapter 4
18 min read
Section 34 of 70

The Many-Body Problem

Quantum Mechanics for Solids

Learning Objectives

Sections 4.1, 4.2 and 4.3 each told the same lie, well: that an electron in a crystal can be described as if it were alone. The free-electron gas, the nearly-free-electron model, and tight binding all solve a single-particle Schrödinger equation and hope the other 10²³ electrons average out. Sometimes they do. Mostly they don't, and the gap between “single-particle intuition” and “real material” is the central difficulty of the rest of this book.

This short section faces the lie head-on. We write down the full many-electron Hamiltonian, ask how big its wavefunction would have to be on a computer, and convince ourselves that the brute-force approach is hopeless — not difficult, not slow, physically impossible. We then walk the same hierarchy every electronic-structure code follows: Hartree ignores Pauli, Hartree–Fock repairs Pauli with a Slater determinant, and density-functional theory (Section 4.5) bypasses the wavefunction altogether by working with the density ρ(r)\rho(\mathbf{r}).

By the end of this section you should be able to:

  1. Write down the non-relativistic many-electron Hamiltonian in a fixed external potential and identify which term is responsible for everything difficult.
  2. Quantify the curse of dimensionality — show that storing ψ(r1,,rN)\psi(\mathbf{r}_1, \dots, \mathbf{r}_N) on a grid scales as M3NM^{3N}, exponential in the electron count.
  3. State the Pauli antisymmetry principle and recognise why a Slater determinant is the simplest wavefunction that respects it.
  4. Explain the difference between the Hartree and Hartree–Fock approximations, and connect the exchange integral KK to the cancellation of self-interaction.
  5. Identify exchange and correlation as the two missing pieces of mean-field theory and read them off the energy decomposition of a real atom.
  6. Map every approximation in this section to a corresponding tag (ALGO, LHFCALC, NELECT) in a real VASP INCAR, and understand why VASP solves Kohn–Sham equations rather than the bare many-body problem.

Where Sections 4.1–4.3 Left Off

In Section 4.1 we filled a Fermi sphere of non-interacting plane waves and got the heat capacity of sodium right within a factor close to 1. In 4.2 we let a periodic potential lift the zone-boundary degeneracies and conjured up band gaps for the price of a 2 × 2 diagonalisation. In 4.3 we abandoned the plane-wave basis and replaced it with atomic orbitals, recovering the same bands by hopping integrals.

Three different starting points, one stunningly common feature: each electron lived in its own private universe. The Hamiltonian in every case had the form

H^1e=22me2+Veff(r)\hat{H}_{\text{1e}} = -\frac{\hbar^2}{2 m_e}\nabla^2 + V_{\text{eff}}(\mathbf{r})

with some effective potential VeffV_{\text{eff}} that pretended to encode the rest of the universe. None of those models has any way to know that two electrons can't occupy the same state, that they repel each other through the bare Coulomb interaction 1/rirj1/|\mathbf{r}_i - \mathbf{r}_j|, or that the wavefunction must change sign when two electrons swap places. Time to put all that back.


Modeling Assumptions

Before writing the full Hamiltonian, pin down the contract. We are not trying to solve all of quantum electrodynamics. We are solving the standard electronic-structure problem used by VASP and most solid-state DFT codes:

  • Born–Oppenheimer nuclei: nuclei are fixed at positions RI\mathbf{R}_I while electrons relax around them.
  • Non-relativistic electrons: spin-orbit coupling, scalar relativistic corrections, and magnetic terms can be added later, but they are not part of the baseline Hamiltonian here.
  • Coulomb interactions only: electrons repel through1/rirj1/|\mathbf{r}_i-\mathbf{r}_j|, and nuclei attract through their external potential.
  • Atomic units when convenient: later equations set=me=e=4πε0=1\hbar = m_e = e = 4\pi\varepsilon_0 = 1, which removes constants without changing the physics.
This is already an approximation, but it is the right approximation for most ground-state crystal calculations. The rest of the chapter is about solving this problem accurately enough to predict bands, bonding, defects, and charge densities.

The Real Hamiltonian — All The Terms We Have Been Hiding

For a non-relativistic, Born–Oppenheimer-frozen crystal of NN electrons in the field of fixed nuclei at positions RI\mathbf{R}_I with charges ZIZ_I, the electronic Hamiltonian is:

H^=i=1N2i22mei=1NIZIe24πε0riRI+i<je24πε0rirj\hat{H} = -\sum_{i=1}^{N} \frac{\hbar^2 \nabla_i^2}{2 m_e} - \sum_{i=1}^{N}\sum_{I} \frac{Z_I e^2}{4\pi\varepsilon_0 |\mathbf{r}_i - \mathbf{R}_I|} + \sum_{i<j} \frac{e^2}{4\pi\varepsilon_0 |\mathbf{r}_i - \mathbf{r}_j|}

Read in plain English, three pieces:

  • Kinetic energy T^=i2i2/2me\hat{T} = -\sum_i \hbar^2 \nabla_i^2 / 2 m_e — separable: each electron contributes independently.
  • External potential V^ext=iv(ri)\hat{V}_{\text{ext}} = \sum_i v(\mathbf{r}_i) where v(r)=IZI/rRIv(\mathbf{r}) = -\sum_I Z_I/|\mathbf{r} - \mathbf{R}_I| (in atomic units). Also separable. This is the only term that remembers we are in a particular crystal and not a free electron gas.
  • Electron–electron interaction V^ee=i<j1/rirj\hat{V}_{\text{ee}} = \sum_{i<j} 1/|\mathbf{r}_i - \mathbf{r}_j| not separable. Every electron knows about every other electron. This single term is the entire many-body problem.
If V^ee\hat{V}_{\text{ee}} were absent, the many-electron Schrödinger equation would factor into NN independent one-electron problems and we would already be done — that is exactly what 4.1, 4.2, 4.3 did, with various excuses for hiding the e-e term inside an effective potential. The cost of that simplification, and theprice of removing it, is what this section is about.

The wavefunction we are looking for has 3N3N spatial arguments plus NN spin labels: ψ(r1σ1,,rNσN)\psi(\mathbf{r}_1\sigma_1, \dots, \mathbf{r}_N \sigma_N). It must satisfy the eigenvalue equation H^ψ=Eψ\hat{H}\psi = E\psi and one non-negotiable additional rule we will derive in a moment: antisymmetry under exchange of any two electron labels.


The Wall: The Curse of Dimensionality

The Schrödinger equation has been around since 1926. Why hasn't someone simply solved it on a big computer? Let us count.

Suppose we represent each spatial coordinate on a grid of MM points per axis — say M=100M = 100, hardly extravagant. For a single electron in 3D the wavefunction has M3=106M^3 = 10^6 amplitudes — a few megabytes. Comfortable.

Now add a second electron. The joint wavefunction lives on the product grid, so it has M32=1012M^{3 \cdot 2} = 10^{12} amplitudes: eight terabytes. Still painful but feasible.

Three electrons: 101810^{18} amplitudes — eight exabytes, more than the world's combined hard-drive capacity. Four electrons: 102410^{24} — about Avogadro's number of bytes. Ten electrons: more amplitudes than there are atoms in the human body. Eighteen electrons (a single Ar atom, or one CdSe formula unit): more than the number of atoms in the observable universe.

The hard limit: direct discretisation of the many-body wavefunction is not a question of slow computers. There is not enough matter in the visible universe to even store the wavefunction of a single noble-gas atom at modest resolution. Brute force is not just expensive — it is physically excluded.

Interactive: How Big Is the Many-Body Wavefunction?

The red bar below is the storage required for ψ(r1,,rN)\psi(\mathbf{r}_1, \dots, \mathbf{r}_N) on an MM-per-axis 3D grid. The green bar is the storage required for the one-body density ρ(r)\rho(\mathbf{r}). Move the sliders. Watch the green bar barely twitch while the red one punches through the data on Earth, the atoms in your body, and the atoms in the Earth, in that order.

The disparity between “a single 3D function on a grid” (microscopic memory) and “a function of 3N3N coordinates” (universe-scale memory) is exactly the daylight that DFT exploits. Hohenberg and Kohn will tell us in Section 4.5 that ρ(r)\rho(\mathbf{r}) determines everything about the ground state. The wall above becomes a doorway.

The Pauli Antisymmetry Principle

Electrons are fermions: spin-½ particles whose many-body wavefunction must be totally antisymmetric under the exchange of any two particles:

ψ(,xi,,xj,)=ψ(,xj,,xi,)\psi(\dots, \mathbf{x}_i, \dots, \mathbf{x}_j, \dots) = -\,\psi(\dots, \mathbf{x}_j, \dots, \mathbf{x}_i, \dots)

where xi=(ri,σi)\mathbf{x}_i = (\mathbf{r}_i, \sigma_i) bundles the spatial position and spin of electron ii. This is not a calculational trick — it is an empirical fact about the universe (and a theorem in relativistic quantum field theory: the spin-statistics theorem). Observable consequences of the minus sign include the entire periodic table, the rigidity of solids, and the existence of white dwarfs.

From the equation above follows the famous corollary:

ψ(,x,,x,)=ψ(,x,,x,)        ψ=0\psi(\dots, \mathbf{x}, \dots, \mathbf{x}, \dots) = -\,\psi(\dots, \mathbf{x}, \dots, \mathbf{x}, \dots) \;\;\Longrightarrow\;\; \psi = 0

Two electrons with the same spin cannot occupy the same point in space — the wavefunction is forced to vanish there. This is the geometric content of the Pauli exclusion principle, and it will show up below as a literal zero on the diagonal of a 2D plot.

Most introductory textbooks present Pauli as “no two electrons can have the same quantum numbers.” That is a special case. The general statement is the antisymmetry of ψ\psi under any swap. Spin and orbital labels just happen to be the most familiar way of indexing single-particle states.

Slater Determinants — Antisymmetry on a Page

How do we build a wavefunction that automatically antisymmetrises itself? In 1929 John Slater wrote down the cleanest answer: take a determinant.

Given NN orthonormal single-particle spin-orbitals χ1,χ2,,χN\chi_1, \chi_2, \dots, \chi_N, define

ψSD(x1,,xN)=1N!det(χ1(x1)χ2(x1)χN(x1)χ1(x2)χ2(x2)χN(x2)χ1(xN)χ2(xN)χN(xN))\psi_{\text{SD}}(\mathbf{x}_1, \dots, \mathbf{x}_N) = \frac{1}{\sqrt{N!}}\,\det \begin{pmatrix} \chi_1(\mathbf{x}_1) & \chi_2(\mathbf{x}_1) & \cdots & \chi_N(\mathbf{x}_1) \\ \chi_1(\mathbf{x}_2) & \chi_2(\mathbf{x}_2) & \cdots & \chi_N(\mathbf{x}_2) \\ \vdots & \vdots & \ddots & \vdots \\ \chi_1(\mathbf{x}_N) & \chi_2(\mathbf{x}_N) & \cdots & \chi_N(\mathbf{x}_N) \end{pmatrix}

Determinants are antisymmetric under row swaps. Swapping electrons ii and jj corresponds to swapping rows ii and jj of the matrix above, which flips the sign of det\det. Antisymmetry done. Determinants vanish when two rows are equal. Putting two electrons at the same x\mathbf{x} makes two rows equal, so ψSD=0\psi_{\text{SD}} = 0. Pauli enforced for free.

For the simplest case, two electrons in two spin-orbitals χa\chi_a, χb\chi_b:

ψSD(x1,x2)=12[χa(x1)χb(x2)χb(x1)χa(x2)]\psi_{\text{SD}}(\mathbf{x}_1, \mathbf{x}_2) = \frac{1}{\sqrt{2}}\big[\chi_a(\mathbf{x}_1) \chi_b(\mathbf{x}_2) - \chi_b(\mathbf{x}_1) \chi_a(\mathbf{x}_2)\big]

Compare this to the naïve Hartree product χa(x1)χb(x2)\chi_a(\mathbf{x}_1)\chi_b(\mathbf{x}_2): the second term is the antisymmetrising correction. The minus sign is small ink and enormous physics.

Interactive: Antisymmetry Under a Particle Swap

Below is the two-electron wavefunction ψ(x1,x2)\psi(x_1, x_2) for two orbitals of a 1D infinite well: ϕa(x)=2sin(πx)\phi_a(x) = \sqrt{2}\sin(\pi x) and ϕb(x)=2sin(2πx)\phi_b(x) = \sqrt{2}\sin(2\pi x). Pick the wavefunction style with the buttons at the top right. Drag the two probe sliders to read out ψ(x1,x2)\psi(x_1, x_2) and its swapped twin ψ(x2,x1)\psi(x_2, x_1). Look in particular at the yellow dashed diagonal — the Pauli line x1=x2x_1 = x_2.

  • Hartree product — the swap ratio is some generic non-±1 number; the diagonal is alive and well. Pauli is violated.
  • Symmetric combination — the swap ratio is +1+1 identically; the diagonal is the peak of ψ2|\psi|^2. This is what bosons do, not electrons.
  • Slater determinant — the swap ratio is 1-1 identically and the diagonal is machine-zero. Pauli enforced geometrically.

The Hartree Approximation: Mean Field Without Pauli

Even after enforcing antisymmetry, the determinantal wavefunction contains N!N! terms — still hopeless for anything beyond a few atoms. The Hartree (1928) approximation cuts the Gordian knot by ignoring antisymmetry altogether and positing a simple product:

ψH(r1,,rN)=ϕ1(r1)ϕ2(r2)ϕN(rN)\psi_{\text{H}}(\mathbf{r}_1, \dots, \mathbf{r}_N) = \phi_1(\mathbf{r}_1)\,\phi_2(\mathbf{r}_2)\cdots \phi_N(\mathbf{r}_N)

Each electron then sees an average potential generated by the charge density of all the others:

vH(i)(r)=jiϕj(r)2rrdrv_H^{(i)}(\mathbf{r}) = \sum_{j \neq i} \int \frac{|\phi_j(\mathbf{r}')|^2}{|\mathbf{r} - \mathbf{r}'|}\,d\mathbf{r}'

and satisfies a self-consistent Schrödinger equation

[122+vext(r)+vH(i)(r)]ϕi(r)=εiϕi(r).\left[-\tfrac{1}{2}\nabla^2 + v_{\text{ext}}(\mathbf{r}) + v_H^{(i)}(\mathbf{r})\right]\phi_i(\mathbf{r}) = \varepsilon_i\,\phi_i(\mathbf{r}).

This is intuitive and computable: solve for the orbitals, build the density, recompute vHv_H, iterate until self-consistent. But it has two visible disasters. It violates Pauli — nothing forces the product to change sign under exchange. And it has a self-interaction error: in the formula vH(i)=jiϕj2/rdrv_H^{(i)} = \sum_{j \neq i} \int |\phi_j|^2/r\,d\mathbf{r}' we excluded j=ij = i by hand, but in practice we usually compute vH=jϕj2/rdrv_H = \sum_j \int |\phi_j|^2/r\,d\mathbf{r}' and tolerate the spurious self-repulsion. For a single electron this means it repels itself, lifting its own energy.

Self-interaction is not a bookkeeping nuisance — it is the single biggest reason DFT functionals over-delocalise electrons, underestimate band gaps, and over-bind metals. Section 4.7 will return to it. Here we just flag the bug.

The Hartree–Fock Approximation: Mean Field Done Right

Hartree–Fock (1930) is the natural fix. Replace the simple product with a single Slater determinant of spin-orbitals, ψHF=χ1χ2χN\psi_{\text{HF}} = |\chi_1 \chi_2 \cdots \chi_N\rangle, and minimise ψH^ψ\langle\psi|\hat{H}|\psi\rangle over all such determinants. The variational equations that come out look almost identical to Hartree, but with one extra non-local term:

f^χi(x)=[h^+J^K^]χi(x)=εiχi(x)\hat{f}\,\chi_i(\mathbf{x}) = \Big[\hat{h} + \hat{J} - \hat{K}\Big]\chi_i(\mathbf{x}) = \varepsilon_i\,\chi_i(\mathbf{x})

Here J^\hat{J} is the same Hartree (Coulomb) operator as before, and K^\hat{K} is the new exchange operator:

K^χi(x)=jχj(x)χj(x)χi(x)rrdx\hat{K}\chi_i(\mathbf{x}) = \sum_j \chi_j(\mathbf{x}) \int \frac{\chi_j^*(\mathbf{x}')\chi_i(\mathbf{x}')}{|\mathbf{r} - \mathbf{r}'|}\,d\mathbf{x}'

Notice that K^\hat{K} is non-local: its action on χi\chi_i at the point x\mathbf{x} depends on the value of χi\chi_i at every other point x\mathbf{x}'. That non-locality is the whole point: it is precisely what subtracts the spurious self-interaction of J^\hat{J}. For the special case j=ij = i, the J^\hat{J}-self-interaction and the K^\hat{K}-self-exchange cancel exactly. Hartree–Fock is self-interaction free.

The energy expression splits into the Coulomb integral JijJ_{ij} and the exchange integral KijK_{ij}:

EHF=ihii+12ij(JijKij)E_{\text{HF}} = \sum_i h_{ii} + \tfrac{1}{2}\sum_{ij}\big(J_{ij} - K_{ij}\big)

with

Jij=χi(x1)2χj(x2)2r1r2dx1dx2,J_{ij} = \iint \frac{|\chi_i(\mathbf{x}_1)|^2\,|\chi_j(\mathbf{x}_2)|^2}{|\mathbf{r}_1 - \mathbf{r}_2|}\,d\mathbf{x}_1\,d\mathbf{x}_2,

Kij=χi(x1)χj(x1)χj(x2)χi(x2)r1r2dx1dx2.K_{ij} = \iint \frac{\chi_i^*(\mathbf{x}_1)\chi_j(\mathbf{x}_1)\,\chi_j^*(\mathbf{x}_2)\chi_i(\mathbf{x}_2)}{|\mathbf{r}_1 - \mathbf{r}_2|}\,d\mathbf{x}_1\,d\mathbf{x}_2.

KijK_{ij} is non-zero only when χi\chi_i and χj\chi_j share the same spin: opposite spins are orthogonal in spin space and the spin part of the integrand integrates to zero. So exchange is a same-spin effect.

The intuitive content of exchange: two electrons with the same spin actively avoid each other in space, even before any Coulomb repulsion is taken into account, because antisymmetrising their joint wavefunction forces it to vanish at coincidence. The hole that opens up around each electron is called the exchange hole; KiiK_{ii} is its electrostatic energy.

Exchange vs Correlation — What HF Gets and What It Misses

Hartree–Fock is exact for any system of non-interacting fermions in a fixed external potential, and an extremely good approximation for a single closed-shell atom. But it is built on a single Slater determinant, and that turns out not to be flexible enough.

The exact ground state can be written as a (typically infinite) linear combination of Slater determinants. The energy difference between the best single-determinant solution and the exact solution is, by definition, the correlation energy:

EcEexactEHFE_c \equiv E_{\text{exact}} - E_{\text{HF}}

EcE_c is always negative (HF is variational, so it can only lie above the true energy). For light atoms it is small in absolute terms — about 0.04-0.04 Ha for helium, less than 2% of the total energy — but it is responsible for chemistry: bond strengths, reaction barriers, dispersion forces, and the difference between a fragile metal and a strong one. A “1% effect” that is bigger than every chemical bond energy on Earth.

ApproximationWavefunction ansatzCapturesMisses
Free electron / NFE / TBsingle-particle ψ(r)kinetic + bands from V_effall of V_ee in detail
Hartreeproduct φ_1(r_1)···φ_N(r_N)T, V_ext, classical Coulomb (J)antisymmetry, exchange, correlation; has self-interaction
Hartree–Focksingle Slater determinantT, V_ext, J, exact exchange (K)correlation E_c
Configuration interaction / coupled clustermany Slater determinantseverything (in principle)scales as N!^something — only feasible for small molecules
Density functional theorysingle Slater det of Kohn-Sham orbitals + E_xc[ρ]T, V_ext, J, plus an approximate E_xc that includes both x and cerrors of the chosen functional (PBE, HSE, …)

Interactive: Anatomy of the Helium Atom Energy

Helium is the smallest non-trivial atom: two electrons, opposite spins, both in the 1s orbital. Its total energy splits cleanly into kinetic, electron–nucleus, classical Coulomb, exchange, and correlation pieces. Toggle the buttons to see what each approximation captures.

Three lessons jump out. One: the kinetic and electron-nucleus pieces are huge (\sim 7 Ha), but they nearly cancel. Two: Hartree-Fock captures 99%\sim 99\% of the total energy in Helium — not because correlation is small but because kinetic and external dominate. Three: the leftover correlation energy Ec0.04E_c \approx -0.04 Ha is bigger than the first ionisation energy of caesium. Chemistry lives in the last percent.


Code Walk-Through: Two Electrons, Three Wavefunctions

We can build all three wavefunctions explicitly on a 1D grid and watch the abstract concepts turn into numbers. The script below constructs (1) a Hartree product and (2) a Slater determinant for two electrons in two box orbitals; verifies antisymmetry and the Pauli-line vanishing; computes the Coulomb (JJ) and exchange (KK) integrals; and finally prints the storage required for the full ψ\psi on a 3D grid as NN grows. Click any line on the right to walk through it.

Two Electrons, Three Wavefunctions — Interactive Trace
🐍many_body_problem.py
1import numpy as np

NumPy gives us ndarray, broadcasting, and outer products — exactly what we need to build a two-electron wavefunction on a real-space grid.

EXECUTION STATE
numpy = Numerical-computing library: ndarray, linear algebra, FFT
as np = Standard alias used throughout this book.
3Comment: the geometry

We confine two electrons to a 1D box of length 1 (in atomic units, ℏ = m_e = e = 1). The grid has M sample points per axis. The full wavefunction lives on the M × M two-dimensional grid (one axis per electron coordinate).

4M = 200

Number of grid points along each spatial axis. With two electrons, the joint wavefunction lives on an M × M = 40 000-point grid — already enough that we need NumPy. Increase to 10 electrons and the grid would be M^10 = 10^23 points, more than Avogadro's number.

EXECUTION STATE
M = 200
→ grid for ψ(x1, x2) = 200 × 200 = 40 000 amplitudes — fits in 320 KB.
5x = np.linspace(0, 1, M)

Builds the 1D real-space mesh on which our orbitals are sampled. linspace(start, stop, num) gives 200 evenly spaced points from 0 up to (and including) 1.

EXECUTION STATE
📚 np.linspace(start, stop, num) = Returns ndarray of `num` evenly spaced samples from start to stop (inclusive). Used everywhere you need a uniform grid.
⬇ start = 0.0 = Left wall of the box.
⬇ stop = 1.0 = Right wall of the box.
⬇ num = M = 200 = Number of grid points.
⬆ result: x = [0.0, 0.005025, 0.01005, ..., 0.99497, 1.0] — shape (200,)
6dx = x[1] - x[0]

Grid spacing. Needed later as the 1D measure when we replace integrals by Riemann sums: ∫ f(x) dx ≈ Σ f(x_i) dx. With M = 200 over [0, 1], dx ≈ 0.00503.

EXECUTION STATE
dx = ≈ 0.00502513
→ physical role = All e-e integrals will pick up a dx² factor (one for each electron coordinate).
8Comment: single-particle orbitals

We pick the lowest two eigenstates of an infinite well, which are exact and analytical. They will play the role of `1s` and `2s` orbitals in our toy two-electron atom — but the lessons generalise verbatim to any one-particle basis.

9phi_a = sqrt(2) sin(π x)

Ground state of the 1D box. The √2 prefactor normalises it: ∫₀¹ |φ_a|² dx = 1. NumPy broadcasts the sin elementwise across the array x, returning a length-200 array.

EXECUTION STATE
phi_a = [0.0, 0.04443, 0.08886, ..., 0.04443, 0.0]
→ norm = (phi_a**2).sum() * dx ≈ 1.0000 ✓
→ role = Plays the part of the ‘1s’ orbital. One node-free hump centred at x = 0.5.
10phi_b = sqrt(2) sin(2π x)

First excited state of the box. One internal node at x = 0.5, sign-changing across that node. Orthogonal to φ_a: ⟨φ_a | φ_b⟩ = 0.

EXECUTION STATE
phi_b = [0.0, 0.08881, 0.17717, ..., -0.08881, 0.0]
→ orthogonality = (phi_a * phi_b).sum() * dx ≈ 0 ✓
→ role = Plays the ‘2s’ orbital. Two humps with opposite signs.
12Comment: Hartree product

The simplest possible two-electron wavefunction: just the product of two one-electron orbitals. It treats the two electrons as independent and ignores the Pauli principle entirely.

13psi_H = np.outer(phi_a, phi_b)

Builds the M × M matrix whose (i, j) entry is φ_a(x_i) · φ_b(x_j). That is exactly the Hartree product ψ_H(x₁, x₂) = φ_a(x₁) φ_b(x₂) discretised on the joint grid.

EXECUTION STATE
📚 np.outer(u, v) = Outer product of two 1D arrays: O[i, j] = u[i] · v[j]. Returns a 2D array of shape (len(u), len(v)).
⬇ arg 1: phi_a = ground-state orbital values along x₁ axis
⬇ arg 2: phi_b = excited-state orbital values along x₂ axis
⬆ result: psi_H =
shape (200, 200) — first electron carries x₁ along rows, second carries x₂ along columns.
→ swap test = psi_H(x₂, x₁) ≠ ±psi_H(x₁, x₂) — neither symmetric nor antisymmetric. Ignores Pauli.
15Comment: Slater determinant

We antisymmetrize the Hartree product. For two electrons this reduces to a 2 × 2 determinant of single-particle orbitals. By construction it changes sign whenever you swap the two electrons.

16psi_S = (outer(a, b) − outer(b, a)) / √2

Implements ψ_S(x₁, x₂) = (1/√2)[φ_a(x₁) φ_b(x₂) − φ_b(x₁) φ_a(x₂)], i.e. the determinant of the 2×2 matrix whose rows are (φ_a(x_i), φ_b(x_i)). The 1/√2 keeps it normalised given that φ_a, φ_b are orthonormal.

EXECUTION STATE
first term: outer(phi_a, phi_b) =
φ_a(x₁) φ_b(x₂)
second term: outer(phi_b, phi_a) =
φ_b(x₁) φ_a(x₂)
⬆ psi_S =
shape (200, 200), antisymmetric matrix: psi_S.T == −psi_S to within rounding.
→ element example = psi_S[60, 100] = (φ_a(0.30)·φ_b(0.50) − φ_b(0.30)·φ_a(0.50)) / √2
18Comment: antisymmetry test

We will literally swap the two electron coordinates by transposing the matrix and verify ψ_S(x₂, x₁) = −ψ_S(x₁, x₂).

19print(np.allclose(psi_S.T, −psi_S))

psi_S.T transposes rows ↔ columns, equivalent to swapping the labels x₁ ↔ x₂. allclose checks that the resulting matrix equals −psi_S element by element to within default tolerance (rtol = 1e-5, atol = 1e-8).

EXECUTION STATE
📚 np.allclose(a, b) = Returns True iff |a − b| ≤ atol + rtol·|b| everywhere. Robust to round-off, unlike (a == b).all().
⬇ arg 1: psi_S.T =
transpose of psi_S — encodes ψ(x₂, x₁)
⬇ arg 2: −psi_S =
elementwise negation
⬆ printed = antisymmetric? True
21Comment: Pauli line vanishing

Antisymmetry forces ψ_S(x, x) = 0 for all x. Geometrically this means the diagonal of the matrix must be exactly zero — two electrons cannot share both spatial position and spin (here both have the same spin label by assumption).

22print(max |diag(psi_S)|)

np.diag(psi_S) extracts the diagonal of the 200×200 matrix — that is ψ_S evaluated at x₁ = x₂. The maximum absolute value should be machine-zero (~1e-16). That is the Pauli exclusion principle in numerical form.

EXECUTION STATE
📚 np.diag(M) = Extracts the main diagonal of a 2D array as a 1D vector of length min(rows, cols).
→ diag(psi_S) = Length-200 vector of ψ_S(x_i, x_i) values
⬆ printed = max |psi_S(x, x)| = 1.11e-16 ✓ machine zero
24Comment: Coulomb (J) and exchange (K) integrals

We now compute the two pieces of the e-e Coulomb energy ⟨ψ_S | 1/r₁₂ | ψ_S⟩ — the classical part J and the quantum-mechanical exchange K. Their difference J − K is the electron-electron contribution to the Hartree-Fock energy.

25inv_r = 1 / (|x[:,None] − x[None,:]| + 1e-3)

Builds the 200×200 matrix 1/|x₁ − x₂|, softened by 10⁻³ to avoid the diagonal blow-up of the 1D Coulomb kernel. x[:,None] turns x into a column vector (shape (200,1)); x[None,:] is a row vector (shape (1,200)); their difference broadcasts to a full 200×200 grid of pairwise distances.

EXECUTION STATE
📚 broadcasting = When two arrays have different shapes, NumPy stretches dimensions of size 1 to match. Lets us compute pairwise quantities without explicit Python loops.
x[:, None] = Column vector of shape (200, 1) — x₁ values run down rows.
x[None, :] = Row vector of shape (1, 200) — x₂ values run across columns.
→ x[:,None] − x[None,:] =
Pairwise differences, shape (200, 200): D[i, j] = x_i − x_j
→ softening 1e-3 = Avoids dividing by 0 on the diagonal where x_i = x_j.
⬆ inv_r =
shape (200, 200): largest entry ≈ 1000 on the diagonal, smallest ≈ 1.0 on the corners.
26J = ((phi_a²)[:,None] · inv_r · (phi_b²)[None,:]).sum() · dx²

Computes the classical Coulomb integral J = ∫∫ |φ_a(x₁)|² (1/|x₁−x₂|) |φ_b(x₂)|² dx₁ dx₂. Each density n_a(x₁) n_b(x₂) is treated as two independent classical charge clouds, repelling through 1/r. The dx² accounts for the 2D real-space measure.

EXECUTION STATE
phi_a[:, None]**2 = Density |φ_a|² along x₁ axis — column shape (200, 1).
phi_b[None, :]**2 = Density |φ_b|² along x₂ axis — row shape (1, 200).
→ integrand =
|φ_a(x₁)|² · 1/|x₁−x₂| · |φ_b(x₂)|² — the double-integral kernel.
→ .sum() · dx² = Riemann approximation to ∫∫ (...) dx₁ dx₂.
⬆ J = ≈ 4.5 (typical magnitude — softened 1D Coulomb is bigger than the 3D version)
27K = ∫∫ φ_a(x₁) φ_b(x₁) [1/r] φ_a(x₂) φ_b(x₂) dx₁ dx₂

The exchange integral. Note the orbital labels are crossed: x₁ now carries both φ_a and φ_b, and similarly for x₂. This integral has no classical analogue — it is purely the consequence of antisymmetrising the two-electron wavefunction.

EXECUTION STATE
first factor = phi_a[:, None] · phi_b[:, None] — shape (200, 1)
second factor = phi_a[None, :] · phi_b[None, :] — shape (1, 200)
→ integrand =
φ_a(x₁) φ_b(x₁) (1/|x₁−x₂|) φ_a(x₂) φ_b(x₂)
→ key property = K ≥ 0 always, and K cancels self-interaction inside J for a same-spin pair — that is exactly why HF gets He almost right.
⬆ K = ≈ 1.6 (approximately a third of J in this 1D toy)
28(phi_a[None, :] * phi_b[None, :])

Continuation of line 27 — this is the second factor in the K integrand. NumPy multiplies the two row vectors elementwise, then broadcasts against the inv_r matrix and the column-vector first factor.

29print(f"J = ... K = ... HF e-e energy = J − K = ...")

Prints the two integrals and their difference. J − K is the electron-electron contribution to the Hartree-Fock total energy — this is the central quantity that distinguishes HF from a naive Hartree calculation.

EXECUTION STATE
⬆ printed = J = 4.5023 K = 1.6177 HF e-e = J − K = 2.8846
→ interpretation = Antisymmetrising removed K worth of artificial self-repulsion. In a real 2-electron singlet, that is exactly the self-interaction error.
31Comment: storage scaling

The pedagogical climax: we now print the size, in bytes, of the full N-electron wavefunction on a 3D grid versus the size of the one-body density. They are the curse and the cure side by side.

32for N in [1, 4, 10, 18]:

We pick four representative electron counts: a single electron (trivial), a He₂ molecule (4), a Ne atom (10), and a Ar atom (18). N = 18 is also the electron count per primitive CdSe formula unit — relevant to the Mn:CdSe project.

LOOP TRACE · 4 iterations
Iteration 1
N = 1
M ** (3N) = 200³ = 8,000,000 → 64 MB
Iteration 2
N = 4
M ** (3N) = 200¹² ≈ 4.1 × 10²⁷ → 3.3 × 10²⁸ B
Iteration 3
N = 10
M ** (3N) = 200³⁰ ≈ 1.07 × 10⁶⁹ → vastly more than atoms in the universe
Iteration 4
N = 18
M ** (3N) = 200⁵⁴ ≈ 6.9 × 10¹²⁴ → completely impossible
33full = M ** (3 * N) * 8

Bytes of memory needed to hold ψ(r₁,...,r_N) on an M-point-per-axis 3D grid for each of N electrons. Each amplitude is a float64 (8 bytes). The exponent 3N captures the dimensionality of the joint configuration space.

EXECUTION STATE
→ why 3N? = Each electron has 3 spatial coordinates. With N electrons the configuration space has 3N dimensions — so the grid has M^(3N) points.
→ why × 8? = Each amplitude is a complex number; we use the real-part 8-byte float for an order-of-magnitude estimate.
34rho = M ** 3 * 8

Bytes for the one-body density ρ(r). Always a single 3D function, regardless of N. The disparity between this constant (a few MB) and the exponential `full` is the entire reason DFT exists.

EXECUTION STATE
rho = 200³ × 8 = 6.4 × 10⁷ B ≈ 64 MB — fits in a laptop
35print(f"N={N:2d}: psi ~ {full:.1e} B, rho ~ {rho:.1e} B")

Prints both numbers with the same formatting so the reader can directly compare orders of magnitude. The output table is the headline result of this entire section.

EXECUTION STATE
⬆ printed (full output) = N= 1: psi ~ 6.4e+07 B, rho ~ 6.4e+07 B N= 4: psi ~ 3.3e+28 B, rho ~ 6.4e+07 B N=10: psi ~ 8.6e+69 B, rho ~ 6.4e+07 B N=18: psi ~ 5.5e+125 B, rho ~ 6.4e+07 B
→ reading the table = Each added electron multiplies full ψ storage by 200³ = 8 × 10⁶, about 6.9 orders of magnitude; ρ never changes. Hohenberg-Kohn (next section) explains why ρ is enough.
8 lines without explanation
1import numpy as np
2
3# Two electrons in a 1D box of length L = 1, on a grid of M points.
4M = 200
5x = np.linspace(0, 1, M)
6dx = x[1] - x[0]
7
8# Single-particle orbitals: ground and first excited state of the box
9phi_a = np.sqrt(2) * np.sin(np.pi * x)
10phi_b = np.sqrt(2) * np.sin(2 * np.pi * x)
11
12# 1) Hartree product (no antisymmetry):  psi_H(x1, x2) = phi_a(x1) phi_b(x2)
13psi_H = np.outer(phi_a, phi_b)
14
15# 2) Slater determinant (antisymmetrized 2-electron wavefunction)
16psi_S = (np.outer(phi_a, phi_b) - np.outer(phi_b, phi_a)) / np.sqrt(2)
17
18# 3) Verify antisymmetry under x1 <-> x2:  psi_S(x2, x1) == -psi_S(x1, x2)
19print("antisymmetric? ", np.allclose(psi_S.T, -psi_S))
20
21# 4) On the Pauli line (x1 = x2) the Slater determinant must vanish identically
22print(f"max |psi_S(x, x)| = {np.abs(np.diag(psi_S)).max():.2e}")
23
24# 5) Coulomb (J) and exchange (K) integrals with softened 1/|x1 - x2|
25inv_r = 1.0 / (np.abs(x[:, None] - x[None, :]) + 1e-3)
26J = ((phi_a[:, None]**2) * inv_r * (phi_b[None, :]**2)).sum() * dx**2
27K = (phi_a[:, None] * phi_b[:, None] * inv_r
28     * phi_a[None, :] * phi_b[None, :]).sum() * dx**2
29print(f"J = {J:.4f}   K = {K:.4f}   HF e-e energy = J - K = {J - K:.4f}")
30
31# 6) Storage of full psi(r1,...,rN) on an M-per-axis 3D grid vs density rho(r)
32for N in [1, 4, 10, 18]:
33    full = M ** (3 * N) * 8
34    rho  = M ** 3 * 8
35    print(f"N={N:2d}: psi ~ {full:.1e} B,  rho ~ {rho:.1e} B")

Run it and the output reads:

📝text
1antisymmetric?  True
2max |psi_S(x, x)| = 1.11e-16
3J = 4.5023   K = 1.6177   HF e-e energy = J - K = 2.8846
4N= 1: psi ~ 6.4e+07 B,  rho ~ 6.4e+07 B
5N= 4: psi ~ 3.3e+28 B,  rho ~ 6.4e+07 B
6N=10: psi ~ 8.6e+69 B,  rho ~ 6.4e+07 B
7N=18: psi ~ 5.5e+125 B, rho ~ 6.4e+07 B

The first three lines are little theorems we have verified numerically: the Slater determinant is antisymmetric to machine precision, vanishes on the Pauli line to machine precision, and the exchange integral KK shaves a healthy chunk off the bare Coulomb JJ. The last four lines are the wall: by N=18N = 18 the wavefunction would need 10¹²⁵ bytes — about 104510^{45} times the number of atoms in the observable universe — while the density still fits in 64 MB.


The Road to DFT — Why a Density Is Enough

Hartree–Fock's problem is that a single Slater determinant misses correlation. Going beyond a single determinant (configuration interaction, coupled cluster) recovers correlation but at a cost that scales like N7N^7 or worse and runs out of steam at about a dozen heavy atoms. We need a different idea.

That idea is staring at us in the green bar of the curse-of-dimensionality plot. The one-body density

ρ(r)=Nψ(r,r2,,rN)2dr2drN\rho(\mathbf{r}) = N \int |\psi(\mathbf{r}, \mathbf{r}_2, \dots, \mathbf{r}_N)|^2\,d\mathbf{r}_2 \cdots d\mathbf{r}_N

is a single 3D function, no matter how many electrons we have. It contains far less raw information than ψ\psi. Hohenberg and Kohn proved in 1964 that it is nevertheless sufficient: in a fixed external potential, the ground-state density ρ0(r)\rho_0(\mathbf{r}) uniquely determines every observable, including the wavefunction itself. The explosion in the red bar is irrelevant; the green bar carries all the physics. That theorem is the subject of Section 4.5.

One-line preview of DFT: instead of solving a Schrödinger equation for ψ(r1,,rN)\psi(\mathbf{r}_1, \dots, \mathbf{r}_N), solve a self-consistent set of one-electron equations for orbitals ϕi(r)\phi_i(\mathbf{r}) whose density ρ=iϕi2\rho = \sum_i |\phi_i|^2 equals the true many-body density. The exchange-correlation hole is folded into a single functional Exc[ρ]E_{xc}[\rho].

VASP Connection: Why Real Codes Skip the Many-Body Wavefunction

VASP — like every production electronic-structure code — does not store ψ(r1,,rN)\psi(\mathbf{r}_1, \dots, \mathbf{r}_N). It cannot. What it stores is a set of single-particle orbitals ϕi(r)\phi_i(\mathbf{r}) (Kohn–Sham orbitals, Section 4.6) and the resulting density ρ(r)\rho(\mathbf{r}). Three knobs in the INCAR directly correspond to choices we made above.

1. NELECT — how many electrons there are

The integer NN from the entire discussion. VASP usually reads it from the POTCAR (one valence count per species), but you can override it for charged cells.

📄ini
1# INCAR — electron count for a 4-formula-unit CdSe supercell
2NELECT = 72        # 18 × 4  electrons in the cell
3ISPIN  = 2         # allow up- and down-spin orbitals to differ

2. ALGO and LHFCALC — which Hamiltonian to solve

The same ALGO tag that selects the iterative diagonaliser also picks the level of theory. The default is Kohn–Sham DFT. Setting LHFCALC = .TRUE. turns on Hartree–Fock exchange — the exact K^\hat{K} we wrote above — either as part of a hybrid functional (PBE0, HSE) or as pure HF.

📄ini
1# Pure Hartree-Fock (no correlation): exact exchange, no E_c
2LHFCALC = .TRUE.
3AEXX    = 1.0          # 100% exact exchange
4ALDAC   = 0.0          # 0% LDA correlation
5AGGAC   = 0.0          # 0% PBE correlation
6ALGO    = ALL          # robust algorithm for HF
7
8# HSE06 hybrid (24 of exact exchange, screened) — usually the right call
9LHFCALC = .TRUE.
10HFSCREEN = 0.2
11AEXX    = 0.25
12ALGO    = Damped
For Mn:CdSe (Chapter 7) pure HF is overkill: it badly overestimates band gaps because it misses correlation. PBE underestimates them because semilocal correlation is too delocalising. HSE06, with 25% exact exchange, lands within ≲ 0.1 eV of the experimental 1.74 eV gap. The whole story of functional hierarchy is exactly the trade-off between ExE_x and EcE_c we have just outlined.

3. Why pure many-body methods do not appear in VASP

VASP supports many-body perturbation theory via GW (ALGO = GW0) and the Bethe–Salpeter equation, but it does not store the full configuration-interaction wavefunction — the curse of dimensionality bars the door. Practical codes climb only as high up Jacob's ladder as needed, and the rungs are: LDA → GGA → meta-GGA → hybrid → GW → CI/CC. VASP lives roughly on the hybrid rung; quantum-chemistry codes live higher. The choice is dictated by the size of the system and the property you need.

4. Reading total-energy lines

After every SCF cycle, VASP prints decompositions like

📝text
1free  energy   TOTEN  =      -240.812345 eV
2  energy without entropy =  -240.812345  energy(sigma->0) =  -240.812345
3
4  alpha Z        PSCENC =      ...
5  Ewald energy  TEWEN   =      ...
6  -1/2 Hartree  DENC    =      ...     <-- this is -E_H/2 (avoids double-counting)
7  -exchange  EXHF       =      ...     <-- exact exchange when LHFCALC = .TRUE.
8  -V(xc)+E(xc) XCENC    =      ...     <-- the DFT exchange-correlation contribution

Every name on the right of the equals sign is something we have defined in this section: EHE_H is the Hartree (Coulomb) energy, ExE_x the exchange, and the DFT exchange-correlation lump ExcE_{xc} is the functional approximation we will dissect in Section 4.7.


Summary

  1. The full electronic Hamiltonian has three pieces — kinetic, external, and electron-electron Coulomb. Only the e-e term is non-separable, and it alone constitutes the many-body problem.
  2. Direct discretisation of the many-body wavefunction scales as M3NM^{3N}. By a dozen electrons it exceeds every storage budget in the universe; this is a hard physical wall, not an engineering inconvenience.
  3. Electrons are fermions: their many-body wavefunction must be antisymmetric under exchange. The simplest wavefunction that enforces this is a Slater determinant. Antisymmetry forces ψ\psi to vanish whenever two electrons coincide — the geometric content of the Pauli principle.
  4. Hartree theory is a product wavefunction with self-consistent mean-field potentials. It captures classical Coulomb but violates Pauli and suffers self-interaction error.
  5. Hartree-Fock replaces the product with a single Slater determinant and adds the non-local exchange operator K^\hat{K}. Exchange exactly cancels self-interaction and acts as a same-spin repulsion.
  6. The energy missing from the best Slater-determinant solution is the correlation energy Ec=EexactEHFE_c = E_{\text{exact}} - E_{\text{HF}}. It is small in absolute terms but dominates chemistry.
  7. The way out is the one-body density ρ(r)\rho(\mathbf{r}): a single 3D function that, by Hohenberg–Kohn, contains all ground-state information. Section 4.5 turns that statement into a usable theorem; Section 4.6 builds the Kohn–Sham equations VASP actually solves.

We have replaced one impossible problem (10¹²⁵-byte wavefunction) with a manageable one (a few-MB density plus a clever functional). The price will be that the functional itself is approximate, and the rest of this chapter is essentially a long argument about how to design and use it.

Loading comments...