In this section, we present in rough outline a purely classical derivation of the non-extensive entropy for an ideal gas considered by Gibbs before "correct counting" (indistinguishability of particles) is accounted for. This is followed by a brief discussion of two standard methods for making the entropy extensive. Finally, we present a third method, due to R. Swendsen, for an extensive (additive) result for the entropy of two systems if they are allowed to exchange particles with each other.
Setup We will present a simplified version of the calculation. It differs from the full calculation in three ways: • The ideal gas consists of particles confined to one spatial dimension. • We keep only the terms of order n \log(n), dropping all terms of size
n or less, where
n is the number of particles. For our purposes, this is enough, because this is where the Gibbs paradox shows up and where it must be resolved. The neglected terms play a role when the number of particles is not very large, such as in
computer simulation and
nanotechnology. Also, they are needed in deriving the
Sackur–Tetrode equation. • The subdivision of phase space into units of the
Planck constant (
h) is omitted. Instead, the entropy is defined using an integral over the "accessible" portion of phase space. This serves to highlight the purely
classical nature of the calculation. We begin with a version of
Boltzmann's entropy in which the integrand is all of accessible
phase space: S = k_\text{B} \ln\Omega = k_\text{B} \ln {\!\! \oint\limits_{H(\mathbf{p}, \mathbf{q}) = E}\!\! d^n\mathbf{p} \, d^n\mathbf{q}} The integral is restricted to a contour of available regions of phase space, subject to
conservation of energy. In contrast to the one-dimensional
line integrals encountered in elementary physics, the contour of constant energy possesses a vast number of dimensions. The justification for integrating over phase space using the canonical measure involves the assumption of equal probability. The assumption can be made by invoking the
ergodic hypothesis as well as the
Liouville's theorem of
Hamiltonian systems. (The ergodic hypothesis underlies the ability of a
physical system to reach
thermal equilibrium, but this may not always hold for computer simulations (see the
Fermi–Pasta–Ulam–Tsingou problem) or in certain real-world systems such as
non-thermal plasmas.) Liouville's theorem assumes a fixed number of dimensions that the system 'explores'. In calculations of entropy, the number of dimensions is proportional to the number of particles in the system, which forces phase space to abruptly change dimensionality when particles are added or subtracted. This may explain the difficulties in constructing a clear and simple derivation for the dependence of entropy on the number of particles. For the ideal gas, the accessible phase space is an
(n − 1)-sphere (also called a hypersphere) in the
n-dimensional \mathbf{v} space: E = \sum_{j=1}^n \frac{1}{2} m v_j^2\,, To recover the paradoxical result that entropy is not extensive, we integrate over phase space for a gas of n
monatomic particles confined to a single spatial dimension by 0. Since our only purpose is to illuminate a paradox, we simplify notation by taking the particle's mass and the Boltzmann constant equal to unity: m = k = 1. We represent points in phase-space and its
x and
v parts by
n and 2
n dimensional vectors: \boldsymbol\xi = [x_1, \dots, x_n, v_1, \dots, v_n] = [ \mathbf{x}, \mathbf{v} ] where \begin{align} \mathbf{x} &= [x_1, \dots, x_n] \\ \mathbf{v} &= [v_1, \dots, v_n]\,. \end{align} To calculate entropy, we use the fact that the (n-1)-sphere, \sum v_j^2=R^2 , has an -dimensional "
hypersurface volume" of \tilde A_n(R)=\frac{n\pi^{n/2}}{(n/2)!} R^{n-1}\,. For example, if
n = 2, the 1-sphere is the circle \tilde A_2(R)=2\pi R, a "hypersurface" in the plane. When the sphere is even-dimensional (
n odd), it will be necessary to use the
gamma function to give meaning to the factorial; see below.
Gibbs paradox in a one-dimensional gas Gibbs paradox arises when entropy is calculated using an n dimensional phase space, where n is also the number of particles in the gas. These particles are spatially confined to the one-dimensional interval \ell^n. The volume of the surface of fixed energy is \Omega_{E,\ell} = \left( \int dx_1 \int dx_2 \cdots \int dx_n\right) \underbrace{\left(\int dv_1 \int dv_2 \cdots \int dv_n\right)}_{\sum v_i^2 = 2E} The subscripts on \Omega are used to define the 'state variables' and will be discussed later, when it is argued that the number of particles, n lacks full status as a
state variable in this calculation. The integral over configuration space is \ell^n. As indicated by the underbrace, the integral over velocity space is restricted to the "surface area" of the dimensional hypersphere of radius \sqrt{2E}, and is therefore equal to the "area" of that hypersurface. Thus \Omega_{E,\ell} = \ell^n \frac{n\pi^{n/2}}{(n/2)!} (2E)^{\frac{n-1}{2}} : After approximating the factorial and dropping the small terms, we obtain \begin{align} \ln\Omega_{E,\ell} &\approx n\ln\ell + n \ln\sqrt{\frac E n} + \text{const.}\\ &= \underbrace{n\ln\frac{\ell}{n} + n \ln\sqrt{\frac E n}}_{\text{extensive}} + \,n\ln n +\text{const.}\\ \end{align} In the second expression, the term n\ln n was subtracted and added, using the fact that \ln\ell-\ln n = \ln (\ell /n). This was done to highlight exactly how the "entropy" defined here fails to be an
extensive property of matter. The first two terms are extensive: if the volume of the system doubles, but gets filled with the same density of particles with the same energy, then each of these terms doubles. But the third term is neither extensive nor
intensive and is therefore wrong. The arbitrary constant has been added because entropy can usually be viewed as being defined up to an arbitrary additive constant. This is especially necessary when entropy is defined as the logarithm of a phase space volume measured in units of momentum-position. Any change in how these units are defined will add or subtract a constant from the value of the entropy.
Two standard ways to make the classical entropy extensive As discussed
above, an
extensive form of entropy is recovered if we divide the volume of phase space, \Omega_{E,\ell}, by
n!. An alternative approach is to argue that the dependence on particle number cannot be trusted on the grounds that changing n also changes the dimensionality of phase space. Such changes in dimensionality lie outside the scope of
Hamiltonian mechanics and
Liouville's theorem. For that reason it is plausible to allow the arbitrary constant to be a function of n. Defining the function to be, f(n)=-\frac32 n\ln n, we have: \begin{align} S = \ln\Omega_{E,\ell} &\approx n\ln\ell + n \ln\sqrt{E} + \text{const.}\\ &= n\ln\ell + n \ln\sqrt{E} + f(n)\\ \ln\Omega_{E,\ell,n} &\approx n\ln\frac{\ell}{n} + n \ln\sqrt{\frac E n} +\text{const.},\\ \end{align} which has
extensive scaling: S(\alpha E,\alpha\ell,\alpha n) = \alpha\, S(E,\ell,n)
Swendsen's particle-exchange approach Following Swendsen, we allow two systems to exchange particles. This essentially 'makes room' in phase space for particles to enter or leave without requiring a change in the number of dimensions of phase space. The total number of particles is N: • n_A particles have coordinates 0. • : The total energy of these particles is E_A • n_B particles have coordinates 0. • : The total energy of these particles is E_B • The system is subject to the constraints, E_A+E_B=E and n_A+n_B=N Taking the integral over phase space, we have: \begin{align} \Omega_{E,\ell,N} &= \underbrace{\left(\int dx_? \cdots \int dx_?\right.}_{n_A\;\text{terms}} \underbrace{\left.\int dx_? \cdots \int dx_N\right)}_{n_B\;\text{terms}} \underbrace{\left(\int dv_1 \int dv_2 \cdots \int dv_N\right)}_{\sum v^2 = 2E_A\;\text{or}\;\sum v^2 = 2E_B} \\[2ex] &= {\left(\ell_A\right)}^{n_A} {\left(\ell_B\right)}^{n_B} \underbrace{\left(\frac{N!}{n_A!n_B!}\right)}_{\text{combination}} \underbrace{\left(\frac{n_A\pi^{n_A/2}}{\left(n_A/2\right)!} {\left(2E_A\right)}^{\frac{n_A - 1}{2}}\right)}_{n_A-\text{sphere}} \underbrace{\left(\frac{n_B\pi^{n_B/2}}{\left(n_B/2\right)!} {\left(2E_B\right)}^{\frac{n_B - 1}{2}}\right)}_{n_B-\text{sphere}} \end{align} The question marks (?) serve as a reminder that we may not assume that the first
nA particles (i.e. 1 through
nA) are in system
A while the other particles (
nB through
N) are in system
B. (This is further discussed in the next section.) Taking the logarithm and keeping only the largest terms, we have: S =\ln \Omega_{E,\ell,N} \approx n_A\ln\left(\frac{n_A}{\ell_A}\sqrt{\frac{E_A}{\ell_A}}\right)+ n_B\ln\left(\frac{n_B}{\ell_B}\sqrt{\frac{E_B}{\ell_B}}\right)+ N\ln N + \text{const.} This can be interpreted as the sum of the entropy of system
A and system
B, both extensive. And there is a term, N\ln N, that is not extensive.
Visualizing the particle-exchange approach in three dimensions The correct (extensive) formulas for systems
A and
B were obtained because we included all the possible ways that the two systems could exchange particles. The use of
combinations (i.e.
N particles choose
NA) was used to ascertain the number of ways N particles can be divided into system
A containing
nA particles and system
B containing
nB particles. This counting is not justified on physical grounds, but on the need to integrate over phase space. As will be illustrated below, phase space contains not a single
nA-sphere and a single
nB-sphere, but instead \binom{N}{n_A} = \frac{N!}{n_A!n_B!} pairs of
n-spheres, all situated in the same -dimensional velocity space. The integral over accessible phase space must include all of these
n-spheres, as can be seen in the figure, which shows the
actual velocity phase space associated with a gas that consists of three particles. Moreover, this gas has been divided into two systems,
A and
B. If we ignore the spatial variables, the phase space of a gas with three particles is three dimensional, which permits one to sketch the
n-spheres over which the integral over phase space must be taken. If all three particles are together, the split between the two gases is 3|0. Accessible phase space is delimited by an ordinary sphere (
2-sphere) with a radius that is either \sqrt{2E_1} or \sqrt{2E_2} (depending which system has the particles). If the split is 2|1, then phase space consists of
circles and
points. Each circle occupies two dimensions, and for each circle, two points lie on the third axis, equidistant from the center of the circle. In other words, if system
A has 2 particles, accessible phase space consists of 3 pairs of
n-spheres, each pair being a
1-sphere and a
0-sphere: \begin{align} v_1^2 + v_2^2 &= 2E_A, & v_3^2 &= 2E_B, \\ v_2^2 + v_3^2 &= 2E_A, & v_1^2 &= 2E_B, \\ v_3^2 + v_1^2 &= 2E_A, & v_2^2 &= 2E_B \end{align} Note that \binom{3}{2} = 3. == References ==