Cours de physique

As the concept of information takes a more fundamental role in the theory, it becomes more of a starting point, and hence an undefined term for which some intuitive meaning has to be assumed. One could say that a bit is the answer to one yes/no question, but this is effectively circular: one now has to say what “one” and “question” mean. “Uncertainty” is similarly undefined - certainty and uncertainty are, at bottom, feelings that we experience, not concepts that we can derive from something else.

Good question. Hopefully the next two sections will shed some light on this. We can discuss it some more in July.

Do you mean, how would you measure the effect of such a (hypothesized) signal and hence get evidence for its existence, or how would you measure the signal itself, using something other than a human as a detector?

Markov blankets (MBs) are defined for classical, causal processes that can be represented as directed, acyclic graphs (DAGs), so diagrams in which states are connected by unidirectional arrows and there are no cycles, meaning no circular (or backwards in time) causation. In this setting, the MB around some state (or set of states) X is the set of states that send arrows to X (”parents of X”), plus the set of states that receive arrows from X (”children of X”), plus any states that are parents of X’s children. Hence all causal inputs to and outputs from X traverse the MB states. Since the MB is a set of states, it is physical. Since it effectively encodes all information that X receives from and sends to the outside world, it is informational. We will see in the June session how a holographic screen functions as an MB, even though it is defined differently. Hopefully your second question will be answered by the end of this course.

Black holes (BHs) are a prediction of classical general relativity (GR). In GR, mass is curvature, so effectively tension, in spacetime. A BH is a region of spacetime in which the curvature is so large that light cannot escape. If spacetime was two-dimensional, it would look like this picture, which I got from the math stack exchange. The x and y axes are spacetime, the z axis is energy, in the form of gravitational potential energy. Anything that falls into a BH increases the BH’s mass and hence its curvature. The standard astrophysical model is that BH are formed when very massive stars run out of fuel, stop radiating energy, and collapse due to their own gravity. There is also the idea that BH form from anisotropies in the very early universe. Both of these ideas are classical. Observationally, BH appear to exist in the centers of galaxies, including our own. These BH are very large, e.g. the size of the whole solar system, so have orders of magnitude more mass than the sun. There is a huge literature on these objects - start with the wikipedia page and follow the links.

Yes - logs make entropy additive and so much easier to think about.

Schrodinger introduced this idea of “resisting entropy” in his book “What is Life” (1944). “Staying together” as an identifiable “thing” takes energy. Relatively simple, solid things like crystals can resist falling apart just with chemical binding energy. Relatively complex, soft things like cells or organisms have to use metabolic energy from controlled chemical reactions. The energy that isn’t stored in chemical bonds is radiated back into the environment as heat, or via the definition dQ = dS/T, as entropy. Hence they are called “dissipative systems” that is, systems that dissipate entropy. This idea is the foundation of theories of self-organization. One can view all of biology as the study of self-organizing systems. Darwinian fitness is, technically, the probability of one’s genotype or phenotype (depending on the formulation) appearing in the next generation, so it depends on successful dissipation of entropy, at least for long enough to reproduce.

Adding action = finite energy in finite time doesn’t increase the number of possible states (i.e. doesn’t change the state space), it just changes the probability distribution of state occupancy. When we add energy, more energetic states have a higher probability of being occupied. Adding or subtracting possible states (changing the state space) typically involves adding or subtracting matter and hence particle degrees of freedom. This is where black holes are a limiting case - adding matter, or even energy in the form of photons, is adding curvature, which makes the horizon beyond which light can’t escape larger. This adds states (i.e. entropy) to the horizon, which is effectively the boundary between the BH and the outside world.

Yes. This will be the main theme of the July session. Whether a behavior is “informative” or “noise” depends on who is looking and how they look. Think about colliding a proton with an antiproton at the LHC. If I look with a complicated detector that measures outgoing particle momenta, charge, and spins, I get a lot of information to test the Standard Model. But if I just measure with a calorimeter, I just get heat, so from my perspective I’ve just increased the system’s entropy. This gets back to Boltzmann’s insight, which is that entropy is a measure of the number of states that I can’t distinguish with whatever measurements I am deploying.

In a sense, yes. Time is perpendicular to our 3 dimensions of space, and matter and energy (and we ourselves) exist in time. Wick is pointing to the intimate relationship between how we measure time and how we determine (or judge) that something maintains its identity through time, i.e. stays “the same thing” through time. To tell time by a clock, I have to be sure I’m looking at the same clock, set the same way, etc. We’ll get into this in July and August.

Yes. Entropy was thought of as objective = observer independent in classical physics because essentially everything was thought of as objective - there was only Galilean and then Einstein’s notions of relativity to take account of. But it did depend from the start of what measurements were made and what the observer knew (in Bayesian terms, what the observer’s priors were) beforehand. A measurement always employs a communication channel, as Shannon recognized. We can think of the channel states as a Markov blanket or a boundary, as the FEP does - the observer is always on one side of this channel and the observed system (black hole, ideal gas, whatever) is on the other side. If the observer is to acquire information and hence change her uncertainty, she has to have a memory she can access. This is what photons don’t have - a photon can encode a memory for me, but it doesn’t encode a memory for itself.

The pollen keeps dancing because it, and the liquid, are embedded in an environment at finite temperature and open to thermodynamic exchange with that environment. It would stop if the whole business was frozen to absolute zero. We can see the dancing because the system is being bathed in light (i.e. energy), some of which is reflected back to us. This is an example of how observation requires thermodynamic exchange, and of why the classical idea of “passive observation” doesn’t actually make sense. To get information about the pollen, we have to make observations over time (e.g. with a video camera) and spend energy writing the results to memory (e.g. the memory chip in the video camera, which requires DC power). This shows how we, as observers, also have to be open to thermodynamic exchange, as do all of our bits of laboratory apparatus. In the presentation, I simplified this by just considering a system and its environment (which in this case includes us, our apparatus, etc). In August or September we will get to an explicit picture of systems that have multiple components (e.g. us, apparatus, the rest of the environment) that exchange classical information. At either level of detail, though, getting and recording information requires thermodynamic exchange.

Philosophers and historians of physics debate this question; a recent example is. My own view is that he is pointing out that the “classical reality” that “laws of physics” are supposed to describe is observer-relative. It’s well worth reading Wheeler’s paper, available at

The interface/MB is indeed the channel. I’m not sure what you mean by “who works the channel” - Alice and Bob write on and read from it, just as we are doing with this Q&A page and the internet as a channel.

Nice observation. The relationship between QT and time is an active area - Rovelli’s papers arxiv:1812.03578 and 2010.05734 are good entry points. A key issue in this relationship is the observer-relativity of entropy or information. From this perspective, dS/dE can be read as the change in information flow to/from the observer (the boundary is informationally symmetric) as the energy expended to get information increases/decreases. Seeing the observer’s internal clock (time reference frame) as a bit counter couples this observer-relative information flow to observer-relative time.

"Local” here means “at some boundary.” QT is a globally deterministic theory, i.e. any isolated system evolves unitarily. Isolated systems are unobservable by definition; they are the abstractions used to get the theory off the ground. If we think of some observer (Alice) and “everything else” (not-Alice), then the joint system comprising these two is isolated by definition. Alice doesn’t observe the joint system; she observes not-Alice (usually called Bob). Think of the boundary between Alice and Bob as an array of qubits as in slide 16 of the talk. Alice and Bob interact by preparing and measuring these qubits. We can think of the qubits as spins, and the preparation and measurement as using the z-spin operator s_z. The z axis is the “up v/s down” direction. Local free choice is the freedom of Alice and Bob to each choose their own z-axis completely independently of each other. If they don’t have this freedom, they are entangled. We’ll discuss this more in the July session.

Good question, one that points to an active area with a long history, going back at least to Laplace. The “empirical” or “frequentist” idea of probability is generally taken to also be “objective” in the sense of observer-independent. The “inferential” or “Bayesian” (see ) idea of probability is “subjective” or observer-specific. The latter perspective works well with observer-relative conceptions of quantum states and operations; see Chris Fuchs, arxiv:1003.5209 or David Mermin arxiv:1809.01639 for good introductions to this debate (both on the Bayesian side). I always these observer-specific readings of entropy, information, measurement, or action. They depend on how the agent interacts with the world, i.e. what reference frames they have available and how they use them. You are right to call this the “quality” of the agent’s generative model. Chat-GPT is a nice example. The July session will focus on these questions of the reference frames used to act and make measurements, and how they also determine what data can be stored in a memory and accessed later.

This is a nice connection to entropy measures in EEG - thanks for bringing it up. As discussed in the context of earlier questions, entropy and hence uncertainty are observer-relative concepts. The question “Whose uncertainty?” always arises. If Alice is measuring EEG, then a variable signal occupies more frequency states than a constant signal, and hence has higher entropy for Alice. This is independent of what the frequency of the signal is telling Alice about the energy and hence the entropy (dE = TdS) of the system being measured. Whether Alice views variation in the signal as “noise” or as informative depends on Alice’s measurement capabilities and priors - if she expects a constant signal but measures a variable one, with some nice, e.g. Gaussian structure in the variation, she may consider the variation “noise” around her expected constant value. But if she expects variation - e.g. she is looking for a phase-coded message in the signal - the variation is no longer noise, but what she’s after.

Good question that brings up a difference between the theories and how they are used. An MB is defined in a causal (directed) network. One can draw a closed boundary anywhere in such a network and define an MB as the set of states with in or out arrows crossing the boundary plus any states not yet counted that have arrows going to the states with incoming boundary-crossing arrows. So if you pick a node, you can define a sequence of larger and larger MBs around that node, each containing all the states of any smaller MBs around that node. In the limit, you get to the boundary of the whole network, outside of which there are no more nodes, so the process stops. The S-matrix takes essentially the opposite approach, starting with idealized “free particle” states at + and - infinity in both space and time from some “scattering center” where an interaction happens. The S-matrix is then the operator S: |in> → |out>. This formulation effectively assumes that “no one is looking” during the actual interaction, which happens “fast” compared to the processes (that are not represented by the S-matrix) of preparing the initial state and measuring the final state. The explicit causal network of the classical picture is replaced by “everything not explicitly forbidden is mandatory” (M. Gell-Mann from T. H. White) with the standard representation being the hierarchy of increasingly complicated Feynman diagrams. The other key difference is unitarity and hence spacetime reversibility: all the Feynman diagrams also work with the arrows reversed.

If “bandwidth” is defined by counting incoming and outgoing arrows, the answer is no - nodes can have high fan-in and low fan-out or the reverse. More subtle metrics, e.g. transfer entropy ( ) can also be defined. The most general way of asking the question would be in terms of conservation of energy, in which case the answer is yes provided there are no sources on sinks inside the MB. A source or sink is effectively a singularity - a node where the energy changes with a step function - so one could require causal networks with no singularities. This is a place where the quantum theory is much more straightforward - holographic boundaries are informationally symmetric by definition. Whether the agents on the two sides of the boundary can use the information in any interesting way depends on their computational capabilities and so is not in general symmetric.

The HP is the statement that interactions between finite systems have not just finite eigenvalues, but finitely encodable eigenvalues. Quantitatively, it sets an upper limit on the entropy associated with a boundary, S <= A/4, A the area in Planck units, with equality only (by definition) for black holes. So the HP only gives you a number for BHs, e.g. lets you associate an entropy with a mass via R = 2M. The HP is also the basis for holographic dualities like AdS/CFT, which effectively lets you change the basis in which a calculation is done. There is a huge literature of applications of AdS/CFT.

In what you quote, I am using the word “science” very broadly, essentially as a synonym for active inference: science (in this usage) is the “practice” of probing the world in order to get new information from it. I suspect that John C. as you quote him was using “science” in the narrower meaning that distinguishes it from other strategies for interacting with the world, as you are in your question #3. In my broad usage, all synchronization strategies are variants or implementations of active inference so they are all “science.” The question of effectiveness can also be asked from multiple perspectives. From an active inference perspective, an effective procedure is one that uses minimal energy to construct a GM with good predictive power. Part of this strategy could involve molding the environment to make it more predictable. But this strategy can clearly get into local minima in which the environment looks more predictable in the short term, but is in fact less predictable in the long term.

I think what you are pointing to here is the fundamentally stipulative nature of QRF definitions. I can only tell an external clock is ticking because I have an internal time QRF, so my use of an external time QRF, even an attosecond resolution light clock, depends on my internal QRF. There is nothing I can measure my internal ticks with (EEG or neural recording for example) that doesn’t (logically and physically) depend on the very internal clock I’m trying to measure. There is a broader issue of which this is an example: no system can reverse-engineer itself. A can be separable from B only if dim(A), dim(B) >> dim(H_AB), which is the (Hilbert-space) dimension of the A-B boundary. But dim(H_AB) is the upper limit of the information A can write to memory. So A doesn’t have the capacity to write anything but a coarse-grained model of itself to memory. Even this model is going to be inferred from interactions with B (since H_AB is the source of all incoming data) and hence will be, like my model of my own internal processes, an inference from observations of systems that A regards as similar to it.

Planck’s constant is (in current theory) the minimal action by definition. It has units of energy x time, which are complementary so cannot be measured simultaneously. But if you separately measure energy and time for some process, in many replicates so you can calculate means, you can compute a mean action. This is what I did for rhodopsin, assuming a minimal (ln2k_B T) energy instead of measuring it. In fact the energy of visible light that rhodopsin responds to is close to this number. The response time is from measurements. Hence I can calculate the action and compare it to the value of hbar. The value of hbar in fundamental (Planck) units is one; the numerical values of 1 sec, 1 Joule, 1 meter, etc are derived from these stipulated values - see.

Yes. There are different time scales for different components and different component scales - e.g. even different molecules have different characteristic times for functionally-relevant configuration changes. It is indeed a complex business! Complex organisms and even single cells have many different interaction modalities with different time scales and different implementation compartments. We can analyze it down to compartments that implement single QRFs and hence just one “language.” We’ll discuss this in the generic case in July, after which you may want to pose an extension of this question.

You are right, a model that ignores the semantics is not viable. The system has to keep track of what QRF outcomes are produced by. This requires (effectively) a labeling scheme imposed by some “meta” component looking at sets of outcomes using its own reference frames. Hence interesting systems have hierarchical processing, with each “layer” doing active inference on its own inputs. We will scratch the surface of this in July.

Yes, A and B can have different time scales, and each will interact with the screen at their own scale. So Alice in your example will not stop “writing” but will overwrite her own last message. Letting t_A = t_B just makes the model easier to understand.

Karl Friston has pointed out that the limit of the classical FEP, the limit of perfect prediction, corresponds to “generalized synchrony” between system and environment - each sends messages that the other can perfectly predict. This notion depends on the system and its environment being separated and hence distinguished somehow. In classical formulations, they are separated in ordinary 3d space - they occupy different locations. The quantum formulation is “background free” meaning it assumes no spacetime embedding - system and environment do not have “different locations” even though they have different sets of degrees of freedom in the overall Hilbert (i.e. state) space. Here the limit of the FEP is maximal entanglement. This does not mean that the system and its environment are “identical” (they aren’t, since they have different degrees of freedom). It means that their joint state |SE> is not separable, for factorable, into a system state |S> and an environment state |E>. Their states are no longer conditionally independent. This is distinct from thermal equilibrium, which means that they have the same temperature while remaining separable, and that their interaction can be characterized as the exchange of thermal fluctuations, i.e. noise.

I suspect that the answer to all general forms of this question is that it is undecidable. Some specific instances where undecidability can be proved will be discussed in the August session. This is clear in some cases; for example, “What is the most efficient way to do science?” is an example of your question. Undecidability leaves us with trying to find good heuristics, which are fiercely debated in the case of how to do science. A spin-off of your question is “How can a system (at least approximately) measure its own VFE?” How good are humans, for example, at assigning subjective probabilities? What is the “sense of certainty” and how is it implemented? These are questions for the October session.

Our models indeed incorporate hypotheses about meaning; we use these every day in conversation. In the theory as presented, an “agent” is just a physical system that is in a separable joint state with its environment. Interesting systems have some level of accessible event memory, which elementary particles lack. Hence “agency” is a distribution, not a bright line.