Entropy in Information Theory and Dreaming

This from Wikipedia

Entropy { Information Theory}

In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable’s potential states or possible outcomes. This measures the expected amount of information needed to describe the state of the variable, considering the distribution of probabilities across all potential states.

Entropy in information theory is directly analogous to the entropy in statistical thermodynamics. The analogy results when the values of the random variable designate energies of microstates, so Gibbs’s formula for the entropy is formally identical to Shannon’s formula. Entropy has relevance to other areas of mathematics such as combinatorics and machine learning. The definition can be derived from a set of axioms establishing that entropy should be a measure of how informative the average outcome of a variable is. For a continuous random variable, differential entropy is analogous to entropy.

———

Introduction

The core idea of information theory is that the “informational value” of a communicated message depends on the degree to which the content of the message is surprising. If a highly likely event occurs, the message carries very little information. On the other hand, if a highly unlikely event occurs, the message is much more informative. For instance, the knowledge that some particular number will not be the winning number of a lottery provides very little information, because any particular chosen number will almost certainly not win. However, knowledge that a particular number will win a lottery has high informational value because it communicates the occurrence of a very low probability event.

The information content, also called the surprisal or self-information, of an event E is a function that increases as the probability of an event p(E) decreases. When p(E) is close to one the surprisal of an event is low, if p(E) is close to zero then the surprisal of an event if it occurs is high.

There is zero surprise when the probability is one.

———-

Are dreams high entropy?

What happens when p(E) is close to zero?

And do dreams contain information of high informational value?

——–

On and off over the years I have had “intuitional discomfort” about the notion of quantum superposition states. Is it physics or our description of physics? Are we making an ersatz by making a quantum supposition, a sticking plaster for some deeper physics we do not yet understand. It does look a bit of a hotch-potch.

Quantum physics has a God-like quality which dare not be questioned…

—-

From Microsoft Quantum

Superposition is a fundamental concept in quantum mechanics, describing the condition in which a quantum system can exist in multiple states or configurations simultaneously. Classical bits can exist in two possible states, typically labeled as “0” and “1”. In contrast, because a qubit is a quantum mechanical system, it can exist in the “0” state, the “1” state, or any state that is a linear combination of 0 and 1.

Mathematically, superposition is a linear combination of “0” and “1” and can be written as:

     |ψ⟩ = α|0⟩ + β|1

where |ψ⟩ is the state of the qubit, |0⟩ and |1⟩ are the basis states (or the computational basis states), and α and β are complex numbers called probability amplitudes. The probability amplitudes determine the probability of measuring the qubit in either state when a measurement is made.

—–

Is the prepared qubit then “natural” or have we imbued it with information which we have decided for it in our labelling. This labelling presupposes no extra physics. A photon might have vertical polarisation, horizontal polarisation or if paired a superposition of both; horizontal and vertical as opposed to horizontal or vertical.

Unless the superposition is somehow aware of our labelling how does it know how we expect it to behave?

If one of a photon pair is measured as horizontal it must communicate that information instantaneously to its erstwhile twin.

Is it physics or information?

A good question.

How much entropy is associated with that information?

If the superposition collapses on measurement then in an idealised physics the probability of the other pair member “knowing” its state is one. It has little informational value as it comes as no surprise.

If eigenfunctions are used to prepare a description of a quantum supposition then they must have a difference in the set of quantum number qualities to be eigenfunctions. They may be energy eigenvalue degenerate in isolation but because of the difference in quantum number that degeneracy must be split, if only slightly under some condition or other, which means that they do not share entirely equal  probability on being prepared into a superposition.

The qubit will have a tiny inherent bias.

———

I keep getting this sense of discomfort that there is something we are not getting and that we are happy sitting with an intuitionally uncomfortable quantum superposition description…it just seems cobbled together.

It is a bit of a mind fuck…

————-

Do dreams bring real world information?

If so is it surprising and of high informational value?

————

I have just re-read a dream of mine from 2008 in which I see a Breton landscape and am chatting with an extroverted French man who is the spitting image physically and in behaviour of my recent physiotherapist.

Was this information, coincidence or my confirmation bias retro-fitting a dream to a current observable?

If dreams do indeed carry information then the implications of this are high.

I personally will trust what my dreams say about a situation over and above what a human being may or may not assert. If there is good physical evidence I will trust that over dream…

This notion of surprise as being a component of information {new} is interesting. The concept of information entropy in a world of fake news and a verbose Trumpian professor, is appealing.

Information and opinion are not the same. It is possible to speak for a long time without giving any reliable information which comes as no surprise. Were there to be anything new {and reliable} that would be very surprising and information rich.

—-

Any description or model must occlude at least partially an occult reality of some kind. Because we do not know the true and extensive nature of that reality we can at best describe it. Such a description must be informative and allow a measure of prediction as to the measurable evolution of events in space-time. We should be honest that our description is not ultimate truth. Behind all known physics there is always something occult, unseen and as yet unknown. History teaches us this

Jut kicking a few ideas around ….

Nothing may come out of it…

Qubit Coherence Time Dream 20-03-2023

This dream came after watching Roger Penrose discus “before the big bang” on YouTube. He suggested that in a thermalized black body compliant universe evidenced by the cosmic microwave background energy spectrum where E=hν = mc2, time or inverse time, frequency, does not exist in the absence of matter. “Time” can be curved to “before” the big bang of our current universe. Similarly at an infinitely expanded universe full only of radiation, photons, time again has no meaning without mass.

The dream starts in the refreshment area or bar off a hall full of poster presentations at a “perimeter” conference on quantum and quantum computing. I am talking with a “famous” prof in his early forties and he is holding court to some post docs and post grads. He is skinny with a beard and unruly hair. He is white. As far as he is concerned, I am a nobody.

I say to him that if I have understood it correctly the biggest drawback for quantum computing is qubit coherence time and thereby quantum fidelity. He says yes.

I say that a qubit is a counter-entropic state artificially manufactured in matter in many systems. In these matter based systems tremendous cooling is required to prevent coupling to the nearby thermal bath states which tend to bring about dephasing and lack of fidelity for the prepared qubit. He agrees.

I say that is why they need fuck off big fridges to get close to absolute zero, which is again a counter entropic state when viewed more than locally. He agrees.

That dephasing happens on a temperature and proximity dependent time scale.

I say that thermodynamics is a property of matter and hence time, without matter entropy as no meaning, nor does thermodynamics.

I ask him why nobody is pursuing fast or ultrafast time domain quantum computing? He does not reply.

I say to him if the prepared qubit state is addressed before dephasing occurs, we can use time to outwit or outpace thermalization to bath. Time domain quantum computing could be the answer.

He nods.

I say that the other problem stems from the cycle time of write to zero of the matter involved in the prepared qubits. It should be possible to use a second pulse similar to the state preparation one to reset the qubit clock on a hundreds of megahertz laser repetition rate. We then have a prepare and subsequent write to zero even if the qubit state has been used or partially used in computation. The matter is then ready for us to contravene entropy by writing a qubit.

As the dream ends, I think that it is a fair way from Guru Rinpoche.

Early Onset Rigor Mortis and Elon Musk Dream 28-02-2025

Yesterday evening I watched a one liner comedian deny that he was suffering from arthritis. He was aged and claimed that it was actually early onset rigor mortis, which is an amusing and attractive idea. I am due to see an orthopaedic surgeon on Monday with a view to an assessment for hip replacement and perhaps some surgery on my lower spine. There is a fly in the ointment, however.

My Carte Vitale has cracked and I have ordered a new one on line. Yesterday they asked me to surrender my Carte Vitale. This is the magic key which unlocks French healthcare and the payments there fore. If you don’t have one you get billed for any hospital stay. These are at more than a grand a day for high care levels. Which means that until the situation is resolved it would be financially unwise to elect for any surgery. The French system is difficult to start and has very high inertia. Once it gets rolling it is a bit of a juggernaut and difficult to stop. In a few hours I will surrender my card and we will enter the twilight zone of French administrative “efficiency”. The pharmacist suggested that we allow two months for this to get sorted…

So, in the meantime I will have to manage early onset rigor mortis…

Last night we watched a documentary asking if Elon Musk wants to rule the world. Given how present he now is in the collective consciousness I am mildly surprised that he has not cropped up before.

This morning, I had a jumbled and chaotic dream with him in. In that dream he lent me one of his cars {Tesla} and gave me the key. He had however forgotten where he had parked it. So, I had to search the city centre of a German town with the key fob, pressing the button to find it.

I then had an extensive discussion with him concerning possible candidates for molecular qubits based upon hindered internal rotation and quantum superposition states of molecular rotors in the gas phase. I was left with the impression that he was rather lonely and bored.