Entropy in Information Theory and Dreaming

This from Wikipedia

Entropy { Information Theory}

In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable’s potential states or possible outcomes. This measures the expected amount of information needed to describe the state of the variable, considering the distribution of probabilities across all potential states.

Entropy in information theory is directly analogous to the entropy in statistical thermodynamics. The analogy results when the values of the random variable designate energies of microstates, so Gibbs’s formula for the entropy is formally identical to Shannon’s formula. Entropy has relevance to other areas of mathematics such as combinatorics and machine learning. The definition can be derived from a set of axioms establishing that entropy should be a measure of how informative the average outcome of a variable is. For a continuous random variable, differential entropy is analogous to entropy.

———

Introduction

The core idea of information theory is that the “informational value” of a communicated message depends on the degree to which the content of the message is surprising. If a highly likely event occurs, the message carries very little information. On the other hand, if a highly unlikely event occurs, the message is much more informative. For instance, the knowledge that some particular number will not be the winning number of a lottery provides very little information, because any particular chosen number will almost certainly not win. However, knowledge that a particular number will win a lottery has high informational value because it communicates the occurrence of a very low probability event.

The information content, also called the surprisal or self-information, of an event E is a function that increases as the probability of an event p(E) decreases. When p(E) is close to one the surprisal of an event is low, if p(E) is close to zero then the surprisal of an event if it occurs is high.

There is zero surprise when the probability is one.

———-

Are dreams high entropy?

What happens when p(E) is close to zero?

And do dreams contain information of high informational value?

——–

On and off over the years I have had “intuitional discomfort” about the notion of quantum superposition states. Is it physics or our description of physics? Are we making an ersatz by making a quantum supposition, a sticking plaster for some deeper physics we do not yet understand. It does look a bit of a hotch-potch.

Quantum physics has a God-like quality which dare not be questioned…

—-

From Microsoft Quantum

Superposition is a fundamental concept in quantum mechanics, describing the condition in which a quantum system can exist in multiple states or configurations simultaneously. Classical bits can exist in two possible states, typically labeled as “0” and “1”. In contrast, because a qubit is a quantum mechanical system, it can exist in the “0” state, the “1” state, or any state that is a linear combination of 0 and 1.

Mathematically, superposition is a linear combination of “0” and “1” and can be written as:

     |ψ⟩ = α|0⟩ + β|1

where |ψ⟩ is the state of the qubit, |0⟩ and |1⟩ are the basis states (or the computational basis states), and α and β are complex numbers called probability amplitudes. The probability amplitudes determine the probability of measuring the qubit in either state when a measurement is made.

—–

Is the prepared qubit then “natural” or have we imbued it with information which we have decided for it in our labelling. This labelling presupposes no extra physics. A photon might have vertical polarisation, horizontal polarisation or if paired a superposition of both; horizontal and vertical as opposed to horizontal or vertical.

Unless the superposition is somehow aware of our labelling how does it know how we expect it to behave?

If one of a photon pair is measured as horizontal it must communicate that information instantaneously to its erstwhile twin.

Is it physics or information?

A good question.

How much entropy is associated with that information?

If the superposition collapses on measurement then in an idealised physics the probability of the other pair member “knowing” its state is one. It has little informational value as it comes as no surprise.

If eigenfunctions are used to prepare a description of a quantum supposition then they must have a difference in the set of quantum number qualities to be eigenfunctions. They may be energy eigenvalue degenerate in isolation but because of the difference in quantum number that degeneracy must be split, if only slightly under some condition or other, which means that they do not share entirely equal  probability on being prepared into a superposition.

The qubit will have a tiny inherent bias.

———

I keep getting this sense of discomfort that there is something we are not getting and that we are happy sitting with an intuitionally uncomfortable quantum superposition description…it just seems cobbled together.

It is a bit of a mind fuck…

————-

Do dreams bring real world information?

If so is it surprising and of high informational value?

————

I have just re-read a dream of mine from 2008 in which I see a Breton landscape and am chatting with an extroverted French man who is the spitting image physically and in behaviour of my recent physiotherapist.

Was this information, coincidence or my confirmation bias retro-fitting a dream to a current observable?

If dreams do indeed carry information then the implications of this are high.

I personally will trust what my dreams say about a situation over and above what a human being may or may not assert. If there is good physical evidence I will trust that over dream…

This notion of surprise as being a component of information {new} is interesting. The concept of information entropy in a world of fake news and a verbose Trumpian professor, is appealing.

Information and opinion are not the same. It is possible to speak for a long time without giving any reliable information which comes as no surprise. Were there to be anything new {and reliable} that would be very surprising and information rich.

—-

Any description or model must occlude at least partially an occult reality of some kind. Because we do not know the true and extensive nature of that reality we can at best describe it. Such a description must be informative and allow a measure of prediction as to the measurable evolution of events in space-time. We should be honest that our description is not ultimate truth. Behind all known physics there is always something occult, unseen and as yet unknown. History teaches us this

Jut kicking a few ideas around ….

Nothing may come out of it…

Secret Garden / Princess Dream 4-3-21

Here is this morning’s dream, it was very vivid.

I am walking in our current garden as it is and stumble upon a wooden door in a stone wall. The door does not exist in our physical plane garden. I reach down into the pocket of my combat trousers and take out the set of keys. I try the keys in the lock and one of them opens the door. I step through the door and close it behind me.

I am now in a magnificent Italianate garden, with stonework, and beautifully created flower beds. It has the feel of northern Italy, of Lago Maggiore. There are carved stone flowerpots and trellises with flowers growing over them. Up ahead is an ornate building which seems unoccupied, the wooden shutters are closed. I go up to that building and again try the keys. The door opens and I enter. I open the shutters, and this reveals a well-appointed ground floor appartement which now looks out onto the sea. I understand that we own this. It is a part of the property we purchased from Mr T. I think we might be able to rent this out. I might also be able to write here.

I see a door which appears to lead nowhere. I again try the key and it opens on to an ornate staircase. I climb the staircase and there is another well-appointed appartement, bigger and grander than the one downstairs. I find a room laid out with a drawing table, for technical drawing. On it is a very fine white cloth envelope. I pick this up and see that there is a message inside for me. It is written in blue copperplate and I decide to open it later. I look around the apartment and see that Mr T. has left many catering supplies from the party he gave just before he departed.

I hear a knock on the apartment door. I go down to investigate and it is the estate agent who is showing around a potential client. The estate agent is female, my age, and has with her a young beautiful woman with long dark hair. The agent is explaining that because of the pandemic she isn’t doing rentals but if we discuss things with the owner, we might be able to agree on something. I nod in agreement.

I show them to the room where the supplies are. The young woman now has a young boy with her, a toddler. I offer him some chicken crisps at which the woman says no he mustn’t. I find some grapes, and these are ok. He eats a few. The estate agent has left. The young woman kisses me on the lips, and I pull away. I say that I am married.

I spend the night with the young woman in the same bed as her and the boy. I am naked and they are clothed. As dawn approaches, I go to the window and look out over the see. I put on a pair of skimpy underpants which have the word FREEDOM written on them. Out of the window I can see the seashore, the sea and some rocks. I see a bright orange/red blob unfurl its wings like a phoenix. It flies over to near the window. The bird like object lands and amidst the feathers it has the face of a black man. It asks me if I am with “the princess”. I say yes because now I know that that is what she is. He says that all will be well and that I must first come with him.

I go to leave and again the princess tries to kiss me. I pull away. Although I am tempted, I know that to have sex with her is not the reason she has found me. She asks me will I return to them. I say that yes but first I must go.

Dream ends.