The free will debate and discussion is a never-ending one - much like any other truly philosophical question, there is no possibility of a “true answer” but rather only an exploration of hypotheses that seem logically sound.
This essay tries to explore a hypothesis that can explain the mechanism of free will with a meager set of assumptions.

It may seem strange and pointless speculation to propose a mechanism for free will , but this is a thought experiment which I believe has some value towards demystifying it. 

Most discussions of free will are stuck at the binary debate of its existence, but to go beyond that, I think the next logical step is to speculate about the “hows” of it.
We have no direct way to understand how free will works, so the best we can do is to try to propose a plausible mechanism for it, which requires as few assumptions as possible. In doing so I believe we can achieve more clarity into the concept of free will itself.

Before going into the details of mechanism, I will attempt to answer what I think are some fundamental questions about free will - I don't claim this to be the absolute truth, but I think these answers are quite satisfactory.
These are essential in order for me to get to the actual mechanism proposed 

To begin with, free will is certainly only possible under a causal framework - causation implies an ordering of events.
In the simplistic view, causation implies the passage of time, since time is a natural universal order that we perceive, however the term causation can be interpreted more generally as a directed edge of a graph where events are nodes, and edges are causal relationships, but for this particular discussion, we will not go into that view.

All of physics is mostly about discovering mathematical equations or models that describe the change of some measurable property across the passage of time.
Perceived time is the universal standard - according to Einstein, the laws of Physics are invariant for all observers. This means that for two observers, even if each sees the other's time as dilated or compressed (i.e. the other seems to live slower or faster), they will observe that their their own subjective time, whether perceived, or measured by any local scientific instrument, does not change. 

Our memories (and any device we can build that remembers things, whether it’s a clay tablet or a hard disk) depend on the notion of entropy increasing with time.
All thermodynamic processes have a natural ordering, with events of lower entropy appearing before those with higher entropy. Every time you record one bit of memory, the entropy of the universe increases just by the physical process of recording it.
This means that time itself for us is defined by what we are capable of remembering - i.e. only lower entropy states.

In short, free will, as well as determinism depend on the idea of time and the fact that some events labeled as “the present” are caused by some other events which are labeled as “the past”

All causation we can know of is temporal causation, but Determinism can be defined timelessly - you could say that all possible events in the universe preexist, much like an entire song on a vinyl record, which only becomes a sequence of sounds when read spirally by the needle.

Free will however, cannot be defined timelessly - since events should not preexist in any sense, until the free willed being causes them.

Thus, free will can be described as a mechanism of temporal causation, but which has to be distinct from the following.

• Deterministic natural laws i.e. Classical physics - for e.g. the almost perfectly predictable movements of the balls on a pool table.
  

• Non-deterministic natural laws i.e. Classical chaos - for e.g. the almost perfectly unpredictable movements of the ocean, which are technically governed by the known laws of physics, but involve an almost infinite number of measurements of infinite precision such that prediction is impossible. 
  

• Un-caused events i.e. Quantum indeterminacy - for e.g. a particle in superposition collapses into one observed state, out of several possible ones - Quantum theory proves that no prior set of measurements of anything in the universe, could even theoretically predict this collapsed state.
  
  

Note that for brevity, in the rest of this essay, the term “random” will refer to true randomness a.k.a. Quantum indeterminacy - as there is no other conceivable mechanism for randomness

To this we could add a fourth mechanism - a free willed event - which can be described as:

• A being must have intended an event
  

• This intention must be recorded in their memory or elsewhere physically

• No objective measurement of any variable done outside of the record (mental or physical) can predict that event

• The event occurs 

The 3rd point requires the same kind of “infinite number of variables of infinite precision” measurement that is needed to predict classical chaos, so it is empirically quite unprovable, but then again, we are in the realm of hypothetical inquiry, where empirical possibility is not considered, but only logical possibility is.

If we accept this definition, much like Quantum indeterminacy, free will affects matter from elsewhere, somewhere far outside the limits of objective measurability.
It does appear as if something immaterial, unmeasurable and intangible, is affecting something material and tangible, but this is how Quantum indeterminacy works too.

Quantum theory tells us that one of realism or locality is false.

• Locality means that things literally have to be in contact for them to affect other things - even forces that act over a distance like gravity are mediated by something moving across space at a finite velocity, and transferring momentum. In fact an object in contact with another only pushes it by virtue of virtual photons travelling from the former to the latter, creating a repulsive force between the electrons of both objects' atoms. The very concept of physical contact is nothing but messages (aka force carrying particles) being sent at the speed of light between objects.
  If locality is false, it would mean anything could possibly cause anything across spacetime which would be quite an absurd and inconceivable state of affairs
  

• Realism says that things exist independently of observers 
  If realism is false then you can have things that affect reality are not within measurable reality - like quantum states...
  
  

The choice most sane physicists accept, is to abandon realism and retain locality - Quantum states are from outside our reality, they don't exist until they are measured

I think it’s not presumptuous to think of free will also as a mechanism that is not within our perceivable reality.

I have deliberately worded the above such that there is no assumption that the being in question initiates the event physically.
For example if I say “I’m going to get a bottle of water”, one would not normally expect the water to appear in front of me.

But should we assume this? I would say that we should not.
After all, if free will triggers the nervous system to make me walk and get the bottle, why should free will not be able to synthesize a bottle of water out of whatever common substrate the material world is made of?
This is something we will discuss in a follow-up post in more depth.

Another aspect is whether intentions need to be declared - I would argue that conscious intentions are remembered, and hence declared at least in the memory of the free willed agent. Actions that you do not remember intending to, are indistinguishable from unintentional ones.

To summarize:

Free will is a mechanism that allows a being to cause an event, with the unique property that the only measurable prior that links to the event, is the declared intention of the free willed being.

By its very nature, one can never prove if a certain being exhibits free-will or not - because whatever is observed in any experiment could just be a predetermined result. This is ideal for classical physics theories, because the basic axiom of Physics is that every event has a predetermined result.

This same thing also applies to randomness, and there exists a hypothesis called super-determinism which posits that everything including the results of quantum measurements are already fixed.

In the case of freewill, we cannot even distinguish free-will from randomness - because by definition, you can neither explain the cause of random behavior, nor of free willed behavior. The best we can do is to define a behavior that cannot be replicated by a standard Turing machine or an enhanced one which also has random number generation.
After we do that, we can hypothesize about what kind of mechanism is needed to allow that.

Our thought experiment takes the form of a simple test - let’s assume 3 distinctive beings taking this test, for illustrations sake:

• A completely classical/deterministic computer running a program, that is robustly designed never to be affected by quantum random phenomena (as almost every modern computer is), but crucially, cannot also rely on anything but an algorithm to generate a random number (as almost every computer prior to the 1990s).
  Such a computer/program falls within the limitation of a Turing machine and cannot produce truly unpredictable sequences of numbers.
  The best they can do is “Pseudo Random Sequence Generation” which relies on some external “seed" value to work
  

• A materialist model human being - defined as what contemporary science believes that the human brain/mind is - i.e. a fairly deterministic genetically programmed brain, that has adapted by learning algorithms, but also subject to random behavior due to the chaotic nature of chemical reactions in the brain

• Our hero, a free willed human being - you know how this feels, you don’t need any external metaphysical system, or religious belief to instinctively know that you can make a free choice - to be able to “Pick a card, any card”
  
  

We need to assume that for free will to be even worth discussing, it should be able to produce better results in the test than the best possible versions of the other two.
Note that all the above characters are goal seeking agents.
If a free-willed being's behavior can be emulated by a computer, then it is superfluous.
If it can be emulated by a random process, it is unlikely to reach a goal at all.
Hence we suggest that if at all freewill exists, it cannot be emulated.  
 

Here’s how the experiment would go

• A random maze with multiple entries and exits is created

• The agents are instructed to pass through the maze

• Points are scored for the shortest path 

In order to keep things simple - these agents will not have any visual input, but only tactile, so that at any given point they only know how many immediate walls surround them. We also obviously assume that they have a persistent memory - i.e. they can remember whatever steps they took.

Now let’s look at how well each agent can theoretically do:

• The deterministic agent can use the simplest algorithm - keep continuously touching the left (or right) wall and move forward (remember this the next time you are in a maze).
  The logic is that any path through a maze divides it into two, and doing the above makes you traverse across one of the halves. 
  There is no other algorithm to do better than this on average, if there are multiple entries and exits - by the nature of this algorithm, one always enters the leftmost entrance and always leaves at the leftmost exit. If you have no randomness, there is no way to explore any shorter path even if it exists.
  
  

• The materialist model human can adopt this same nice strategy, but by definition, they are bound to deliberately or involuntarily incorporate some randomness when moving.
  This leads to a couple of things:

• Potentially reaching dead ends and backtracking, and/or traversing every part of the maze in the worst case scenario

• Potentially stumbling onto a shorter path which the first agent cannot do.
  
  

• The free willed human must do something (discussed later) that lets them beat the first two agents on average

We’d run these agents through multiple random mazes ad nauseam, and see which had the lowest average distance traversed.

The deterministic algorithm will certainly beat the second agent, because the mazes are random - it will hit the shortest path once in a while, but the second agent will commit errors due to randomness. Adding randomness doesn’t help the cause for the second agent - I would certainly love to hear from any expert, if there is a way to add randomness such that the second agent can have any advantage.

Now, what I intend to propose is that, under my hypothetical description of the mechanism of free will, the “free willed” agent will perform the bext, and I will attempt to explain how it could.

From what we know about computation, or just from plain reason, there seems to be no way to outdo the deterministic program (if you know a better algorithm, it still doesn't really change things), but is it somehow possible to make this program do better?

Definitely - for e.g. with some relaxation of rules:

• We can allow vision - this allows agents to see dead ends from afar and avoid needless backtracking.

• We can give a spotlight view - like a real-time circular diagram of the maze centered on the agent - this allows discarding many non-optimal choices.
  If this spotlight was as big as the maze, the perfect path will be trivially computable.

We see a pattern here - more information available about the maze allows a better outcome, and complete information allows the optimal outcome.

Consider autonomy - this is the level of robustness that a goal seeking entity has in resisting external influences. Computers are especially autonomous - they are physically designed in a way to be insulated from physical failures, and (usually) programmed in a way that rogue inputs are rejected.

The highest level of autonomy is present perfect free will - in as much as nothing can prevent such a being from reaching its goal.
Autonomy is a necessary, but not sufficient condition for free will - after all there is nothing free about making a choice, if you can’t execute it due to external circumstances. Also, the highly autonomous computer can only make choices at the points in the code where the programmer set it up as some function of the input data, or some random variable - hence its autonomy is merely towards executing the code without straying.

Now, we will actually delve into speculation and hypothesizing, but before we go down that path, let me establish some axioms that are necessary for further elaboration of my hypothesis.

• The near future is uncertain and not predetermined

• There are several alternate possible futures whenever a decision is made

Now, the maze we used was a spatial one, but what if we consider causal existence to be structured as a temporal maze?
We do live trapped helplessly in the present, with very little foreknowledge of what will come to pass, with not even an option to pause and plan, unlike in a spatial maze.
Much like the overhead view of the maze that spreads across space allows better traversal, a view that spreads temporally allows better traversal across the timeline 

This brings us to the essence of my hypothesis as a one liner:
Free will is a mechanism that allows a being to gather information across time from multiple possible futures

Disclaimer: From this point onward, the rhetoric is purely opinionated, biased to my personal view of the universe. It can be considered a hypothesis, but I present it without any obligatory what ifs 

Consciousness/Awareness is the substratum of the multiverse, a consciousness that transcends all spaces and times.
The individual consciousness is an undivided, yet somehow divided part of it, much like a wave, or Tsunami, or trans-oceanic current in the water.

Let’s consider that a microcosmic being, much like the macrocosm, has awareness too, but somewhat limited.

This limitation extends to:

• The immediate present for inanimate objects - these behave like a deterministic computer, using only the current state of the world as an input
  
  

• The immediate present and experienced/recorded past for living things - trees and most animals have the genetically encoded view of the distant past and their memory and conditioned learning of the recent past - together this drives their behavior
  
  

• The immediate present, recorded past, as well as the extrapolated future - computers and artificial intelligence which fine tune the above mechanism, and have a concept of a directed goal seeking, an extrapolated future model, with some degree of self monitoring, correcting and self-training mechanisms

The fourth category is the free willed being whose consciousness extends not only to the experienced/recorded past, but also beyond that, and extends not only to the predicted/extrapolated future, but also the actual future. Something that extends and exists across a temporal period, not necessarily constrained by the physical form that it is attached to. This consciousness could be termed the soul.

If I, as a being, have access to the true probability of possible futures, I have a power that transcends computation, as well as those who have a shorter view of the future.
This information lets me behave in a manner that overcomes the limits of deterministic rules, while also not falling prey to random speculation about outcomes.
If the information crosses a longer span of time, I literally have the ability to control the future I would like to be in.

Something like this is often exemplified in fiction - whether it is the protagonist of Groundhog day living the same day over and over, till he has complete mastery over the events, or Dr Strange living out a dozen million possible futures so he can direct events into the one future he desires, in Infinity war.

Free will is essentially a temporal limb, the height your head extends over the walls of the infinite temporal maze, metaphorically speaking

What else does this hypothesis explain or imply?

• The process of growth towards the supreme consciousness is nothing but extension of awareness over larger and larger extents. A temporal lengthening of the soul so to speak, leading to disassociation with things related to shorter finite spacetime boundaries - whether it is material objects, human life, the lineage and species you were born as, the very spacetime bubble you live in.
  The culmination is to extend to the entirety of reality, the Vaishvaanara or Viraat who is individual yet free of identity, aware of everything else and yet of nothing, since there is nothing else

• The concept of higher Gods (as per Sanaatana) are nothing but beings who evolved their consciousness to transcend the lifespan of physical universes like ours.
  The term Dheerga Darshin used by our ancients to refer to evolved persons suggests they thought on similar lines.
  
  

• There is nothing sacred about biology or organic life - biological life is just one medium where computation, self awareness, as well as this cross-temporal perception can be channeled. There is nothing to say that some configurations of metal or silicon could not do the same.
  Unlike the contemporary notion of consciousness being an emergent phenomenon of matter, rather it is that matter is merely an emergent objective manifestation of consciousness, which seems to channel itself as identity, free will and life, when it has a certain structure.
  
  

• Since a soul identifies with some vehicle of experience i.e. a body, its choices are naturally limited to expression via that medium - hence no human can choose to fly like a bird. Free will is on a spectrum, and action is limited to expression in its medium.
  Brahma could create many universes at will, but still cannot make a human body disobey gravity, because Brahma’s choice of making a creation with Physics inbuilt, limits it to not allow a levitating human. This is what our ancients termed Rta
  Each level of consciousness has its own medium of expression, through which the formless churns out forms and actions.
  The ultimate will of the supreme consciousness is to manifest in myriad unlimited ways - this is not unlike Max Tegmark's theory that every possible mathematical model that can manifest as a universe, does manifest as a universe