Metamagical Themas: Questing for the Essence of Mind and Pattern

I: Snags and Snarls

1. On Self-Referential Sentences

• Many systems have the capability to represent or refer to themselves somehow, to designate themselves (or elements of themselves) within the system of their own symbolism. Whenever this happens, it is an instance of self-reference.
• Self-reference is often associated with paradox, but this is not necessarily the case.
• The classic paradox is 'Epimenides the Cretan said "All Cretans are liars."'
• It seems that all paradoxes involve, in one way or another, self-reference, whether it is achieved directly or indirectly.
• The philosophical problem of the connections among Platonic ideas, mental activity, physiological brain activity, and the external symbols that trigger them is vividly raised by these disturbing sentences:
  • I am the meaning of this sentence.
  • I am the thought you are now thinking.
  • I am thinking about myself right now.
  • I am the set of neural firings taking place in your brain as you read the set of letters in this sentence and think about me.
  • This inert sentence is my body, but my soul is alive, dancing in the sparks of your brain.
• Content is just fancy form. Content is just a shorthand way of saying "form perceived by a very fancy apparatus capable of making complex and subtle distinctions and abstractions and connections to prior concepts.
• When self-reference (or reference in general, for that matter) is indirect, mediated by form, then fluidity is required. The understanding of such sentences involves a mixture of deriving the content and yet retaining the form in mind, letting qualities of the form in mind, letting qualities of the form conjure up flavors and enhance the meaning with a halo of not-quite-conscious pseudo-meanings, connotations, flavors, that flicker in the mind, not quite out of reach.

2. Self-Referential Sentences: A Follow-Up

• David Moser's: "This is the Title of this Story, Which is Also Found Several Times in the Story Itself":
  • This is the first sentence of this story. This is the second sentence.
  • This sentence is questioning the intrinsic value of the first two sentences. This sentence is to inform you, in case you haven't already realized it, that this is a self-referential story, that is, a story containing sentences that refer to their own structure and function.
  • I mean, there is such a thing as free will, there has to be, and this sentence is proof of it!
  • This is the last sentence of the story. This is the last sentence of the story. This is the last sentence of the story. This is.
  • Sorry

3. On Viral Sentences and Self-Replicating Structures

• Walton and Going saw self-replicating sentences as similar to viruses - small objects that enslave larger and more self-sufficient "host" objects, getting the hosts by hook or by crook to carry out a complex sequence of replicating operations that bring new copies into being, which are then free to go off and enslave further hosts, and so on. "Viral sentences, as Walton called them, are "those that seek to obtain their own reproduction by commandeering the facilities of more complex entities."
• Both W and G were struck by the perniciousness of such sentences; the selfish way in which they invade a space of ideas and, merely by making copies of themselves all over the place, manage to take over a large portion of that space. Why do they not manage to overrun all of that idea-space? A good question. The answer: competition from other self-replicators. One types of replicator seizes a region of the space and becomes good at fending off rivals; thus a "niche" in idea-space is carved out.
• Roger Sperry in 1965, "Mind, Brain, and Humanist Values":
  • "Ideas cause ideas and help evolve new ideas. They interact with each other and with other mental forces in the same brain, in neighboring brains, and, thanks to global communication, in far distant, foreign brains. And they also interact with the external surroundings to produce in toto a burstwise advance in evolution that is far beyond anything to hit the evolutionary scene yet, including the emergence of the living cell."
• Jacques Monod in 1970, "Chance and Necessity":
  • The performance value of an idea depends upon the change it brings to the behavior of the person or the group that adopts it. The human group upon which a given idea confers greater cohesiveness, greater ambition, and greater self-confidence thereby receives from it an added power to expand which will insure the promotion of the idea itself. Its capacity to "take", the extent to which it can be "put over" has little to do with the amount of objective truth the idea may contain. The important this about the stout armature a religious ideology constitutes for a society is not what goes into its structure, but the fact that this structure is accepted, that it gains sway. So one cannot well separate such an idea's power to spread from its power to perform.
  • The spreading power - the infectivity - of ideas, is much more difficult to analyze. let us say that it depends upon preexisting structures in the mind, among them ideas already implanted by culture, but also undoubtedly upon certain innate structure which we are hard put to identify. What is very pain, however, is that the ideas having the highest invading potential are those that explain man by assigning him his place in an immanent destiny, in whose bosom his anxiety dissolves.
• Richard Dawkins in 1976, "The Selfish Gene":
  • As a library is an organized collection of books, so a memory is an organized collection of memes. And the soup in which memes grow and flourish is the soup of human culture.
  • Memes should be regarded as living structures, not just metaphorically but technically. When you plant a fertile meme in my mind you literally parasitize my brain, turning it into a vehicle for the meme's propagation in just the way that a virus may parasitize the genetic mechanism of a host cell. and this isn't just a way of talking - the meme for, say, "belief in life after death" is actually realized physically, millions of times over, as a structure in the nervous systems of individual men the world over.
  • The survival value of the god meme in the meme pool results from its great psychological appeal. It provides a superficially plausible answer to deep and troubling questions about existence. It suggests that injustices in this world may be rectified in the next. The "everlasting arms" hold out a cushion against our own inadequacies which, like a doctor's placebo, is none the less effective for being imaginary. These are some of the reasons why the idea of God is copied so readily by successive generations of individual brains. God exists, if only in the form of a meme with high survival value.
  • There need not be an exact copy of each meme, written in some universal memetic code, in each person's brain. Memes, like genes, are susceptible to variation or distortion - the analog to mutation. Various mutations of a meme will have to compete with each other, as well as other memes, for attention - for brain resources in terms of both space and time devoted to that meme. Not only must they compete for inner resources, but, since they are transmissible visually and aurally, they must compete for radio and television time, billboard space, newspaper and magazine column-inches, and library shelf-space.
  • Unconscious memes have ensured their own survival value by virtue of those same qualities of pseudo-ruthlessness which successful genes display. The idea of hell fire is, quite simply, self-perpetuating, because of its own deep psychological impact. It has become linked with the god meme because the two reinforce each other, and assist each other's survival in the meme pool.
  • Faith - Means blind trust, in the absence of evidence, even in the teeth of evidence... Nothing is more leghal for certain kinds of meme than a tendency to look for evidence... The meme for blind faith secures its own perpetuation by the simple unconscious expedient of discouraging rational inquiry.
• A manuscript sent to an editor may be considered viral, even though it contains no explicit self-reference, because it is attempting to secure its own reproduction through an appropriate host; the same manuscript sent to someone who has nothing to do with publishing may have no viral quality at all.
• In a democracy, nearly any idea will tend to replicate since the only way to win an election is to convince other people to share your ideas. Most political ideas are not properly self-replicating, since the motive for spreading the idea is separate from the idea itself. But ideas can sometimes take on a life of their own and drive their own propagation.
• Some statements like "The bourgeoisie is oppressing the proletariat." drive a desire to propagate them through a wish to protect a victim figure from a villain figure.
• Most extremist mass movements are based on a belief of victimhood.
• "The self-replicating ideas are conspiring to enslave our minds" - this paranoid statement is the Epimenides Paradox.
• Groups of memes are "meme complexes" for Dawkins, or "schemes" for Hofstadter.
• When a thing - a sentence, book, system, person - seems to refer to itself but does so only by allusion to something resembling itself, it is called indirect self-reference.
• This depends upon the ease with which our perceptual systems convert a mirror image into its reverse, and upon other qualities of translation without being aware of the layers - like looking through many feet of water and seeing not the water but only what lies at its bottom.
• Reagan evoked the memory of Truman to implicitly menace Iran with nuclear attack.
• Can self-reference really be direct or indirect? Or are these two points on a continuum? Is there even "self"-reference? One thing refers to another whenever, to a conscious being, there is a sufficiently compelling mapping between the roles the two things are perceived to play in some larger structures or systems. Conscious being, here, is an analogy-hungry perceiving machine that gets along in the world thanks to its perceptions; it need not be human or even organic. The mapping of systems and roles that establishes reference need not actually be perceived by any such being; it suffices that the mapping exist and simply be perceptible to such a being were it to chance by.
• In the movie of the The French Lieutenant's Woman, as two stories unfold in parallel, a number of coincidences arise that suggest ever more strongly that a mapping should be made. But it is left to the movie viewer to carry this mapping out; it is never called for explicitly. After a time though it simply becomes unavoidable.
• Indirect reference of the artistic type is much less precise than indirect reference of the formal type. The latter arises when two formal systems are isomorphic - they have strictly analogous internal structures, so that there is a rigorous one-to-one mapping between the roles in the one and the roles in the other. The existence of genuine reference becomes as clear to us as in the case of someone talking about their mirror-image: we take it as immediate, pure self-reference, without even noticing the indirectness, the translational steps mediated by the isomorphism. In fact, the connection may seem too direct even to be called "reference", some may see it simply as identity.
• This perceptual immediacy is the reason that Gôdel's famous sentence G of mathematical logic is said to be self-referential.
• Indirect self-reference suggests the idea of indirect self-replication, in which a viral entity, instead of replicating itself exactly, brings into being another entity that plays the same role as it does, but in some other system, perhaps its mirror image, perhaps its translation into French, perhaps a string of the product numbers of all its parts, together with pre-addressed envelopes containing checks made out to the factories where those parts are made, and a list of instructions telling what to do with all the parts when they arrive in the mail.
• The arbitrary and peculiar aspect of the Quine sentence is that its seed is half as complex as the sentence itself. But is there one which is made of smaller, less complex parts? Is there a self-documenting or self-building sentence that builds both its halves - its quoted seed and its unquoted building role - out of words or letters? All structure in the built object must arise exclusively out of some principle enunciated in the building rule, not out of the seed's internal structure.

4. Nomic: A Self-Modifying Game Based on Reflexivity in Law

• Reflexivity dilemmas of the Protagoras vs Euathlus (I'll teach you law and you pay me when you win your first case) type and problems of conflicting omnipotence crop up with astonishing regularity in the down-to-earth discipline of law.
• Reflexivity can include:
  • The self-reference of signs
  • The self-applicability of principles
  • The self-justification and self-refutation of propositions and inferences
  • The self-creation and self-destruction of legal and logical entities
  • The self-limitation and self-augmentation of powers,
  • Circular reasoning, circular causation, vicious and benign circles, feedback systems, mutual dependency, reciprocity, and organic form
• A good scheduling system strives to be equitable, but all kinds of conflicts can arise, in which interrupts interrupt interrupts and are then themselves interrupted. The scheduler has to be able to run its own internal decision-making programs with high priority, but not so high that nothing else ever runs. It can get bogged down in "introverted" computation deciding what it should spend its time doing and meanwhile very little real computation is going on.
• In government, supermajorities and the concurrence of many bodies are necessary to protect the foundations of the system from hasty change, but that protective purpose is frustrated if those foundations are reachable by rules requiring merely a simple majority of one legislature. Although all the rules in the American system are mutable, it is convenient to refer to the less mutable constitutional rules as immutable, and to the ones below them in the hierarchy as mutable.
• Normal social life is just such a system of indefinite tiers. Near the top are actual laws, rising through case precedents, regulations, and statutes, all the way up to constitutional rules. At the bottom are rules of personal behavior that individuals can amend unilaterally without incurring disapprobation or censure. Above these are rules for which amendment is increasingly costly, starting with costs like furrowed brows and clucked tongues, and passing through indignant blows and vengeful homicide.
• Nomic:
  • After a few rounds the game being played by the players may in a certain sense be different from the one they were playing when they started. In a similar way, human beings undergo constant development and self-modification, and yet continue to be convinced that it makes sense to refer, via such words as "I", to an underlying stable entity. The more immediately perceptible patterns change, whereas deeper and more hidden patterns remain the same. From birth to maturity to death, however, the changes can be so radical that one may sometimes feel that in a single lifetime one is several different people. Similarly, in law, many have acknowledged that an amendment clause, even a clause limited to piecemeal amendment, could, through repeated application, create a fundamentally new constitution.
  • The continuing identity of the game, like that of a nation or a person, is due to the fact (if fact it is) that all change is the product of existing rules properly applied, and that no change is revolutionary. Once could even argue that revolutionary change is just more of the same, where rules that were assumed immutable are simply rendered mutable by other rules that are more deeply immutable, but that previously had been taken for granted and hence had been invisible or tacit.
  • Stare decisis - precedents should be followed.
  • Nomic affords the blurring between constitutive rules (which define lawful play) and rules of skill (which define artful play), ie a blurring between the permissible and the optimal.
  • A game can embrace anything at the vote of the players. The line between play and non-play may shift at each turn, or it may apparently be eliminated. Players may be governed by the game when they think they are between games or when they think they have quit.
  • For most games, there is an infallible decision procedure to determine the legality of a move. In Nomic, by contrast, situations may easily arise where it is very hard to determine whether or not a move is legal.
  • Some rules, all of which can become immutable:
    • Immutable (113): A player always has the option to forfeit the game rather than continue to play or incur a game penalty. No penalty worse than losing, in the judgment of the player to incur it, may be imposed.
    • Immutable (114): There must always be at least one mutable rule. The adoption of rule changes must never become completely impermissible.
    • Mutable (202): One turn consists of proposing a rule and having it voted on and throwing a die and adding the number to your score.
    • Mutable (213): If the rules are changed so that further play is impossible, or if the legality of a move is impossible to determine with finality, or if by the Judge's best reasoning, not overruled, a move appears equally legal and illegal, then the first player who is unable to complete a turn is the winner. This rule takes precedence over every other rule determining the winner.
  • Some possible rule changes:
    • Remove the unanimity requirement for changing mutable rules
    • Add new tiers above, below, or between the two initial tiers.
    • Add special procedures for changing some rules (incomplete self-entrenchment)
    • Devise sunset rules that expire automatically after a certain number of turns
    • Allow private consultation on future amendments
    • Allow secret ballots
    • Allow constitutional conventions or revolutions
• How we make analogies determines how we make choices, and that is the essential nature of all judgment.
• The crux of any legal system is the ability of people to distinguish between the incidental qualities and the essential qualities of various events and relations, resulting in recognition of what a given item is, which category it belongs to. This is "choice".
• Coming to grips with the contrast between explicit rules and implicit principles or guidelines is of great importance if one wants to characterize how flexible category recognition (choice) takes place. In any attempt to make a machine capable of choice, one runs headlong into the problem of inconsistencies, level-collisions, and reflexivity tangles.
• In any modality, perception consists of many layers of processing, from the most primitive or syntactic levels, to the most abstract or semantic levels. The zeroing in on the semantic category to which a given raw stimulus belongs is carried out not by a purely bottom-up (stimulus-driven) or purely top-down (category-driven) scheme, but rather by a mixture of them, in which hypotheses at various levels trigger the creation of new hypotheses or undermine the existence of already-existing hypotheses at other levels. This process of sprouting and pruning hypotheses is a highly parallel one, in which all the levels compete simultaneously for attention, like billboards or radio commercials, or advertisements on the subway.
• Any recognition program must have at its core a tiered structure precisely like that of government or Nomic in which there are levels that are easily mutable, moderately mutable, almost mutable, and so on, and so a "choice" program is inevitably to be riddled with reflexivity.

II: Sense and Society

5. World Views in Collision: The Skeptical Inquirer vs The National Enquirer

• At its core, rationality will always depend on inscrutables: the simple, the elegant, the intuitive.
• Human perception and categorization underlie all that we take for granted in terms of common sense, and in more primordial ways that are so deeply embedded that we even find them hard to talk about. Such things as:
  • how we break the world into parts,
  • how we form mental categories,
  • how we refine them certain times while blurring them other times,
  • how experiences and categories are clustered associatively,
  • how analogies guide our intuitions,
  • how imagery works,
  • how valid logic is and where it comes from,
  • how we tend to favor simple statements over complex ones, and so on.
• This question of the interaction of form and content fascinates me deeply. I believe that if one has the right "terraced scan" mechanisms, one can go very far in separating the wheat from the chaff.
• Where does one draw the line? Where is the borderline between open-mindedness and stupidity? Or between closed-mindedness and stupidity? Where is the optimum balance? That is such a deep question that I could not hope to answer it. Even if we have no adequate theory to formalize such decisions, we nonetheless are all walking instantiations of such decision-making beings, and we make decisions for which we could not formally account in 1m years. Such decisions include all decisions of taste. We do not yet know how we make such decisions, but that does not mean we have to allow in indecisiveness.

6. On Number Numbness

• If we can develop a sense for the number of chairs in a room, why not as good a sense for the number of zeroes in a numeral?
  • 10 - (1) - Tens
  • 100 - (2) - Hundreds
  • 1,000 - (3) - Thousands
  • 10,000 - (4) - Tens of Thousands
  • 100,000 - (5) - Hundreds of Thousands
  • 1,000,000 - (6) - Millions
  • 10,000,000 - (7) - Tens of Millions
  • 100,000,000 - (8) - Hundreds of Millions
  • 1,000,000,000 - (9) - Billions
  • 10,000,000,000 - (10) - Tens of Billions
  • 100,000,000,000 - (11) - Hundreds of Billions
  • 1,000,000,000,000 - (12) - Trillions
• Some other kinds of big numbers
  • 100m cells in your retina
  • 100b neurons in your brain
  • 1t glia in your brain
  • 70t cells in your body
  • 6b trillion hemoglobin molecule in your body (400t being destroyed and the same number being created each second
• When numbers get too big, our perceptual relaity begins to shift. We cannot visualize the actual quantity and our perceptual reality becomes the string of zeroes
• Your estimate should be within ten% of the correct answer at the level of your perceptual reality

7. Changes in Default Words and Images, Engendered by Rising Consciousness

• The ability to ignore what is very unlikely - without even considering whether or not to ignore it - is part of our evolutionary heritage, coming out of the need to be able to size up a situation quickly but accurately.

8. A Person Paper on Purity in Language

• What if linguistic sexism were linguistic racism?

III: Sparking and Slipping

• Human thoughts have a way of slipping easily along certain conceptual dimensions into other thoughts, and resisting such slippage along other dimensions. A given idea has slightly different slippabilities - predispositions to slip - in each different human mind that it comes to live in. Yet some minds' slippabilities seem to give rise to what we consider genuine creativity, while others' do not. What is this precious gift? Is there a formula to the creative act? Can spark and slippability be canned and bottles? In fact, isn't that just what a human brain is - an encapsulated creativity machine? Or is there more to creativity and mind than can ever be encapsulated in any finite physical object or mathematical model?

9. Pattern, Poetry, and Power in the Music of Frédéric Chopin

• Phenomena perceived to be magical are always the outcome of complex patterns of nonmagical activities taking place at a level below perception. The magic behind magic is pattern. The magic of life itself is a perfect example, emerging as it does out of patterned but liefless activities at the molecular level. The magic of music emerges from complex, nonmagical - metamagical - patterns of notes.

10. Parquet Deformations: A Subtle, Intricate Art Form

• What's the difference between music and visual art? Temporality. Pieces of music consist of sounds intended to be played and heard in a specific order and at a specific speed. Music is thus fundamentally 1-D - it is tied to the rhythms of our existence. Works of visual art, by contrast, are generally 2-D or 3-D. Paintings and sculptures seldom have any intrinsic "scanning order" built into them that the eye must follow. Your are free to come and go as you please.
• Regrouping - in which the boundary line of the uit cell shifts so that structures jump out at the eye that before were completely submerged and invisible - while conversely structure that a moment ago were totally obvious have now become invisible, having been split into separate conceptual pieces by the act of regrouping, or shift of perceptual boundaries. it is both a perceptual and conceptual phenomenon, a delight to that subtle mixture of eye and mind that is most sensitive to pattern.
• Is there an architecture to creativity? Is there a plan, a scheme, a set of principles that, if elucidated clearly, could account for all the creativity embodied in the collection of all parquet deformations, past, present, and future?
• The essence of any artistic act resides not in selecting particular values for certain parameters, but far deeper: it's in the balancing of a myriad intangible and most unconscious mental forces, a judgmental act that results in many conceptual choices that eventually add up to a tangible, perceptible, measurable work of art.
• Once the finished work exists, scholars looking at it may seize upon certain qualities of it that lend themselves easily to being parametrized. Anyone can do statistics on a work of art once it is there for the scrutiny, but the ease of doing so can obscure the fact that no one could have said, a priori, what kinds of mathematical observables would turn out to be relevant to the capturing of stylistic aspects of the as-yet-unseen work of art.
• What kind of architecture is responsible for all of these ideas? Or is there any one architecture that could come up with them all? I would say that the ability to design good parquet deformations is probably deceptive, in the same way as the ability to play good chess is; it looks more mathematical than it really is.
• Good chess moves spring from the organization of a good chess mind: a set of perceptions arranged in such a way that certain kinds of ideas leap to mind when certain subtle patterns or cues are present. This way that perceptions have of triggering old and buried memories underlies skill in any type of human activity, not only chess. But in chess the skill is particularly deceptive, because after the fact, it can all be justified by a logical analysis, a fact that seems to hint that the original idea came from logic.
• For computers to act human, we will have to wait until we have good computer models of such human things as perception, memory, mental categories, learning, and so on.
• For a machine to make simple variants of a given design, it must possess an algorithm for making that design which has explicit parameters; those parameters are then modifiable. But the way people make variations is quite different. They look at some creation by an artist and then they abstract from it some quality that they observe in the creation itself. This newly abstracted quality may never have been thought of explicitly by the artist yet it is there for the seeing by an acute observer. This perceptual act gets you more than half the way to genuine creativity; the remainder involves treating this new quality as if it were an explicity knob: "twiddling" it as if it were a parameter that had all along been in the program that made the creation.
• That way, the perceptual process is intimately linked up with the generative process; a loop is closed in which perceptions spark new potentials and experimentation with new potentials opens up the way for new perceptions; The element lacking in current computer are is the interaction of perception with generation. Computers do not watch what they do; they simply do it.
• Each of us is at the very center of our own personal web, and each one of us things "I am the most normal, sensible, comprehensive individual." And our identity - our shape in personality space - springs largely from the way we are embedded in that network - the identities (shapes) of the people we are closest to. We help to define others' identities even as they help to define our own.

11. Stuff and Nonsense

• What marks something off as being nonsense?

12. Variations on a Theme as the Crux of Creativity

• In the mind of each person who perceives a Rubik's Cube there arises a concept that we could call "Rubik's-Cubicity". It's not the same concept in each mind, just as not everyone has the same concept of asparagus or of Beethoven. The variations that are spun off by a given cube-inventor are variations on that concept. In a discussion of perception and invention, this distinction between an object and some mind's concept of the object is simple but crucial.
• Invention is more like falling off a log than like sawing one in two. A mind follows its path of least resistance, and it's when it feels easiest that it is most likely being its most creative. Mozart: Things should "flow like oil".
• Making variations on a theme is really the crux of creativity.
• If a concept is a machine with knobs on it, we have to stretch the very concept of "knob". New knobs must be able to sprout, depending on the settings of other knobs. On each concept, there are potentially an infinite number of knobs, and at any moment, some new knobs may get revealed as a consequence of the settings of other knobs.
• Nondeliberate yet nonaccidental slippage permeates our mental processes, and is the very crux of fluid thought. This subconscious manufacture of "subjunctive variations on a theme" is something that goes on day and night in each of us, usually without our slightest awareness of it. It is one of those things that, like air or gravity or 3-D, tend to elude our perception because they define the very fabric of our lives.
• It is very hard to slip mentally into a world in which people would not think by slipping mentally into other worlds - very hard to make a counterfactual world in which counterfactuals were not a key ingredient of thought.
• The hidden fault lines of the mind, which things are solid and which things can slip. Nothing is reliably unslippable. Context contributes an unexpected quality to the knobs that are perceived on a given concept. The knobs are not displayed in a nice, neat little control panel, forevermore unchangeable. Instead, changing the context is like taking a tour around the concept, and as you get to see it from various angles, more and more of its knobs are revealed. Some people get to be good at perceiving fresh new knobs on concepts where others thought there were non, just as som people get to be good at perceiving mushrooms in a forest where others see none, even when they stare mightily.
• One of Knuth's main theses is that with computers, we now are in the position of being able to describe not just a thing in itself, but how that thing would vary.
• I call it the implicosphere (implicit counterfactual sphere) referring to things that never were but that we cannot help seeing anyway (or sphere of implications surrounding any given idea.
• One way to imagine how slippability might be realized in the mind is to suppose that each new concept begins life as a compound of previous concepts, and that from the slippability of those concepts, it inherits a certain amount of slippability. that is, since any of its constituents can slip in various ways, this induces modes of slippage in the whole.
• As the space of possibilities of the new concept - the implicosphere - is traced out, the most common and useful of those slippages becomes more closely and directly associated with the new concept itself, rather than having to be derived over and over from its constituents. This way, the new concept's implicosphere becomes more and more explicitly explored, and eventually the new concept becomes old and reaches the point where it too can be used as a constituent of fresh new young concepts.
• The test of whether a concept has really come into its own, of its genuine mental existence, is its retrievability by that process of unconscious recall. That's what lets you know that it has been firmly planted in the soil of your mind. It is not whether that concept appears to be "atomic", in the sense that you have a single word to express it by. That is far too superficial.
• When one story from a friend triggers the retrieval of an old memory, the two subjects can appear to be completely unrelated, but at some deeper conceptual level, these really are the same idea. There is just one idea here, and this idea I call a conceptual skeleton. Try to verbalize it. It's certainly not just one word. It will take you a while. And when you do come up with a phrase, chances are it will be awkward and stilted - and still not quite right.
• The point is that the concept itself has been reified - this much is proven by the fact that it acts as a point of immediate reference; that my memory mechanisms are capable of using it as an "address" (a key for retrieval) under the proper circumstances. The vast majority of our concepts are wordless in this way, although we can certainly make stabs at verbalizing them when we need to.
• Variations on a them - the notion encompasses knobs, parameters, slippability, counterfactual conditionals, subjunctives, "almost"-situations, implicospheres, conceptual skeletons, mental re-ification, memory retrieval, and more.
• My own mental image of the creative process involves viewing the organization of a mind as consisting of thousands, perhaps millions, of overlapping and intermingling implicospheres, at the center of each of which is a conceptual skeleton. The implicosphere is a flickering, ephemeral thing, a bit like a swarm of gnats around a gas-station light on a hot summer's night, perhaps more like an electron cloud, with its quantum-mechanical elusiveness, about a nucleus, blurring out and dying off the further removed from the core it is. if you have studied quantum chemistry, you know that the fluid nature of chemical bonds can best be understood as a direct consequence of the curious quantum-mechanical overlap of electronic wave functions in space, wave functions belonging to electrons orbiting neighboring nuclei. In a metaphorically similar way, it seems to me, the crazy and unexpected associations that allow creative insights to pop seemingly out of nowhere may well be consequences of a similar chemistry of concepts with its own special types of "bonds" that emerge out of an underlying "neuron mechanics'.
• Making variations is not just twiddling a knob before you; part of the act is to manufacture the knob yourself. Where does a knob come from? The question amounts to asking; How do you see a variable where there is actually a constant? More specifically; What might vary, and how might it vary? It's not enough to just have the desire to see something different from what is there before you. Often the dullest knobs are a result of someone's straining to be original, and coming up with something weak and ineffective. So where do good knobs come from? I would say they come from seeing one thing as something else. Once an abstract connection is set up via some sort of analogy or reminding-incident, then the gate opens wide for ideas to slosh back and forth between the two concepts.
• When two things can both be seen as instances of one abstract phenomenon, it is a very exciting discovery. Then ideas about either one can be borrowed in thinking about the other, and that sloshing-about of activity may greatly illumine both at once.
• Projection of oneself into a situation: "How would that be for me?" People are remarkably fluid at seeing themselves in roles that they self-evidently could never fill, and yet the richness of the insights thus elicited is beyond doubt.
• Experience with a wide variety of things refines your category system and allows you to make incisive, abstract connections based on depp shared qualities.
• Seeing clear to the essence of something unfamiliar is often best achieved by finding one or more known things that you can see it as, then being able to balance these views.
• Once you have decided to try out a new way of viewing a phenomenon, you can let that view suggest a set of knobs to vary. The act of varying them will lead you down new pathways, generating new images ripe for perception in their own right.
• Like a planet orbiting a star, and whose orbit brings it so close to another star that it gets" captured" and begins orbiting the second star. As it swings around the new star, perhaps it find itself coming bery close to yet another star, and fickly changes allegiance.
• You can think of concepts as stars, and knob-twiddling as carrying you from one point on an orbit to another point. If you twiddle enough, you may well find yourself deep within the attractive zone of an unexpected but interesting concept and be captured by it. You may thus migrate from concept to concept. Knob-twiddling carries your from one concept to another, taking advantage of their overlapping orbits.
• Can "concept" become a legitimate scientific term? This goal could be taken to be the central goal of cognitive science.
• Of course not all implicospheres have the same radius. Some people's tend to have bigger radii than others, and consequently they overlap more. This can be good but it can be overdone:
  • Too much overlap and all you have is a mush of vaguely associated ideas, a tasteless mental goulash
  • Too little and you have a very thin, watery mind, one with few big surprises
  • There is, in other words, an optimum amount of overlap for useful creative insight.
• When a new idea is implanted in a mind, an implicosphere grows around it. Since this means, in essence, the linking up of this new idea with older ideas, I call it diffusion in idea-space.

13. Metafont, Metamathematics, and Metaphysics

• The mathematicalization of categories - the idea that any abstraction or Platonic concept can be captured as a software machine with a finite number of knobs, like a meta-font, a meta-waltz, a meta-shoe.
• But to fill out the full "space" defined by a category such as "chair", or "waltz" or "face" is an act of infinite creativity, and no finite entity will ever be capable of producing all possible "A"s and nothing but "A"s.
• This amounts to positing that any conceptual or semantic category is a productive set, namely a set whose elements cannot be totally enumerated by any effective procedure without overstepping the bound of that set.
• A system that contains at least one unprovable truth is said to be incomplete, and a system that not only contains such truths but that cannot be rescued in any way from the fate of incompleteness is said to be essentially incomplete (or productive). The requirement that one must stay within the bounds of a conceptual category could be called consistency.
• The frame problem is about the question: What variables (knobs) is it within the bounds of normalcy to perceive?
• Trying to capture the essence of each separate concept in a separate "knobbed machine", to isolate the various Platonic spirits will not work. Those spirits overlap and mingle in a subtle way.
• No letter is an island. There has to be much mutual knowledge spread about among all the letters. Letters mutually define each others' essences, and this is why an isolated structure supposedly representing a single letter in all its glory is doomed to failure.

IV: Structure and Strangeness

14. Magic Cubology

• This Magic Cube is much more than just a puzzle. It is an ingenious mechanical invention, a pastime, a learning tool, a source of metaphors, and inspiration. It now seems an inevitable object, but it took a long time to be discovered. Somehow, though, the time was ripe, because the idea germinated and developed nearly in parallel in Hungary and Japan and perhaps even elsewhere.
• It is necessary to find a general algorithm for doing it from any scrambled state. No one can restore a messed up Magic Cube to its pristine state by mere trial and error. Anyone who gets back to START has built up a small science.

15. On Crossing the Rubicon

• The cube has some kind of primordial appeal:
  • It is small and colorful, fits snugly in the hand and has a pleasing feel. Twisting is a fundamental and intriguing motion that the hand performs naturally. There are not many puzzles or toys that give the mind and fingers a genuine 3-D workout.
  • That it manages to stay in one piece when it has so many independent ways of twisting is initially amazing, and remains mysterious even after you've seen its "guts".
  • It's a miniature incarnation of that subtle blend of order and chaos that our world is.
  • There are plenty of patterns, some attainable, some unattainable. Sometimes they are simple to generate, but one can't see how they emerge. Sometimes they are hard to generate, yet one clearly understands how they arise.
  • They are many routes to any state, and the shortest is nearly always completely unknowable.
  • The Cube is a rich source of metaphors. It furnishes analogies to particle physics, to biology, to problem-solving in everyday life, to entropy and path-finding, and even touches theology.
  • There is a strong contrast between the algebraic or mathematical approach (long sequences of operations compounded out of shorter sequences, which become increasingly abstract) and the geometric approach (using eye and mind to compose twist after twist along a carefully charted pathway.
  • Aside from cubies and modes of twisting, there is flippedness and twistedness.
• Getting into a scrambled state and getting out of it are operations of different computational complexity. It is easier to find routes out than routes in, even though there are the same number of each. Clearly there is something deeply asymmetric about such a situation, and the whole thing smells of the second law of thermodynamics, stating that entropy will tend to increase with time in a closed system.

16. Mathematical Chaos and Strange Attractors

• The motion of a point in phase space must always be non-self-intersecting. This arises from the fact that a point in phase space representing the state of a system encodes all the information about the system, including its future history, so that there cannot be two different pathways leading out of one and the same point.
• All systems that exhibit turbulent behavior are dissipative: they dissipate or degrade energy from more usable forms such as electricity into the less usable form of heat
• Metrical universality (as opposed to structural universality) shows the parabolic case with the onset of chaos happening through periodic-doubling attractors.
• When you are searching for a memory that eludes you but that you kno is there, what you in effect do is release ten "dogs" in your brain and let them go "sniffing" in parallel. Each dog will start to rummage around here and there, sometimes going in circles, sometimes smelling down wrong alleys, but since there are a bunch of them you can afford to let them smell out many false pathways. They don't need to be very bright; they just need to have had a whiff of the original idea, and they will follow that spoor high and low. Eventually it is likely that one dog or another will trot home carrying the desired memory in its mouth.
• Locking-in - a system that seeks and gradually settles into its own most stable states, and the mechanism whereby it seeks and attains such loci of stability is feedback. A system that locks into a state is in a stable equilibrium, which means that if you perturb it somehow, it will swiftly return to the state is was in - there are restoring forces that push it back.
• As long as we're willing to "up" the dimensionality of the space, we can store more and more information in a single point. Thus fixed points and stable orbits are very close concepts.
• Generally speaking, the stablest behavior of a system seems also to be its simplest behavior.
• Starting somewhere random and relying on feedback to get you somewhere better is the most likely way to discover a fixed point.
• The tangledness of one's own self is a perfect metaphor for understanding what renormalization is all about. Begin by imagining yourself as a 0th order person - someone totally unaware and inconsiderate of all others, a baby. Then imagine how you would be modified if you started to take other people into account, always considering others as perfect babies, or 0th order people. This gives a first-order version of you. You are beginning to have an identity emerging from this modeling of others inside yourself. Now iterate: second order people are those who take into account the identities of first-order people. An on it goes. The final result is renormalized people: people who take into account the identities of renormalized people. I know it sounds circular, and indeed it is, but paradoxical it is not.
• Memory retrieval - How do things that are only vaguely similar to each other stir up rumblings of recollection, and eventually trigger the retrieval of amazingly deep abstract resemblances? If the initial input is a seed - a vector in a very high-dimensional space, then the seed is fed into memory-retrieval mechanisms, which convert it into an output vector that is then ged back in again. This cyclic process continues until it either converges on a stable fixed point - the desired memory trace - or is seen to be wandering erratically without any likelihood f locking in, tracing out a chaotic sequence of "points" in mind-space
• Perception - the bottom-up processing is complemented by concurrent top-down processing driven not by the input, but by expectations of what is "out there" to be recognized. The swirling activity in which bottom-up and top-down processes seek a reconciliation with each other leads to a gradual kind of "crystallization" in which many small pieces of evidence align with, and mutually reinforce, each other. The ultimate justification for some of them resides, of course, in the raw perceptual input, while for others of them it resides in the richness of previous experiences stored in memory. The combination of all these mutually confirming hypotheses results in a globally optimal interpretation of the input: an act of recognition. Once again, locking-in carries the day.
• Locking in seems to be a key to the metamagics of snarls, of society, of slipping, of strangeness, of substrate, of stability, of survival.

17. Lisp: Atoms and Lists

• Psychologically, one of the great powers of programming is the ability to define new compound operations in terms of old ones, and to do this over and over again, thus building up a vast repertoire of ever more complex operations:
  • It is quite reminiscent of evolution, in which ever more complex molecules evolve out of less complex ones, in an ever-upward spiral of complexity and creativity.
  • It is also quite reminiscent of the industrial revolution, in which people used very simple early early machines to help them build more complex machines, then used those in turn to build even more complex machines, and so on, once again in an ever-upward spiral of complexity and creativity.
  • At each stage, whether in evolution or revolution, the products get more flexible and more intricate, more "intelligent" and yet more vulnerable to delicate bugs or breakdowns.

18. Lisp: Lists and Recursion

• Speaking recursively, an n factorial is simply the product of n and the previous factorial. It reduces the given problem to a simple sort of the same type. That simpler one will in turn be reduced, and so on down the line, until you come to the simplest problem of that type, which I call the embryonic case or the bottom line. People often speak, in fact, of a recursion bottoming out.
• Shapes with substructures that goes on indefinitely, never bottoming out in ordinary curves, are called fractals.

19. Lisp: Recursion and Generality

• Since the whole thing is recursive, every needle (in the Tower of Brahma puzzle) will be switching hats many times over during the course of the transfer. That's the beauty of this puzzle and in a way it's the beauty of recursion.
• In Lisp, one has the ability to "elevate" an inert, information-containing data structure to the level of "animate agent", where it becomes a manipulator of inert structures itself. This program-data cycle, or loop; can continue on and on, with structures reaching out twisting back, and indirectly modifying themselves or related structures.
  • Certain types of inert, or passive, information-containing data structures are sometimes referred to as declarative knowledge (aka "knowledge that") - because they often have a form abstractly resembling that of a declarative sentence, and encode facts about the world in some way, accessible by looking in an index in somewhat the way "book-learned" facts are accessible to a person
  • By contrast, animate, or active, pieces of code are referred to as procedural knowledge, since they define sequences of actions that actually manipulate data structures, and can be viewed as embodying the program's set of skills, something like a human's unconscious skills that were once learned through long, rote, drill sessions (aka "knowledge how").
• This distinction should remind biologists of that between genes - relatively inert structures inside the cell - and enzymes, which are anything but inert. Enzymes are the animate agents of the cell; they transform and manipulate all the inert structures in indescribably sophisticated ways. Genes dictate the form of enzymes, and enzymes manipulate genes (among other things). Thus Lisp's procedural-declarative program-data loop provides a primitive but very useful and tangible example of one of the most fundamental patterns at the base of life: the ability of passive structures to control their own destiny, but creating and regulating active structures whose form they dictate.
• If you know enough English, you can bootstrap your way further into English; there is a point beyond which explanations written in English about English are indeed quite useful, and that point is not too terribly far beyond the beginning level. You need a kernel, and then you can begin to life yourself by your own bootstraps. For children, it is an exciting thing when, in reading, they begin to learn new phrases all by themselves, simply by running into them several times in succession. Their vocabulary begins to grow by leaps and bounds. So it is once there is a Lisp kernel in a system; the rest of the Lisp interpreter can be - and usually is - written in Lisp.
• When one interpreter runs on top of another one, there is always the question of what level one chooses not to look below. I personally seldom think about what underlies the Lisp interpreter, so that when I am dealing with the Lisp system, I feel as if I am talking to someone whose native language is Lisp. Similarly, when dealing with people, I seldom think about what their brains are composed of; I don't reduce them in my mind to piles of patterned neuron firings. It is natural to my perceptual system to recognize them at a certain level and not to look below that level.
• For some reason, many people in AI seem to have a deep sense that recursivity in some form or other is connected with the trick of intelligence.
• Gödel's construction revealed in a crystal clear way that the line between direct and indirect self-reference/reference is completely blurry, because it pinpoints the essential role played by isomorphism (or coding) in the establishment of reference and meaning.
• Semantics is an emergent quality of complex syntax.
• Certain orally produced screeches or manually produced scratches (such as word, say, language, sentence, reference, grammar, meaning, etc) can stand for elements of language itself - pieces of language talking about language. How amazingly magical it must have felt to our ancestral cave people when such powerful concepts as words about words first sparked. In some sense, human consciousness began then and there.
• Gödel got a system to talk - in code - about itself. He had snuck self-reference into systems that were presumed to be incapable of self-reference.
• The central data structure of Lisp, the list, was at the core of Gödel's work, and the crucial need to distinguish between atoms and lists was resolved by him with his odd-even distinction. Also critical were the idea of quoting and recursive functions.
• Gödel's remarkable series of 46 function definitions is, IMO, the world's first serious computer program - and it is in Lisp
• A programmer's instinct says that you can cumulatively build a system, encapsulating all the complexity of one layer into a few functions, then building the next layer up by exploiting the efficient and compact functions defined in the preceding layer. This hierarchical mode of buildup would seem to allow you to make arbitrarily complex actions be represented at the top level by very simple function calls. In other words, the functions at the top of the pyramid are like cognitive events, and as you move down the hierarchy of lower-level functions, you get increasingly many ever-dumber sub-routines, until you bottom out in a myriad calls on trivial subcognitive ones.
• But this is a crazy vision - the top-level behavior of the overall system must emerge statistically from a myriad independent pieces, whose actions are almost as likely to cancel each other out as to be in phase with each other.

20. Heisenberg's Uncertainty Principal and the Many-Worlds Interpretation of Quantum Mechanics

• The main thing to keep in mind is that science is about classes of events, not particular instances. Science explains through abstractions that underlie a potentially unlimited number of concrete phenomena.
• Many people think the quantum-mechanical uncertainty principle actually applies to everyday phenomena. Nothing could be further from the truth!
• In some ways, light waves are simpler than water waves:
  • While water waves of different wavelengths travel at different speeds, all light waves travel at one speed, c.
  • The medium that light waves travel through is nondispersive: all wavelengths travel at exactly the same speed. Vacuum is light's medium - how peculiar for waves to wave even when there's nothing to wave!
• De Broglie proposed the equation p = h/λ:
  • Where p = momentum, h = Planck's constant, and λ = wavelength
  • It is proposed as universal, applying to all matter, so that anything has a quantum-mechanical wavelength whose value would depend on how fast it is moving.
  • Wavelength is inversely proportional to momentum.
  • Arbitrarily accurate measurement of either position or momentum is possible; you just can't get both.
• The uncertainty principle is more than an epistemological restriction on human observers; it is a reflection of uncertainties in nature itself. Quantum-mechanical reality does not correspond to macroscopic reality. It's not just that we cannot know a particle's position and momentum simultaneously; it doesn't even have definite position and momentum simultaneously!
• As long as no measurement is made of a system, a physicist cannot know which eigenstate the system is in. In a very fundamental sense, the system itself does not know this and only decides - at random - at the moment the observer observes. And up to this moment it is not in any eigenstate.
• There is a clear connection between the imaginary worlds of our minds and the alternate worlds evolving in parallel with the one we experience.
• When a novelist simultaneously entertains a number of possible ways of extending a story, are the characters not, to speak metaphorically, in a mental superposition of states? If the novel never gets set to paper, perhaps the split characters can continues to. evolve their multiple stories in their author's brain. Furthermore, it would even seem strange to ask which story is the genuine version. All the worlds are equally genuine.

V: Spirit and Substrate

• How and when could mind and emotions - surely the essence of the animate - emerge from complex inanimate substrates.
• Where is the borderline between the highest inanimate flexibility and the lowest animate sentience. When does a system or organism have the right to call itself "I", and to be called "you" by us?

21. Review of Alan Turing: The Enigma

• This image of a machine that jumped from one state to another according to a finite set of rules became uppermost in Turing's mind. What fascinated him was the idea that such meaningless actions could also be viewed as having meanings. For instance, one rule-obeying machine might be viewed as making moves of chess, another as producing truths of mathematics, and yet another as writing poetry.
• What Gödel had left unsettled was the question of whether, given an axiomatic system and an arbitrary proposition in it, one could determine mechanically whether that proposition was undecidable in that system.
• To Turing's surprise, he discovered that for very Gödelian reasons, no machine could be built that could infallibly recognize undecidable propositions.
• First he supposed that a machine for recognizing undecidable propositions exists; then he showed how that assumption leads to self-contradiction - a universal Turing machine being fed its own description number would instantly send it into a dizzying loop of computational vertigo.
• Undecidable propositions run through mathematics like ineradicable threads of gristle that crisscross a steak in such a dense way that they cannot be cut out without the entire steak's being destroyed.
• Mathematics was revealed to be incompletely mechanizable, no matter how complex the machine involved.
• People, no matter how aware they are of their minds, cannot fully take their own complexity into account in attempting to understand themselves, and, quite like Turing machines baffled by their own descriptions, may be plunged into a vertigo of the psyche when they attempt to calculate their own hypothetical or future acts.
• Lady Lovelace's objection - computers cannot originate anything, but can do only what we explicitly tell them to do. Turing's answer to this - that one does not know what one has programmed a machine to do, except in the most superficial and general way - has a depth that eludes many good minds.

22. A Coffeehouse Conversation on The Turing Test

• Starquakes and earthquakes can both be subsumed into a new, more abstract category. And that's how science constantly extends familiar concepts, taking them further and further from familiar experience and yet keeping some essence constant. The number system is the classic example - from positive numbers to negative nmbers, then rationals, reals, complex numbers, "and on beyond zebra".
• Though, even more than hurricanes, ia an abstract structure, a way of describing some complex events that happen in a medium called a brain. But actually, thought can taked place in any one of several billion brains. There are all these physically very different brains, and yet they all support "the same thing": thinking. What's important, then, is the abstract pattern, not the medium. The same kind of swirling can happen inside any of them, so no person can claim to think more "genuinely" than any other. Now, if we come up with some new kind of medium in which the same style of swirling takes place, could you deny that thinking is taking place in it?
• People give other people credit for being conscious simply because of their continual external monitoring of other people - which is itself something like a Turing Test.
• I think emotions are an automatic by-product of the ability to think. They are entailed by the very nature of thought.
• In my view, consciousness requires a certain way of mirroring the external universe internally, and the ability to respond to that external reality on the basis of the internally represented model. What's really crucial is that it should incorporate a well-developed and flexible self-model.
• Any intelligence has to have motivations. Machines, when they look at a scene, will have to focus and filter that scene down into some preconceived categories, just as a person does. And that means seeing some things and missing others. It means giving more weight to some things than to others. This happens on every level of processing.
• People talk to each other out of conviction - not out of hollow, mechanical reflexes. Even the most intellectual conversation is driven by underlying passions. There's an emotional undercurrent to every conversation - it's the fact that the speakers want to be listened to, understood, and respected for what they are saying.
• When things get complicated enough, you're forced to change your level of description, to adopt the intentional stance. Really interesting things will happen in IA when the program itself adopts the intentional stance toward itself.
• The smarter computers get, the more they'll be in a position to tackle messy real-life domains, so they'll be more and more likely to have inaccurate models. To me, mistake-making is a sign of high intelligence.
• Parry (a paranoid) maintains strict control so that no one can truly probe him. For reasons like this, simulating a paranoid is a whole lot easier than simulating a normal person.
• Each time I could postulate some not too sophisticated mechanical underpinning that would allow that particular thing to happen. I kept on trying to come up with rationalizations for the fact that this program was doing so well. My conclusion was that it was a very vast and quite sophisticated bag of tricks, no one of which was terribly complex.
• The class was willing to view anything on a video terminal as mechnically produced, no matter how sophisticated, insightful, or poetic an utterance it might be.

23. On the Seeming Paradox of Mechanizing Creativity

• Creativity is part of the very fabric of all human thought, rather than some esoteric, rare, exceptional, and fluky by-product of the ability to think, which every so often surfaces in places spread far and wide.
• To me, noncreative intelligence is a flat-out contradiction in terms.
• Creativity is an automatic consequence of having the proper representation of concepts in a mind.
• If you have succeeded in making an accurate model of concepts, you have thereby also succeeded in making a model of the creative process, and even of consciousness.
  • It is the organization of memory that defines what concepts are.
  • What established the "concepthood" of something is the way it is integrated into memory.
  • Nothing is a concept except by virtue of the way it is connected up with other things that are also concepts.
  • The property of being a concept is a property of connectivity, a qulity that comes from being embedded in a certain kind of complicated network. Concepts sound like structural or even topological properties of vast tangly networks of sticky mental spaghetti.
• What is the relationship between a general, or Platonic, concept, such as that of tree and the concept you form of some specific tree, between semantic or perceptual categories and the representations of individual instances of them?
• Only through a deep understanding of the organization of memory - which is to say, only by answering the question "What is a concept?" - will we be able to models of the creative process.
• A distinction between
  • spexishness - supremely unconscious behavior (after the wasp)
  • antispexishness - where consciousness is the possession of this to the highest degree.
• To have more antispexishness is to have a greater sensitivity to patterns, an ability to spot patterns of unanticipated types in unanticipated places at unanticipated times in unanticipated media. All human beings have this readiness to see sameness, this alertness, and that is what makes them so antispexish. Whenever they get into some kind of loop, they quickly sense it. Something happens inside their heads - a kind of loop detector (or rut detector or sameness detector) fires.
• The possession of the ability to break out of loops of all sorts seems the antithesis of the mechanical. Or, the essence of the mechanical seems to be in its lack of novelty and its repetitiveness, in its trappedness in some kind of precisely delimited space. This is why the wasp, the dog, even some humans seem so mechanical.
• Clearly there is something lacking in the machine that allows it to have this unbounded tolerance for repetitive actions, an ability to watch oneself as one deals with the world, to perceive in one's own activities a pattern, and to be able to do so at many levels of abstraction.
• Watching one's own internal microscopic patterns is bound to be boring. What is critical is to be able to watch activities on a completely different level - the collective level, in which huge patterns of activity of these many components assume regular behaviors perceptible on their own. A hurricane is a huge pattern of activity of tiny atomes, but one that has such regularity and pattern that we can predict hurricanes without ever thinking of their constituent atoms. A thought is a huge pattern of activity of tiny calles of which much the same can be said.
• Antispexishness has to do with self-perception at this kind of level. Rather than watching its neurons or transistors or registers, an antispexish being watches its own high-level patterns, looking for similarities somewhat the way meteorologists might look for one hurricane following another in a regular way.
• It is in the nature of human pattern perception to be able to detect infinite regresses and to stop them short before they ever get anywhere. But what about the hypothetical self-watching computer, with its infinitely many layers of watchers?
• Once the pattern is perceived by a watcher, that watcher can form the general concept of "all the levels seen at once", associated with the concept of "all the natural numbers conceived of at once", or omega (ω), which we can take as the name of a new watching level that looks out for patterns in this potentially infinite tower of watchers.
• The "halting problem" - Can there exist a program that can inspect other programs before they run and reliably predict if they will go into infinite loops. No.
• Dynamic diagonalization - a slef-watching program - that seems to be closely connected with what makes a human being so utterly different from a stuck record or a Sphex wasp.
• We are not really so concerned with the mathematical perfection of our self-watching system as with its likelihood of survival in a complex world; after all, that's what intelligence is about. If there is a mathematical theorem telling us that no program whatsoever will be a perfect self-watcher, able to catch itself in any conceivable kind of infinite regress, that is simply a statement that perfect intelligence is unreachable.
• BOOLE "Breaking Out of Loops Everywhere" - the idea that humans are self-watchers of infinite depth.
• Each of us, even the Mozarts among us - exhibits a "cognitive style" that in essence defines the ruts we are permanently caught in. Far from being a tragic flaw, this is what makes us interesting to each other. We celebrate individual styles, rather than seeing them negatively, as proofs of inner limits. What in fact is curious is that those people who are able to put on or take off styles in the manner of a chameleon seem to have no style of their own and are simply saloon performers, amusing imitators. We accord greatness to those people whose "limitations", if that is how you want to look at it, are the most apparent, the most blatant.
• The point is that Mozart and Ravel, and you and I, are all highly antispexish, but not perfectly so, and it is at that fuzzy boundary where we can no longer quite maintain the self-watching to a high degree of reliability that our own individual styles, characters, begin to emerge to the world.
• The degree of nonmechanicalness that one perceives in a being is directly related to its ability to self-watch in ever more exquisite ways.
• Critical to the way our memory is organized is our automatic mode of storing and retrieving items, our knowledge of when we know and do not know, of how we know or why we wouldn't know. Such articles of metaknowledge are fluidly integrated into the way our concepts are meshed together. They are not some sort of "extra layer" added on top by a second-generation programmer who decided that metaknoledge is a good thing, over and above knowledge! No, metaknowledge and knowledge are simmering together in a single stew, totally fused and flavoring each other richly. This makes self-watching an automatic consequence of how memory is structured.
• How can we create a program that, like a human brain, is all of a piece, a program that is not simply a stack of ever-higher other-watchers, but is truly a seamless self-watcher, where all levels are collapsed into one.
• The seemingly distinct levels of watcher and watched are totally fused, in the Gödel construction, exactly as Lucas would have it occurring in the minds of all conscious beings. But the ability to fold around and see oneself in the wonderfully circular Gödelian way does not, cannot, bring with it total antispexishness. That is a chimera.
• John Myhill "Three Classes of Ideas":
  • Effective - There is a way, given a candidate for membership, of deciding without any doubt whether that object is or is not a member.
  • Constructive - Some means exists whereby members of the category can be churned out one by one, so that you will eventually see any particular member if you wait long enough, but no means exists for doing the complementary operation - churning our nonmembers one by one
  • Prospective or Productive - It eludes production by any finite set of rules. However, it can be approximated to a higher and higher degreee of accuracy by a series of bigger and better sets of generative rules. Tarski and Gödel establish that truth has this open-ended prospective character. You can produce all sorts of examples of truths - unlimitedly many - but no set of rules is ever sufficient to characterize them all
• Beauty admits of a succession of ever-better approximations, but is never fully attainable. If we see the aim of art as the production of all possible objects of beauty:
  • Each artist contributes objects in a particular style, in which she is trapped.
  • Artists in groups form movements or schools or periods, and their boundaries are wider and the school is less sphexish - more conscious - than any of its members, but it has its limits and starts to stagnate, and a new school begins to form
  • At the level of society, art progresses towards an every wider vision of beauty - a prospective vision of beauty - by a series of diagonalizations - processes of recognizing and breaking out of ruts. This is the process of joosting (jumping out of the system) to ever wider worlds. This endless jootsing is a process whose totality cannot be formalized.
• I believe that reasoning is far from the core of thought and so expert systems with complex systems of rules and metarules will not work for AI. Instead of the topless tower of bureaucracies and meta-bureaucracies above, we must make rule-like behavior emerge out of a mult-level bubbling broth of activity below. Give up the idea of trying to explicitly tell the system as a whole how to run itself and instead define explicity micro-behaviors that will interact in vast numbers, and let them go, carefully watching what ensues. And then tweak and iterate.
• The difference between such statistically emergent macrobehavior and rigidly constrained macrobehavior - you can program the latter, but with the former, all you can do is tailor its micro-elements and then release them and see what happens.
• An intelligence is a system meant to be able to deal with the unpredictable. But how can any set of rules frozen into a machine's design do that?

24. Analogies and Roles in Human and Machine Thinking

• Within the internal representations of concepts there are substructures that have a kind of independence of the structures of which they are part. such a substructure is modular - exportable from its native context to alien contexts. It is an autonomous structure in its own right, and we call these modules roles. A role is a natural "module of description" of something, a sort of bite-sized chunk that seems to be comfortable moving out of its first home and finding homes in other places, some of them unlikely at first glance.
• Something terrible happens to a concept (like Britain's equivalent of America's "First Lady") as it gets more flexible. The definition not only should be general, but also should incorporate some indication of what the spirit of the idea is. Computers have a hard time getting the spirit of things.
• If a role is left implicit, nonverbalized, it has more fluidity in the way it transfers than if it is "frozen" in an English phrase.
• Most analogies arise as a result of unconscious filterings and arrangings of perceptions, rather than as consciously sought solutions to cooked up puzzles. To be reminded of something is to have unconsciously formulated an analogy.
• Meta-analogy - an analogy between analogies.
• There are competing urges: first to stay with the exact original concept, and second to flex and bend when it "feels right", when it would seem rigid and stodgy to insist on established conventions over simple and "natural" extensions. But it is just these sorts of terms - flex, bend, feels right, rigid, natural, and so on - that are so extraordinarily hard to put into programs, logical though programming might be.
• Any mapping is doomed to be imperfect, so how can we do it best (that is with the least pain or frustration)?
• Sometimes you need a willingness to let go of qualities that had seemed important, a willingness to bend gracefully under pressure. Fluid analogies are not a game for rigid minds.
• There is often no Gloriously Right Answer, but there are certainly ideas that seem good and ideas that seem shaky.
• In a play, the various roles all mingle together in scenes. A scene is a larger-scale structure than an individual role; it is a place where several roles coexist and interact. In our analogy problems, one might try to conceive of the two structures involved as if they were two enactments of a single scene, portrayed by different directors working with different actors.
• The less salient an object is inside a larger structure, the harder it is to characterize in an exportable way? But what makes something salient? As a rule, it is its proximity, in some sense, to a "distinguished" element of the larger structure.
• A distinguished item is something we can get at via an elegant, crisp exportable-sounding description. A nearly distinguished item is something we can get at by first pointing to a distinguished item, and then, in an exportable way, describing a short "jog" that leads to it. When giving directions, some places are more salient and some less so. Some roles in a complex conceptual structure are highly distinguished and easily exportable, others are very hard to describe. Although they may have certain idiosyncratic qualities in their local context, nothing makes them stand out globally.
• Don't press an analogy too far, because it will always break down.
• Analogy and reminding, whether they are accurate or not, guide all our thought patterns. Being attuned to vague resemblances is the hallmark of intelligence, for better or for worse.
• The fact we use words and ready-made phrases shows that we funnel the world down into a fairly constant set of categories. Often we end up with one word, such as "kitchen"
• Our language provides for mappings of many degrees of accuracy.
• It is in the nature of categories that their validity can at best be partial. No matter at what level of detail you cut off your scrutiny, your perception amounts to filtering out some aspects and funneling the remainder into a single conceptual target, a mental symbol often labeled with just one word (such as "word") or stock phrase (such as "stock phrase"). Each such mental symbol implicitly stands for the elusive sameness shared by all the things it denotes.
• When a friend recounts a romantic woe, we can usually map it onto some experience of our own. Romances are incredibly detailed and idiosyncratic. The point is that we throw many details away; we skim off some abstractions and are careful not to try to carry the resemblance too far. And certainly we ignore the trivial aspects.
• Precisely in the nonverbalizability of an analogy lies its fluidity, its flexibility.
• A frozen verbal phrase is like a snapshot that gives a perfect likeness at one moment but fails to show how things can slip and move. There is something much more fluid in the way a mind represents the role internally. Various features are potentially important in defining the role, but no until and example comes up and makes one feature explicity does that feature's relevance emerge.
• Words like "this" and "that" and phrases like "that sort of thing" are even better at picking up intangible, flexible, implicit meanings that can be transported across the borders of situations differing widely from each other. And that's the name of the game in thought.
• Boundary separation - Where do you draw the lines separating one substructure from another one? And what are good ways of restructuring or regrouping if a first try fails?
• One can be more flexible or less in how one interprets "Do this!" What does one take literally, what does one see as playing a role in a foreordained and familiar structure? What kinds of familiar structures is one willing to see as identical to each other? When is it necessary to start inventing new ways of perceiving a given situation in order to fit it into pre-existent frameworks, which then allow already-familiar roles to emerge? What remains firm, and what slips? What sticks, and what gives? These kinds of questions sound rather abstract, but when real analogies are manufactured, they are the chief concerns of the analogy maker.
• Central problems of analogy making:
  • How literally to take references
  • What structures are worth perceiving
  • Which aspects are most relevant
  • How literally to take roles
  • Choosing the most elegant way of viewing a situation against another.
• Symmetry, uniformity, good substructures, boundary strength, and so on - these are the kinds of forces that push and pull you to and from analogical choices.
• Some answers are "sick", others are "healthy" in terms of the survival value that an analogy-making capacity confers on its possessors. After all, our brains got to be the way they are only by helping our forbears to survive better than their rivals in this unforgiving world. And analogy-making is at or near the pinnacle of our mental abilities:
  • Law sanctifies analogy as the ultimate justification for making a reasonable and even a wise decision. "Precedent" is just a legalistic way of saying "well-founded analogue".
  • Questions about what to buy, what to think of someone, whom to marry, whether to move to a new city, how to talk, etc - all are influenced in a myriad ways by prior experiences of the same general sort. And even in cases with no obvious analogy, all the categorization of the situation is being made by a mind exposed to many thousands of words, and the purpose of words is to label situation types and thus implicitly to make use of stored analogical mappings.
  • The boundary line between making creative analogies and recognizing pre-existent categories is very blurry. Most common words hide an enormous degree of analogical abstraction.
  • Samenesses that we have absorbed into our perception as being obvious were once great insights.
  • Far from being a matter of taste, variations in analogy-making skill can spell the difference between life and death!
  • People who make "sick" analogies to the letter puzzles will have a rough time in life because they lack the means to size up a situation and catch its essence in the mind's mesh, letting the trivial pass through. Something about their analogy-making mechanisms is defective.
  • There are better and words answers to analogies. Elegance is more that just a frill in life; it is one of the driving criteria behind survival. Elegance is about getting at the essence of situations.
  • What should be taken literally and what slipperily?
  • Elegance is a drive toward abstract symmetry.
• In an analogy where real insight is needed, often some initial forays are made that reveal some literal-minded ways of seeing the situation, but they ring false or overly crude, and so one does not stop there. Instead one continues searching, guided by a number of small cues, and at some unpredictable moment, something simply snaps, and a host of things fall into place in a new conceptual schematization of what is going on. What was once important becomes suddenly trivial, and a new essence emerges, an organizing concept or set of concepts that seem far superior.
• But beware! Being literal-minded is not to be avoided at all costs. If jumping to rarified levels of abstraction were always preferable, then we would wind up never making any distinctions between situations. Rigidity comes in many forms, and a rigid drive toward abstraction is no less stupid than a rigid refusal to abstract. The clearest and cleanest statement of the problem that analogy poses is that there are always fights between forces pushing for literality and forces pushing for abstraction.
• Seeking the balance point is far more than just esthetic play; it probes the very core of how people perceive abstractions, and it does so without their even knowing it.
• Even in scientific discovery, conflicting pressures push around one's percepts and concepts, letting things bounce against each other until, all at once, something galls into place and then, presto! A sense of certainty crystallizes, so powerful that you know you have found the right way to look at things. Mini-breakthroughs and maxi-breakthroughs have precisely the same texture.
• What characterized Einstein's unique view of space and time was that he had decided that certain things were more unslippable than others; in particular that the speed of light was unslippable but the notion of absolute simultaneity of events separated in space was slippable. Or rather, he didn't decide that simultaneity was slippable, but was forced into that conclusion, since his stronger intuitive belief in the invariance of c compelled him to accept its consquences, strange and counterintuitive though they might be. He did not begin with the idea of simultaneity being nonabsolute, but when he had to confront that possibility, he let it slip. This fluidity of mind, guided by a certainty about the deepest, most unslippable concepts, gave rise to the creative insights of special relativity.
• A reliable nose for what might slip and what ought not marks the difference between a great mind and a small one.
• In a certain way, translation is the quintessential form of analogy.
• Translating a novel's title may sometimes be the most challenging aspect of translating the whole novel.
• Is this one phenomenon manifesting itself in two different ways, or is it simply a pretty coincidence? Such questions can occasionally not be answered, but very often our minds come to conclusions on such matters without our ever noticing it. Reification of new categories in words is a telltale signal, and one of the most important of mental events.
• The task of compressing a piece of text one has written into fewer and fewer words forces one to struggle to define the essence of what one is trying to get across. Up to a point, a piece of text may actually be improved by having some fat trimmed here and there, but this can be carried too far, and meaning will certainly begin to suffer.

25. Who Shoves Whom Around Inside the Careenium?

• Were you to take an average over a million marbles, you could find out how each pin statistically affects the way the marbles descend to the bottom. Pathways aren't just allowed or disallowed; rather some are more likely, some are less likely, depending on how the pins are arrayed. There are fewer degrees of freedom for the motions in a maze. But suppose that in your maze, one of the effects of the people moving were that the hedges gradually shifter position, as if the maze were formed of movable partitions that constrain the maze runners, yet the maze runners' paths gradually move the partitions, thus changing the maze.
• There are two scales in time and space operating here, each affecting the other. The heavy hockey-puck-like pins appear almost stationary to the light marbles. To a casual observer who's following the motions of the marbles, the massive pins would appear to be determining the light marbles' motions, to be telling them where to go, shoving them around.
• The marbles are launched according to conditions outside the table. If there's a red light near a marble-launching station, that station slows down its firing rate; if a green light is near it, it speeds it up.
• Say there are no pucks at all. There are only marbles and a number of larger stiff yet malleable mobile metal strips (stiff yet malleable membranes).
• So these SYMMs will now be knocked around along with the marbles that ar bashing into one another. Will the SYMMs occasionally get wrapped around some group of marbles and form a circular membrane, separating out a group of marbles from the rest?
• So who pushes who comes down to summballs vs marbles. And I can twiddle the speed control on the projector and watch the film fast or slow.
• Symballs are constantly forming and unforming, like blocks of ice melting down into chaotically bounding water molecules - and then new ones can form, only to melt again. This kind of phase transition view of the activity is very apt. And it introduces a third time scale for the projector, one where it is running much faster and even the motions of the symmballs would start to blur. Symmballs have a dynamics, a way of forming, interacting, and splitting open and disintegrating, all their own. Symmballs can be seen as reflecting, internally to the careenium, the patterns of lights outside of it. They can store "images" of light patterns long after the light patterns are gone - thus their configurations can be interpreted as memory, knowledge, and ideas.
• I guess I can't see how I myself could be a system like this inside my cranium. I feel alive. I have thoughts, feelings, desires, sensations!
• "Thought" is a shorthand for the activity of the symmballs that you see when you run the movie fast: the way they interact and trigger patterns of motions among themselves.
• Add an artificial mouth and throat, just as we added an arm.
• The careenium voice is giving a dynamic description of what is perceived in the vicinity.
• When you perceive something "out there", you cannot help but mirror that even inside you somehow. Without that internal mirroring event, there would be no perception. The trick is to know what kind of external event triggered it, and to describe what you felt out loud in public language that refers to something external.
• The most convenient language gets straight to the external source. We are constructed in such away as to be unaware of our brain's internal activity underlying perception, and therefore we map it outward.
• Verbalizing its inner state employs words that you and I think refer to objects and facts outside the careenium. In fact, it too thinks so. But you could very well argue that it is just making sounds that mirror its internal state in some very complex way. It could be deluding itself. There might be nothing out there to refer to!
• As a member of the human race, you were constructed in such a way as to enjoy mimicry. You heard your own voice. Your were then able to compare the sounds you'd just made with your memory of the sounds you'd heard. By playing this exciting new game, you were learning the English words for objects.
• Gradually, I internalized a huge set of external, public, aural conventions - namely the English words attached to particular states of my own brain, states that were correlated with things "out there". But instead of conceiving that the words described my brain state, it was easier to conceive of them as describing things out there directly. By omitting a level in my interpretation of my own brain's state, I cast internal images outward.
• Could there be inward-directed transducers that focus on symmballs and come up with a representation of symmball activity? That would be a sort of sixth sense - an inward-directed sense. You could call it the "inner eye".
• Mindquakes would be, after all, just as tangible to the system as is an increase of marble-firings on any side. Both are simply internal events, even though the one is set off by something internal.
• We're constantly thinking thoughts - some fresh, some stale - constantly mentally alive and aware - partly of the external world, partly of our own state - for example, how confused or tired we are, what something reminds us of, how bored we are...
• Every organism has to monitor itself in terms of hunger. But primitive organisms don't use much information about the external environment they're in; they just flap about and hope to encounter some food. Pretty unconscious. A more complex organism will have an elaborate representation of its environment inside itself, so it also has a lot of options when it detects internal hunger.
• The total interaction of symbols at that point we might call "consideration" or "deliberation" or "reflection" - as distinguished from "reflexes. Now does this help you to see why such a careenium might have a self?
• You can only do what your brain will allow you to do, ad that is very crucial.
• You can't decide to do "anything"; you are limited to being able to decide to do only things you want. Worse yet, you are in fact limited to doing at any time, the one thing that you want most to do!
• People can pick up habits that begin to interfere with the rest of their lives. Once the wife starts twiddling a rubik's cube, a part of her takes over. She gets obsessed, or possessed by one of her own subsystems! There is a kind of inner inertia that makes her want to continue, even when there are other things she would also like to do. It's as if some part of her just "slips away" from a higher level of control - some subsystem gets out of control and won't obey the soul on top - like a bucking bronco unwilling to obey its rider.
• We'd like to say that symmballs can decide to do arbitrary things, but they are constrained. The "heavyweight" entities, hedges, pins, symmballs, constrain the "lightweight entities - maze runners, pinballs, symms; but in revenge, the little ones, acting together control how the high-level ones are arrayed.
• A "snap decision" is a coarse-grained manner of talking about a huge cloud of neural activity, like a huge blurry cloud of symms in a careenium projected at a high speed on the screen. And sometimes the activity of neurons inside a cranium or of symms inside a careenium, lends itself arbitrarily to such a high-level, coarse-grained, symbolic description.
• I find I'm most creative when I feel my brain consisting of such halfway-coalesced symbols - neurons acting somewhat independently, somewhat collectively. It's a happy medium where neural bubblings cooperate with symbolic channelings and yield the most creative, fulfilling, semi-chaotic sense of aliveness.
• It's hard to assign credit or blame once you start analyzing thought mechanistically. I see that decision and choice are very subtle concepts that somehow have to do with the ways in which constraints on two different levels affect each other reciprocally, and at two different time scales, inside a cranium or a careenium.
• When your leg falls asleep its a chance to be privy to all the tiny goings-on inside your leg, feeling the mingling tingling of trillions of cells all buzzing at once. Do you suppose that's what being alive really is like, and most of the time we're numb to it? Can you imagine if all of your body were always as fuzzy and tingly as that?
• Sometimes the decisions I make seem to be slow percolating processes, things that are utterly out of my control.
• In emotionally wrenching cases, you can hardly decide what you will feel. Something just happens inside you. Subtle forces shift deep inside you, hidden, subterranean. You find out what sort of stuff you're made of. It's more passive than active - or the action is on levels of yourself that are far lower - far more microscopic - than you have direct control over.
• In cases of judgement, the top level pretty much has to wait for decisions to percolate up from the bottom level. The masses down below are where the decision really gets made, in a time of brooding and rumination. Then the top level may struggle to articulate the seething activity down below, but those verbalized reasons it comes up with are always a posteriori. Words along are never rich enough to explain the subtlety of a difficult choice. Reasons may sound plausible but they are never the essence of a decision, the tip of the iceberg. Every reason has its army.
• I guess that the real conviction of having free will would arise when, repeatedly and reliably, a collection of symmballs wants something and then watches its desire getting carried out. It must seem like magic!
• The continuity and strength of the feeling of "being someone" come from identification with past and future versions of the same system, from the way the system sees itself as a unitary thing moving and changing through time - it's an outcome. In a way, it's nature's hoax: the illusion of selfsameness.
• When an organism is so complicated that it is forced to ascribe beliefs and purposes to itself, you could say it is adopting the auto-intentional stance. This is its best way of understanding itself. It creates a self-perpetuating delusion, the notion of a unitary self then reenters the system as one of its own beliefs, and the more the illusion is cycled through the system the more established and hardened and locked-in it becomes.
• Once the concept of a self is locked in, this fact determines much of the system's own future behavior. And this same level-crossing feedback loop takes place in every careenium of sufficient complexity, so they all have notions of selfhood.
• The circling-back of a complex representation of the system together with its representations of all the rest of the world is the single mechanism underlying this I-ness. And the way you do this cycling and the way you represent the world determines which I you are.

26. Waking Up from the Boolean Dream, or, Subcognition as Computation

• Various phrases could be substituted for being conscious:
  • Thinking
  • Having a soul
  • Having an inner life
  • Having semantics (as opposed to mere syntax)
  • Having content (as opposed to mere form)
  • Having intentionality
  • Being something it is like something to be
  • Having personhood
• Hofstadter's position: "Everything of interest in cognition happens below the 100ms level - the time it takes to recognize your mother."
• The problem of intelligence is to understand the fluid nature of mental categories, to understand the invariant cores of percepts such as your mother's face, to understand the strangely flexible yet strong boundaries of concepts such as chair or the letter a.
• Here is the very hingepoint of thought, the place where one thing slips into alternate, subjunctive, variations on itself.
• How can such a fantastic recognition machine as our brain be so terrible at rendition? We can accept things as members of categories and perceive how they are members of those categories, yet not be able to reproduce those things from memory.
• Carefully constructed simple analogy problems probe close to the core mechanisms of intelligence.
• AI programs are carrying out cognitive activities in the absence of any subcognitive activity. There is no substrate that corresponds to what goes on in the brain. There is no fluid recognition and recall and reminding. These programs have no common sense, little sense of similarity or repetition or pattern. They can perceive some patterns as long as they have been anticipated - and particularly, as long as the place where they will occur has been anticipated - but they cannot see patterns where nobody told them to look. They do not learn at a high level of abstraction.
• This is in complete contrast to how people are. People perceive patterns anywhere and everywhere, without knowing in advance where to look. People learn automatically in all aspects of life. Common sense if not an area of expertise, but a general domain-independent capacity that has to do with fluidity in representation of concepts, an ability to sift what is important from what is not, and ability to find unanticipated analogical similarities between totally different concepts (reminding)
• Finding anagrams - Using intuitions about letter affinities, plausible clusters and their stabilities, syllable qualities, etc
• Short term memory is an epiphenomenon, a consequence that emerges out of the design of the system, a product of many interacting factors, something that was not necessarily known, predictable, or even anticipated to emerge at all.
• What you see at the top level need not have anything to do with the underlying swarm of activities bringing it into existence. Something can be computational at one level, but not at another level.
• People see symbols as lifeless, dead, passive objects - things to be manipulated by some overlying program. I see symbols - representational structures in the brain - as active. They flow and act on their own. There is no higher level agent or program that reaches down and shoves them around. Active symbols must incorporate within their own structures the wherewithal to trigger and cause actions.
• The brain does not manipulate symbols - it is the medium in which the symbols are floating and in which they trigger each other. There is no central program. There is simply a vast collection of teams - patterns of neural firings that, like teams of ants, trigger other patterns of neural firings.
• The symbols are not down there at the level of individual firings but up here where we do our verbalization. We feel them churning in us like we feel our stomach. We do not do symbol manipulation by some sort of act of will, let along some set of logical rules of deduction. We cannot decide what we will next think of, nor how our thoughts will progress.
• We are manipulated by our symbols! Cognition, that rational-seeming level of our minds is a consequence of much deeper processes of myriads of interacting subcognitive structures.
• Thought is not a formal activity whose rules exist at the level of predicate calculus and other formalisms.
• What good would it do a spear thrower to be able to calculate parabolic orbits? They need to imagine a cluster of approximations of what may happen and to anticipate some plausible consequences of them
• Mathematical simulation has to be replaced by abstraction, which involves discarding the irrelevant and making shrewd guesses based on analogy with past experience. Rather than playing out a scenario perfectly, play it out in a few probable or plausible ways or bring in some scenarios from the past that may have no obvious relevance other than as metaphors.
• Categories do not point to specific physical objects, but they can be used as masters from which copies - instances - can be rubbed, and then those copies are activated in various conjunctions; these activations then automatically trigger other instance-symbols into activations of various sorts. The overall activity will be semantic - meaningful - if it is isomorphic, not necessarily to some actual event in the real world, but to some event that is compatible with all the known constraints on the situation
• Any perceived situation seems to be surrounded by a cluster, a halo, of alternative versions of itself, of variations suggested by slipping any of a vast number of features that characterize the situation, and this is at the dead center of thinking.
• Until AI has been stood on its head and is 100% bottom-up, it won't achieve the same level or type of intelligence as humans have. Subcognition at the bottom will drive cognition at the top. And, perhaps, most importantly, the activities that take place at that cognitive top level will neither have been written nor anticipated by any programmer. This is the essence of what I call statistically emergent mentality.
• As fragments start to fit together coherently the system continues to turn down its randomness knob. Gradually, larger conceptual structures begin to form and to confirm each other in a benign, self-reinforcing loop. These higher-level structures now bias the probabilities of random activation of lower-level fragments, so that the thermal activity, though still random, is more directed. The system is settling into a stable state that captures, in some internal code, the salient external realities.
• The idea of stochastically guided convergence to what is called a globally optimum state seems to have arisen (as do so many good ideas) in the minds of several people at once.
• This new model comprises three notions:
  • Asynchronous parallelism
  • Temperature-controlled randomness
  • Statistically emergent active symbols
• Why should the intangible world of the intellect be any less real than the tangible world of the body? Does it have less structure? No, not if you get to know it. Every type of complexity in the physical world has its mirror image in the world of mathematical constructs, including time.
• Symbolic events are not the primitives of thought.
• What we are looking for, in explaining cognition, is a bridge between the formal and the informal.

VI: Selection and Stability

27. The Genetic Code: Arbitrary?

• An enzyme is a protein that does something.
• All proteins are strangely curled molecules made up of amino acids. A protein likes to be curled up in its little ball. That 3-D shape is called its tertiary structure, and each tertiary structure is unique. The sequence of amino acids totally determines the tertiary structure; it is what makes the protein fold up every time into the same shape. The 1-D sequence of amino acids is the protein's primary structure and the primary structure determines the tertiary structure
• Certain parts of an enzyme are called its active sites. They are where the enzyme fastens itself, leechlike to the lumps it is going to act on.
• An enzyme is very specific; it is tailor-made for a certain task and for no other. Once it is hooked up to its substrates, it starts churning, like a washing-machine into which the proper coins have been inserted. It may rip parts off one substrate and attach them to another, it may bind two substrates together, etc
• But enzymes are not protein-builders. That job is so delicate and specialized and critical that a different kind of machine exists to do it, called a ribosome, of which there are thousands in any living cell.
• A ribosome is simply a general-purpose amino-acid hooker-upper. But then somebody must tell it which amino acids to hook up and in what sequence. This is the job of messenger RNA (mRNA) consisting of thousands of nucleotides strung together like beads.
• A ribosome is a translating mechanism between the nucleotide and amino acid languages, between codons and amino acids. In normal circumstances, it can translate about 20 codons/sec in a bacteria cell (but only 1/sec in a rabbit cell).
• DNA is a set of blueprints for all kinds of cellular constituents, lumps and doers alike. The piece of DNA that codes for some specific entity (a tRNA molecule, a protein, some RNA that will become part of a ribosome) is that entity's gene. Whatever the constituent is, there is a gene for it in the long, twisty DNA molecule. That is why DNA is so long. It consists of a sequence of thousands, millions, or even billions of codons, constituting anywhere from a handful of genes to many thousands of them arranged sequentially, like sentences following one another in a book, or songs in the grooves of a record.
• Two levels of decoding for DNA, for a tRNA modelcule, it is shallow decoding, or transcription, but to make a protein, it uses both stages, with the second being translation, which lays at the dead center of life.
• The DNA incorporates coded versions of all the tRNA molecules, of all the synthetases and polymerases, not to mention the constituents of the ribosomes. It contains coded versions of its own decoders.
• A more compact, more elegant, more efficient language will be more able to keep up with real-time needs - Efficiency matters
• Having variants matters, so the variants can fight it out and the best will survive while the weaker ones are pruned.
• But shortness and efficiency aren't all - we need to be able to easily distinguish between words, so clear distinctions may act against ultimate concision.

28. Undercut, Flaunt, Pounce, and Mediocrity: Psychological Games with Numbers

• How universal, how primitive is the blend of goodwill and rivalry that one feels in a highly competitive games with friends, that feeling of pitting one strategy against another and watching them fight it our. Dogs certainly seem to experience this feeling pleasure.
• Since the game Undercut is completely symmetric for both players, there can be no winning strategy in game theory, for otherwise both players could use it and be guaranteed to beat each other. But there is an optimal strategy, which will guarantee you long-term parity with your opponent.
• The game theory version of the game knew nothing of temptations or teasing. It just played blindly on, and the longer the game, the more surely it would break even.
• For humans, impatience and audacity are both important psychological elements, and an optimal strategy does not take those into account.
• To encourage pattern-flaunting, it seemed reasonable to award pattern points whenever that are not picked up on by the enemy. This version is "Flaunt".
• Variant rules - "stop when the two players numbers differ by 2" and "the game is over if either player wins five turns in a row" lead to the version "Pounce"
• "Mediocrity" - A variant with 3 players where the middlemost number is rewarded. And after a certain number of rounds, the player whose score was in the middle should win. To win at level 2, it's best to be a mediocre player at L1. Whereas before it was desirable to be extremely mediocre at choosing mediocre numbers, now it's desirable to be mediocrely mediocre at choosing mediocre numbers. How perverse! How wonderful!
• The strategy for winning at L2 gets confusing very quickly and L3 is completely beyond reach. If such simple games are so baffling, how much more complex are the games of international bargaining, bluffing, and war-making. One can hardly help feeling like a single cell in some vast organism whose strategy was set long ago, the consequences of which one can only watch, hoping all will turn out for the best

29. The Prisoner's Dilemma Computer Tournaments and the Evolution of Cooperation

• Does logic prevent cooperation? This is the issue of the Prisoner's Dilemma.
• In the iterated Prisoner's Dilemma, entities interact repeatedly over time.
• In a collective sense, it would be best for both of you to always cooperate. But suppose you have no regard whatsoever for the other. There is no "collective" good you are both working for. You are both supreme egoists.
• In the canonical PD payoff matrix:
  • 3 = R (the reward for mutual cooperation)
  • 1 = P (the punishment)
  • 5 = T (the temptation)
  • 0 = S (the sucker's payoff)
  • Temptation > Reward > Punishment > Sucker's Payoff
  • (Temptation + Sucker's Payoff) / 2 < Reward
• How you should play depends on who you're playing
• Can totally selfish and unconscious organisms living in a common environment come to evolve reliable cooperative strategies? Can cooperation emerge in a world or pure egoists? In a nutshell, can cooperation evolve out of noncooperation?
• TIT FOR TAT's tactics are:
  • Cooperate on move 1
  • Thereafter, do whatever the other player did the previous move.
• TIT FOR TAT is "nice": It will never be the initial cause of a breakdown of trust
• Even expert strategists from political science, sociology, economics, psychology, and mathematics made the systematic errors og being too competitive for their own good, not forgiving enough, and too pessimistic about the responsiveness of the other side.
• There is no single strategy for all environments, so that winning in one tournament is no guarantee of success in another. T4T has the advantage of being able to "get along well" with a great variety of strategies, while other programs are more limited in their ability to evoke cooperation.
• The general lesson is "Be nice, provocable, and forgiving."
• T4T did not beat anyone. It cannot beat anyone, but it still wins out overall. It wins by eliciting behavior from the other player which allowed both to do well. Like a business partner who never cheats anyone, T4T never beats anyone - yet both do well for themselves.
• Having behavior so complex as to be incomprehensible is very dangerous.
• How does cooperation emerge?
  • Initial viability - Even if there are only small clusters of cooperators, that is enough to gain a toehold.
  • Robustness - Niceness, provocability, forgiveness, and clarity tend to flourish even in unpredictable and shifting environments.
  • Stability - Once cooperation has established itself, it is permanent and not penetrable by meanies
• Mutual cooperation can emerge in a world of egoists without central control, by starting with a cluster of individuals who rely on reciprocity.
• How can the human race, whether for selfish or more cosmopolitan ends, understand and control the seemingly blind forces of history?
• Should I abide by the rules until they're changed, or help speed the change by breaking them? Better start rushing before the rush begins!

VII: Sanity and Survival

30. Dilemmas for Superrational Thinkers, Leading Up to a Luring Lottery

• The mental pressures are not completely explicit; they are evoked by, but not totally spelled out by, the wording of the letter. Each person brings a unique set of images and associations to each word and concept, and it is the set of those images and associations that will collectively create, in that person's mind, a set of mental pressures like the set of pressures inside the earth in an earthquake zone. Wen people decide, you find our how all those pressures pushing in different directions add up, like a set of force vectors pushing in various directions and with strengths influenced by private or unmeasurable factors. The assumption that it is valid to extrapolate has to be based on the idea that everybody is alike inside, only with somewhat different weights attached to certain notions.
• Emotionally it cannot help but feel a little peculiar since it flies in the face of free will and regards people's decisions as caused simply by combinations of pressures with unknown values. On the other hand, perhaps that is a better way to look at decisions than to attribute them to "free will", philosophically murky notion at best.
• To what extent does my choice inform me about the choices of the other participants?
• If reasoning dictates an answer, then everyone should independently come to that answer, like it is a math problem. Seeing this fact is itself the critical step in the reasoning toward the correct answer, but unfortunately it eluded nearly everyone to whom I sent the letter. Once you realize this fact, then it dawns on you that either all rational players will choose D or all rational players will choose C. This is the crux.
• Any number of ideal rational thinkers faced with the same situation and undergoing similar throes of reasoning agony will necessarily come up with the identical answer eventually, so long as reasoning alone is the ultimate justification for their conclusion. Otherwise reasoning would be subjective, not objective as arithmetic is. A conclusion reached by reasoning would be a matter of preference, not of necessity. Now some people may believe this of reasoning, but rational thinkers understand that a valid argument must by universally compelling, otherwise it is simply not a valid argument. (at least for me, this is a surprising conclusion or position for H to hold)
• Schoolchildren do not choose what 507/13 is; they figure it out. Analogously, my letter really did not allow choice; it demanded reasoning. Thus a better way to phrase the "voodoo" statement would be this: "If reasoning guides me to say C, then, as I am no different from anyone else as far as rational thinking is concerned, it will guide everyone to say C."
• The corresponding foray into the opposite world ("If I choose D, then everyone will choose D") can be understood more clearly by likening it to a musing down by the schoolkid before she divides. "Hmm, I guess it's about 49, maybe 39. I guess I need to work it out but when I do, I'll have the right answer.
• Choosing D undermines your reasons for doing so.
• One entrant treated his own brain as a simulation of other people's brains and ran the simulation enough to get a sense of what a "typical person" would do.
• As long as you feel your decision is independent of others' decisions, you should defect. But cool-headed people should take into account that cool-headed calculating people should take into account that cool-headed calculating people should take into account that cool-headed calculating people should take into account that...
• Another entrant expected that there is enough flakiness in people that he would prefer not to rely on the rationality of 19 other people. But of course in assuming the flakiness of others, he would be his own best example
• Reverberant doubt - an amazing and disturbing slide from certain restraint to certain pushing? It is a cascade, a stampede, in which the tiniest flicker of a doubt has become amplified into the gravest avalanche of doubt. And the brighter you are, the more quickly and clearly you see what there is to fear. A bunch of amiable slowpokes might well be more likely to unanimously refrain and get the big payoff than a bunch of razor-sharp logicians who all think perversely recursively reverberantly. It's that smartness to see that initial flicker of a doubt that triggers the whole avalanche and sends rationality a-tumblin' into the abyss. (see Intriguing Words, Expressions, Phrases)
• We have nothing to fear but fear itself!

31. Irrationality is the Square Root of All Evil

• A defection is an action such that, if everyone did it, things would clearly be worse (for everyone) than if everyone refrained from doing it, and yet which tempts everyone, since if only one individual (or a sufficiently small number) did it while others refrained, life would be sweeter for that individual (or select group).
• Apathy at the individual level translates into insanity at the mass level.
• Garret Hardin's The Tragedy of the Commons. Two types of rationality, which are in an inevitable and eternal conflict:
  • Local - that strives for the good of the individual
  • Global - that strives for the good of the group
• But if they are fully aware of their joint situation, and yet in the face of it they blithely continue to act as if their situation were not a communal one, then I maintain that they are acting totally irrationally. In other words, with an enlightened citizenry, local rationality is not rational, period. It is damaging jot just to the group, but to the individual. Thus it pays a lot to reflect carefully about one's situation in the world before defecting, that is, jumping to do the naïvely selfish act. You had better be prepared for a lot of other people copping out as well, and offering the same flimsy excuse.
• This short-sighted race for first place reveals the way in which people in a huge crowd erroneously consider their own fancies to be totally unique.
• In an era when resources are running out in a way humanity has never had to face before, new kinds of social arrangements and expectations must be imposed, Hardin feels, by society as a whole. He is a dire pessimist about any kind of superrational cooperation, emphasizing that cooperators in the birth-control game will breed themselves right out of the population.
• Conscience is self-eliminating.
• I sometimes wonder whether there haven't been many civilizations Out There, in our galaxy and beyond that have already dealt with just these types of gigantic social problems. Most likely some would have survived, some would have perished. And perhaps the ultimate difference in those societies may have been the survival of the meme that, in effect, asserts the logical, rational validity of cooperation in a one-shot PD. It may be that lack of conscience is self-eliminating, provided you wait long enough that natural selection can act at the level of entire societies.
• Evolution is a merciless pruner of ill logic.
• Most philosophers and logicians are convinced that truths of logic are analytic and a priori; they do not like to think that such basic ideas are grounded in mundane, arbitrary things like survival. They might admit that natural selection tends to favor good logic - but they would certainly hate the suggestion that natural selection defined good logic! Yet truth and survival value are all tangled together, and civilizations that survive certainly have glimpsed higher truths than those that perish.
• Ultimately, beliefs are to be grounded in experience, whether that experience is the organism's or its ancestors', or its peer group's. My feeling is that the concept of superrationality is one whose truth will come to dominate among intelligent beings in the universe simply because its adherents will survive certain kinds of situations where its opponents will perish. Let's wait a few spins of the galaxy and see. After all, healthy logic is whatever remains after evolution's merciless pruning.

32. The Tale of Happiton

• The Disobedi-ant refused to believe that its powerful impulses to play instead of work were anything but unique expressions of its very unique self, and it went its merry way, singing, "What I choose to do has nothing to do with what any-ant else chooses to do! What could be more self-evident?
• Coincidentally enough, so went the reasoning of all its colony-mates. In fact the same refrain was independently invented by every last ant in the colony and each ant thought it original. It echoes throughout the colony, even with the same melody.
• The colony perished.

33. The Tumult of Inner Voices, or, What is the Meaning of the Word "I"?

• Which is right? Am I really here at my terminal, typing, or am I really here, standing before you? I can't decide.
• My mind is "shoved around" by internal agents of my brain that have become activated after a long dormancy. And where my brain shoves, so must my mind follow. How can I be saying these strange things? What is the meaning of "mind as slave to brain"?
• As a consequence, the behavior of such a system has some striking properties that go under the general heading of "collective phenomena".
• A traffic jam is just not on the level of an individual car. It is a pattern composed of cars, a pattern that moreover has deep repercussions on the cars it is composed of.
• The nature of collective phenomena is that they are patterns composed of parts, and they in turn exert powerful influences on their parts, acting to keep them in line. Think hurricanes, life, intelligence.
• What is the dynamics of a collective phenomenon? How does a thought get generated and propagate inside a brain? Nobody really knows.
• During much of any country's existence, it does not act as a single, polarized, black-and-white entity, but rather as a much more complex entity composed of dozens and dozens of smaller entities - domains - all of which are pursuing their own goals and coexisting in one way or other with each other.
• A brain is much like this. A brain, with its billions of neurons, resembles a community made up of smaller communities, each in turn made up of smaller ones, and so on. The highest-level communities just below the level of the whole are what I like to call "subselves" or "inner voices". Not in the sense of schizophrenia, but rather, in the sense of competing facets that try to commandeer the whole, something like hijackers, although often benevolent hijackers. Perhaps it is more like a vote of the passengers on a plane where they want to go, after it is in the air!
• There is too much bottom-up momentum to put the lid on. You cannot keep the lid on an angry populace, like the Iranian people or the Salvadoran people or the Chilean people. No matter how tightly you press down the pressure from inside will in the end overwhelm you.
• And thus it is within the brain. Competing subselves cannot be held in check indefinitely. They cannot be clamped down, forbidden to act. For each "inner voice" is in actuality composed of millions of smaller parts, each of which is active, and under the proper circumstances, those small activities will someday all "point in the same direction", and at that moment the inner voice will crystallize, will undergo what is called a phase transition, will emerge from obscurity and proclaim itself an active member of the community of selves. And if it is powerful enough, it will try to exert pressure and to be recognized. It will attempt to seize power. It will not want to relinquish power, once it has it. That's what I mean by "commandeering the soul".
• "Selective interruptibility" is one of the most critical characteristics of a hierarchically organized system such as a brain, which has evolved to deal with a world in which there are events of various priorities that have to be dealt with sequentially. Choices must be made, so there must be a highest-level body whose purpose is to make choices rapidly and reliably, one that sorts out the priority of subselves and allows only the one deemed most important to take charge.
• An interesting problem arises when even this deciding agent must be preempted by some sort of extreme emergency situation that arises. The decider can be in the midst of trying to sort out some ordinary conflict of subselves when -
• Phase transitions take place in simple physical systems, schools of fish, individual brains, and in countries as well, when there are sufficiently strong and numerous interactions between the components of the system, and when those interactions add up in such a way as to make for large-scale correlations, or, put another way, long-distance effects despite the short-range nature of the direct interaction. When such long-distance effect occur, then a new kind of entity springs up, an entity on a higher level of organization than its constituents, and that entity obeys certain laws of its own.
• There are no clear boundary lines to be drawn between that subself, this subself, and any other subselves of DRH the person. All of them are fictions, because the only real thing is the sum total, the integrated person.
• When trying to get out of a warm bed on a cold morning, says William James:
  • We suddenly find that we have got up. A fortunate lapse of consciousness occurs; we forget both the warmth and the cold; we fall into some revery connected with the day's life, in the course of which the idea flashes across us, "Hollo! I must lie here no longer!" - an idea which at that lucky instant awakens no contradictory or paralyzing suggestions, and consequently produces immediately its appropriate motor effects. It was our acute consciousness of both the warmth and the cold during the period of struggle, which paralyzed our activity then and kept our idea of rising in the condition of wish and not of will. The moment these inhibitory ideas ceased, the original idea exerted its effects.
• It's more passive than active - or more accurately put, the action is on levels of yourself that are far lower - far more microscopic - than you have direct control over.