Media and Memory

Some Analogies

     In the page containing quotes, there was a quote from John Locke that read, "Let us then suppose the mind to be, as we say, white paper, void of all characters, without any ideas; ..." An empty sheet of paper is used here as an analogy to help us think about the following question....What is the medium that serves as a basis for storing and remembering events, or ideas?

     Compared to John Locke's day, we now have many different types of media. Paper, blackboard, photographic film, magnetic audio tape, CD's, random access memories (RAM), hard disks, optical disks, etc. are some of the media that we use to 'remember' or store information. These media differ from each other in many ways. Consider first audio tape and photographic film. Both of these storage media are designed to be sensitive to the physical energy that comes in contact with the media. (see the page What kind of memory is a photograph? for a simple discussion of how black and white film records an image). The physical energy interacts directly with the media (the magnetic tape or the film) and as a result of the physical laws that govern this interaction an acoustic or visual "image" is laid down. The media, the acoustic tape or film, has no ability to exert any control over this process of storing a "memory". The memory stored is solely a function of the physical energy and the sensitivity of the medium to the physical spectrum of energy.

     Contrast this with the white paper of John Locke or a blackboard. It is true that a physical process must be used to create marks on the paper or blackboard. But unless it is a work of art...a drawing...the marks are not the point of the memory. The marks 'stand for' or represent a memory ...and, it really doesn't matter if the marks are in black, yellow or green; thick or thing; angular or loopy; ... because the memory is not about the marks, but what the marks represent.

     But, let us return to the magnetic tape and film; memories that are layed down as a result of direct interaction with physical energy. Let us consider what is involved when magnetic tape is used to "remember" the acoustic events of say a live performance of music.

     How might we remember this line shown above. One way would be to place a stick at the leftmost point of the line and then trim the stick at the rightmost point of the line. The stick would then be 'about' the same length as the line. We could never guarantee that it would be exactly the same length. But our stick would now constitute a pretty good memory for the length of the line. Note the stick is not the line...just a media used to remember something about the line. It won't remember the color or thickness of the line. But if we have a crayon, we could use the crayon to create something that resembles aspects of our remembered line. We simply take a piece of paper, place our stick on the paper and trace along the length of the stick with our crayon. The crayon may not be the right color....or thickness....and we may have some tremor in our hands as we trace the line. Consequently, artifacts (aspects that weren't there in the original line) may be introduced in the process of "remembering."

     Now, with this very simple example we have, I hope, demonstrated several features that seem to be characteristic of this attempt to store information about physical events or energies with these type of media; namely,

• the physical event that is 'stored or remembered ' is distinct from what is stored in the media;
• what is stored is always partial information about the physical event;
• if the physical event is continuously valued, this value can only be approximated by the storage media;
• the 'retrieval' of the memory for use in recreating the event may (and usually does) introduce artifacts.

     In the case of acoustic tape, it stores the continuous physical energy that constitutes the sound, the magnetic media is typically not capable of 'remembering' all of the physical information. Rather it is designed to primarily record the physical energy in the acoustic range of the spectrum. And, it is limited with respect to the accuracy with which the energies are recorded. The information recorded is said to be analog information. The way we used the stick to record and reproduce the remembered stick illustrates what is meant by analog. Note, that if we asked how long in meters was the original line, we couldn't answer that question with this stick memory. The stick "is" the length. And, because it directly reflects the length, we were able to use it as a basis for drawing the "remembered" line.

     Magnetic tape used to record sound is similar to our stick. It remembers physical aspects of the original events and these physical aspects provide the basis for reproducing the original event. Of course you need to amplify the signal and drive some quite complicated devices called speakers which then reproduce the remembered sounds. And, even if you spend a great deal of money, artifacts will be introduced when the events are recreated.

     Is this a good analogy for the mind. Does the mind have a medium which physically interacts with physical energy to "remember" those physical events? The answer is, probably yes; but a highly qualified yes. For example,the ear drum does vibrate as a consequence of the physical energy that strikes the ear drum. But, it doesn't appear that we remember these vibrations. They are simply a first step and this analog information is quickly changed to digital information. And to the degree we can remember something as complex as a musical performance, that memory is more similar to the way sound is "remembered" on a CD, than to our magnetic audio tape. (Magnetic tape can, of course, be used as the medium to "remember" digital information, and in this case it would be similar to the CD format.) But exactly how does a "CD memory" differ from an "analog magnetic tape memory?"

     The compact disk (CD) was probably the earliest use of the digital encoding of information (in this case sound) that appeared on the mass market. Think for a moment about a CD of a live performance that you might have purchased.When you purchase such a CD you are purchasing a "memory" for some events that occurred at a particular place over a particular period of time. The events, 'sounds,' are changes in physical energy in the acoustic spectrum. A memory for these complex events must involve some medium which can be altered in a way that allows the events to be reproduced. Recall, that the physical energy that we recognize as sound is energy described by values that are said to be continuous. But the CD's "memory" is digital. Consequently, it is an approximation of the continuous information and the digital format is, in some sense, an arbitrary code or language within which to remember acoustic events. It is arbitrary in the sense that other codes could have been chosen. Indeed, codes are constantly changing in consumer electronics much to the bewilderment of the consumer.

     Let us look a bit more closely at the nature of the code used in CDs. The format for audio CDs is 16 bit/44.1 kHz, stereo and is known as the Red Book standard. The 44.1 kHz refers to the fact that each second of audio contains 44,100 samples. The 16 bit refers to the fact that this many bits are used to encode the information in each of these 44,100 samples.

     The figures to the right will help you to visualize this method of encoding acoustic energy. Each of these figures shows a sound recorded over a 6 second interval. (This is not a "CD-quality" encoding. The format is 8 bit/22 kHz; but this is sufficient for illustrative purposes.) In the top figure a tuning fork engineered to vibrate at 5623.3 hertz (middle C) is recorded. The second figure is a recording of an electronic tuning device that is playing the A below middle C. A is typically defined as 440 hertz. Finally, the bottom figure is a male baritone voice singing an A.

     At the top of each figure is a graph where the X co-ordinate is time measured in seconds. The dots represent the sampling of the acoustic energy over time.If these were pictures of the physical sounds, then rather than dots we would have a jumble of waveforms that represented the changing physical energy over time. But these waves have to be sampled; and the dots represent the result of the sampling.

     The graph below each figure is the result of computing a Fast Fourier Transform of the sound sample that corresponds to the sound spectra that the red line passes through. The X axis corresponds to hertz or the sound frequency. The Y axis corresponds to the amount of energy at that Hertz measured in db.

     The Fourier transformation is a way of decomposing the acoustic spectra into its frequency components. Human hearing does something quite analogous to this and is why we can hear differing pitches. (Note that our vision system does not decompose the visual spectrum of energy into its components.) Notice that the greatest energy appears at the frequency of C in the case of the tuning fork, and A in the case of the electronic tuner and the human voice. But the actual distribution of the energy over the spectrum and over time is what allows us to easily distinguish these differing sound sources.

     Technically, a great deal of information is lost when instead of remembering the actual physical signal we rather remember a digital code that stands for or represents the original physical information. But note we could just remember that an A was heard. In this case, we have lost all of the information that distinguishes the middle panel from the lower panel representing the human voice.

Now, you have heard music stored on CD's played. It sounds very much as it was intended to sound; namely like the musical events it encodes. And most of us would have difficulty telling the difference between the same music recorded as an analog signal on tape and that recorded on a CD.

     But, don't be fooled by this "functional equivalence"...that is, they both do a good job of "remembering" musical events. There is a fundamental difference. The 1's and 0's of the digital language of the CD are an "arbitrary" language that is used to "remember" the musical events. This language of 1's and 0's can also be used to encode color, images, text, or numbers, shape, or whatever. For example, I see no reason, in principle, why a musical CD couldn't be played back as a dynamically changing array of colors.

     Since this code is arbitrary, we must associate with the code a set of instructions that can serve to appropriately interpret the code...in this case, to interpret the code as musical events. Consequently, every CD player has a computer chip which contains the instructions that must be followed to interpret the 1's and 0's as musical information that must be mapped back into an analog representation that can be amplified and drive the speakers of your sound system.

To reiterate, the point of all this is that:

An arbitrary language used to represent X, requires a set of instructions on how to interpret expressions in the language and an agent capable of executing the instructions.

     An arbitrary language, the 1's and 0's of our example, is a language which bears no intrinsic relation to what is represented. In our example the instructions are written into the microprocessor which also serves as the "agent" that carries out those instruction. Hopefully, you can now see that if we assume that the mind possesses an "arbitrary" language of thought, then that commits us to believing that our memories are interpreted.

Arbitrary Languages, Interpreters, and "Compression"

     The animation stores the sequence of pictures by taking into account the fact that much of the information is the same from one picture to the next. This same information needn't be stored each time as long as we can instruct our interpreter to repeat the same information appropriately. Thus, the animation requires about half the space required to store all of the pictures. Some of this space is taken to store control parameters, that are passed to the interpreter. For example, the interpreter is provided information which determines how long each frame is displayed, how often the animation is repeated, and the like. The language or format for these memories varies and in the commercial world fortunes are made or lost depending on which standards (languages) are adopted. The animation pictures are stored in a format called GIF and it is one of the standard formats used for pictures displayed on the Web. The other is JPEG. MPEG is a format for encoding moving pictures. DVD and I believe the direct satellite providers use an MPEG format.

     But, let us return to the mind and consider whether the mind uses these kinds of principles for storing information. To consider this we will return to use of the 7x7 squares considered earlier. However, we will continue this on a new page since otherwise this page will take forever to load. Onward to Structure, Memory and Figures.