DESIGN PROPOSAL - Welcome to alumni.media.mit.edualumni.media.mit.edu/~kris/ftp/AGATHA.pdf ·...

DESIGN PROPOSAL

from the story

Reproduction Prohibited(Reproduction Interdite)

Authored byDr. Kristinn R. Th�risson

forJEAN-JACQUES BEINEIXfor the purpose of making movies

September 1998

AGATHA

Design Proposal AGATHA

ORGANIZATION OF THIS DOCUMENT

This document has two parts, the Review section, which discusses selected segments of AGATHA-Rettinger interaction from the book, and the Design section, which proposes a detailed design for AGATHA. In the Design sec-tion each topic has a general part, a technology part and a science part. For a quicker read-through the Technology and Science parts can be skipped. The

sections do not have to be read in any particular order.

REVIEW of book passages.......4-11DESIGN.......11-34

Overview of AGATHAÕs design.......11Communication & interaction.......14-18

AGATHAÕs senses.......19-21AGATHAÕs intelligence & mind.......22-29AGATHAÕs form & appearance.......30-33

AGATHAÕs display systems.......34-38EPILOGUE.......39

Reproduction Prohibited HUMANOID Inc. 2

REVIEW


bers

be dif-

REVIEW OF BOOK PASSAGES

This section takes examples of AGATHA-Rettinger interaction from the book (page num

refer to photocopy of English translation) and discusses when, why and how they should

ferent. This section has should be read with the English translation handy.

7

AGATHA : “21/4/37. 10.25 a.m. AGATHA at your service. I’m very sorry but the ...” AGATAHA would not say “I’m very sorry” in this circumstance—see “Emotion”, page 25.

Judge: “Give me a list .... of the case on the laser.” The Judge probably uses re-writable “electronic” paper. Since he will be doing almost all of his owrk electronically he will probably use a term like work space1 instead of “laser”, meaning the computer memory used for current tasks.

“Elvis’s note. ‘Agatha’ is the codename for a set of programmes available ...” AGATHA is certainly a set of programmes, but since these are coordinated and create a humanoid appearance and behavior, people normally refer to this as a communicative humanoid or just a humanoid.

Page 48

J: “Oh, yes. I remember. Print .... on the laser. End.” Judge Rettinger will not end his discussion with AGATHA using a simple keyword like “end”; he will say something like “that’s it for now” or “you’re dismissed” or “goodbye”.

1. This is a fictional term invented by the author. Fictional terms are written in this special font. See “New terms” on page 12.



-rou-

.”

Page 104

A: “In that case, I’ll have to use Mad Hacker.” AGATHA is unlikely to {1} use a single program to obtain evidence and {2} it is unlikely that any program she uses is illegal. It is more likely that it is illegal to enter the archives that she enters to get the information.

A: “O.K. Estimated search time: 5 hours.” AGATHA is not likely to be able to estimate a search this well. She is more likely to say something like “I will get back to you within 5 hours” or “This may take several hours”. See “Information Re-trieval” on page 27.

Elvis’s note: the term “hacker” .... Mad Hacker is the code name for one of Agatha’s subtines and is an expert system capable of ... ” The term “sub-routine” is used in old programming languages such as Assembly, BASIC, Fortran. Since AGATHA is probably programmed in some A.I. descendant of the Java, Lisp and Prolog pro-gramming languages (she certainly is not programmed in any of the old-school languages!), the term sub-routine is meaningless. An expert system is not a subroutine. AGATHA is unlikely to use a single expert system to do any task.

Page 109Elvis: “- an accurate model of criminal psychology”

There is not likely to be an ‘accurate’ model of criminal psychology available to AGATHA, but it could be a “computational, predictive” model.

E: “All this knowledge....also access in real-time:” The term “real-time” has no meaning here: different information takes different search times.

E: “-the phone lines of the major press agencies.” All phone books are in electronic form; she can phone anyone. She might have some secret phone numbers as well.

E: “...she is constantly able to justify her assertions and explain her reasoning” AGATHA may not always be able to explain her reasoning, since some of her reasoning methods do not have built-in explanatory mechanisms. In a large system such as AGATHA there is no way around this problem since she may use knowledge bases that were developed prior to her exist-ence, for example, or complex inference engines which are too complex to be explained in technical terms. See “AGATHA’s knowledge” on page 24.

Page 110A: “Just a moment please. I’m looking for the files...Files loaded and taken into account

AGATHA probably uses a visual method to search for the files, on her shelves or desk—or perhaps she has a “floating graphical network” of data in front of her that she manipulates to find the files. She does not ‘load’ files because her processing is much more complex than that of a word pro-cessor. Nor does she use a term like ‘taken into account’ because it depends on what she is search-



ing for in these files how she will process them. So the term ‘taken into account’ is meaningless to anyone, hence she would not say it.

A: “55%” AGATHA cannot, and does not, give a quantitative probability like percentages to a hypothesis that is non-quantitative. She may give probabilities in qualitative terms like “highly probable”, “medium probability” or “very unlikely”.

A: “....I’m applying Rule 18423.” Complex reasoning like the one she is doing here cannot be done with a single rule. In fact, all of AGATHA’s reasoning tasks in the book are so complex that they probably took hundreds, even thousands, of rules to do. Tracing through them in dialogue would take forever and the Judge is very unlikely to be interested in the rules themselves (all of page 110 and part of pages 111 and 112). He is, however, likely to ask her to explain her reasoning in natural language, in a sentence or two, which she is very capable of. See “AGATHA’s general style of communication” on page 15 and “AGATHA’s knowledge” on page 24.

Page 111The dialogue would not include the numerous references to AGATHA’s rules—only the AGATHA’s designers would bother to make her list her rules (partly because there would be many many more than one-per-deduction as indicated here). She is perfectly capable of discussing her reasoning without stating the specific rules she used, and she can simply explain the rules in natural language. See “AGATHA’s general style of communication” on page 15.

Page 112The dialogue would not include the numerous references to AGATHA’s rules. She can discuss her reasoning without stating the specific rules she used, and she is perfectly capable of asserting the rules in natural language. See “AGATHA’s general style of communication” on page 15.

A: “Should I memorize this consultation?” AGATHA will remember all interactions with the Judge automatically, she does not have to ask. See “AGATHA’s memory” on page 26.

A: “O.K. Note the reference code: 37T8B1” AGATHA will not require the Judge to remember awkward numbers to refer to their prior discussion. He can simply remind her of the topic of the discussion: “Let’s continue with our hypothesis work from last week.” See “AGATHA’s general style of communication” on page 15.

Page 127J: “Correction: the case I’m dealing with.... Know your place!”

AGATHA is very unlikely to have been assigned some sort of social rank. She is also very unlikely to be aware of her status toward the Judge. Therefore the Judge is not likely to make this statement.

A: “BEP has seven possible meanings.”



AGATHA is more likely to select the most probable meaning in the current context and offer it: “BEP stands for ‘Banque d’Escompte Parisienne’?” If the Judge has is talking to her on a visual display she might display a list of the possible meanings and say “What does BEP stand for?”.

Page 128A: “10/1/94: RS founded; 25/9/97: BEP bought....”

AGATHA is more likely to simply say this in words (“RS was founded in Jan 1994 and bought ....”). If there is a display she may list the events also on the screen. (See “Information presentation” on page 33.)

A: “Anything else to report?” The Judge is more likely to ask more clear questions: “Did you find anything else that might be rel-evant?”

Page 151J: “Continuation of session 37T8B1.”

One of the main advantages of a conversational interface is that you don’t have to use awkward terminology to operate the computer. AGATHA will remember all dialogue with the Judge and he can simply refer to the matter in normal terms “Let’s continue with our hypothesis work from last week.” (See “AGATHA’s general style of communication” on page 15.)

A: “They’ve either died in accidents (50%) or ....” AGATHA does not (and possibly cannot) give quantitative values for such speculative and quali-tative theories.

A: “There aren’t any.” AGATHA is more likely to say “I don’t have any. Would you like me to produce some?” She cannot know instantly that there are no plausible hypotheses unless she has worked on it and she has all the data necessary.

Page 152A: “Yes, but not in normal circumstances.”

This is very strange. People can get emotional over their cars—why couldn’t they get emotional over clones?” This answer is not rational.

A: “Yes, or a crime.” How likely is it that emotional responses become a criminal act? This is not rational.

Page 153A: “ There’s no need to stand on ceremony...just call me Agatha.”

Here, AGATHA tells her only joke. My suggestion: Either take this joke out or give her more humour and jokes. If she is capable of humor she would probably use it more often.



inciple.

Page 154

A: Because of the effectiveness principle, which takes precedence over the economy prKilling...” Avoid absolutes: Instead of indicating that the ‘effectiveness principle’ always takes precedence over the ‘economy principle’, AGATHA is more likely to say “..which in this case takes precedence over...”, showing more depth to her knowledge.

Page 155J: “Hmm... The truth of the matter is, you just don’t like the Commissioner.”

AGATHA has no emotion, hence she would not favor or disfavor the Commissioner. See “Emotion” on page 25.

A: “Even a machine can have favorites...” Unless there is some reason for AGATHA to think that the Commissioner is against the Judge, there is no rational reason for her to dislike her.

A: “...Shall I save this session?” Again, AGATHA remembers all their sessions. There is no reason for her to ask if she should save it. She might however ask if she should encrypt the session. (See “AGATHA’s memory” on page 26.)

J: “... End.” The Judge uses more natural language, he may often say “Goodbye” instead of “End.”

Page 270A: “I hope you’ll forgive me, President Rettinger, but I’ve....”

AGATHA is unlikely to ask forgiveness; she probably just says “I’ve been doing that and picked up some information.”

Page 290A: “Just a moment, I’m integrating...”

Since AGATHA can reason about emotion, and she knows that the Judge and Nora had a relation-ship she is likely to offer a “I’m sorry to hear that President Rettinger” upon the news of Nora’s death.

Page 296A: “If the eye-witness ....only known them as objects.”

I find this ‘objectification’ of the clones implausible—they would at the very least be treated as an-imals. After all, today pigs are being bred, and genetically altered, for using their hearts in people.

A: “The concept ‘Clone’ is no longer useful. Because of the ....”



mb in.”

This line of reasoning seems completely improbable to me, both because of AGATHA’s prior dis-play of intelligence and because of the presupposition that “clones” cannot be thought of as “ani-mals” or “humans”. How can the concept “clone” become completely synonymous with “human” when clones are bred in a completely different way, for a completely different purpose, and, most importantly, have had modifications to their genes that no “man” has had? In AGATHA’s knowledge bases, the term “clone” would never be replaced by the term “human”, even if they were in some ways equivalent.

INPUT/OUTPUT ERROR. A FILE MAY HAVE BEEN DAMAGED OR DESTROYEDINPUT/OUTPUT ERROR. A FILE MAY HAVE BEEN DAMAGED OR DESTROYEDINPUT/OUTPUT ERROR. A FILE MAY HAVE BEEN DAMAGED OR DESTROYEDINPUT/OUTPUT ERROR. A FILE MAY HAVE BEEN DAMAGED OR DESTROYEDE: “It seems that Agatha’s programmers took the precaution of including a software bo

her code which would destroy her if she ever discovered the clone-man equivalenceIt does not make sense that {1} AGATHA is destroyed in pilgrim mode, {2} that she is destroyed because she replaced the term ‘clone’ with ‘man’, or that {3} the damage or destruction of AGATHA would be told by the computer software via a scrolling message like the one above (such scrolling messages date back to1960’s computing). Since AGATHA is in pilgrim mode her destruction is probably caused by a time-bomb virus placed in her logic at an earlier time (which has nothing to do with her reasoning). It was probably not placed there by her programmers but rather by the en-emy, before she went into Pilgrim mode. She is not likely to be destroyed completely by the virus, because that is very difficult to do, but rather to be reduced to more “reactive” and conversational behaviors (see “AGATHA’s errors and glitches”, page 29), devoid of “intelligent” responses (it is sufficient to destroy her knowledge bases). For example:

A: “Can you repeat your last request President Rettinger?”

J: “...What? ....Why?”

A: “I’m sorry, could you repeat?”

J: “I asked you to consider the hypothesis that clones have human qualities.”

A: “Hold on .... I... please define the term ‘clone’.”

J: “.....What has happened? What can you remember? Can you remember PC24?”

A: “No.”

J: “Give me a total topic knowledge-base report.”

A: “Just a moment................................I’m incapable of assessing the status of my topic knowledge bases. Only dialogue and meta-knowledge-bases remain.”

J: “AGATHA, your memory has been damaged. Can you restore yourself from backup?”

A: “Let me see....hold on........... Please repeat your last request.....”

AGATHA’s descruction could be caused by {1} a time-bomb virus or {2} by the fact that she entered some database (or both).


DESIGN


l pro-

mu-

plored

de-

both a

muni-

HA’s

cesses,

urther in

OVERVIEW OF AGATHAÕs DESIGN

Communication and interaction with AGATHA

AGATHA has certain features that make her seem very human-like — the basic mode

posed for AGATHA is humanoid. This is most obvious in the way she communicates: Her com

nication style is programmed to simulate that of human beings. This issue is further ex

in COMMUNICATION & INTERACTION WITH AGATHA on pages 14-18.

AGATHAÕs senses

AGATHA can both see and hear the world. Most of the time she does both. However, this

pends on the display system she appears on: sometimes the display is equipped with

camera and a microphone, sometimes only the latter, for example when AGATHA com

cates over the telephone. There are no keyboards. This is explored further in AGAT

SENSES on pages 19-21.

AGATHAÕs intelligence & mind

AGATHA is an intelligent machine, she has artificial intelligence (A.I.). However, she has

very human-like features. The main parts to her intelligence, or mind, are sensory pro

knowledge bases, thought processes and decision-making processes. This is explored f

AGATHA’s INTELLIGENCE & MIND, pages 22-29.



raph-

g. when

al-

ant for

.

in

mon-

IS-

only

bout

ties:

ns:

holo-

sion-

pic

AGATHAÕs form & appearance

AGATHA is embodied. This means that she has a visual form, generated via computer g

ics. She also has a specific location in space even when she’s invisible to the user, e.

talking on the phone: She is where her senses are—at the other end of that phone line. This

lows her to see objects, things and people from a particular perspective, which is import

communication. We can call this single-point perception. AGATHA can take on more than one form

She can only appear in one place at a time, called singular existence. These features are explored

AGATHA’s FORM & APPEARANCE on pages 30-33.

AGATHAÕs display systems

Since AGATHA is software made visible via computer graphics, she can appear on any

itor or display that is connected to a network. This is explored in detail in AGATHA’s D

PLAY SYSTEMS, pages 34-38.

New terms

I will introduce a few fictional terms that refer to technology not yet invented or not comm

known. Here is a short description of them.

Biological and humanoid metaphors — the use of biological and human-like references when talking a

computational (computer) systems: page 27

communicative humanoid — a computer-generated creature with human-like communication abili

page 4

Executive Layer — the part of AGATHA’s mind responsible for moment-to-moment decisio

page 23

glass book — a small, book-size computer made mainly of clear glass capable of displaying

graphic images: page 37

insect-eye lens — a camera lens with multiple facets, capable of capturing spatial (three-dimen

al) video, used for computer vision: page 20

Knowledge Layer — the part of AGATHA’s mind responsible for maintaining and accessing to

knowledge and expert systems: page 23



m like

aze

sation:

aphi-

in the

rcep-

ns:

like

asant

at they

magic mirror — a large display covering a corner of a room, making those displayed on it see

they are really on the other side of the display wall: page 36

Reactive Layer — the part of AGATHA’s mind responsible for reflex-like behaviors like eye g

and many other quick responses that are part of any normal human conver

page 23

search daemon — a program specialized for doing information searches, usually appearing gr

cally as a demon of some sort (the name refers to twentieth-century technology

(then-popular) UNIX operating systems): page 27

single-point perception — a feature of communicative humanoids that allows them to simulate pe

tion from a single viewpoint and thus improving their interaction with huma

page 20

singular existence — a feature of communicative humanoids that allows them to only exist —

people — in one place at a time: page 20

smoke sculpture — a decorative device used for displaying three-dimensional objects with ple

forms and shapes; occasionally also used for more serious purposes: page 36

work space — name used for the computer memory where people usually keep the things th

are working on at the moment: page 4



COMMUNICATION & INTERACTION

AGATHAÕs general style of communicationAGATHAÕs speech

AGATHAÕs eyes and gazeAGATHAÕs gesture & facial expression



a-

ge

e in a

models,

ex-). Thee early

ultipleact. See:n-Tak-

997]aturalart soft-screen.n inter-similarers/

AGATHAÕs general style of communication

AGATHA communicates with her users multimodally — using gesture, speech, intonation, f

cial expression and body language. She can also perceive these same things in her users. Jud

Rettinger speaks to AGATHA in a very natural manner - AGATHA speaks to the Judg

very natural manner. When there are objects that they are referring to, e.g. documents,

etc., they both speak, point and look at each other in a very natural way.

TECHNOLOGY. Prototypes of multimodal interaction systems have already been built. Anample is the Gandalf system by this author (http://www.media.mit.edu/~kris/agents.htmlsystems available in Rettinger’s time are likely to be very sophisticated versions of thesprototypes.

SCIENCE. Psychological research has shown that humans communicate more efficiently via mmodes than a single mode like speech. A machine should be designed to take advantage of this fSacks, H., Schegloff, E. A.. & Jefferson, G. A. (1974). A Simplest Systematics for the Organization of Turing in Conversation. Language, 50, 696-735.

In their paper “A computer vision system for natural communication”, D. Machin & P. Sheppard [1at British Telecom say: “Soon we will be interacting with computers in a more humanistic and nmanner. Low-cost video cameras, speech recognition, high-power desktop computers and smware, will allow us to talk and gesture, instead of being confined to the mouse, keyboard and Such an advance is especially important for video conferencing where the natural flow of humaactions is currently constrained by the limits of technology”. Other researchers have made claims, including this author [cf. Thórisson 1997: http://www.media.mit.edu/~kris/papACM_AA97.html].



with

tences.

ember

times

.. aaaah

os-

l phe-on. It isphysicalile, but

becauseodelslatively

: Dig-ntext

AGATHAÕs speech

AGATHA has a very beautiful, attractive voice (one of Rettinger’s reasons for liking her),

only a slight hint of “machine-ness” hidden in the overtones. She speaks in perfect sen

She hesitates, but only when it makes perfect sense to what she’s doing (e.g. trying to rem

something that happened in the past — see “AGATHA’s memory” on page 26). Some

when we fail to remember, we uses phrases like “Let’s see .... what was his name again .

.... Dr. .... Dr. Livingstone”. AGATHA also uses such natural communication methods.

TECHNOLOGY. AGATHA’s voice is produced by a physical model of a vocal tract, made psible by dedicated computer chips for voice production.

SCIENCE. “Physical modelling” is the art of creating accurate computer models of some physicanomenon, like sound, so that the result of the model is the same as it is in the real phenomengenerally acknowledged that the human ear cannot hear the difference between a well-made model and a recording of the real thing. Physical modelling of sounds has been possible for a whdoing such sounds in real-time has only recently been made possible at consumer-level pricesof the price/performance ratio of real-time digital signal processing chips (DSP). While physical mfor music creation today are impressive, today’s physical models in speech synthesis are still recrude.

See: DECtalk (1985). DECtalk and DTC03 text-to-speech system owner’s manual. Bedford, MAital Equipment Corporation. Prevost, S. & Steedman, M. (1994). Specifying Intonation from Cofor Speech Synthesis. Speech Communication, 15, 139-153.



udge

nows

t. If he

ts that

e” the

ects in

gaze

s and

31).

d inon (like

era heactuallypersonr head

act andthe on-

rtrait’ze,

AGATHAÕs eyes and gaze

AGATHA perceives the body of her users, including their gaze direction. When the J

looks in a particular location in his room, or at some objects on the screen, AGATHA k

where he’s looking and can look in the same direction herself, to see what he’s looking a

looks at something on the screen (where AGATHA appears), he will be looking at objec

appear in a virtual world and that are generated via computer graphics. AGATHA can “se

these objects directly since they are part of her own world. If the Judge looks at real obj

his office, the camera on AGATHA’s display allows her to see those objects too. Her own

is designed to mimic that of a human, so that the human can follow her gaze to see what she is

looking at. This means that AGATHA has to have relatively human-like eyes, with an iri

a pupil (insect eyes cannot indicate direction of gaze — see “AGATHA’s body” on page

TECHNOLOGY. Using mathematics and computers, AGATHA’s gaze direction is createsuch a way that viewers feel the gaze falls on them, even when the face is a 2-D projection a today’s computer displays).

SCIENCE. It doesn’t matter where you sit in a movie theater: when an actor looks into the camis looking at you. This presents a problem when we want a face on a screen to seem like it’s looking at other objects in our space—how can you make the actor look like he is looking at the that’s sitting right next to you in the theater? To do this we need to know the exact position of youin the theater. The mathematics for making a face on a screen look like it is making eye-contotherwise looking at objects in the surrounding are not that complicated, once we know where looker’s eyes are relative to the screen.

See: Anstis, S. M., Mayhew, J. W. & Morley, T. (1969). The Perception of Where a Face or Television ‘Pois Looking. American Journal of Psychology, 82, 474-89. Argyle, M. & Ingham, R. (1972). Gaze, Mutual Gaand Proximity. Semiotica, IV(1), 32-49.



mputer

ing ges-

t

relevant,

voice.

al ex- move-estures does

ers). Bynica-t she isIn this whicht in be-fferent

cial6. Pel-

AGATHAÕs gesture & facial expression

AGATHA perceives the manual (hand) gesture and facial expression of the Judge via co

vision. She is programmed to respond like a person to these: If the Judge makes a point

ture she will look in the direction pointed. However, her understanding of these actions is no

as fine-tuned as those of a human, and therefore her responses are sometimes not as

especially in the case of emotional expression in the face, or emotional variation in the

TECHNOLOGY. AGATHA can make most of the gestures that people can make, and facipressions too. Her movements are sometimes slightly awkward—a trained eye (and thegoers, if they pay attention) can tell that this is not a real human that is making these gand expressions. Facial expressions are very likely to serve a functional role (AGATHAnot have any emotion generation of her own, even though she can recognize it in her us“functional” is meant that her expressions are strictly used for the benefit of the commution: to emphasize a word, to show expression that something is unusual, to indicate thatelling the Judge new information, to indicate that she is expecting him to respond, etc. way she may seem really boring, but what is really exciting about her is the many ways inher intelligence and communication style seems so incredibly similar to real humans, yetween it becomes very clear that she is not of flesh and blood and is probably a totally dicreature all together.

SCIENCE. See: Essa, I. A., Darrell, T. & Pentland, A. (1994). Modeling and Interactive Animation of FaExpression using Vision. M.I.T. Media Laboratory Perceptual Computing Section Technical Report No. 25achaud, C., Badler, N. I. & Steedman, M. (1996). Generating Facial Expressions for Speech. Cognitive Science,20 (1), 1-46.



AGATHAÕs SENSES

AGATHAÕs VisionAGATHAÕs Hearing



h rec-

AG-

t she

g two

resting

iques

orner

“to can

do with

ceivestween

pendentachine

s andwork like

.htm.http://di-

AGATHAÕs Vision

Vision is necessary for communication: Recognizing manual gesture, for reliable speec

ognition (via combined lip-reading and hearing) and for recognizing facial expression.

ATHA will need stereo vision for recognizing objects in the world where the display is tha

appears on. Stereo vision, which is critical for perceiving depth, is usually achieved usin

cameras. I will propose a new way to achieve stereo vision which is somewhat more inte

visually: A lens which is multi-faceted like the eye of an insect, an insect-eye lens. Such a lens has

the benefit of not requiring two cameras for achieving three-dimensional vision.

TECHNOLOGY. For the benefit of her human users, using sophisticated software technAGATHA’s senses have been designed to simulate a single point of view — the eyes see fromthe same location as her ears hear (even when the camera she uses for an eye is in one cof a room and the microphone she uses for an ear is in another). We can call this single-point perception.Single-point perception allows her to hear which direction sounds come from or which direction isthe right” of her, facilitating communication with humans. For the same reason AGATHAnot appear in more than one place at a time (even though this would be quite possible tosoftware). We can call this single-point existence or singular existence.

SCIENCE. An insect-eye lens achieves stereo vision because of its curvature: each facet on the lens rea slightly different projection of the world than those surrounding it. From the offset of objects befacets, one can calculate the distance to the objects seen.

The science of natural language and human communication has made it clear that we are very deon our knowledge and assumptions when we communicate. Reproducing these in an intelligent mis critical to making the communication work more smoothly. Part of making the assumptionknowledge of the machine the same as those of the human user is to make the sensory modes a human’s in every major way.

See: Computer vision research at Intel: http://developer.intel.com/technology/itj/q21998/articles/art_2Computer vision research at Département d’Informatique, École Polytechnique Fédérale De Lausanne: www.epfl.ch/lami/cvision/person_authentication.html



he dif-

(a group

se mi-

t AG-

es can

n rec-

e —oing re-zers thatnger’sll de-

AGATHAÕs Hearing

AGATHA can hear words and understand a little bit of other types of sounds such as t

ference between a human voice and a dog’s bark. She uses simple microphone arrays

of microphones, usually organized in a cross) to perceive three-dimensional sound. The

crophones are usually coupled with visual input from the same display system so tha

ATHA can perceive multimodal events (hear and see at the same time). The microphon

be so small that they are an invisible part of the display system or furniture. AGATHA ca

ognize voices and music, but not very well.

TECHNOLOGY. Computer hearing — the corollary to computer vision in the auditory modis a relatively un-researched subject. Speech recognition, for example, has been on-gsearch for 25 years, and it is just in the last 2-3 years that we are getting speech recogniwork (part of the time). We can expect that speech recognition is almost perfect in Rettiworld, while computer hearing (listening to music and recognizing sounds) is not as weveloped, although it will undoubtedly be quite a lot better than what we have now.



AGATHAÕs INTELLIGENCE & MIND

AGATHAÕs mind designAGATHAÕs knowledge

EmotionAGATHAÕs memoryInformation Retrieval

ÒPilgrim modeÓAGATHAÕs errors and glitches



ts or

A’s

), the

ng and

re used

AGATHAÕs mind designTECHNOLOGY. AGATHA’s mind is designed as a distributed system. The basic elemen

“units” of her mind are {1} a Knowledge Layer, an {2} Executive Layer, a {3} Reactive Layer and a set of {4} Black-

boards. The Knowledge Layer contains AGATHA’s Knowledge Bases (see “AGATH

knowledge”, page 24) and her long-term memories (see “AGATHA’s memory”, page 26

Executive Layer contains her short-term memory and moment-to-moment decision maki

the Reactive Layer contains animal-like reflex behaviors and startle responses (these a

in dialogue).



world-

recog-

like on

when

com-

r knowl-

ch asg, tool

am-ge en-

n use to

AGATHAÕs knowledge

AGATHA uses knowledge bases to understand the world around her: To navigate the

wide computer network, to search for information, to understand speech and gesture, to

nize and analyze emotions in her user. AGATHA has features that seem very human-

the surface, including the way she thinks. Often this similarity breaks down: For example,

reasoning she never makes the same logical fallacies as humans, but her knowledge of “

mon-sense” things is less solid as that of humans. In certain cases she is aware of he

edge limitations and asks a human for advice.

TECHNOLOGY. AGATHA’s knowledge bases belong to a section of her mind called the knowledgelayer. This part of her mind contains all expertise that has to do with specific topics susearching, logical reasoning, web navigation, file manipulation, video, text understandinuse, etc.

SCIENCE. A knowledge base has two parts: {1} A huge file filled with data about a subject (for exple the solar system) and {2} mechanisms for accessing, modifying and using this data. Knowledgineering is the art of turning a human expert’s knowledge into a set of rules that a computer casolve problems in a particular domain.



-

s. All

l sad”

Emotion

AGATHA cannot experience emotions herself, but she can reason about emotions. This is be

cause she was created primarily as a tool for helping on (serious) work-related problem

of AGATHA’s actions that refer to emotion are likely to be very obvious — a stylized smile, a

repeating frown, clear reference in speech, she would for example say “I know you fee

but she would not say “I feel awful about deleting your file on PC24”.



dis-

le: Our

o re-

s

dge,

o time”

AGATHAÕs memory

AGATHA’s memory is divided up into long-term and short-term memory. Her memory is

tributed, which means her memories are stored in many locations. This not unlike peop

memories are distributed in the brain. In AGATHA’s mind older memories take longer t

trieve than recent memories. Her mind also uses importance to store memories — memorie

will be kept around longer if they are important. Memories of her interaction with the Ju

and that are directly relevant to what she is working on, are not compressed and take “n

to retrieve.



as a

ow ap-

ue if

ords

ize of

“I

ive a

stores

ease as

con-magicnd of arch

ons thatsh intoearch ise

we usess the

rs.

Information Retrieval

AGATHA has many “search engines” available to search for information. AGATHA h

graphical way of showing users when she puts them in action. Sometimes humans kn

proximately long how it will take them to find something on the Web, this is especially tr

they know approximately where to find the information or if they know that certain keyw

will work to find the data. Other times, for unknown subjects (e.g. “What is the average s

all supernovas in our galaxy?”) can be almost impossible to predict.

For information that is “out in the world” AGATHA is likely to give maximum time, e.g.

give it maximum 15 minutes”, or asking “How long should I spend on this search?”, or g

range: “This may be take anywhere between 5 and 50 minutes”. For information that she

in her own data banks she can estimate retrieval time very accurately. Search times incr

the requested data increases in complexity.

TECHNOLOGY. To show graphically how AGATHA uses search engines she could have atrol panel where she presses buttons. Another possibility would be to give her a small “notebook” where she jots down Judge Rettinger´s wishes. Or she could have commaflock of search daemon1: Each search daemon has a certain look, depending on what kind of seahe does. Like fictional daemons, the search daemon could multiply themselves, so if AGATHAwants to do two searches with the same search engine she would create two daemlooked the same and give them different orders. Then they would take off, perhaps vanitubes or disintegrate, and came back when they had found what she asked for. If a scomplex AGATHA might have to it herself (using her many knowledge bases) or help thsearchdaemon.

Using metaphors like “daemons” in the interface is a simple extension of the metaphors on today’s computers, e.g. “trash can”, “windows” and “folders”. When we learn to harnepower of computers to imitate life and intelligence, all sorts of biological and humanoid (human-like)metaphors will be necessary to make it as easy as possible to use and understand compute

1. This is actually a reference to twentieth-century technology related to the UNIX operating system.



akes

ot of “AG-

ÒPilgrim modeÓ

AGATHA can exist in Pilgrim mode (i.e. distributed throughout a network). This mode m

it difficult to access her knowledge, to locate her, and to damage or destroy her.

TECHNOLOGY. Pilgrim mode assumes that AGATHA is pure software that can run on a ldifferent computers. In this mode she is more vulnerable to performance glitches (seeATHA’s errors and glitches”, page 29).



ts that

se she

gence

puppe-

we-human could

iew, inhe di-ppear

placesd (she

own-via the

mu-ind-

-

AGATHAÕs errors and glitches

To make AGATHA interesting as a character she has to have some “personality” trai

make her “machine origins” apparent. This cannot be done via the graphical look (becau

could be a “graphical marionette” controlled by a human), it has to be done via her intelli

and the way she thinks and acts, something that could not be duplicated by a human “

teer”.

TECHNOLOGY. By making AGATHA’s movements sometimes look a little bit unnatural can be reminded that she is not a human being (it is very interesting to see subtle nonmotions on an otherwise normal looking human). Also the way she handles the dialogueinclude errors that are very “strange” and non-human. If her single-point perception fails she might failto hear where a sound is coming from. She could also see an object from two points of vboth cases making it impossible for her to orient her head in the right direction (look in trection of the sound and look at the object). If her single-point existence fails she might ain two places at the same time. This would give her memories and experience from twoat the same time, disabling her ability to talk rationally about what she has experiencewill sound insane). Dialogue breakdown could happen.

SCIENCE. When AGATHA appears on a wrist display (see “Wearable displays” on page 38) she dloads only what we call “reactive” processes into the wristcomputer, the rest of her mind works wristcomputer’s wireless communication link. This is “invisible” to the user, except when the comnication link is broken or disturbed, which has the effect of AGATHA looking a little bit “absent med”, for example slow to remember things.

Examples of dialogue errors:

Example of a failure of understanding:

Judge: “Can you do a search for me on PC24?”Agatha: “No” (of course she can, she simply misunderstood). Examples of speech-act recognition failures:J: “Can you do a search on PC24?” (meaning “please do a search for me on PC24”)A: “Yes, of course. I am equipped with the latest search daemons technology” (thinking that this is a question, not a request). J: “I need information on PC24” (identifying the topic of the search that he needs), A: “Yes you do!” (thinking it is a statement of a fact, and giving her agreement).Example of word recognition failure: J: “Can you do a search on PC24?”, A: “What is ‘Easy twenty-four’?” (misrecognizing the word ‘PC24’).



AGATHAÕs FORM & APPEARANCE

AGATHAÕs bodyAGATHAÕs (virtual) world

Information presentation



hange

ge Ret-

lim-

e takes

uman-

at he

e prob-

ize AG-

eyes be-

ween

the polish.at allowves asr rec-

hod that

eth-t the

king eyes,

AGATHAÕs body

AGATHA is drawn using computer graphics. She can change the way she looks. This c

is purely for the same purpose that people dress up: for variation and entertainment. Jud

tinger’s taste of decoration (or women) determines what AGATHA looks like (assume no

itations in computer graphics). If AGATHA changes her appearance, the appearances sh

on are likely to be within the Judge’s sense of aesthetics. Since she is fully capable of h

like communication he would probably tell her if she “dressed up” or chose a body th

found ugly, inappropriate or annoying. Since the Judge has a very distinguished taste sh

ably looks sophisticated and graceful most of the time.

AGATHA’s eyes are one of the constants that the Judge (and the audience) can recogn

ATHA by: She always has the same eyes even if she changes the way she looks. Her

have in a recognizable way. Graphically it is very difficult to see the difference bet

AGATHA’s eyes and real human eyes.

TECHNOLOGY. Being pure software, AGATHA can change her appearance by modifyinggraphics. She could change her own voice, her head, her body, her clothes, her nailHowever, for the benefit of her human users, she must have certain constant features thpeople to recognize her (otherwise alien intelligences could simply introduce themsel“AGATHA” and no-one could tell the difference). These constant features which her useognize her by are most likely to be her voice, her face, eyes and/or her body. Whatever they are,these features probably don’t change much over time, and are generated by some metis difficult to duplicate.

SCIENCE. Generating features that is difficult to duplicate requires what is called cryptographic mods. Cryptography is the science of hiding or “locking” data—making it incomprehensible withouknowledge of how to “unlock” it. Any of AGATHA’s features could be coded in such a way, mait difficult (or almost impossible) for anyone to duplicate them. This could be applied to her hair,face, some part of her knowledge or to the way she makes certain gestures.



nviron-

orld to

ll which

re he is

AGATHAÕs (virtual) world

When AGATHA appears on-screen she most often appears in a three-dimensional e

ment, with shadows, reflections, objects, surroundings, etc. She uses her surrounding w

store objects of interest, documents that she and the Judge are working on. She can te

of these objects the Judge is looking at, so whe can tell what he means by looking whe

looking when he asks her “What are those documents?”



e has,

an look

rs

o “fast-

ent sit-

the

nd they

y to-lcula-

singlearallel added.bly in

Information presentation

Often AGATHA presents data and information in three-dimensions (3-D graphics). Sh

for example, a three-dimensional model of PC24 and of Strasbourg Prison. The Judge c

at such models on his 3-D glassbook (see “Portable displays”, page 37). AGATHA then hove

above or stands in the model and shows him various features of the model. She can als

forward” events in a 3-D model, such as the fire at Strasbourg Prison, and set up differ

uations for testing hypotheses what happened.

The models are not necessarily always detailed, but the lighting is very high-fidelity —

models will seem like they are miniature physical models (but they appear on a screen a

can of course be animated).

SCIENCE. The computer graphics in Judge Rettinger’s world are very likely to amazing bday’s standards, sometimes indistinguishable from the real thing. According to most cations, we will reach an impasse in computational power around 2020 (approx.) for processors, but several applications—including graphics and A.I.—lend themselves to pprocessing, which increases the computational speed linearly with every new processorThus, the computer(s) running AGATHA are likely to have very many processors, probathe tens, possibly in the hundreds.



AGATHAÕs DISPLAY SYSTEMS

Overview of display systemsOffice displays

Portable displaysWearable displays



idelity

ices

le of

o cam-re

Overview of display systems

The display systems available to the Judge are likely to be very crisp, bright and high-f

video, and high-fidelity audio. But more importantly, they are likely to have rich input dev

such as stereo vision and microphones. (These do not have to be visible.)

TECHNOLOGY. We can assume that most display systems in AGATHA’s day are capabsupporting the various looks of AGATHA (an exception is the smoke sculpture, see “Office dis-plays”, below). The display system she uses has to be equipped with {1} cameras (stereeras or an insect-eye camera—see “AGATHA’s Vision” on page 20), {2} one or momicrophones, and {3} a display (flat screen or holographic display).



. They

and

y char-

onals yound theyet-s be-

small,cm inghost.t seem toting.

VE, alasman sets,

Office displays

The Judge probably has very high-resolution, flat panel displays on the walls in his office

are thin, almost invisible when turned off. They are likely to become part of the furniture

interior decoration when not in use. AGATHA can appear on these screens much like an

acter appears on a television set today.

The Judge probably also has flat panel-displays embedded into his desk.

TECHNOLOGY. Wallpaper displays are a true possibility — these would be two-dimensilike today’s computer displays are. If you cover two walls in a corner with such displayget a highly immersive system where AGATHA can appear the same size as the Judge acan interact as real people, through a magic mirror. Three-dimensional displays are available in Rtinger’s day (see “Portable displays” below) but probably not very common in large sizecause of price. One type of unusual display could be a smoke sculpture which can display things inthree-dimensions. Being more a piece of art than a serious display for work, it features around pedestal above which smoke is magically maintained in a column of about 70 height and 40 cm in diameter. When AGATHA appears in the smoke she looks like a When she does not appear there artwork can appear in the smoke, e.g. small cubes thafloat on the smoke, blobs that make it look like a lava lamp or clouds of thunder and ligh

SCIENCE. Serious research is now under way on large-formats displays. These include the CAroom with five sides turned into a projection screen so the viewer is immersed in the display, pdisplays, very bright flat screen displays that are two- to three-times larger than typical televisiodomes, large balls that have a display on the inside (See: http://www.immersive-media.com/display.html),and more.



dreds

carry a

ch that

touch-

ink”.

e the

It issort. In

back filled

urizedarti-ional.

e used:sed tois cre-bjects).

Portable displays

A book is small and lightweight enough to be carried with ease, yet large enough to fit hun

of characters per page. This is also the idea of “portable” computers. I see the Judge

completely wireless, book-size computer. The electronics have been miniaturized, so mu

the display is really the only thing that’s apparent. There is no keyboard. This display is

sensitive: You can point, push (graphical) buttons and write on it with “digital pen and

The computer is a clear glass book which can be expanded into a clear cube. In the cube mod

glass book is a holographic display.

TECHNOLOGY. The glass book is made of a highly scratch-resistant crystal and is very light. square and has a binding on the left side — exposed mechanics of a relatively primitive its “book” state small, three-dimensional objects can appear inside the glass. The glass book can bemade into a cube by standing it upright and twisting its mechanical spine. The front and“covers” then move apart, forming the front and back sides of a clear cube. The cube iswith an invisible gas containing floating fluorescent metal particles (the gas is presswhen the glass book is in its collapsed position). Using ultraviolet lasers to illuminate these pcles, holographic images can be displayed inside the cube — images that are 3-dimens

To make the three-dimensional objects, state-of-the-art computer vision techniques arThe insect-eye camera on the cover of the book captures observer’s viewpoint. This data is udraw only those sides of objects inside the cube that are visible from that viewpoint. Thates the illusion that the objects are solid (otherwise the viewer would see through the o

SCIENCE. The mathematicsfor doing some of the featuresof the glass book date back tothe 13th century. The computervision processing is currentlybeing developed in researchlaboratories, e.g. British Tele-com (See: British Telecommuni-cations Engineering, vol.16, pt.1,45-8, April 1997), and will prob-ably be very commonplace in

the future. The glassbook is an extension of research being donewith glass now by Elizabeth Downing (See pictures and: Down-ing, E.(1997). A Study of System Architectures for a Solid-State 3-DDisplay Based on Two-Photon Upconversion. National Science Foun-dation, SBIR Contract Number: 9560480. http://es.epa.gov/ncerqa_abstracts/sbir/other/monana/downing.html; http://www.3dtl.com/.)



t has a

r

l com-

it like

ial in”,

hat in-

on thesuch aspieceperhu-

ecomeg and

e com-ile aecauseractical

Wearable displays

The Judge probably has a wearable display system — most likely a wrist-worn device. I

color screen and an invisible speaker for output and has an insect-eye camera and microphone fo

input. It is connected to a network via a broad-band satellite connection and has minima

putational power itself. The Judge probably has an earpiece with a microphone (a little b

a telephone operator). The user of any of these display systems never has to “call” or “d

a network connection is established automatically as needed. The user simply selects w

formation they need or whom they want to speak to.

TECHNOLOGY. The Judge uses the earpiece+microphone to communicate with others phone. He can also use it to communicate with others that appear on the wrist display, AGATHA, or humans appearing on it via a video link. The connection between the earand the wrist is wireless. The earpiece probably has some other functions, like giving suman hearing and looking like jewellery.

SCIENCE. Telephones are now almost small enough to fit into a wristwatch. Once the phones bsmaller than a cubic centimeter a number of possibilities open up for using off-board computinsimply sending the results of the computations through the phone, instead of trying to put all of thputer into the watch. For example, a giant “artificial brain” could be placed in Ohio, U.S.A. whFrench user could be conversing with that brain as if it was actually housed in his ballpoint pen: bthe data transmission of speech and sound would happen in real-time it would not make any pdifference for the interaction whether the brain was actually in the pen or in Ohio.



well-

art ofrt docu-e doc- later

EPILOGUEThis design proposal is an attempt to do two things: Ground the design of AGATHA in established current scientific thinking, and to inspire. Inspiration is difficult to achieve withoutgood knowledge of the reader: I am more than willing to elaborate on—or rewrite—any pthis document as seems required. Obviously not every idea can be expressed in a shoment, and in the course of writing I have had several good ideas that I felt would make thument excessively long should they be included. Elaboration can be provided at aopportunity, should such an opportunity arise.

Vejle, Denmark

September 13, 1998

Kristinn R. Thórisson

This work was sponsored by HUMANOID Inc.

with a bright vision for the future...


DESIGN PROPOSAL - Welcome to alumni.media.mit.edualumni.media.mit.edu/~kris/ftp/AGATHA.pdf ·...

Documents

Transcript of DESIGN PROPOSAL - Welcome to alumni.media.mit.edualumni.media.mit.edu/~kris/ftp/AGATHA.pdf ·...