# Man-computer symbiosis

> An annotated edition of this paper on Fermat's Library, where scientists and readers explain its ideas, equations and history in the margins. Annotated edition: https://fermatslibrary.com/s/man-computer-symbiosis

- **Authors:** J. C. R. Licklider
- **Topic:** Computer Science
- **Original:** https://s3-us-west-1.amazonaws.com/fermatslibrary/uploads%2F1492376398965-sz4vjtnqn-188225edd9c6c12a1d7cd19367317484%2FLicklider+-+Man-Computer+Symbiosis.pdf
- **Pages:** 8
- **Annotations:** 7 comments in 7 threads
- **Featured:** Fermat's Library Journal Club
- **Annotated edition:** https://fermatslibrary.com/s/man-computer-symbiosis

## Annotations

### Annotation 1 (page 1)

Beside the passage: “Man-Computer Symbiosis*”

**Annotation** by Tymor Hamamsy (2017-04-28):

> J.C.R. Licklider’s article, “Man-Computer Symbiosis,” was the opening piece in the inaugural issue (March 1960) of the electrical engineering journal, IRE Transactions on Human Factors in Electronics. The journal ceased publications just 2 years later in 1962.

### Annotation 2 (page 1)

Beside the passage: “'iIi lie coupled together very tiglhtly, anid tlclt. the iesuilting partnershil) xvill tliink as ino hluimian brain has ever tlhotiglit and process data in a way not approached by the information-handling machinies w-e kniow todlay.”

**Annotation** by Tymor Hamamsy (2017-04-28):

> Joseph Carl Robnett Licklider (March 1915 – June 1990), was an American psychologist, and computer scientist and is one of the most important figures in the history of computer science and computing. He is one of the first people to foresee modern-style interactive computing and its application to all manner of activities; and he was an internet pioneer who had an early vision of a worldwide computer network long before it was built. His predictions and actions actually initiated the internet by funding research which led to much of it, including today's graphical user interphase, and the ARPANET, the direct predecessor to the Internet.
>
> Among other things, he has been called “computing’s Johnny Appleseed", for planting the seeds of computing in the digital age; Robert Taylor, founder of Xerox PARC's Computer Science Laboratory and Digital Equipment Corporation's Systems Research Center, noted that "most of the significant advances in computer technology—including the work that my group did at Xerox PARC—were simply extrapolations of Lick's vision. They were not really new visions of their own. So he was really the father of it all”.

### Annotation 3 (page 1)

Beside the passage: “1. INTRODUCTION A. .ipibiosis r v HE fig tree is pollinated only by the insect Bla.sto- /li(aJa gr'ossor'ItnR2. Tile larva of time insect lives in the ovary of the fig tr ee, and tlhere it gets its food. The tree andl time insect, ar e thus heavily interdle)epndent: the tree cannot. relrodluce xvitlhout the insect; …”

**Annotation** by Tymor Hamamsy (2017-05-08):

> We are close to achieving the complete man-computer symbiosis that Licklider envisions here- where humans process data in unthinkable ways with a tight coupling to machines. There are now many companies and initiatives ranging from Facebook's brain-computer interface, to Darpa projects to Elon Musk's Neuralink company that are all working towards this tight coupling. Neuralink in particular is working on a cloud based artificial intelligent extension of our brains, an augmentation of human intelligence that is closest to Licklider's vision. Although we can perform incredible computations far exceeding human processing with the power of smartphones and personal computers, we have not yet achieved his ultimate vision.

### Annotation 4 (page 1)

Beside the passage: “1. INTRODUCTION A. .ipibiosis r v HE fig tree is pollinated only by the insect Bla.sto- /li(aJa gr'ossor'ItnR2. Tile larva of time insect lives in the ovary of the fig tr ee, and tlhere it gets its food. The tree andl time insect, ar e thus heavily interdle)epndent: the tree cannot. relrodluce xvitlhout the insect; …”

**Annotation** by Tymor Hamamsy (2017-05-08):

> Licklider's paper not only presents a remarkably prescient prediction of the future, but also inspires many people, like Robert Taylor, to work towards achieving his vision. Licklider spent much of his career pushing towards man-computer symbiosis.
>
> In 1962, Licklider was asked to lead to ARPA departments: Behavioral Sciences and Command and Control. One of his main tasks with this position was to find a better use for computers other than numerical calculations. In order to accomplish this, Licklider sought out top computer research institutions and opened research contracts with them. Among these institutions were Stanford, UCLA, and Berkeley. This group, named the Intergalactic Computer Network by Licklider, were eventually part of the group that created ARPANET; the precursor to the internet.
>
> Licklider directed ARPA, a government agency, from 1962 to 1964. Using government funds, he supported inventors whose creativeness led to: personal computing, creating the first online community; the mouse, onscreen windows, hypertext, word processing; and the ARPAnet, the prequel to the Internet.

### Annotation 5 (page 2)

Beside the passage: “proving, problem-solving, chess-playing, and patternrecognizing programs (too iiany for complete reference4-15) capable of riv aling humiian intellectual performance in restricted areas; and Newell, Simon, and Shaw'sl6 'general prolblem solver" may r emove some of the restrictions. In slhort, it seems worthwhile to …”

**Annotation** by Tymor Hamamsy (2017-05-08):

> Licklider's predictions are a bit ambitious in terms of their time horizon (it is crazy to think that the technology could improve so exponentially from 1960 when he wrote this article to 1980) but they have for the most part come true. Problem solving and thinking of military significance is currently done by military strategists with access to incredible computing ability/power, and artificial intelligence is not yet the standard for decision making. It is reasonable to project that for military decisions that need to be made on a minute/second time scale, computers and artificial intelligence will no doubt outperform humans soon. The use of military AI will not doubt be a matter of immense debate. Organizations like Open AI are currently researching different ethical issues of AI in anticipation of this future.

### Annotation 6 (page 2)

Beside the passage: “proving, problem-solving, chess-playing, and patternrecognizing programs (too iiany for complete reference4-15) capable of riv aling humiian intellectual performance in restricted areas; and Newell, Simon, and Shaw'sl6 'general prolblem solver" may r emove some of the restrictions. In slhort, it seems worthwhile to …”

**Annotation** by Tymor Hamamsy (2017-05-08):

> Many believe that the years of man-computer symbiosis will be the most creative and exciting in the history of mankind as we make scientific and intellectual discoveries at an exponential pace. The rapid rise of computing power with Moore's law and our generation of bigger and bigger data sets has already impacted every scientific discipline; the man-computer symbiosis driven by artificial intelligence would lead to our exponential ability to analyze these data sets and make even more discoveries.
>
> It is interesting that Licklider puts a time table on our ability to use man-computer symbiosis. He says we will have "15 years to use it. The 15 may be 10 or 500". What occurs after those 10 to 500 years is presumably an artificial intelligence that outperforms even a man-computer symbiosis.

### Annotation 7 (page 3)

Beside the passage: “case-hiistor-y studies of the process of ineto, ol findc nothiing comparab le to a t.ime-an(ndinotion-study anail"Ysis of the mienital wNor-k of 'a person engagedl in a scientific ortechniical enterl)rise. In the sp)inMg and sumiimier, of 1957, therefore, I tried to) keep) track of wh-lat onle mlo(ler-atelv technical …”

**Annotation** by Tymor Hamamsy (2017-05-08):

> Licklider didn't know it then, but this unofficial experiment prepared the way for the invention of interactive computing.

## Full text

Extracted from the PDF. [Annotation N] marks where each annotation above sits.

#### Page 1

IRE TRANSACTIONS ON HUMiAN FACTORS IN ELECTRONICMr;h

Man-Computer [Annotation 1] Symbiosis*

J. C. R. LICKLIDERt

Summary-Man-computer symbiosis is an expected development in cooperative interaction between men and electronic computers. It will involve very close coupling between the human and the electronic members of the partnership. The main aims are 1) to let computers facilitate formulative thinking as they now facilitate the solution of formulated problems, and 2) to enable men and computers to cooperate in making decisions and controlling complex situations without inflexible dependence on predetermined programs. In the anticipated symbiotic partnership, men will set the goals, formulate the hypotheses, determine the criteria, and perform the evaluations. Computing machines will do the routinizable work that must be done to prepare the way for insights and decisions in technical and scientific thinking. Preliminary analyses indicate that the symbiotic partnership will perform intellectual operations much more effectively than man alone can perform them. Prerequisites for the achievement of the effective, cooperative association include developments in computer time sharing, in memory components, in memory organization, in programming languages, and in input and output equipment.

1. INTRODUCTION A. .ipibiosis r v HE fig tree is pollinated only by the insect Bla.sto- /li(aJa gr'ossor'ItnR2. Tile larva of time insect lives in the ovary of the fig tr ee, and tlhere it gets its food. The tree andl time insect, ar e thus heavily interdle)epndent: the tree cannot. relrodluce xvitlhout the insect; the insect cannot eat wsitlhout the tree; together, tlheyr consitutte not only a vialile but a productive and thriving p)artnershilp. This cooperative "living togethler in intimnate [Annotation 3] association, or even close uinioni, of txvo dissimiiilar organismnis" is called symliiosis.1 Man-computer symbiosis is a stibelass of manimaeline systemns." There ar e many milanimachl-inie systemiis. At present, hoxvever, there are no mnan-comliuter svimbioses. The piurposes of this paper are to present the concept and, hopefully, to foster the development of man-computer symbiosis [Annotation 4] a-Ilalyzin;g some probilem's of interaction between miien an(I computing machines., calling attention to ap)plicable prineililes of miian-machline eniginiecring, anci l)ointing otit a fexquiestions to which resetarch ansxvers are needledl. The hoiie is tlhat, in not too many years, humiiiian blrain.s and computing imiachiines

MaNinuscrlipt receivedl by the PGHFEk Jmnuary 13. 1960; revise(l manuscript received, Januiary 18. 1960. The backgrouind worik on which this paper is basdcl wv'as suipported largely by the Behavioral Sciences Di-isioni, Air Force Office of Scientific Researcl, 'Air Research and Development Command, thiotigh Contract No. AF-49(638)-355. t Bolt Ber-anek and Newiman Inc., Caml1br idge, ailss. 1 "Welister's New International Dietionary," 2n(l e(i., G. andl C. Mlerriam Co., Springfield, Mass., p. 2555; 1958.

'iIi lie [Annotation 2] coupled together very tiglhtly, anid tlclt. the iesuilting partnershil) xvill tliink as ino hluimian brain has ever tlhotiglit and process data in a way not approached by the information-handling machinies w-e kniow todlay.

P). Betwveen? "'Mechanicalt E.Lxted 11e(Ilan and' l "'A rtificial In telliyen ce' As a concelpt, mnaii-computer syml)iosis is diflerent inI an important way frioimi whaIt North2 has called "iechanicallyv extendeci imian." In the miiain-mlaelline sNsteims of the l)ast, the lhtumiian operator suI)l)lie(l the initiative, the (dlirection, the integration, an1 the criterion. Th1e imiechlaIiceal l)arts of thie syst'ems wvere mnere extellnsions. fir;st. of the lhumiiani armii. tlhen of the hluiima-n eye. These sy.stems certalinly did Inot con,sist. of "dissimilar organisms livinog together . There wa-as onfly, one kiindl of organlism-man 1la ndl . the r est was there onlyv to help

In onie sense of cotur.se, any mlaii-iiia(le system is illtended to lhell) mlan, to hielp) a imaan or iien outsidle the svstemi. If wN-e focus upon the htumi-ani operator (s) w\ithIinl the system, however, w -e see that, in some areas of tech-inology, a fanitastic clhainge lias taken l)lace clur ing, the last, few y 7ears. "Mlechanical extension" hias given way to replacement of miien, to automatio, an(l the miieni wh-llo remain are tlher e more to liel) tlhani to be helped. In sonce instances, particularly in large comp)uter-centeredl in formiiiatioIn andi contirol systems. thle hiumiian operators arle resl)onsiile mainly for functions thiat it mroved inifeasibile to automiiate. Such 11svstenis ('humanlv extenidedl mnachines', Noirtlh iimiglht call them) arc nIot symii'iotiC systems. Tlhe are "semi-aut,omatic' systems, systems th1at! stcarted otit, to he fully atutoiimatic hut, fe sllor t of thle gotal. MI\an-computer symbiosis is prol)al)ly not the ultimIate par.adigmin for complex technological systems. It, seeins entit-ely )ossible tlat, in dlue course, electronic oi' chemic;al "'miacllines' vill outdo thle lhunmai brain in m-iost of thlle fullntions we now consi(der exclusivelv wN-ithlini its> province. Even now, (eclernter's IBAI-704 pr'ogramil forl pr'oving theorems In plane geometry lp)roceedls at talout tihe same pace as Brooklyn hiiglh selool students, awld makes silmilar errors.3 There are, in fact, several thieoiell-e-

2J. D. Northi, 'The rational behlavior of mechanically extended man," Boulton Paul Aircraft Ltd.. Wolverhampton. Eng.; Septenmber, 1954. 2 H. Gelernter, "Realization of a. G(eomnetnr Theorem Pro-ing Machine." ITnesco, NS, ICIP, 1.6.6, Internatl. Conf. on Informnatioil Proves-s.ing, Paris.i, France: Jine. 1959.

4 lHarch.

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 2

Licklider: Man-Computer Symbiosis

proving, problem-solving, chess-playing, and patternrecognizing programs (too iiany for complete reference4-15) capable of riv aling humiian intellectual performance in restricted areas; and Newell, Simon, and Shaw'sl6 'general prolblem solver" may r emove some of the restrictions. In slhort, it seems worthwhile to avoid argument Nitlh (other) entlhusiasts for artificial intelligence by conceding [Annotation 5] dcominance in the distant future of cerebration to miiachlines alone. There ill nevertheless be a faiirly long interimii during wlhiel the miiain intellectual advances will be miiade by miien anl comlputers working togetlher in intimiiat*e association. A miiultidisciplinary st.udy groul), examiiining future research and development problems of the Air Force, estimated that it would be 1980 before dev-elopmiients in artificial intelligence miiake it )ossible for miiachines alone to do much thinking or problem solving of military significance. That would leave, say, fiv-e years to develop man-computer symbiosis and 1.5 years to use it. The 15 miiay be 10 or 500, but those years slhould be intellectually the most creative and exciting in the [Annotation 6] hiistory of mankind.

II. AIMiS OF i\IAN-COMPUTER SYMBIOSIS Present-day computers are designed primarily to solve preformulated problems or to process data according to predeterminedl procedures. The course of the computation may be conditional upon results obtained during the computation, but all the alternatives must be foreseen in advance. (If an unforeseen alternative arises,

4.A. Newell and J. C. Shaw, "Programming the logic theory machine," Piroc. WJCC, pp. 230-240; March, 1957. 5 P. C. Gilmore, "A Program for the Production of Proofs for Theorems Derivable Within the First Order Predicate Calcultus from Axioms," Unesco, NS, ICIP, 1.6.14, Internatl. Conf. on Information Processing, Paris, France; June, 1959. 6 B. G. Farley and W. A. Clark, "Simtulation of self-organizing systems by digital computer s," IRE TRANS. ON INFORMATION THEORY, vol. IT-4, pp. 76-84; September, 1954. 7R. M. Friedberg, "A learning machine: Part I," IBM J. Res. & Dev., vol. 2, pp. 2-13; January, 1958. O. G. Selfridge, "Pandemonium, a paradigm for learming," Proc. Symp. Mechanisation of Tho?ight Processes, Natl. Physical Lab., Teddington, Eng.; November, 1958. 9 W. W. Bledsoe and I. Browning, "Pattern Recognition and Reading by Machine," presented at the Eastern Joint Computer Conf., Boston, Mass., December, 1959. 10 C. E. Shannon, "Programming a computer for playing chess," Phil. Mag., vol. 41, pp. 256-75; March, 1950. 11 A. Newell, "The chess machine: an example of dealing with a complex task by adaptation," Proc. TVJCC, pp. 101-108; March, 1955.

12 A. Bernstein and M. deV. Roberts, "Computer versus chessplayer," Scientific American, vol. 198, pp. 96-98; June, 1958. 13 A. Newell, J. C. Shaw, and H. A. Simon, "Chess-playing programs and the problem of complexity," IBM J. Res. & Dev., vol. 2, pp. 320-335; October, 1958. 14H. Sherman, "A Quasi-Topological Method for Recognition of Line Patterns," Unesco, NS, ICIP, H.L.5, Internatl. Conf. on Information Processing, Paris, France; June, 1959. 15 G. P. Dinneen, "Programming pattern recognition," Proc. WVJCC, pp. 94-100; March, 1955. 16 A. Newell, H. A. Simon, and J. C. Shaw, "Report on a general problem-solving program," Unesco, NS, ICIP, 1.6.8, Internatl. Conf. on Information Processing, Paris, France; June, 1959.

the whole process comes to a halt and awaits the necessary extension of the program.) The requirement for preformulation or predeterimination is sometimes no great disadlvantage. It is often said that programming for a computing miiachine forces one to think clearly, that it disciplines the thouglht process. If the user can think hiis problem througlh in adv ance, symbiotic association Nwitlh a computing machine is not necessary. Howev-er, many problems that can be thlought tlhrough in adv-ance are very difficult to think tlhrough in advance. They would be easier to solve, and tlhey could be solve(l faster, through an intuitively guided trial-anderror procedure in wNlhich the computer cooperated, turning up flaws in the reasoning or revealing unexpected turns in the solution. Other problems simiiply cannot be formulated without computing-machine aid. Poincare anticipated the frustration of an important group of would-be computer users wlhen lhe said, "The question is not, 'Wlhat is the answer?' The question is, 'What is the question?''Y One of the main aims of man-computer symlbiosis is to bring the computing maclhine effectively into the formulative parts of technical problems. The other main aim is closely related. It is to bring comiputing iimachines effectively into processes of thinking that miiust go on in "real time," time that moves too fast to permit using computers in conventional ways. Imagine trying, for example, to direct a battle with the aid of a computer on such a schledule as this. YoU formulate your problemi today. Tomnorrow you spend witlh a programmer. Next week the computer devotes 5 minutes to assembling your program and 47 seconds to calculating the answer to your problem. You get a sheet of paper 20 feet long, full of numbers that, instead of providing a final solution, only suggest a tactic that should be explored by simiiulation. Obviously, the battle would be over before the second step in its planning was begun. To tlhink in interaction with a computer in the same way that you think witlh a colleague whose competence supplements your own will require much tighter coupling between mian and maclhine than is suggested by the example and than is possible today.

III. NEED FOR COMPUTER PARTICIPATION IN FORMULATIVE AND REAL-TIME THINKING

The preceding paragraphs tacitly made the assuml-ption that, if they could be introduced effectively into the thought process, the functions that can be performed by data-processing machines would improve or facilitate thinking and problem solving in an important way. That assumption may require justification.

A. A Prelimninary and Infornmal Timne-and-MlIotion Analysis of Technical Thinking

Despite the fact that there is a voluminous literature on thinking and problem solving, including intensive

5 1960

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 3

IRE TRANSACTIONS ON HUMilAN FACTORS IN ELECTRONIC,'S 1lrh

case-hiistor-y studies of the process of ineto, ol findc nothiing comparab le to a t.ime-an(ndinotion-study anail"Ysis of the mienital wNor-k of 'a person engagedl in a scientific ortechniical enterl)rise. In the sp)inMg and sumiimier, of 1957, therefore, I tried to) keep) track of wh-lat onle mlo(ler-atelv technical person actually do (lI( dring thec hourns hie regardle(d as dlevotedl to work. Althouigh I wvas. aware of the inadcc{uiacv of thle s~ainpling, I servedl as 1iiv ownN1 suihrect. It, 510011 became applarlclit that the maIc-in thinlg I did was-- to keel) records, andth p rojcect wvouldh hiave become anti inifinite regress if the keeplingo of records hiad been carried t lir-oughin the detail (nvisae(ld In th-e initiRal. plan. It, wa's nlot. 'Neverthlele"ss I obta-inled a p)icture of ny activities thiat. gave mie pauise. Perhaps mly spect i'um1 is niot. typical-I hope it is niot, b..ut I fearit, is. About, 85 per cenit of my 'thin-king' time was spen-t, 0'oettHioitito a p)osition to thintk, to matke a decision, to l earni somet-hing I nieedled to kniowN. Mutch more timeA wnNCit. inlto find(inlg orobtainling in1forma1,-tion than1 inIto (dioestingr it. Hours-, wNent inito the pilottli-no of graph-s, a-nd otlier lioirs inito ins,1tructingIf an11 assistan-t how to p)lot. Wh71en the grzaphis were finsishd, tie relations.~ were oh)viouis at onice, but. the plottinop hadto he (lone in ordler to mi-ake thiem so. At onie l)oint., it, wvasneicessary to compare s,ix exlperimnental (leterminatiomlis of a funcitioni relatinspeech-intelligibility to speeclh-to0-noise~ rattio. N o tw N'o expierimnenters lia'id usedi the same (leffinltion or eaur of speech-to-nioise ratio. Several houirs of c-alculatingY were requtired to get, the dhata inito conmparab)le formii. When they wNere InI comparable formii, it took only1 afew second(s to cletermiine wvha t I nieed(le to kniow. Throughout, the period I cxaimineol, in zshort, my i'thiniking'' tlime was (devote(l main-ly to activities that wvere essenitially clerical orimcchanlical: seuachingo calcuilatling, plottinig, transfornming, (leternini-ng thie logical orolvnaniici conseq~uences of a set of assuimptioils 01 liylpot-heses, preparing the waly fora (decision 01, ain ii-isiht. Moreover, myv choices of what to .attempt and xIi wht, not, t) a -ttempt wvere dleterni'ine( to an eiiubarma-inohioml reat extent by considlerations of clrclfea-sibilitv, not in1- tellectutal. calpability. The mtalin suggestioii comiveycol by the finings lust, described is that the operations that fill mos,-t of thie tlime allegedlyv(devotedl to technical thinikinig ar-e operatioins thiat, ca-n be lperformed miorec effectively by machines thlan by mieni. 'Severe lprobilems are posed by tIme f(act. thalit these olperat ions have to be performedl upon oliVerlse variables andii(i unfor,eseen amI-i continiually chianginig sequtencees. If thlose problemscani be solved [Annotation 7] in sucha wy asto reae a symbiotic relat-ion betw-eena matn anid a fast, information--retrieval and (ldata-processinig imacchinie, however, it, seems eviden-t, thtat thle cooperative initer-actioni would greatly improve tIme thiniking- I )1O((55

B. Comnparatice Capabilitiles of ilIeko and Computers It miay be applrol)riate to acknowledge, aIt. t his powint. that wve ar-e using the termi- 'computer' to cover a wvid.e clcass~ of ecalculating, dat a-processing, and iiiforum "t ioi.mstorge-nd-rtrivalmaclines. The Capabilitiesof miachiines~ in this class arc increcasingalmost tla-ilIv. It is therefore hazardlous to niake gener.al. statemenmts, about calal)lites f te cass 1~rhaps it. is, equally hiazardous; to ma-tke general statements about, tihe cap)abilities~ ot mieni. Neverthieless. certalin geniotypic differenices InI cLapability between mieni awll compliuters do standII ou.t. andt they have a b.earingr oni the niat.ure of possibleman comnputer symbiosis and the p otentialI valuec of' achlievino, it.

As has been sai(.l in various ways. mieii are. noisv, niarrow-b-and (devices, but theirl nier"Vous svstenis haveN VeryV Imiani parallel and simultaneouslyI active chuannlsc. 14lelative to macai, computing macWhlimes are very l'ast anid veryv accurate, I ut th1ey are cons,trainedI to performn oiily one or a fewN element arv operatoions~ at. a. time. 'Menl are1 flexih mc. calpable Of "programm.1ingII-o themse"lves cit ingetlv" on t le basi~s of newly received in formation. Compuitinog macills

Zfesingole-minidled, cntrained byv thell progamamimig. ' Je naturally s,pecak redi ln.lamiguagces orglaniztd aroundI unitary obj ects amnd colicremit acions, aim( emlyn 0t 0 elementaryv symbols. Coniputers niaturially' spicak nonredlundant. Ianouages. usuialk lv wit Ii onily twNo elementary symb ols land nio inher- (nt appr)1eciation eithier of uni,1ta-ry obi ects orof cohereint act iomns. TIo hw r'iooromislv correct. those chiamacterizat ions wvould huave to iincludce imany qualifiers. -Nevertheless, tIme piU.- ture of dhissimilairity andc tlimerefom-e pot-ential sup)plemaent-ation that they present is essentially val1id(. Comnputimig miachinies can (10 reatdily, well, and r.apidly nianv thminos., thiat are tJifficult om' impossible for miani, and incan can do readiil and well, thioughniot. rapidly, many tIingst' that ar-c (lifficuilt orimp)ossiblhe for, comphuters. Tha-t suggeststhmait a symbiotic coopierat ion, if successful ini inteogratinog thle positive charcacteristics of miieii anml compluters, would be of gr-eat. value. Time dJifferences in spieed andl In .lanlguage,"C of cours11e, pose difficulties, that muust. be ov-ercomle.

I V. S~-'EPARABLE FuN-1,CrIioNs, OF MEIN AND COMPU-TERS IN THEF A-NTICIPATED SYMBIOTIC Ass-OCIATmON .It seemls likely that tlIme contrilbutions of humiiani opemlators aia(l equipmenit. will1 blend together so comuphetehy in nmanv operations, that, it will be (lifficuilt. to sep.arate thieii n'eatly Imn analysis. Thlat womild be tHie case if, InI gathering (lata oiWhuich bo ase adecision, for exape both tIme mian and tIme eonihut.er canine upl with relevant lprece(lents fr-omi experience and if thie comlputer thieni suggestedl Ca couirse of actioni that, agreed withi time inians" initutiv e udN't(gimnent. (In theorem-proving programis, cojinputers find p)recedents in e'xperience, andl in thec SAGE

March

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 4

Licilider: lMan-Computer Symbiosis

System, thev suggest courses of action. The foregoing is Inot a far-fetchied examiple.) In otlher operations, however, thie contributions of miien and equipment Nill be to some extent separable. Men Nwill set the goals and supply the motivations, of coturse, at least in the early years. They wrill forml-ulate ylN)otheses. Thley will ask quiestions. They wrill think of mechanisms, procedures, and models. They will iemembet thiat stuchl-anid-suclh a person didl soiime possibly relevant work on a topic of interest back in 1947, or at any rate shortly after World W1'ar II, and they Nill have an icdea in what joturnals it mliglht have been publislhed. In genieral, they Nill make approximate and fallible, but leading, contributions, and they wrill define criteria and serve as evaluators, judging the contributions of the equipment and guiding the general line of tlhouglht. In addition, men will lhandle the very-low-probability situations wlhen such situations do actually arise. (In current man-machine systemiis, that is one of the human operator's most iinportant functions. The sum of the probabilities of very-low-probability alternatives is often imiuclh too large to ineglect.) Men will fill in the gaps, eitiher in the problem solution or in the computer program, when the computer lhas no mode or routine that is applicable in a particular circumstance. The information-processing equipment, for its part, will convert hypotlheses into testable models and then test the models against data (which the lhutman operator miiay designate roughly and identify as relevant when the computer presents them for hiis approval). The equipment will answer questions. It will simulate the miechanisms and models, carry out the procedures, and display the results to the operator. It w ill transform data, plot graphs ("cutting the cake" in whatever way the lhuman operator specifies, or in several alternative ways if the human operator is not sure what he wants). The equipment will interpolate, extrapolate, and transforimi. It will convert static equations or logical stateimients into dynamic models so the lhuman operator can examine their behavior. In general, it will carry out the routinizable, clerical operations that fill the intervals between decisions. In addition, the comlputer will serve as a statisticalinference, decision-tlheory, or gamue-theory machine to miiake elementary evaluations of suggested courses of actioIn wlhenever there is enouigh basis to support a formal statistical analysis. Finally, it will do as muclh diagnosis, pattern matching, and relevrance recognizing as it profitablv can, but it will accept a clearly secondary status in those areas.

y!. PREREQUISITES FOR REALIZATION OF AMAN-COMPUTER SYAIBIOSIS

The data-processing equipm-nent tacitlr postulated in the preceding section is not available. The colnputer pro-

gramns lhavre not been wsritten. Tlhere are in fact several lhurdles that stand betwteen the nonsymbiotic present and the anticipated symiibiotic future. Let tus examine somiie of tlheimi to see more clearly wh-liat is needled andl wlhat the clhances are of achieving it.

A. Speed llistinatch Between lien and Com)ipiters Any present-day large-scale comlputer is too fast anid too costly for real-timiie cooperative tlinkiing witlh one mian. Clearly, for the sake of efficiency andl economiy, the computer miiust divide its time amiioing miiany users. Timie-slharing systemiis are cuirrently under active develol)ment. Tlhere are even arrangements to keep users from "clobbering" anythiing but their own personal programs. It seemiis reasonable to enivision, for a time 10 or 15 yeairs lhencee, 'tlhinking center" that will incor)orate the functions of present-day libraries together with anticipated advances in inforimation storage and retrieval and the symbiotic functions suggested earlier in this paper. The picture readily enlarges itself into a network of sucl centers, connected to one anothier bv wide-band coImlmunication lines and to individual users by leased-wire servrices. In suchi a systemii, the speedi of the computers would be balanced, and the cost of the gigantic memiories and the sophisticated programs would be divided by the nulmlber of users.

B. Alemoary Har}dtare Requiremnents Wlhen we start to think of storing any appreciable fraction of a technical literature in computer memory, we run into billions of bits and, unless things change narkedly, billions of dollars. The first tlhing to face is that we shall not store all the techlnical and scientific papers in computer memory. We nay store the parts that can be summarized most succinctly-the quantitative parts and the reference citations-but not the whole. Books are among the most beautifully engineered, and human-engineered, components in existence, and they will continue to be functionally important within the context of man-computer symbiosis. (Hopefully, the computer will expedite the finding, delivering, and returning of books.) The second point is that a very important section of miiemory will be periimanent: part indelible mzemiiory and part published nmemiory. The computer will be able to write once into indelible miemory, and then read back indefinitely, but the computer will not be able to erase indelible memory. (It may also over-write, turning all the O's into l's, as though marking ovrer what was written earlier.) Published memory will be "read-only" memory. It Nill be introdutcedl into the coimiptuter already structured. The coml1puter wN ill be able to r-efer to it repeatedly, btut not to change it. These types of miiemory will becomiie mioime and miiore imiiportant as comiiputers grow larger. They can be made miore complact than core, tlhin-film, or

1960 7I

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 5

8 ~~~IRE TRANSACTION'S ON HL 'MAN FACTORS IN ELECTRONI(CS ii((i

ev~en tap)e memory, and( thiev wxill bie miuchi less expensive. Tue main eginieeringo problems -will concerni selection eircultrv. In s,;o fain a'. othtic 1sl)ects of memory r'equirm'nent, are concerniedl, XXv( iVIav (outiit upo0n the coniiitnino'( (levelopincnt. of or'dinau'y 'cientific and( hulsinlesS computing mai-- chines. There is oine prospewct thal-t miemiorv elemients NvI'II l)ecome as fast as process~ino' ilooic) elements. Tha-tt lJevclopiment wouldl have a revolutiona-ry effect. upo in the olesign of comnputers.

C . ifl[mo'y Oya'q-abo&itw Reqiuiremileuid.

Implicit in the idlea of inan-cominpteci symbiosis are thie requirements that information be iet rievable both by name and by pattern antI thiat. it. he accessible thlrough, procedlure imuch fasterthian serial searchl. At least, hliaf of t-Ime prob1lem of memnory or-ganiizationi appears to resi(le in the.s tor-age procedlure. 'Most of the remainder seemsl to be xvi'rapped upl in the problemi of pattern recognition with finte storage miechianismi ormiediumi. DetailedI discus.sion of thiese problems 'Is beyondI the present s.cope(. However, a l)rief outtline of onecpromising Idea.. "trie memory," may servie to indicate the greneral. na-iture of anticpted (levelopments,. Tiie memory011 is so calledI by it-s or-iginiator, Fredkin.17 bec-ause it is tlesigne(I to facilitate retr-ieval of informat.ion and lbecause thie b)ranching storage strutuetre, when developed, resembles,- a tr-ee. Most commiionmlen-ior-vs'- tentis store functions of arguments, at locations designated i)y time arguments. (TIn one sense, thiey (10 not store tlhe agument,s at. aill. TIn another and m-iore reali~sticsne they stoice all the possible argruments, in the framework Structure of tIme memory.) The tr-ie memory system, oni the othlerhand, stores both tIme funcitionis anid the airgtimnents, TIme argument is introduced into thie memory first4.. one character at a tlime, starting at, a standatrd iniitialreg-ist.er. Eachi argument register im s one cell foreachi character of the ensemblle (e.q., two foriniformiation enicoded in binary formii) and( aeah charaicter cell hias wNithini it. storage space for. thie aiddress of the next. reg'ister. Thel ai-rgumient is storedl by wr-1itling a series, of addresses-, each onie of which t,ells' wh-lere to find( thie next. At, th-e endI of thec ar-gumiient is a special 'end(-of-arguimient" marker. Thien follow, dIirectionis to the funcition, wichiil is stored in one or anotheri of s-,everal ways,, either' further tr'ic structure or- "list, structiture" often l)cing, miost. effective. Thie trie memory scheme is inefficient foi' smIall miemtories, but, it. becomes, increasingly efficient, in using available storage space as miemiory size Increases. Tile attr-active feat.ures of the scemiiie are thiese: 1) The retrieval process is ext,remely simple. Given t le argument. cuite r thie stanidard initia1 register wAithl the first character., and lpick upj thle addIress of thie second. Thien go to thle seconid register, an(l pick up the add(ress of the thiird, etc. 2.) If

1 '- Fr'edkimn, T m'ie. m oem no my%,'

twNo arguments~ have I n It Ialt c harancters InI eon11(IIno, fley uise t.he samie. storage spa,fce forthose characters. 3) T1he lengthis of the arguments ncedi ot be tIme 'saint1, and neced niot be. specified in. advance. 4 N'o r-oomini IIstorage is rese,~rve(l for, or' used. by any aroounintit tliitil it is actunallv stored. The triei structure is crea,ted as theo items aic, int-roduced inito the inemory. 5) A function czan he used as an alrgfumlent, foranlother function, 'and tImatt lunct iou aI5 an rguent for, the next. -Thus, for, cxaimph vetr ing. wNitli tilie a.-roninen t., 'imat trix immmmit mplicint ioi in' ne immigli1t ict rieve the enltire p-rogmraill foi pertoimo1110 "iii a nlt iilx inult ipdiclation Onl thle computer. (6) Byx cxaiim ing'11 Hthe storaaeo at agvnlevel, one (ain dleteirmine w hat flim~- fai' siniiiai' items1.- liave beeni stored. Fomuxamplh if there isnoctat'ion for- Eg-an. .J. P., it is. hut a step ortwom :ik- Ward,( to pick til the trail of Egan, James. TheI properties *i tist described (10 not Include all, Ih dcs,ii'ed IoticeS, but t 1ev 1briig comlpliter stor,age nit I) resomaiewtlIi Inunan-Il opcrat ois aiid 1 h1cii p ired ilect i t deigntet uns i nanung or poiiit ing.

I.). Thle Lau qpiae Problemi Th'le basic (lissimiiliarity between humanzi lanuptages, aiiul coinmutetr languages maty be tile most 5scrious ob)stacle to titme svyimihioss. It is retassur~ing,, however, to niote -what goreat st ride~s have al ready been imade, thI roughm iiterpretive prog-ramls aind p.am'ticuh'larl through assemnbly oi' comlpiling lirogram.,s 5ic(i as, FRTWrAN, to adaICpt computer-s to limimiaii 1,-language form ms. The -In formua,ttion Processinig LangumaQ'e of Sl1iaw, Newvell, Sirmon, (and Ellistm repies,ents atiiotlmerline of ma ppiroclhemment Amid, in ALG()L am nI i'clatedI systems, men ( ame provingu their flexibility by adopiting standard formnulas~ of r(1presenit'ation aim I txpr'ess,iomn lthat ai'e r'ea(lily tr.anslattable iiito machine langu-tage. For thie pumrposes of meald-time coopemration l.etweetn men and( compulters.~ it. wXill be necesiary. lhowevei', to iiiake tmse of ami additional1 and matlier ditfemrmit principle of comimmunication and( control. The idea iniv ibe highviligh-tedI by comparing' Inistriuctiomis omrdinarily adl-dressed to initelligrenit hm1anl lbeiiig's Xvith imistiructions om'(iniai'i lv tisedI wXitli coniptuter's. The lattfer specify preccisely tIme individuaxl steps to take and( the ~sejuenled inl X 1imici tdi t`ake thiem. The foriim'r pr'esent oiimuply somiet.ltiniig albout iii- (cenitive ormotivat.ion, and( tIhey supplv a cr-iter-ioii by wh'liclh tIme hitmman exectutor ot the inistrtuctions XX'ill know XX'hmenilit' liaxs axccoinplished liis task. In short instri'ietions directed to comiputmttr spemix'v courses istoIt'- tionis (lirected to hitnianl-i beings (,c f oals. i\[cn appeam' t.o thumiik momrei ritumi ally, antiesiyi termums of gToals th-an mintiu of coturses. Tirue, tliey usually know sometihing, aliout. directiomns in wh'lilh to travel or lines along wh'iieh to XX'ork, bumt fewv star't, ouit. wXit.Ii lirecisely formutlated itinera-ries. Whrlo, fom' example. wXould_l

j1; J. C. Sillaxx' A.. Newell. H. A. Simci,om lnmid T. 0. E'lUis. 'A c'omimand stri'etumre fom' complex mInformiatmion i!'csml. prol' 11700C. pp. 119-128; May, 1958.

8 March

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 6

Licklider: Man-Computer Symbiosis

depart from Boston for Los Angeles with a detailed specification of the route? Instead, to paraphrase Wiener, mi-en bound for Los Angeles try continually to decrease the amount by which they are not yet in the smog. Computer instruction through specification of goals is being approached along two paths. The first involves problemn-solving, hill-climbing, self-organizing programs. The second involves real-time concatenation of preprogrammed segments and closed subroutines which the human operator can designate and call into action simply bv name. Along the first of these paths, there has been promising exploratorv work. It is clear tlhat, working within the loose constraints of predetermined strategies, computers will in due course be able to devise and simplify their oNn procedures for achieving stated goals. Thus far, the achievements lhave not been substantively important; they have constituted only "demonstration in principle." Nevertheless, the implications are far-reaching. Although the second path is simpler and apparently capable of earlier realization, it has been relatively neglected. Fredkin's trie memory provides a promising paradigm. We may in due course see a serious effort to develop computer programs that can be connected together like the words and phrases of speech to do whatever computation or control is required at the moment. The consideration that holds back such an effort, apparently, is that the effort would produce nothing that, would be of great value in the context of existing computers. It would be unrewarding to develop the language before there are any computing machines capable of responding meaningfully to it.

E. Ilput and Output Equipmnent The departimient of data processing that seems least adv,anced, in so far as the requirements of man-computer symbiosis are concerned, is the one that deals with input and output equipment or, as it is seen from the human operator's point of view, displays and controls. Immediately after saying that, it is essential to muake qualifying comments, because the engineering of equipment for highspee(l introduction and extraction of information has been excellent, and because soimie very sophisticated display and control techlniqtues lhave been developed in such research laboratories as the Lincoln Laboratory. By and large, in generally available computers, however, there is almost no provision for any more effective, immediate man-machine coillmmunication than can be achieved with an electric typewriter. Displays seemito be in a somewhat better state than contirols. Mlany com-iputers plot graphs on oscilloscope screens, and a few take advantage of the remarkable capabilities, graphical and symbolic, of the charactron display tube. Nowhlere, to my knowledge, however, is there anything approachiing the flexibility and convenience of the pencil and doodle pad or the clhalk and blackboard used by men in technical discussion.

1) Desk-Surface Display and Control: Certainly, for effective man-computer interaction, it will be necessary for the man and the computer to draw graphs and pictures and to write notes and equations to each other on the same display surface. The man should be able to present a function to the computer, in a rough but rapid fashion, by drawing a graph. The computer should read the man's writing, perhaps on the condition that it be in clear block capitals, and it should immediately post, at the location of each lhand-drawn symbol, the corresponding chiaracter as interpreted and put into precise typeface. With such an input-output device, the operator would quickly learn to write or print in a manner legible to the machine. He could compose instructions and subroutines, set them into proper format, and check them over before introducing them finally into the computer's main memory. He could even define new symbols, as Gilmiiore and Savell19 have done at the Lincoln Laboratory, and present them directly to the computer. He could sketch out the format of a table roughly and let the computer shape it up with precision. He could correct the computer's data, instruct the machine via flow diagramns, and in general interact with it very much as he would with anothler engineer, except that the "other engineer" would be a precise draftsmiian, a liglhtning calculator, a mnemonic wizard, and many other valuable partners all in one.

2) Comiputer-Posted TWalt Display: In some technological systemus, several men slhare responsibility for controlling vehicles wlhose behaviors interact. Some information Imlust be presented simultaneously to all the men, preferably on a commiion grid, to coordinate their actions. Other infor-mation is of relevance only to one or two operators. There would be only a confusion of uninterpretable clutter if all the information were presented on one display to all of them. The information must be posted by a computer, since manual plotting is too slow to keep it up to date. The problem just outlined is even now a critical one, and it seems certain to become more and more critical as timiie goes by. Several designers are convinced that displays with the desired characteristics can be constructed with the aid of flashing lights and time-sharing viewing screens based on the light-valve principle. The large display should be supplemented, according to most of those who have thought about the problem, by individual display-control units. The latter would permit the operators to modifv the wall display without leaving their locations. For some purposes, it would be desirable for the operators to be able to communicate with the computer through the supplementary displays and perhaps even through the wall display. At least one scheme for providing such communication seems feasible.

19 J. T. Gilmore and R. E. Savell, "'The Lincoln Writer," Lincoln Laboratory, M.I.T., Lexington, Mass., Rept. 51-8; October, 1959.

9 1960

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 7

IRE 'l'RAA'SA'(.'TIONS ON HULilAN FACTORsS' IN ELEC'TROANJI(.11t

Thlie large xa11 dlisplay and its associated systemii are relevant, of course, to symbiotic cooperaition betweeni a conul)ter' anti a teamii of mien. Iahoratory experiments halv-e ind(icatedI rep)eatedly thatm informal, pai'allel at'- iraiigements of operators, coordinating tleilr activities thl-rotughl reference t.o a lig e situation display, hlave impoit ant adivantages over thie am'rangement, more wi(lely 1se(d. that. locates thie opei'atoi's at ind(isvidutal consoles and.1 attemipts to correlate tlheir actionis througlh the agelncy of a compiuter. This one of several operator-team prol)- leius in mueedl of careftil sttudv. 3 AlIto?(tti c Speech PIrodtulctioa and Recoginit:ion: How desirable anid how feasible is speeel-c commliiiuication between humiiiian operattons anuid computimig mnachines? That coin)ound qIuestion i,s aslked wv ienexer sophisticated data- P rocessing systems ame discusse. Engineers wlho work andl live wN-itlh computers take a conservative attitude toward tlie desiirahilit.y. Engineeis whlo ht.ave hia(l experieince in tile fie]ld of automatic s)eech iecognition take a consemvttive attittdle toward the feasibility. Yet thlere is conltinuliino interes, t in the idlea of talkin-g with computing niachine.s;. In laige <paint, the interest stenvis frolm iecalizatiOlo tlalit onte can-m hardly taket inilitarv comman(Jei' or a coitpo)ratioCn president awav fm-omni his wlomk to teacll him to type. If Co111pltinlg macl>teihmleS ar'e eve'r to b.e usedl di- 'ectliy by to)p-level decision maker's. it m1ay b)e woitli- Nxxliile t.o lprovide coiimumiicat.oioi vi.a thle m-no.st nat,ural nicans,, even at comisidemralde cost. Preliiminairv analvsi of his problems anud1 tiiime scales suggests thlat, a corpl)oration piresidlent wouldl be intferested in a svinhiofic associatioIn withl a comiputer only s anl avocation. Bsiiness situat ions usiual iv ove sleIVly eniouiglh thalt there is tim,e for h)riefings anidI conferen1ees. It seeniis iea sonable. tlerefore. for comptiter specialists to be the ones wxlho interact (lireetly withl computers in btmsiness offices. Time military comumander, oni the otheml hand, faces a, grealter probability of h-aving to mi-iake critical decisions in shmoitt iintervals of timine. It is easy to overdramatize thle not,ion of time ten-minute wamr, but it, would be dangemous to couint on having moire tbhia teIn minliutes in wvhich to nmake a critical (lecisiorn. As military system ground(c enironments and control centers grow in eapah)ility and complexity, ther'efore, a real reqcuiremeent for automatic speecl l)ro(ltdction andI r'ecognition in conmpuiters seems likely to (develop. Certainly', if thle equipmiienit were alm'eadtv developed, reliable, anii av'ailalble, it xxvould be used. In so far as feasibility is concerned, speeeh lpoduction poses less severe pmoblerns of a teclhnical nattuie than does automatic recogniition of sh)eecli souinds. A comiimercial electm'onic digital voltmeter noIw' ieads alotud its incications, dligit by dligit.. For eight or ten vears, at. the Bell Telephone Laboratomies. the Royal Institute of Technologv (Stockholm), thie Signals Research and Development Establishment (Chmristcimrch) f thie Haskins Laboratory,

aind the M\as-9sachusetts Institute of Tecelhnologv, Dtuinnl.2" Fant,21 Lawrence,-2 toii)e',": Stetx'ei)'/4 anti tlie il (C0- workers, hi.ave demonstiatted suecessive generation's of in.- telligible auitomiiatic t.alkers. Recent: work at. the liaskli,-ns Laboratory htns led to thle dlevelol)pment of a igitat 0o ic, suitable for uise by coniputing mnachines, that ia-lkes an tIUt(-)lleltitY0i(*(tItt.etl. illt(ljig'i_)je C )llllj,t'('(te (tji9(.(-jl '2t-- aut.omlatic voic tQ inelglecilctd lieii's C The. feasibility of automatatic speech reco"nit ion Icpendsi heavily upon the size of thle voc abul 11 y of wodsc to be recognized and upon tlhe dlivenlsity of talkers tindi accent.s withi which it, must wvork. Nin(tNe hiht per (c(ent Correct, recoonlitionl of naturally spoken du no ii dgiats4 Wtas denllonstrate(d sever,al ye(ars ago at tli(e Iie lreicphone Laboratories anald t the Ininc oln Laboratoiv To go ot a,tep up thlescale of i (v ciaulary Size, wec 11tay say tihiat an aL-tutoniatic i'ecognizer of Clelyks Z ok iI al tiiumcrlical characters cani almost urelhbe developed inowx oni thlie basis of existing knowxIleoe.t 1. `n11ee untilail-n11 operators cani r(e-ad at leat a4 s napidli iasz trained olCis can type, i suci 1i a Cvice would lie .l convenient too1)l illn :dnmos-t aiiy c(iliputer' instalklati0)m. Flor real-time inteCilactionl OIl a truly s5Vliio)it level, however, a voc abularv of about. 2000() wods, e.q. 1(000 Wor(m's of smllnetfliang like basic English ai(l( 1000 telinical t,erins. would prolbably he re uire(l. That com istit at Cs a cliallening p)ro)lecm. hli the con.,;sen-s.us of acoust ileans an1 linguists, construction of a recognizer of 2000 wvords (c.1tm11Ot lie a(.co ni)|i shCdl 1,loW. fowever, there aic several o rg anizations t1hatt x oull happily uidllCrttik ( to dvelop and alitomliatic rectolOnizer 1o Such a voealablary oni a five- -eam' basis. They w ould stipulate thatit. thle s peech he clear speech1 dicta. tioni style witho1lt. 1unusual accenl t. Altlioug-h detailed (lisellbsionl of tc('hni( lies of antomatic ;pIeeIh recognitmon m1j beyo td the lrescuit. scope it iS fitt ino to iiote 'liLt 1)1 o11niting ina('chinms a icling aL (d011ii1iitl.t role inl the dlevelop)melmt. of alto latic sipeccli rTcoO zr1 s. Th1ey have contr iibut ed tie iinpeijetus that

20I H. K. Duiiin, "Tlhe calculation of vowel reolulmces, ail(l nll electrical vocal tract," J . Acoast. Soc. A.m x ol. 22, pp). 740-7d53: November, 1950. 1G. Falnt, "On thle Acotustic of Spee i," paper repseslted at the Third Internati. Congress on A\ouostius, St.littgart, C'er.: Septembier, 1959. 22 XV. Lawr,ence, et (l., ;"Mlethods an(l Puirposes of' Seellh Synthlesis," Signals Res. and Dev. Estab., Ministry of Supply, Chlistchulrlichi, Hants England, Rept. 56/1457; MIarch, 1956. 23 F. S. Cooper, et al.. "Some experiments on the percep)tion of synthetic speech sounds," J. Acousf. Soc. A ier., vol. 24. pp. 597- 606: November, 1952. 24 K. N. Stevens, S. Kasowski, and CI. . Fant, "Electric analog of t.he vocal tract," J. Aco.st. Soc. .4Amner., vol. 25, pp. 734-742; July, 1953. A. M. Libenlnan. F. Ingel-anai, L. LiskerC, 1). Delatre1, andI F. S . Cooper, "MIinimal Irtules for sy\nthesizing speecl,' .J. Acoust. Soc. Amen.' vol. 31, pp. 1490-1499; N4ovember, 1959. K6 K. H. Davis, R. Biddulph, and S. Balashek, "Automatil m ecogn'ition of spoken digits," in XV. Jackson, "4Comnmnication Theoi'y," Butterworths Scientific Puhlictattions, London. Eng., pp. 433-441: 1953. 2 J. W. Forgie and C. D. Forgic, "Results obtained fi'omn a vowel iecognition coipluter proginam," J. Acoust. Soc. Amer.. -ol. 31, pp. 1480-1489; November, 1959.

W( ,.l .1arch'1

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

#### Page 8

Bush., et al.: Pattern Recognition and Display Characteristics

accounts for the present optimism, or rather for the optimism presently found in some quarters. Two or three years ago, it appeared that automatic recognition of sizeable vocabularies would not be achieved for ten or fifteen years; that it would havre to await much further, gradual accumulation of knowledge of acoustic, phonetic, linguistic, and psychological proceeses in speech communication. Now, howvever, many see a prospect of accelerating the acquisition of that knowledge with the aid of com-

puter processing of speech signals, and not a few workers have the feeling that sophisticated computer programs will be able to perform well as speech-pattern recognizers even witlhout the aid of muclh substantive knowledge of speech signals and processes. Putting those two considerations together brings the estimate of the time required to achieve practically significant speech recognition down to perlhaps five years, the five years just mentioned.

Pattern Recognition and Display Characteristics*

W. R. BUSHt, R. B. KELLY4, AND V. Ml. DONAHUEt

Summary-This paper reports experimental results of human operator performance in a visual recognition task. The work began with a method of generating families of complex patterns to simulate certain characteristics of visual sensor displays, such as radar and infrared returns. The experimental effort was directed toward establishing criteria for predicting human operator performance in a map-matching task. The operators' task was to recognize which of four patterns presented simultaneously with a reference pattern belonged to the reference pattern family. The measure of performance was the time in seconds taken by the operator to make a selection. Response times were more rapid when the reference pattern was less complex than the comparison than when the reference pattern was the more complex. Analysis of the display characteristics led to the selection of four physical measures to be used in predicting operator performance. These measures-pattern length, pattern density, and two measures of pattern complexity-correlated highly with response time, were not highly intercorrelated, and were applicable to natural sensor returns. The four measures were found to account for a high degree of the total variance. Regression equations were derived which predict performance from known values of the four measures.

INTRODUCTION

ECENT developments in high-speed, high-altitudce aircraft have imposed increased demands on the visual performance of the human operator. Typical operational tasks include the monitoring of complex navigational, target and predictive displays. A representative problem encountered in the study of visual

* Manuscript received by the PGHFE, December, 1959; revised manuscript received, January 13, 1960. This research was performed under the sponsorship of the Radio Corporation of America.

t Missile Electronics and Controls Div., Radio Corporation of America, Burlington, Mass. i Dunlap and Associates, Santa Monica, Calif.; formerly with Missile Electronics and Contr ols Div., Radio Corporation of America, Burlington, Mass.

performance under field conditions is that of "mapmatclhing," by which radar operators orient with the ground. This task involves the comparison of two complex pattern displays in order to determine whether a reference pattern is contained within a comparison. If performance is to be predicted and improved in this situation, research must be conducted to identify the relevant variables and any interactions wlhich may exist among them. Hake1 lhas discussed a number of the factors influencing pattern vision. Among these are the amount or density of information contained in the display, time requirements, contrast between coimponent parts, adaptation, and a number of operator conditions, including training and expectancies. Little past research relates directly to the field conditions discussed, either in terms of time restrictions or of the type and complexity of the stimulus materials. As Hake points out, results depencd to a considerable extent upon specific methodology. This fact would seem to justify a close correspondence between experimental conditions and that particular situation about which recommendations are to be made. Boynton and Bush,2 using various sorts of geometric forms, have determined that the probability of recognition of a critical target embedded within other forms decreases linearly with the logarithlm of the number of confusion forms. It is predicted from these data that performance will differ depending on the number of elements

1 H. W. Hake, "Contribtutions of Psychology to the Studsof Pattern Vision," Wright Air DeNr. Center, Dayton, Ohio, Teclh. Rept. 57-621; October, 1957. - R. M. Bovnton and W. R. Buish, "Recognition of forms against a complex backgrotund," J. Opt. Soc. Amer., vol. 46, pp. 758-763; September, 1956.

1960 11

Authorized licensed use limited to: Cornell University. Downloaded on September 6, 2009 at 23:16 from IEEE Xplore. Restrictions apply.

---

Source: [Fermat's Library](https://fermatslibrary.com/) — https://fermatslibrary.com/s/man-computer-symbiosis. More annotated papers: https://fermatslibrary.com/llms.txt
