These excerpts were chosen for their relevance to the issue of ethics and technology, and their specific relationship to the two other pieces of material pointed at for this lecture: the professional society code of ethics and the Bill Joy article and reactions to it. (Each numbered paragraph is included either in its entirety or not at all. No particular effort has been made to present samples of the author's complete argument, though, which is broader than what we will discuss in class.)
You should not take either selection of a paragraph or the omission of a paragraph to indicate anything about what my own views might be. I have made a serious effort to attempt to remove any personal biases from this selection process; this material, as well as the other posted material, are topics for discussion, not alleged facts.
1. The Industrial Revolution and its consequences have been a disaster for the human race. They have greatly increased the life-expectancy of those of us who live in "advanced" countries, but they have destabilized society, have made life unfulfilling, have subjected human beings to indignities, have led to widespread psychological suffering (in the Third World to physical suffering as well) and have inflicted severe damage on the natural world. The continued development of technology will worsen the situation. It will certainly subject human beings to greater indignities and inflict greater damage on the natural world, it will probably lead to greater social disruption and psychological suffering, and it may lead to increased physical suffering even in "advanced" countries.
2. The industrial-technological system may survive or it may break down. If it survives, it MAY eventually achieve a low level of physical and psychological suffering, but only after passing through a long and very painful period of adjustment and only at the cost of permanently reducing human beings and many other living organisms to engineered products and mere cogs in the social machine. Furthermore, if the system survives, the consequences will be inevitable: There is no way of reforming or modifying the system so as to prevent it from depriving people of dignity and autonomy.
29. Here is an illustration of the way in which the oversocialized leftist shows his real attachment to the conventional attitudes of our society while pretending to be in rebellion against it. Many leftists push for affirmative action, for moving black people into high-prestige jobs, for improved education in black schools and more money for such schools; the way of life of the black "underclass" they regard as a social disgrace. They want to integrate the black man into the system, make him a business executive, a lawyer, a scientist just like upper-middle-class white people. The leftists will reply that the last thing they want is to make the black man into a copy of the white man; instead, they want to preserve African American culture. But in what does this preservation of African American culture consist? It can hardly consist in anything more than eating black-style food, listening to black-style music, wearing black-style clothing and going to a black-style church or mosque. In other words, it can express itself only in superficial matters. In all ESSENTIAL respects more leftists of the oversocialized type want to make the black man conform to white, middle-class ideals. They want to make him study technical subjects, become an executive or a scientist, spend his life climbing the status ladder to prove that black people are as good as white. They want to make black fathers "responsible." they want black gangs to become nonviolent, etc. But these are exactly the values of the industrial-technological system. The system couldn't care less what kind of music a man listens to, what kind of clothes he wears or what religion he believes in as long as he studies in school, holds a respectable job, climbs the status ladder, is a "responsible" parent, is nonviolent and so forth. In effect, however much he may deny it, the oversocialized leftist wants to integrate the black man into the system and make him adopt its values.
45. Any of the foregoing symptoms can occur in any society, but in modern industrial society they are present on a massive scale. We aren't the first to mention that the world today seems to be going crazy. This sort of thing is not normal for human societies. There is good reason to believe that primitive man suffered from less stress and frustration and was better satisfied with his way of life than modern man is. It is true that not all was sweetness and light in primitive societies. Abuse of women and common among the Australian aborigines, transexuality was fairly common among some of the American Indian tribes. But is does appear that GENERALLY SPEAKING the kinds of problems that we have listed in the preceding paragraph were far less common among primitive peoples than they are in modern society.
47. Among the abnormal conditions present in modern industrial society are excessive density of population, isolation of man from nature, excessive rapidity of social change and the break-down of natural small-scale communities such as the extended family, the village or the tribe.
48. It is well known that crowding increases stress and aggression. The degree of crowding that exists today and the isolation of man from nature are consequences of technological progress. All pre-industrial societies were predominantly rural. The industrial Revolution vastly increased the size of cities and the proportion of the population that lives in them, and modern agricultural technology has made it possible for the Earth to support a far denser population than it ever did before. (Also, technology exacerbates the effects of crowding because it puts increased disruptive powers in people's hands. For example, a variety of noise-making devices: power mowers, radios, motorcycles, etc. If the use of these devices is unrestricted, people who want peace and quiet are frustrated by the noise. If their use is restricted, people who use the devices are frustrated by the regulations... But if these machines had never been invented there would have been no conflict and no frustration generated by them.)
50. The conservatives are fools: They whine about the decay of traditional values, yet they enthusiastically support technological progress and economic growth. Apparently it never occurs to them that you can't make rapid, drastic changes in the technology and the economy of a society with out causing rapid changes in all other aspects of the society as well, and that such rapid changes inevitably break down traditional values.
87. Science and technology provide the most important examples of surrogate activities. Some scientists claim that they are motivated by "curiosity," that notion is simply absurd. Most scientists work on highly specialized problem that are not the object of any normal curiosity. For example, is an astronomer, a mathematician or an entomologist curious about the properties of isopropyltrimethylmethane? Of course not. Only a chemist is curious about such a thing, and he is curious about it only because chemistry is his surrogate activity. Is the chemist curious about the appropriate classification of a new species of beetle? No. That question is of interest only to the entomologist, and he is interested in it only because entomology is his surrogate activity. If the chemist and the entomologist had to exert themselves seriously to obtain the physical necessities, and if that effort exercised their abilities in an interesting way but in some nonscientific pursuit, then they couldn't giver a damn about isopropyltrimethylmethane or the classification of beetles. Suppose that lack of funds for postgraduate education had led the chemist to become an insurance broker instead of a chemist. In that case he would have been very interested in insurance matters but would have cared nothing about isopropyltrimethylmethane. In any case it is not normal to put into the satisfaction of mere curiosity the amount of time and effort that scientists put into their work. The "curiosity" explanation for the scientists' motive just doesn't stand up.
88. The "benefit of humanity" explanation doesn't work any better. Some scientific work has no conceivable relation to the welfare of the human race - most of archaeology or comparative linguistics for example. Some other areas of science present obviously dangerous possibilities. Yet scientists in these areas are just as enthusiastic about their work as those who develop vaccines or study air pollution. Consider the case of Dr. Edward Teller, who had an obvious emotional involvement in promoting nuclear power plants. Did this involvement stem from a desire to benefit humanity? If so, then why didn't Dr. Teller get emotional about other "humanitarian" causes? If he was such a humanitarian then why did he help to develop the H-bomb? As with many other scientific achievements, it is very much open to question whether nuclear power plants actually do benefit humanity. Does the cheap electricity outweigh the accumulating waste and risk of accidents? Dr. Teller saw only one side of the question. Clearly his emotional involvement with nuclear power arose not from a desire to "benefit humanity" but from a personal fulfillment he got from his work and from seeing it put to practical use.
89. The same is true of scientists generally. With possible rare exceptions, their motive is neither curiosity nor a desire to benefit humanity but the need to go through the power process: to have a goal (a scientific problem to solve), to make an effort (research) and to attain the goal (solution of the problem.) Science is a surrogate activity because scientists work mainly for the fulfillment they get out of the work itself.
90. Of course, it's not that simple. Other motives do play a role for many scientists. Money and status for example. Some scientists may be persons of the type who have an insatiable drive for status (see paragraph 79) and this may provide much of the motivation for their work. No doubt the majority of scientists, like the majority of the general population, are more or less susceptible to advertising and marketing techniques and need money to satisfy their craving for goods and services. Thus science is not a PURE surrogate activity. But it is in large part a surrogate activity.
91. Also, science and technology constitute a mass power movement, and many scientists gratify their need for power through identification with this mass movement (see paragraph 83).
92. Thus science marches on blindly, without regard to the real welfare of the human race or to any other standard, obedient only to the psychological needs of the scientists and of the government officials and corporation executives who provide the funds for research.
114. As explained in paragraph 65-67, 70-73, modern man is strapped down by a network of rules and regulations, and his fate depends on the actions of persons remote from him whose decisions he cannot influence. This is not accidental or a result of the arbitrariness of arrogant bureaucrats. It is necessary and inevitable in any technologically advanced society. The system HAS TO regulate human behavior closely in order to function. At work, people have to do what they are told to do, otherwise production would be thrown into chaos. Bureaucracies HAVE TO be run according to rigid rules. To allow any substantial personal discretion to lower-level bureaucrats would disrupt the system and lead to charges of unfairness due to differences in the way individual bureaucrats exercised their discretion. It is true that some restrictions on our freedom could be eliminated, but GENERALLY SPEAKING the regulation of our lives by large organizations is necessary for the functioning of industrial-technological society. The result is a sense of powerlessness on the part of the average person. It may be, however, that formal regulations will tend increasingly to be replaced by psychological tools that make us want to do what the system requires of us. (Propaganda [14], educational techniques, "mental health" programs, etc.)
115. The system HAS TO force people to behave in ways that are increasingly remote from the natural pattern of human behavior. For example, the system needs scientists, mathematicians and engineers. It can't function without them. So heavy pressure is put on children to excel in these fields. It isn't natural for an adolescent human being to spend the bulk of his time sitting at a desk absorbed in study. A normal adolescent wants to spend his time in active contact with the real world. Among primitive peoples the things that children are trained to do are in natural harmony with natural human impulses. Among the American Indians, for example, boys were trained in active outdoor pursuits -- just the sort of things that boys like. But in our society children are pushed into studying technical subjects, which most do grudgingly.
121. A further reason why industrial society cannot be reformed in favor of freedom is that modern technology is a unified system in which all parts are dependent on one another. You can't get rid of the "bad" parts of technology and retain only the "good" parts. Take modern medicine, for example. Progress in medical science depends on progress in chemistry, physics, biology, computer science and other fields. Advanced medical treatments require expensive, high-tech equipment that can be made available only by a technologically progressive, economically rich society. Clearly you can't have much progress in medicine without the whole technological system and everything that goes with it.
123. If you think that big government interferes in your life too much NOW, just wait till the government starts regulating the genetic constitution of your children. Such regulation will inevitably follow the introduction of genetic engineering of human beings, because the consequences of unregulated genetic engineering would be disastrous. [19]
124. The usual response to such concerns is to talk about "medical ethics." But a code of ethics would not serve to protect freedom in the face of medical progress; it would only make matters worse. A code of ethics applicable to genetic engineering would be in effect a means of regulating the genetic constitution of human beings. Somebody (probably the upper-middle class, mostly) would decide that such and such applications of genetic engineering were "ethical" and others were not, so that in effect they would be imposing their own values on the genetic constitution of the population at large. Even if a code of ethics were chosen on a completely democratic basis, the majority would be imposing their own values on any minorities who might have a different idea of what constituted an "ethical" use of genetic engineering. The only code of ethics that would truly protect freedom would be one that prohibited ANY genetic engineering of human beings, and you can be sure that no such code will ever be applied in a technological society. No code that reduced genetic engineering to a minor role could stand up for long, because the temptation presented by the immense power of biotechnology would be irresistible, especially since to the majority of people many of its applications will seem obviously and unequivocally good (eliminating physical and mental diseases, giving people the abilities they need to get along in today's world). Inevitably, genetic engineering will be used extensively, but only in ways consistent with the needs of the industrial-technological system. [20]
127. A technological advance that appears not to threaten freedom often turns out to threaten freedom often turns out to threaten it very seriously later on. For example, consider motorized transport. A walking man formerly could go where he pleased, go at his own pace without observing any traffic regulations, and was independent of technological support-systems. When motor vehicles were introduced they appeared to increase man's freedom. They took no freedom away from the walking man, no one had to have an automobile if he didn't want one, and anyone who did choose to buy an automobile could travel much faster than the walking man. But the introduction of motorized transport soon changed society in such a way as to restrict greatly man's freedom of locomotion. When automobiles became numerous, it became necessary to regulate their use extensively. In a car, especially in densely populated areas, one cannot just go where one likes at one's own pace one's movement is governed by the flow of traffic and by various traffic laws. One is tied down by various obligations: license requirements, driver test, renewing registration, insurance, maintenance required for safety, monthly payments on purchase price. Moreover, the use of motorized transport is no longer optional. Since the introduction of motorized transport the arrangement of our cities has changed in such a way that the majority of people no longer live within walking distance of their place of employment, shopping areas and recreational opportunities, so that they HAVE TO depend on the automobile for transportation. Or else they must use public transportation, in which case they have even less control over their own movement than when driving a car. Even the walker's freedom is now greatly restricted. In the city he continually has to stop and wait for traffic lights that are designed mainly to serve auto traffic. In the country, motor traffic makes it dangerous and unpleasant to walk along the highway. (Note the important point we have illustrated with the case of motorized transport: When a new item of technology is introduced as an option that an individual can accept or not as he chooses, it does not necessarily REMAIN optional. In many cases the new technology changes society in such a way that people eventually find themselves FORCED to use it.)
128. While technological progress AS A WHOLE continually narrows our sphere of freedom, each new technical advance CONSIDERED BY ITSELF appears to be desirable. Electricity, indoor plumbing, rapid long-distance communications . . . how could one argue against any of these things, or against any other of the innumerable technical advances that have made modern society? It would have been absurd to resist the introduction of the telephone, for example. It offered many advantages and no disadvantages. Yet as we explained in paragraphs 59-76, all these technical advances taken together have created world in which the average man's fate is no longer in his own hands or in the hands of his neighbors and friends, but in those of politicians, corporation executives and remote, anonymous technicians and bureaucrats whom he as an individual has no power to influence. [21] The same process will continue in the future. Take genetic engineering, for example. Few people will resist the introduction of a genetic technique that eliminates a hereditary disease It does no apparent harm and prevents much suffering. Yet a large number of genetic improvements taken together will make the human being into an engineered product rather than a free creation of chance (or of God, or whatever, depending on your religious beliefs).
129 Another reason why technology is such a powerful social force is that, within the context of a given society, technological progress marches in only one direction; it can never be reversed. Once a technical innovation has been introduced, people usually become dependent on it, unless it is replaced by some still more advanced innovation. Not only do people become dependent as individuals on a new item of technology, but, even more, the system as a whole becomes dependent on it. (Imagine what would happen to the system today if computers, for example, were eliminated.) Thus the system can move in only one direction, toward greater technologization. Technology repeatedly forces freedom to take a step back -- short of the overthrow of the whole technological system.
131. Technicians (we use this term in its broad sense to describe all those who perform a specialized task that requires training) tend to be so involved in their work (their surrogate activity) that when a conflict arises between their technical work and freedom, they almost always decide in favor of their technical work. This is obvious in the case of scientists, but it also appears elsewhere: Educators, humanitarian groups, conservation organizations do not hesitate to use propaganda or other psychological techniques to help them achieve their laudable ends. Corporations and government agencies, when they find it useful, do not hesitate to collect information about individuals without regard to their privacy. Law enforcement agencies are frequently inconvenienced by the constitutional rights of suspects and often of completely innocent persons, and they do whatever they can do legally (or sometimes illegally) to restrict or circumvent those rights. Most of these educators, government officials and law officers believe in freedom, privacy and constitutional rights, but when these conflict with their work, they usually feel that their work is more important.
145. Imagine a society that subjects people to conditions that make them terribly unhappy, then gives them the drugs to take away their unhappiness. Science fiction? It is already happening to some extent in our own society. It is well known that the rate of clinical depression had been greatly increasing in recent decades. We believe that this is due to disruption fo the power process, as explained in paragraphs 59-76. But even if we are wrong, the increasing rate of depression is certainly the result of SOME conditions that exist in today's society. Instead of removing the conditions that make people depressed, modern society gives them antidepressant drugs. In effect, antidepressants area a means of modifying an individual's internal state in such a way as to enable him to toelrate social conditions that he would otherwise find intolerable. (Yes, we know that depression is often of purely genetic origin. We are referring here to those cases in which environment plays the predominant role.)
147. To start with, there are the techniques of surveillance. Hidden video cameras are now used in most stores and in many other places, computers are used to collect and process vast amounts of information about individuals. Information so obtained greatly increases the effectiveness of physical coercion (i.e., law enforcement).[26] Then there are the methods of propaganda, for which the mass communication media provide effective vehicles. Efficient techniques have been developed for winning elections, selling products, influencing public opinion. The entertainment industry serves as an important psychological tool of the system, possibly even when it is dishing out large amounts of sex and violence. Entertainment provides modern man with an essential means of escape. While absorbed in television, videos, etc., he can forget stress, anxiety, frustration, dissatisfaction. Many primitive peoples, when they don't have work to do, are quite content to sit for hours at a time doing nothing at all, because they are at peace with themselves and their world. But most modern people must be contantly occupied or entertained, otherwise the get "bored," i.e., they get fidgety, uneasy, irritable.
152. Generally speaking, technological control over human behavior will probably not be introduced with a totalitarian intention or even through a conscious desire to restrict human freedom. [28] Each new step in the assertion of control over the human mind will be taken as a rational response to a problem that faces society, such as curing alcoholism, reducing the crime rate or inducing young people to study science and engineering. In many cases, there will be humanitarian justification. For example, when a psychiatrist prescribes an anti-depressant for a depressed patient, he is clearly doing that individual a favor. It would be inhumane to withhold the drug from someone who needs it. When parents send their children to Sylvan Learning Centers to have them manipulated into becoming enthusiastic about their studies, they do so from concern for their children's welfare. It may be that some of these parents wish that one didn't have to have specialized training to get a job and that their kid didn't have to be brainwashed into becoming a computer nerd. But what can they do? They can't change society, and their child may be unemployable if he doesn't have certain skills. So they send him to Sylvan.
157. Assuming that industrial society survives, it is likely that technology will eventually acquire something approaching complete control over human behavior. It has been established beyond any rational doubt that human thought and behavior have a largely biological basis. As experimenters have demonstrated, feelings such as hunger, pleasure, anger and fear can be turned on and off by electrical stimulation of appropriate parts of the brain. Memories can be destroyed by damaging parts of the brain or they can be brought to the surface by electrical stimulation. Hallucinations can be induced or moods changed by drugs. There may or may not be an immaterial human soul, but if there is one it clearly is less powerful that the biological mechanisms of human behavior. For if that were not the case then researchers would not be able so easily to manipulate human feelings and behavior with drugs and electrical currents.
158. It presumably would be impractical for all people to have electrodes inserted in their heads so that they could be controlled by the authorities. But the fact that human thoughts and feelings are so open to biological intervention shows that the problem of controlling human behavior is mainly a technical problem; a problem of neurons, hormones and complex molecules; the kind of problem that is accessible to scientific attack. Given the outstanding record of our society in solving technical problems, it is overwhelmingly probable that great advances will be made in the control of human behavior.
159. Will public resistance prevent the introduction of technological control of human behavior? It certainly would if an attempt were made to introduce such control all at once. But since technological control will be introduced through a long sequence of small advances, there will be no rational and effective public resistance. (See paragraphs 127,132, 153.)
160. To those who think that all this sounds like science fiction, we point out that yesterday's science fiction is today's fact. The Industrial Revolution has radically altered man's environment and way of life, and it is only to be expected that as technology is increasingly applied to the human body and mind, man himself will be altered as radically as his environment and way of life have been.
170. "Oh!" say the technophiles, "Science is going to fix all that! We will conquer famine, eliminate psychological suffering, make everybody healthy and happy!" Yeah, sure. That's what they said 200 years ago. The Industrial Revolution was supposed to eliminate poverty, make everybody happy, etc. The actual result has been quite different. The technophiles are hopelessly naive (or self-deceiving) in their understanding of social problems. They are unaware of (or choose to ignore) the fact that when large changes, even seemingly beneficial ones, are introduced into a society, they lead to a long sequence of other changes, most of which are impossible to predict (paragraph 103). The result is disruption of the society. So it is very probable that in their attempt to end poverty and disease, engineer docile, happy personalities and so forth, the technophiles will create social systems that are terribly troubled, even more so that the present one. For example, the scientists boast that they will end famine by creating new, genetically engineered food plants. But this will allow the human population to keep expanding indefinitely, and it is well known that crowding leads to increased stress and aggression. This is merely one example of the PREDICTABLE problems that will arise. We emphasize that, as past experience has shown, technical progress will lead to other new problems for society far more rapidly that it has been solving old ones. Thus it will take a long difficult period of trial and error for the technophiles to work the bugs out of their Brave New World (if they ever do). In the meantime there will be great suffering. So it is not all clear that the survival of industrial society would involve less suffering than the breakdown of that society would. Technology has gotten the human race into a fix from which there is not likely to be any easy escape.
171. But suppose now that industrial society does survive the next several decade and that the bugs do eventually get worked out of the system, so that it functions smoothly. What kind of system will it be? We will consider several possibilities.
172. First let us postulate that the computer scientists succeed in developing intelligent machines that can do all things better that human beings can do them. In that case presumably all work will be done by vast, highly organized systems of machines and no human effort will be necessary. Either of two cases might occur. The machines might be permitted to make all of their own decisions without human oversight, or else human control over the machines might be retained.
173. If the machines are permitted to make all their own decisions, we can't make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines. It might be argued that the human race would never be foolish enough to hand over all the power to the machines. But we are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines decisions. As society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decision for them, simply because machine-made decisions will bring better result than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won't be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide.
174. On the other hand it is possible that human control over the machines may be retained. In that case the average man may have control over certain private machines of his own, such as his car of his personal computer, but control over large systems of machines will be in the hands of a tiny elite -- just as it is today, but with two difference. Due to improved techniques the elite will have greater control over the masses; and because human work will no longer be necessary the masses will be superfluous, a useless burden on the system. If the elite is ruthless the may simply decide to exterminate the mass of humanity. If they are humane they may use propaganda or other psychological or biological techniques to reduce the birth rate until the mass of humanity becomes extinct, leaving the world to the elite. Or, if the elite consist of soft-hearted liberals, they may decide to play the role of good shepherds to the rest of the human race. They will see to it that everyone's physical needs are satisfied, that all children are raised under psychologically hygienic conditions, that everyone has a wholesome hobby to keep him busy, and that anyone who may become dissatisfied undergoes "treatment" to cure his "problem." Of course, life will be so purposeless that people will have to be biologically or psychologically engineered either to remove their need for the power process or to make them "sublimate" their drive for power into some harmless hobby. These engineered human beings may be happy in such a society, but they most certainly will not be free. They will have been reduced to the status of domestic animals.
175. But suppose now that the computer scientists do not succeed in developing artificial intelligence, so that human work remains necessary. Even so, machines will take care of more and more of the simpler tasks so that there will be an increasing surplus of human workers at the lower levels of ability. (We see this happening already. There are many people who find it difficult or impossible to get work, because for intellectual or psychological reasons they cannot acquire the level of training necessary to make themselves useful in the present system.) On those who are employed, ever-increasing demands will be placed; They will need more and m ore training, more and more ability, and will have to be ever more reliable, conforming and docile, because they will be more and more like cells of a giant organism. Their tasks will be increasingly specialized so that their work will be, in a sense, out of touch with the real world, being concentrated on one tiny slice of reality. The system will have to use any means that I can, whether psychological or biological, to engineer people to be docile, to have the abilities that the system requires and to "sublimate" their drive for power into some specialized task. But the statement that the people of such a society will have to be docile may require qualification. The society may find competitiveness useful, provided that ways are found of directing competitiveness into channels that serve that needs of the system. We can imagine into channels that serve the needs of the system. We can imagine a future society in which there is endless competition for positions of prestige an power. But no more than a very few people will ever reach the top, where the only real power is (see end of paragraph 163). Very repellent is a society in which a person can satisfy his needs for power only by pushing large numbers of other people out of the way and depriving them of THEIR opportunity for power.
176. Once can envision scenarios that incorporate aspects of more than one of the possibilities that we have just discussed. For instance, it may be that machines will take over most of the work that is of real, practical importance, but that human beings will be kept busy by being given relatively unimportant work. It has been suggested, for example, that a great development of the service of industries might provide work for human beings. Thus people will would spend their time shinning each others shoes, driving each other around inn taxicab, making handicrafts for one another, waiting on each other's tables, etc. This seems to us a thoroughly contemptible way for the human race to end up, and we doubt that many people would find fulfilling lives in such pointless busy-work. They would seek other, dangerous outlets (drugs, , crime, "cults," hate groups) unless they were biological or psychologically engineered to adapt them to such a way of life.