Find counter arguments by entering NameVs… or …VsName.
The author or concept searched is found in the following 4 entries.
| Disputed term/author/ism | Author |
Entry |
Reference |
|---|---|---|---|
| Ethics | Bostrom | I 257 Ethics/morals/morality/superintelligence//Bostrom: No ethical theory commands majority support among philosophers, so most philosophers must be wrong. ((s)VsBostrom: It is not a question of applause as to which theory is correct.) I 369 Majorities in ethics/Bostrom: A recent canvass of professional philosophers found the percentage of respondents who “accept or leans toward” various positions. On normative ethics, the results were deontology 25.9%; - consequentialism 23.6%; - virtue ethics 18.2%. On metaethics, results were moral realism 56.4%; - moral anti-realism 27.7%. On moral judgment: cognitivism 65.7%; - non-cognitivism 17.0% (Bourget and Chalmers 2009(1)) >Norms/normativity/superintelligence/Bostrom, >Ethics/superintelligence/Yudkowsky. Morality models: I 259 Coherent Extrapolated Volition/CEV/Yudkowsky: Our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted. >Ethics/superintelligence/Yudkowsky. I 266 VsCEV/Bostrom: instead: Moral rightness/MR/Bostrom: (…) build an AI with the goal of doing what is morally right, relying on the AI’s superior cognitive capacities to figure out just which actions fit that description. We can call this proposal “moral rightness” (MR). The idea is that we humans have an imperfect understanding of what is right and wrong (…) ((s)VsBostrom: This delegates human responsibility and ultimately assumes that human decisions are only provisional until non-human decisions are made.) I 267 BostromVsYudkowsky: MR would do away with various free parameters in CEV, such as the degree of coherence among extrapolated volitions that is required for the AI to act on the result, the ease with which a majority can overrule dissenting minorities, and the nature of the social environment within which our extrapolated selves are to be supposed to have “grown up farther together.” BostromVsMR: Problem: 1. MR would also appear to have some disadvantages. It relies on the notion of “morally right,” a notoriously difficult concept (…). I 268 2. (…) [MR] might not give us what we want or what we would choose if we were brighter and better informed. Solution/Bostrom: Goal for AI: MP: Among the actions that are morally permissible for the AI, take one that humanity’s CEV would prefer. However, if some part of this instruction has no well-specified meaning, or if we are radically confused about its meaning, or if moral realism is false, or if we acted morally impermissibly in creating an AI with this goal, then undergo a controlled shutdown.(*) Follow the intended meaning of this instruction. I 373 (Annotation) *Moral permissibility/Bostrom: When the AI evaluates the moral permissibility of our act of creating the AI, it should interpret permissibility in its objective sense. In one ordinary sense of “morally permissible,” a doctor acts morally permissibly when she prescribes a drug she believes will cure her patient - even if the patient, unbeknownst to the doctor, is allergic to the drug and dies as a result. Focusing on objective moral permissibility takes advantage of the presumably superior epistemic position of the AI. ((s)VsBostrom: The last sentence (severability) is circular, especially when there are no longer individuals in decision-making positions who could object to it. >Goals/superintelligence/Bostrom. I 312 Def Common good principle/Bostrom: Superintelligence should be developed only for the benefit of all of humanity and in the service of widely shared ethical ideals. I 380 This formulation is intended to be read so as to include a prescription that the well-being of nonhuman animals and other sentient beings (including digital minds) that exist or may come to exist be given due consideration. It is not meant to be read as a license for one AI developer to substitute his or her own moral intuitions for those of the wider moral community. 1. Bourget, David, and Chalmers, David. 2009. “The PhilPapers Surveys.” November. Available at http://philpapers.org/surveys/ |
Bostrom I Nick Bostrom Superintelligence. Paths, Dangers, Strategies Oxford: Oxford University Press 2017 |
| Goals | AI Research | Bostrom I 126 Goals/superintelligence/AI Research/Bostrom: Is it possible to say anything about what a superintelligence with a decisive I 127 strategic advantage would want? I 129 Motivation/intelligence/superintelligent will/orthogonality/Bostrom: Intelligent search for instrumentally optimal plans and policies can be performed in the service of any goal. Intelligence and motivation are in a sense orthogonal: we can think of them as two axes spanning a graph in which each point represents a logically possible artificial agent. Some qualifications could be added to this picture. For instance, it might be impossible for a very unintelligent system to have very complex motivations. I 130 Def Orthogonality thesis/Bostrom: Intelligence and final goals are orthogonal: more or less any level of intelligence could in principle be combined with more or less any final goal. According to the orthogonality thesis, artificial agents can have utterly non-anthropomorphic goals. -Predictability through design: I 131 (…) even before an agent has been created we might be able to predict something about its behavior, if we know something about who will build it and what goals they will want it to have. -Predictability through inheritance. If a digital intelligence is created directly from a human template (as would be the case in a high-fidelity whole brain emulation), then the digital intelligence might inherit the motivations of the human template. -Predictability through convergent instrumental reasons: (…) we may be able to infer something about its more immediate objectives by considering the instrumental reasons that would arise for any of a wide range of possible final goals in a wide range of situations. I 132 Def Instrumental convergence thesis/Bostrom: Several instrumental values can be identified which are convergent in the sense that their attainment would increase the chances of the agent’s goal being realized for a wide range of final goals and a wide range of situations, implying that these instrumental values are likely to be pursued by a broad spectrum of situated intelligent agents. >Goals/Omohundro. Where there are convergent instrumental values, we may be able to predict some aspects of a superintelligence’s behavior: -Self-preservation: Most humans seem to place some final value on their own survival. This is not a necessary feature of artificial agents: some may be designed to place no final value whatever on their own survival. -Goal-content integrity: If an agent retains its present goals into the future, then its present goals will be more likely to be achieved by its future self. This gives the agent a present instrumental reason to I 133 prevent alterations of its final goals. For software agents, which can easily switch bodies or create exact duplicates of themselves, preservation of self as a particular implementation or a particular physical object need not be an important instrumental value. Advanced software agents might also be able to swap memories, download skills, and radically modify their cognitive architecture and personalities. I 141 Orthogonality thesis/Bostrom: (see above) the orthogonality thesis suggests that we cannot blithely assume that a superintelligence will necessarily share any of the final values stereotypically associated with wisdom and intellectual development in humans (…). I 270 Goals/ethics/morality/superintelligence/Bostrom: Consider, for example, the following “reasons-based” goal: Do whatever we would have had most reason to ask the AI to do. ((s)VsBostrom: Here it is assumed that the AI has no reason to falsify our intentions. I 272 Bostrom: components for choices of behavior: -Goal content: What objective should the AI pursue? How should a description of this objective be interpreted? -Decision theory: Should the AI use causal decision theory, evidential decision theory, updateless decision theory, or something else? -Epistemology: What should the AI’s prior probability function be (…).What theory of anthropics should it use? -Ratification: Should the AI’s plans be subjected to human review before being put into effect? If so, what is the protocol for that review process? >Ethics/superintelligence/Bostrom, >Ethics/superintelligence/Yudkowsky, >Norms/Bostrom. |
Bostrom I Nick Bostrom Superintelligence. Paths, Dangers, Strategies Oxford: Oxford University Press 2017 |
| Goals | Bostrom | I 126 Goals/superintelligence/AI Research/Bostrom: Is it possible to say anything about what a superintelligence with a decisive I 127 strategic advantage would want? I 129 Motivation/intelligence/superintelligent will/orthogonality/Bostrom: Intelligent search for instrumentally optimal plans and policies can be performed in the service of any goal. Intelligence and motivation are in a sense orthogonal: we can think of them as two axes spanning a graph in which each point represents a logically possible artificial agent. Some qualifications could be added to this picture. For instance, it might be impossible for a very unintelligent system to have very complex motivations. I 130 Def Orthogonality thesis/Bostrom: Intelligence and final goals are orthogonal: more or less any level of intelligence could in principle be combined with more or less any final goal. According to the orthogonality thesis, artificial agents can have utterly non-anthropomorphic goals. -Predictability through design: I 131 (…) even before an agent has been created we might be able to predict something about its behavior, if we know something about who will build it and what goals they will want it to have. -Predictability through inheritance. If a digital intelligence is created directly from a human template (as would be the case in a high-fidelity whole brain emulation), then the digital intelligence might inherit the motivations of the human template. -Predictability through convergent instrumental reasons: (…) we may be able to infer something about its more immediate objectives by considering the instrumental reasons that would arise for any of a wide range of possible final goals in a wide range of situations. I 132 Def Instrumental convergence thesis/Bostrom: Several instrumental values can be identified which are convergent in the sense that their attainment would increase the chances of the agent’s goal being realized for a wide range of final goals and a wide range of situations, implying that these instrumental values are likely to be pursued by a broad spectrum of situated intelligent agents. >Goals/Omohundro. Where there are convergent instrumental values, we may be able to predict some aspects of a superintelligence’s behavior: -Self-preservation: Most humans seem to place some final value on their own survival. This is not a necessary feature of artificial agents: some may be designed to place no final value whatever on their own survival. -Goal-content integrity: If an agent retains its present goals into the future, then its present goals will be more likely to be achieved by its future self. This gives the agent a present instrumental reason to I 133 prevent alterations of its final goals. For software agents, which can easily switch bodies or create exact duplicates of themselves, preservation of self as a particular implementation or a particular physical object need not be an important instrumental value. Advanced software agents might also be able to swap memories, download skills, and radically modify their cognitive architecture and personalities. I 141 Orthogonality thesis/Bostrom: (see above) the orthogonality thesis suggests that we cannot blithely assume that a superintelligence will necessarily share any of the final values stereotypically associated with wisdom and intellectual development in humans (…). I 270 Goals/ethics/morality/superintelligence/Bostrom: Consider, for example, the following “reasons-based” goal: Do whatever we would have had most reason to ask the AI to do. ((s)VsBostrom: Here it is assumed that the AI has no reason to falsify our intentions. I 272 Bostrom: components for choices of behavior: -Goal content: What objective should the AI pursue? How should a description of this objective be interpreted? -Decision theory: Should the AI use causal decision theory, evidential decision theory, updateless decision theory, or something else? -Epistemology: What should the AI’s prior probability function be (…).What theory of anthropics should it use? -Ratification: Should the AI’s plans be subjected to human review before being put into effect? If so, what is the protocol for that review process? >Ethics/superintelligence/Bostrom, >Ethics/superintelligence/Yudkowsky, >Norms/Bostrom. |
Bostrom I Nick Bostrom Superintelligence. Paths, Dangers, Strategies Oxford: Oxford University Press 2017 |
| Norms | Bostrom | I 258 Norms/normativity/superintelligence/Bostrom: Indirect normativity is a way to answer the challenge presented by the fact that we may not know what we truly want, what is in our interest, or what is morally right or ideal. Instead of making a guess based on our own current understanding (which is probably deeply flawed), we would delegate some of the cognitive work required for value selection to the superintelligence. >Ethics/superintelligence/Bostrom. (…) one could generalize this idea (…): Def Principle of epistemic deference: A future superintelligence occupies an epistemically superior vantage point: its beliefs are (probably, on most topics) more likely than ours to be true. We should therefore defer to the superintelligence’s opinion whenever feasible. ((s)VsBostrom: (1) Here the tacit assumption is made that moral decisions are subject to progress. (2) This delegates human responsibility and ultimately assumes that human decisions are only provisional until non-human decisions are made.) >Ethics/morality/superintelligence/Yudkowsky. I 259 Ethics/morality/superintelligence/Yudkowsky: Yudkowsky has proposed that a seed AI be given the final goal of carrying out humanity’s “coherent extrapolated volition” (CEV), which he defines as follows: CEV/Yudkowsky: Our Coherent Extrapolated Volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted. >Ethics/superintelligence//Bostrom. |
Bostrom I Nick Bostrom Superintelligence. Paths, Dangers, Strategies Oxford: Oxford University Press 2017 |
| |||
No results. Please choose an author or concept or try a different keyword-search.