|Association: linking of sensations, ideas, thoughts or beliefs that is acquired by repetition (even unconsciously). Unlike causality._____________Annotation: The above characterizations of concepts are neither definitions nor exhausting presentations of problems related to them. Instead, they are intended to give a short introduction to the contributions below. – Lexicon of Arguments. |
|Norvig I 762
Association/AI Research/Norvig/Russell: Hopfield networks (Hopfield, 1982)(1) are probably the best understood class of recurrent networks. They use bidirectional connections with symmetric weights (i.e., wi,j = wj,i), all of the units are both input and output units, the activation function g is the sign function, and the activation levels can only be ±1.
A Hopfield network functions as an associative memory: after the network trains on a set of examples, a new stimulus will cause it to settle into an activation pattern corresponding to the example in the training set that most closely resembles the new stimulus. For example, if the training set consists of a set of photographs, and the new stimulus is a small piece of one of the photographs, then the network activation levels will reproduce the photograph from which the piece was taken. Notice that the original photographs are not stored separately in the network; each
Norvig I 763
weight is a partial encoding of all the photographs. One of the most interesting theoretical results is that Hopfield networks can reliably store up to 0.138N training examples, where N is the number of units in the network. >Boltzmann Machines.
1. Hopfield, J. J. (1982). Neurons with graded response have collective computational properties like those of two-state neurons. PNAS, 79, 2554–2558._____________Explanation of symbols: Roman numerals indicate the source, arabic numerals indicate the page number. The corresponding books are indicated on the right hand side. ((s)…): Comment by the sender of the contribution. The note [Author1]Vs[Author2] or [Author]Vs[term] is an addition from the Dictionary of Arguments. If a German edition is specified, the page numbers refer to this edition.
Stuart J. Russell
Artificial Intelligence: A Modern Approach Upper Saddle River, NJ 2010