The capability which human brain has is unrealizable even by present computer. Neural network is the technology which reproduced human brain in false. Time-sereies such as sound are widely studied using this. In general, Elman's network is used for time-siries learning. We introduced the connection weights to the context layer in Elman's network. This layer is called an internal memory layer. And complicated time-series are dealt with by this network. In this research, the network with an internal memory as a form of a network was considered, and studied regarding this. The conventional back-propagation was not effective for learning the network. And so the ``delay learning method'' was newly devised. Using this method, the neural network with internal memory can learn complicated time-series. It turns out that the delay learning method is effective for making it learn. A supervisory signal can be taken out if connection weight after successful study is used. When the internal memory layer is used, a supervisory signal is reproducible, even if the error is included in the input signal. And the value of an internal memmory cell is in accord with a supervisory signal. This is the reason the network with an internal memorry can learn complicated time-series.