Linsker's multi-layer neural network model is considered; in particular, development of an oriented receptive field of a cell on layer G is studied mathematically. We attempt to investigate which features in his model are essential for developing an oriented receptive field. Here we consider a slightly simplified version of Linsker's model and analyze its process of changing synaptic weights by using the Fourier transform of the formula of modifying synaptic weights. Through our mathematical analysis, we obtained the following observations: (1) the correlation function that is a DOG function extracts the Fourier coefficients of a certain frequency, (2) the arbor function determines either a Fourier sine coefficient part or a Fourier cosine coefficient part dominates the updating process, and (3) if the Fourier cosine coefficient part is dominant, then a circular symmetric receptive field is developed, and on the other hand, if the Fourier sine coefficient part is dominant, then a bi-lobed receptive field is obtained. These observations are also justified by our computer simulations. We also consider the case that a Gaussian is used as the arbor function and show that a Fourier cosine coefficient part is dominant in this case and hence a circular symmetric receptive field is developed. Finally, we discuss the role of Linsker's parameters k_1 and k_2.