In this paper, Miller's correlation based model of orientation selectivity is studied, in particular, a development of an oriented receptive field is considered and analyzed mathematically. Our computer simulations show that (1) the final shape of a receptive field most likely appears just after the first update of synaptic weights, and (2) synaptic weights are updated monotonously preserving the shape of a receptive field that is already determined at the first step. We try to explain these two properties and give some mathematical justifications by considering a simplified model. As a result, we show that (1) if a learning efficacy is small enough, synaptic weights almost always grow monotonously, and (2) if the width of an arbor and the range of a correlation is similar, then the increment at the first step forms three separated subregions of {\small\it ON} and {\small\it OFF}. Moreover, we also show that if the range of a correlation is much smaller than the width of an arbor, then the incremental forms several small clusters of {\small\it ON} and {\small\it OFF}.