This research work proposes a novel technique for Automatic face recognition (AFR) system using cascaded structures and clustering network. Human beings at a very early age are capable of recognizing the varying facial features, due to the Human Visual System (HVS). But it’s difficult to depict the human visual system using computer vision system. The basic idea used in this proposed work is to use divide and conquer method, where we design a particular approach for each processing stage and then embedding the entire strategy for AFR system. In this proposed work, two important factors namely cost efficiency and applied technology for varying characteristics of input image are considered respectively, irrespective of the traditional factors such as accuracy, retrieving rate etc. For facial detection, a heterogeneous cascaded detector based on various features is designed to increase the processing capability and detecting efficiency respectively. For facial feature extraction, sparse graph, component based direct fitting and component based texture fitting methods are used to extract the various features at different orientations.