The face recognition of medically altered faces under gender transformation is performed. Medical alteration is the gender transformation performed by undergoing hormone replacement therapy (HRT). Face images that are undergoing HRT based gender transformation over a period of several months to three years are taken. By altering the balance of sex hormones HRT achieves gender transformation. HRT causes changes in the physical appearance of the body and face. Changes in the face due to HRT affect the face recognition system. Existing system uses a face recognition system and the studies reveal that the Periocular region is invariant to the changes due to HRT. So a Periocular based face recognition system is developed. Initially perform face and eye detection then alignment and cropping is performed. Face images are represented using patch based local binary patterns. Face recognition is performed by using the binary classifier SVM Classifier.