The AI-based Generative adversarial networks or the GANs can generate hyper-realistic image using the random noise of low dimension. This led to the emergence of the term DeepFake which not only includes forged images but doctored videos and audios. The generated fake images with inappropriate content propagated through social networks and social media can have adverse effect on the society, politics, and economy. Thus, to effectively mitigate the DeepFake issues, there is a pressing need for an advanced deep learning-based detector. Detection of GAN generated fake images is a challenge for the conventional detection techniques as the images are reconstructed from manipulation of the source image. However, recent studies have revealed that CNN based detection architectures like VGG16, InceptionV3 and others have often showed high accuracy in classification tasks. This paper proposes a CNN based architecture, the Segment-based model to perform the classification between real and fake images generated using GANs. The DCGAN, DRAGAN, LSGAN and WGAN are used to generate the fake images. The Segment-based model is a novel method of segmenting the images in four quadrants followed by localization of fake features. The proposed architecture is compared with the pre-trained VGG16 and InceptionV3 through the evaluation matrices Classification accuracy and AUC. The Segment-based model is in parity with the popularly used VGG16 and InceptionV3.