Potato diseases are one of the primary causes of decreased agricultural production quality and quantity. With ongoing changes in potato structure and cultivation techniques, new diseases are constantly arising on potato leaves. In this work, we have reviewed many CNN articles on detecting potato disease detection. CNN models are trained on image data are the most effective method for detecting early leaf detection. But here we work upon a specific plant disease i.e. potato plant disease like – early blight, late blight. In this study we use CNN models for feature extraction and segmentation, where we can get the CNN model as a pre-trained deep learning model. Here we also use a model for classification i.e. LSTM (long short-term memory) which is an updated version of RNN model. The experiments are carried out using the popular publicly available dataset Plant Village dataset and potato leaf disease dataset which has about 2152 images of early blight, Late blight and healthy leaves.