Why my Neural Network Accuracy is 100%?

Question

I wanna ask about Neural Network.

I have a research and programming about Neural Network using Backpropagation algorithm for prediction. My input data using binary form. But, i'm still confuse, because i get some input data from expert. I have separate some data for training and the remaining data i have use to testing. I get accuracy 100% for training and testing. I think 100% for training is possible but 100% for testing it's something weird.

Is it possible if the data I get from expert including linear separability?

If the data is linear separability, is my neural network fail or my neural network works because the pattern too easy?

Thanks..

Answer 1

If the data is linearly separable then yes, it's possible. Take one of these scatter plots which show the blue points and the red points and the line between them.

(image stolen from here)

If your neural network got the line right, it is possible it can have a 100% accuracy. Remember that a neuron's output (before it goes through an activation function) is a linear combination of its inputs so this is a pattern that a network consisting of a single neuron can learn. But the data you are using is clearly synthetic, it's rather unlikely for real life data to be perfectly linearly separable.

Answer 2

If your machine learning classifier is obtaining perfect or almost perfect predictions (accuracy = 1.0 and MCC = +1.0), probably it means that there is a feature in the input dataset that is duplicate of the target feature you want to predict.

If you represent your dataset with M rows as data instances, N features as columns, and 1 target feature as an additional column, it means that one of the N columns has the same values of the target column.

Your machine learning algorithm is able to make perfect predictions because, after some iterations, it "understands" that it can make accurate predictions by just looking at the next value of the feature duplicating the targets.

If this happens, you machine learning system becomes quite useless: anyone (even without a computer) can become able to make perfect predictions, by just looking at the duplicate feature.

An example can be found in the paper entitled "An approach based on probabilistic neural network for diagnosis of mesothelioma’s disease" (Er et al, Computers & Electrical Engineering, 2012), where the authors used a neural network to predict the diagnosis of 324 patients having 34 features and one target label. The target column is "class of diagnosis", that states if each patient (row) has mesothelioma or not. But the dataset contains another feature, called "diagnosis method", which has exactly the same values of "class of diagnosis" (you can check this aspect by taking a look to their dataset, which they made available on the University of California Irvine Machine Learning Repository).

The authors include that "diagnosis method" in their input dataset, and therefore obtain spectacularly high accuracy prediction rates: top accuracy = 0.98

This prediction does not make much sense, because biased on the duplication of the target labels. They basically included the ground truth labels into the dataset they wanted to predict. If they removed that feature, prediction scores would be much lower.

So, if you see your machine learning algorithm predicting too well, always wonder: am I including any feature duplicating the targets in my input dataset?