A neural network approach to obtaining upper and lower bounds on the Bayes error rate for pattern recognition problems is presented. The approach is developed using the key concept of resubstitution and leave-one-out testing from conventional nonparametric error estimation techniques. The neural network approach is evaluated by applying it to several 8D, two-class `toy' problems, where the Bayes error rate is known. The neural network error estimate for a high-dimensional problem with an unknown Bayes error rate is also compared to error estimates obtained using conventional nonparametric estimation techniques. Using the neural network procedure, the upper bound of the Bayes error rate is reliably found for problems with complex decision boundary surfaces. Alternative testing approaches are suggested for reducing the difference between the bounds and the true Bayes rate.