(18)F-flutemetamol is a positron emission tomography (PET) tracer for in vivo amyloid imaging. The ability to classify amyloid scans in a binary manner as 'normal' versus 'Alzheimer-like', is of high clinical relevance. We evaluated whether a supervised machine learning technique, support vector machines (SVM), can replicate the assignments made by visual readers blind to the clinical diagnosis, which image components have highest diagnostic value according to SVM and how (18)F-flutemetamol-based classification using SVM relates to structural MRI-based classification using SVM within the same subjects. By means of SVM with a linear kernel, we analyzed (18)F-flutemetamol scans and volumetric MRI scans from 72 cases from the (18)F-flutemetamol phase 2 study (27 clinically probable Alzheimer's disease (AD), 20 amnestic mild cognitive impairment (MCI), 25 controls). In a leave-one-out approach, we trained the (18)F-flutemetamol based classifier by means of the visual reads and tested whether the classifier was able to reproduce the assignment based on visual reads and which voxels had the highest feature weights. The (18)F-flutemetamol based classifier was able to replicate the assignments obtained by visual reads with 100% accuracy. The voxels with highest feature weights were in the striatum, precuneus, cingulate and middle frontal gyrus. Second, to determine concordance between the gray matter volume- and the (18)F-flutemetamol-based classification, we trained the classifier with the clinical diagnosis as gold standard. Overall sensitivity of the (18)F-flutemetamol- and the gray matter volume-based classifiers were identical (85.2%), albeit with discordant classification in three cases. Specificity of the (18)F-flutemetamol based classifier was 92% compared to 68% for MRI. In the MCI group, the (18)F-flutemetamol based classifier distinguished more reliably between converters and non-converters than the gray matter-based classifier. The visual read-based binary classification of (18)F-flutemetamol scans can be replicated using SVM. In this sample the specificity of (18)F-flutemetamol based SVM for distinguishing AD from controls is higher than that of gray matter volume-based SVM.