Abstract: Objective To evaluate rectal toxicity of radiotherapy for prostate cancer using a novel predictive model based on multi-modality and multi-classifier fusion. Methods We retrospectively collected the clinical data from 44 prostate cancer patients receiving external beam radiation (EBRT), including the treatment data, clinical parameters, planning CT data and the treatment plans. The clinical parameter features and dosimetric features were extracted as two different modality features, and a subset of features was selected to train the 5 base classifiers (SVM, Decision Tree, K-nearest-neighbor, Random forests and XGBoost). To establish the multi-modality and multi-classifier fusion model, a multi-criteria decision-making based weight assignment algorithm was used to assign weights for each base classifier under the same modality. A repeat 5-fold cross-validation and the 4 indexes including the area under ROC curve (AUC), accuracy, sensitivity and specificity were used to evaluate the proposed model. In addition, the proposed model was compared quantitatively with different feature selection methods, different weight allocation algorithms, the model based on single mode single classifier, and two integrated models using other fusion methods. Results Repeated (5 times) 5-fold cross validation of the proposed model showed an accuracy of 0.78 for distinguishing toxicity from non-toxicity with an AUC of 0.83, a specificity of 0.79 and a sensitivity of 0.76. Conclusion Compared with the models based on a single mode or a single classifier and other fusion models, the proposed model can more accurately predict rectal toxicity of radiotherapy for prostate cancer.