Objective To investigate the efficacy of anlotinib plus S-1 for treatment of patients with recurrent or metastatic esophageal squamous cell carcinoma with failed first-line chemotherapy. Methods Twenty-six patients with recurrent or metastatic esophageal squamous cell carcinoma patients who experienced progression after first-line paclitaxel plus platinum chemotherapy in our hospital between July, 2018 and February, 2020 were enrolled in this study. The patients received oral anlotinib along with S-1 treatment (anlotinib at 12 mg once daily and S-1 at 50 mg twice daily for two weeks; 3 weeks per cycle). The objective response rate (ORR), disease control rate (DCR), progression-free survival (PFS) and adverse effects were evaluated for all the patients. Results No complete remission (CR) was observed in the 26 patients. Partial remission (PR) was achieved in 6 cases, stable disease (SD) in 12 cases, and progressive disease (PD) occurred in 8 cases, with an ORR of 23.1% and a DCR of 69.2% in these patients. The median PFS was 4.5 months (95%CI: 2.7-6.4 months). Univariate analysis showed that the patients with moderate or high tumor differentiation had significantly longer PFS than those with low tumor differentiation (6.1 months vs 1.9 months, P<0.05). Multivariate analysis suggested that pathological differentiation grade (HR=6.778, 95%CI:1.997-23.012) was an independent factor for a prolonged PFS. The adverse effects in the patients included mainly fatigue, hypertension and hand- foot syndrome, mostly of grade 1 to 2. Conclusion Patients with recurrent or metastatic esophageal squamous cell carcinoma can benefit from a second-line anlotinib plus S-1 treatment, which has relatively mild adverse effects with a good safety profile.