Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the following Registration Statements:

(1) Registration Statement (Form S-8 No. 333-61190) pertaining to the Delta Apparel, Inc. 2000 Stock Option Plan and Delta Apparel, Inc. Incentive Stock Award Plan, and

(2) Registration Statement (Form S-8 No. 333-172018) pertaining to the Delta Apparel, Inc. 2010 Stock Plan

(3) Registration Statement (Form S-3 No. 333-214783) of Delta Apparel, Inc.

of our reports dated November 19, 2018, with respect to the consolidated financial statements of Delta Apparel, Inc. and subsidiaries and the effectiveness of internal control over financial reporting of Delta Apparel, Inc. and subsidiaries, included in this Annual Report (Form 10-K) of Delta Apparel, Inc. and subsidiaries for the year ended September 29, 2018.

/s/ Ernst & Young LLP

Atlanta, Georgia

November 19, 2018