Navigate to your Content > Videos section in your Admin Area.
Click on the checkbox at the top of your video listing.
This will select all checkboxes on that current view (meaning, on that page of pagination, if you have more than one page).
You can then select the Delete option.
You'll be prompted with a confirmation that you truly want to delete all the videos selected. Confirm, and that's it!