Go to Settings
Hover over the contact and click on the 3 vertical dots
Select Delete contact
If you have used a contact in an invoice, bank transaction, tax invoice or credit note you will not be able to delete the contact. However you can deactivate the contact.