If you’ve bought goods or services from a VAT registered supplier then (unless the nature of those goods or services mean that they fall within an exemption or zero rating), the supplier will be required to charge you VAT. I can’t think of any circumstances where a customer not being registered for VAT would mean that a supplier didn’t have to charge it.
The supplier is required to pay the VAT amount charged over to HMRC, normally based on when the date that they issue their invoice, even if they’ve yet to receive payment from the customer. So if you haven’t paid the VAT element of an amount that you’ve been charged, that is a cost to the supplier, and they’ll take the same action as they would to pursue any unpaid debt.
But if I’ve misunderstood the contractual arrangements between you and this company, please provide a bit more detail.