If the goods concerned never leave the EU / enter the UK, it’s hard to see how the supply could be subject to UK VAT. However it could still be subject to VAT in the EU.
If the goods remain within the same member state it’s likely that you may need to register for VAT there. If the goods cross a border, the position may be dependant on whether the transaction is B2B or B2C – as with B2B it’s possible that the customer may be required to account for the VAT as acquisition tax. But I would suggest that you need to check guidance issued by the tax administrations of the member state(s) concerned, as rates, thresholds, etc. vary between countries.
As an example, section 8 onwards of HMRC’s notice 700/1 deals with NETP (Non-Established Taxable Persons) based outside the UK selling goods within the UK. I suggest you need to see the equivalent guidance for the member state(s) concerned.