I'm seeking guidance on establishing general regulatory compliance with our cloud vendors. What should I be aware of and include? Are there any best practices or templates available to assist me in this process?

Building upon what others have said, each of the major cloud providers offer policy "overlays" for cloud environments. Say you need to be PCI compliant, both AWS and Azure have reviewed areas of risk and control specific to that framework. When you apply that governance framework to your instance, it will highlight your compliant and non-compliant configurations.

At the higher vendor management level, we ask any vendor we work with to complete a security questionnaire which explores topics of security, patching, data management, change management, etc.. Smaller providers will tend to answer directly. Larger providers will tend to have a set of governance documents, such as a SOC2 report. We will review those and potentially accept them in lieu of our questionnaire.

I hope this helps.

A good starting point would be to check your vendor's compliance management details. AWS provides the "Artifact" service https://aws.amazon.com/artifact/ for this, Azure has a comprehensive list of compliance documents https://learn.microsoft.com/en-us/azure/compliance/ and GCP has their compliance resource center https://cloud.google.com/compliance?hl=en. 

You should also be mindful of the SLAs, data security and access policies in their shared responsibility models to ensure that you are covering any possible gap that they are not covering. There are generic SLAs but you might have agreed something more specific in your contract.

For templates, a good starting point would the the Cloud security alliance Cloud Controls Matrix (https://cloudsecurityalliance.org/research/cloud-controls-matrix) and the NIST compliance templates. 

As compliance requirements vary a lot depending on the country where your company provides services and per industry, consider also looking into more specific guidance like the ISO/IEC 27001 Toolkit, the GDPR Compliance Checklist or the PCI DSS Self-Assessment Questionnaire.