What are the most significant challenges that organizations face when it comes to managing third-party risks effectively?

not complying with the baseline security poses a huge risk.

I have been in TPRM process for more than 5 years and consistent challenge which I found is collaboration and response timelines from vendor contacts. 

Two important areas of focus in managing third-party risks are Service Continuity and Management Due Diligence. Each covers many individual aspects.

Service Continuity
In the Service Continuity area, the focus should be on measures that will ensure the services on which your organization depends will be available. What will your organization do in the event of a third-party service interruption or failure? 

Your contingency plans should be informed by the relative importance of the service, with critical services justifying more investment than non-critical services. In practical terms, this may mean procuring primary and secondary services from different providers, if the service permits. Operational testing of fail-over to secondary services must be routinely performed as well.

In the case of more complex third-party service arrangements, redundancy may not be possible, in which case much depends on the third party's management and operating effectiveness. Selecting the right third party and effectively managing their arrangement are key success factors.

Management Due Diligence

Management Due Diligence describes appropriate oversight and attention given to third party evaluation, selection, performance, and disposition. 

Evaluation and selection are heavily dependent on a clear understanding of your organization's requirements and tolerances and must include scrutiny of third-party management and operations. This will typically include review of audit certifications like SOC1/SOC2, ISO, or others. The third party's financial stability and market performance should also be evaluated to ensure reasonable expectation of their longevity and continued ability to provide the service. 

After a vendor has been selected and engaged, sufficient attention must be given to the vendor's performance over the life of the contract. Service failures must be regularly reviewed to ensure that risks of repeat failures are minimized, and that appropriate financial compensation is provided according to the terms of your agreements. 

Disposition, or the terminal phase of a third-party arrangement, is an often-overlooked aspect of third-party arrangements, but is a vital aspect of risk mitigation. The most important aspect of this phase is a practical plan for extraction of organizational data from the third-party to enable transition to an alternate service, where applicable. Without such a plan in place, your organization may become overly dependent on a third party, increasing associated risk and placing you at a strategic disadvantage.

Effective third-party management must also include sufficient legal protections for the organization in the event the third party engages in unethical or unlawful practices. Internal legal review of all third-party agreements and addenda should be a standard part of any engagement or renegotiation to address legal and reputational risk. 

Other Resources
ITIL Supplier Management - Supplier Management | IT Process Wiki (it-processmaps.com)
COBIT - Using COBIT to Govern and Manage Third-Party Risk (isaca.org)