We want to do a new advertising agency review and send out for RFP to a closed group of agencies without hiring an agency to help us find the right agencies to apply. Does anyone have suggestions on how to search for the right agency beyond the Google search?

Agree as well that personal references are the best. Outline what you are looking for first in terms of not just the capabilities but also any other preferences in terms of workstyle, geographical preference, response time, budget, etc. then ask people in your network that you trust. Also if you belong to a peer organization such as CMO Collaborative, you can post a note in their Slack channel asking others to share their recommendations.

I agree with Lucy. Referrals are the way to go here. I've often reached out to people I admire or past colleagues to get some names. Then, I do my own research to determine initial fit-- because 1) I don't want an agency to waste their time on preparing for an RFP they have no chance of winning AND 2) I don't want to spend my time reviewing more RFPs than from whom I think are great contenders. 

I'd say this is the time to work your network! Contact those in your professional network who have likely faced the same challenge and ask them if they've worked with an agency that they would recommend. You should probably tell them a little about what you are looking for--website redo, email marketing, rebranding, for example.

If you're in a major metro area, you might want to look at local business' advertising and then reach out and tell them that you love their advertising/messaging, and ask if they would mind sharing the name of the agency that does their creative.  

While the agency doesn't necessarily need to be local, it can be handy if you have product photo shoots, and other on-site assets that they will be creating. 

You may also be able to get referrals from business networking events--we have some very general ones in Chicago for marketing/advertising types, C-level, and other professional organizations. A referral (or an unhappy warning) from someone is way better than Google.