How do you tackle implementing a new AWS Service? When the documentation is so vast that it really takes a while to read all, digest it & implement it?

I have mostly tried things out myself or asked someone on the team to try it out. We take it from there. Since most of the services have hourly or per second billing, it makes sense to do this. Video trainings works sometimes but not always and is very time consuming. Qwiklabs also helps to get a hands on training but mostly GCP and few AWS related courses on it.

Implementation of any new service in a product depends on the time given to achieve that feature. The methodology of learning & implementation depends upon time, skill etc. If we have enough time the developer can get into details and learn, however it may be (text, video, audio). But if we have less time then asking an expert to jump in for quick implementation helps. 

If I don't have the knowledge in-house yet
A) I try hiring consultants for the first round of implementation and have people (who will later own the application or platform) learn from them by doing it with them. The next round can then be done by the in-house team. 
B) In case I don't have/get budget for consultants, then I'd look for the kind of to-be owner Peter Webber has described.

Major task migration on-prem to AWS. Have a dedicated person to manage the project. You will also need SME's to handle DB migration, set-up Network and permissions, set-up monitoring systems and establish alerting mechanisms. Break down the project in smaller pieces and be open to trial and error. Have cut-over and roll-back plan in place. There will be a lot of learning involved fortunately AWS provides lots of documentation. Though initially overwhelming they almost always help reach solution. 
Good Luck 

Hire subject matter experts