If you've been piloting Microsoft Copilot, what advice would you give to other CIOs considering implementing Copilot?

You need to understand what Copilot is first of all. It's not one product. It is made up of multiple components including a lightweight Chat, M365 Copilot, Copilot Studio to create Agents, AI observability, out of the box purpose built agents etc. But irrespective of the tooling/product, I'd be thinking more about organisational preparedness to take advantage of AI (Copilot or other). Leadership and strategy (culture, use cases and ROI, AI strategy tied to business strategy etc) / Human and process change (Skilling, change management, feedback channels) / Business & Tech readiness (CoEs, data foundations, security/risk) / Governance (responsible AI, risk management, observability etc). This will be a business and process transformation more than a technological one.

Tools alone is not enough, it’s a quick win but after initial productivity gain getting value is difficult without process re-engineering

I second the comments made by Sean Burke.  A few additional comments for our organization.  

* Consider opt in if you're not prepared to pay $30/user/month and then monitor for usage. It's helping us manage by identifying who actually needs M365 Copilot vs general Microsoft Copilot. We've also required a brief virtual training to ensure people who've requested M365 learn more about the possible uses to increase the value.

*Educate your executive peers. Our AI team provided a hands-on training on AI/Gen AI. To help execs embrace the possibilities, they used Copilot as an easy entry point. It was well received and has generated support for funding Copilot and other AI.

*Legal and Compliance implications are evolving. One particular area of focus for public companies is related to open records acts requests. Our Legal department is drafting limitations to recording Teams meetings. This is one example where technology advances and AI are outpacing laws and policies. The result is a growing concern in how to comply with legislation, protect data/security, while using technology to support efficient operations. 

I'm seeing answers for two different AI tools in the ever-evolving ever-name-changing Microsoft AI Stack

"Microsoft Copilot" (formerly Bing Chat Enterprise) is what I call a 'web search on steroids' AI tool.  It's big benefit is that it keeps the data from your employees' interactions out of the model, so you don't have to worry about your company information appearing in someone else's dialog with the AI.  On the downside, it requires a bit more savvy on the user-side to wield it well.

Microsoft 365 Copilot is the 'suite' of Copilots for the Microsoft 365 stack--with Copilots for the usual Office apps, Teams, and Outlook.  They're in GA now, but in varying degrees of maturity.  M365 Copilot searches across all the data (that an individual has rights to) in your M365 tenant, as well as the Microsoft LLM and (optionally--you have to enable it tenant-wide) Internet sources.  Some key thinking for Piloting M365 Copilot (some of this has been mentioned):

   * Cost -- $30/user/month in small volumes, but can add up quickly.  An M365 E3 or E5 license is also required for each Copilot user.
   * Data Security -- Since M365 Copilot looks across the content in your tenant, having appropriate permissions on your data is VERY important, just as it is with the traditional MS Graph search. 
   * Legal/Compliance implications -- Copilot for Teams requires meeting recording and transcription to be enabled.  If your organization restricts that, you may not see full value from the Team Copilot
   * Latest version of the Office apps -- The M365 versions of all the Office apps are required (no MSI's), and the more aggressive release ring is recommended
   * Identifying business SME's/champions -- this is not something that IT can lead people through, unless you have a lot of solutioning/consulting capacity in your organization.  Identifying those early adopters in the business that can coach others is key.

My advice for CIOs would likely include thorough testing, evaluating compatibility with existing systems, assessing security implications, and considering the impact on developer workflows and productivity.