What lessons have you learned from the GenAI “boom” so far? What challenges did you encounter during initial adoption, and what early successes did you achieve?

A challenge we've encountered is ensuring developers understand the non-deterministic nature of AI. While AI systems may perform as expected most of the time, they can occasionally produce unexpected results. It's important for our team to delve deeper into understanding AI to avoid misuse or misunderstandings.

Maintaining business context across multiple queries is crucial. The way prompts are written must capture this context, especially when dealing with sequential questions. This is particularly important for use cases like Robo Advisors in finance, where responses must consider prior interactions. Addressing this challenge will be key as we develop more tools in 2025.

One key lesson was the need to update data policies and governance. GenAI's role extends beyond data engineers, affecting everyone involved in production. This challenged us to rethink data policies from an engineering perspective. We also had to revisit procurement processes with vendors like Microsoft, as GenAI is now integrated into many products. This required us to renegotiate contracts and ensure data interactions are secure, without risking the company's brand.

Generative AI has significantly disrupted traditional development processes, shifting the mindset from writing code from scratch to optimizing baseline constructs. Initially, we faced security concerns and negative press, which made us hesitant to pilot GenAI. However, as Microsoft Copilot improved controls and trust, we began to define specific use cases for GenAI. A major challenge was prompt engineering; a simple change in a prompt can drastically alter the response. Training engineers to construct effective prompts was crucial. While productivity gains are evident, costs remain high, and until AI becomes an integrated service rather than an add-on, adoption may be limited.