Gartner Insights Abstract

利用模型护栏规范GenAI的行为和输出

Published: 11 July 2024

Summary

随着生成式人工智能模型的创造性和通用性日益提升,幻觉等非预期行为和输出成为了企业机构担忧的主要问题。数据和分析领导者应部署模型护栏,验证并矫正GenAI模型的输入和输出,提高模型的可靠性。

Included in Full Research

概述

主要发现
  • 生成式人工智能(GenAI)的创造性和通用性利弊兼具。一方面,这两个强大的特性使GenAI可以在某些场景生成亮眼的结果;另一方面,在GenAI的设计和实施中过分追求这两个特性,会增加模型的行为风险以及生成非预期结果(如出现幻觉)的可能性

  • 负责AI工作的数据和分析(D&A)领导者在将GenAI部署到生产环境时,普遍关注的一个问题是风险缓解,他们需要采取切实可行的方法来完成这一任务。

建议

负责管理GenAI计划的D&A领导者应:

  • 确定企业机构的GenAI解决方案在创造性和通用性两个方面的定位。适应业务部门的风险容忍度,在提升创造性/通用性与管理风险和保持控制之间取得最佳平衡。

  • 利用模型护栏等工具来平衡GenAI解决方案的创造性和通用性,并通过控制模型的输入和输出来降低其他风险。

Clients can log in to view the entire document.

To access these insights and much more, talk to Gartner today.

Fill out the form to connect with a specialist to learn how we can help you.

By clicking the "Continue" button, you are agreeing to the Gartner Terms of Use and Privacy Policy.

Access Insights

Already a Gartner client?

AskGartner: The AI-powered tool our clients use to find trusted Gartner insights, instantly.

AskGartner | Discover the new AI-powered tool to access trusted Gartner insights

Gartner research: Trusted insight for executives and their teams

What is Gartner research?

Gartner research, which includes in-depth proprietary studies, peer and industry best practices, trend analysis and quantitative modeling, enables us to offer innovative approaches that can help you drive stronger, more sustainable business performance.

Gartner research is unique, thanks to:

Independence and objectivity

Our independence as a research firm enables our experts to provide unbiased advice you can trust.

Actionable insights

Not only is Gartner research unbiased, it also contains key take-aways and recommendations for impactful next steps.

Proprietary methodologies

Our research practices and procedures distill large volumes of data into clear, precise recommendations.

Gartner research is just one of our many offerings.

We provide actionable, objective insight to help organizations make smarter, faster decisions to stay ahead of disruption and accelerate growth.

Tap into our experts

We offer one-on-one guidance tailored to your mission-critical priorities.

Pick the right tools and providers

We work with you to select the best-fit providers and tools, so you avoid the costly repercussions of a poor decision.

Create a network

Connect directly with peers to discuss common issues and initiatives and accelerate, validate and solidify your strategy.

© 2026 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form without Gartner's prior written permission. It consists of the opinions of Gartner's business and technology insights organization, which should not be construed as statements of fact. While the information contained in this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner business and technology insights may address legal and financial issues, Gartner does not provide legal or investment advice and its insights should not be construed or used as such. Your access and use of this publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its insights are produced independently by its business and technology insights organization without input or influence from any third party. For further information, see Guiding Principles on Independence and Objectivity. Gartner publications and other content may not be used as input into or for the training or development of generative artificial intelligence, machine learning, algorithms, software or related technologies.