Gartner Research

Maverick* Research: Your Smart Machine Has Been Conned! Now What?

Published: 15 October 2015

ID: G00278261

Analyst(s): Patrick Hevesi, Anton Chuvakin, Erik Heidt

Summary

Smart machines and AI pose huge future risks that derive from malicious humans using or abusing them to achieve their goals. Here, we focus on identifying and reducing those risks. (Maverick research deliberately exposes unconventional thinking and may not agree with Gartner's official positions.)

Table Of Contents

Analysis

  • *Maverick Research
  • Definition of Smart Machines
  • Morality and Ethics in AI
  • Establishing, and Verifying, Trust Across People and Machines
  • Smart Machine Attack Taxonomy
    • Data Channels Are a Prime Avenue for Attack
    • Smart Machine Workflow Is Vulnerable
    • Smart Machine Logic Is Vulnerable
    • Smart Machine Morality Is Vulnerable
  • Trust in Smart Machines Versus Trust in Humans
  • Deceived-AI Threat Scenarios
    • Social Engineering Stock Market AI to Manipulate World Stocks for Financial Gain
    • Using Robotic Cars to Assassinate
    • Conning the Military's Medical AI Drones
  • Next-Generation Defenses Needed for Smart Machines
    • Controlling Smart Machine Input
    • Controlling Machine Output
    • Implementing Smart Machine Checks and Balances
    • Building Internal Smart Machine Constraints for Ethical and Safe Behavior
    • Designing Features That Protect Against Developer and Operator Malicious Intent
    • Detecting AI Tampering and Self-Tampering
    • Building Transparent and Auditable Smart Machines
  • Conclusion

Gartner Recommended Reading

©2020 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form without Gartner’s prior written permission. It consists of the opinions of Gartner’s research organization, which should not be construed as statements of fact. While the information contained in this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such. Your access and use of this publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its research is produced independently by its research organization without input or influence from any third party. For further information, see Guiding Principles on Independence and Objectivity.

Already have a Gartner Account?

Become a client

Learn how to access this content as a Gartner client.