Published: 04 January 2023
The software engineering discipline transcends feature development. Software engineering leaders must balance the prioritization of new feature development with investments in digital immunity and resilience. These vendors offer products to help achieve this goal.
Software engineering leaders struggle to balance feature development and delivery of product resilience.
Software testing frequently omits validation of nonfunctional requirements in favor of functional correctness.
The triage and debugging of complex systems is difficult and further burdened by difficulties that many product teams are experiencing in hiring and laying off engineers.
Software engineering leaders who want to sharpen team rigor beyond yet more feature development must:
Support product teams through platform investments that support a digital immunity strategy.
Mitigate the software testing bottlenecks by using machine learning (ML) models to augment testing activities, such as predicting risk and authoring and maintaining the automated tests.
Leverage proactive automated verification and make system behavior visible by instrumenting observability, easing cognitive load on their teams.
In Gartner’s 2021 Software Engineering Leaders Survey, 46% of software engineering leaders identified balancing feature delivery, tech debt, bugs and risk as one of their top three areas of concern.Despite such concerns, it is common to hear these leaders emphasize acceleration over discipline. Without applying a level of rigor, short-term gains lead to a painful life span for digital products and the teams that own them. Digital immunity is a strategy for building reliable and resilient systems while mitigating the operational and security risks for the business.Digital immunity interlinks practices from the areas of observability, resilience, AI-augmented testing, chaos engineering, site reliability engineering and supply chain security of applications (for more information, see Figure 1 and ).This Cool Vendor report highlights technologies that assist in three practices within a digital immunity strategy.
San Mateo, California ()
Analysis: Mark O’Neill
Why Cool:Akita is unique because it enables API observability, even for APIs that your organization was unaware it had. To achieve this, Akita infers API endpoint structure based on live API traffic. Akita models API traffic to provide a fast and easy method for API discovery and monitoring. Generation of API models aids in automatic API monitoring, making it possible for software teams to quickly understand issues across every endpoint without making any code changes. Dashboards created by Akita allow organizations to take advantage of this functionality.
Challenges:
Akita’s product is based on extended Berkeley packet filter (eBPF). Akita monitors unencrypted HTTP traffic using BPF, which is not supported on platform-as-a-service systems like Heroku. Encrypted traffic further limits the supported platforms.
Akita does not yet have official Windows support for the Akita Agent, limiting the runtimes where this technology can be deployed to.
Akita has strong competition from API management solutions, which typically also include API monitoring solutions, and are widely used.
Competitors such as Optic also provide automated, traffic-based analysis of API behavior.
Who should care:Software engineering leaders who want to move quickly to ship features while providing their teams with an accessible, API-centric view of their production systems. The advantage of Akita is that organizations are not required to have a deep understanding of their own systems, or of APM tools. Therefore, it also is valuable for app developers, as well as app architects who require visibility of API flows.
San Francisco, CA ()
Why Cool: Sentry offers a shift-left approach to developer-focused application monitoring by focusing on the developer experience. Starting with its flagship error monitoring and error tracking that bypasses the cumbersome approach of log analysis, Sentry’s application performance monitoring (APM) platform enables development teams to maximize developer productivity and drive innovation through performance monitoring, error monitoring and code coverage, and offers developer-focused application monitoring and error tracking, delivered as SaaS. Achieving digital immunity involves employing resilient applications and the infrastructure that supports them. But resiliency requires capturing errors at all levels of the application stack, which can be difficult, whether they are generated by applications in front-end frameworks such as React or Vue, or in back-end applications created using Java, .NET or node.js. Sentry’s code-level observability is fundamentally different because it allows software engineers to focus on exceptions within the analysis and minimizes the noise of informational logging.
Challenges:
While Sentry offers APM capabilities, its performance capabilities are geared toward allowing developers to remediate issues in their code, and is distinct and limited in scope compared to traditional APM vendors.
Sentry does not work with commercial, off-the-shelf applications, so monitoring proprietary, closed-source workloads such as SAP and Oracle ERP is not possible.
Who Should Care: Software engineering leaders who want to improve error detection mechanisms and accelerate fixes with shift-left techniques should evaluate Sentry. Sentry’s product is particularly useful for identifying errors before a wide array of users is impacted.
San Francisco, California, U.S. ()
Analysis: Jim Scheibmeir
Why Cool: testRigor provides a new approach to test automation that allows nontechnical testers to describe a test case in plain English. testRigor will execute this test case, emulating how a human would do it. This capability is not bound by any format, such as the given-when-then format used in the behavior-driven development approach. Additionally, testRigor offers a JavaScript agent that can be included in web application deployments to track user journeys. These user journeys are then automatically converted into automated test cases that are based on real application usage. Regardless of the authoring technique, these capabilities ease test automation maintenance through robust identification algorithms, so that applications experiencing UI changes don’t lead to flaky tests. If quality engineers prefer to refactor tests from a coded approach, testRigor offers features that aid in refactoring the automation at the code level.
Challenges:
The test automation market is highly competitive and has established players that are both developing and acquiring new capabilities.
Additionally, the traditional buyer of a test automation tool — the leader of a quality assurance organization — is disappearing from many organizations, as the function of testing becomes distributed into product teams.
Open-source software (OSS) continues to be a major influence in the test automation market. When we last surveyed OSS adoption for test automation in 2019, 50% of those surveyed would select OSS test automation tools first. Inquiry about OSS for test automation has not slowed since that time.
Who Should Care: Software engineering leaders who want to improve their QA process should evaluate testRigor for its no-code, ML-driven test automation features. These capabilities accelerate test creation and reduce maintenance. Additionally, teams who rely on manual regression testing ought to consider testRigor to aid in the development life cycle and ensure that manual testers can participate in test authoring.
Fairfax, Virginia, U.S. ()
Analysis: Jim Scheibmeir
Why Cool: Verica transcends the chaos engineering practice into continuous verification. Continuous verification allows software engineering teams to optimize for availability and security, among other nonfunctional characteristics and benefits. The strengths of Verica are ideal for teams that are operating software on top of Kubernetes container orchestration, or who are using Kafka event store and streaming services. Verica utilizes experimentation to assure that configurations and deployments are optimized to meet business objectives.
Challenges:
Some engineering leaders and teams will consider continuous verification as a “nice to have” rather than a must.
Similarly, software engineering teams and leaders often look at chaos engineering and experimentation activities as requiring an exceptional level of maturity.
Within the market for chaos engineering, there are many available technologies and even more open-source solutions.
Who Should Care:
Software engineering leaders whose teams are accountable for distributed systems that run on Kubernetes or utilize Kafka streaming services should utilize Verica. Similarly, to grow talent and to help hire skilled engineers, software engineering leaders should budget for cool tools to support their teams and to build happy and brag-worthy teams to work in.
2021 Gartner Software Engineering Leaders Survey: This survey was conducted to understand the challenges and responsibilities of software engineering leaders. The research was conducted online from April through June 2021 among 314 respondents from North America (n = 155), Western Europe (n = 103) and Asia/Pacific (n = 56). Respondents were screened as responsible for at least one team of software engineers at organizations with over $20 million in worldwide revenue across organizations from all industries, except construction, natural resources, energy, some manufacturing subindustries, local or regional government, and wholesale.Disclaimer: Results of this survey do not represent global findings or the market as a whole, but reflect the sentiments of the respondents and companies surveyed.
Analysts:
Gartner research, which includes in-depth proprietary studies, peer and industry best practices, trend analysis and quantitative modeling, enables us to offer innovative approaches that can help you drive stronger, more sustainable business performance.
Our independence as a research firm enables our experts to provide unbiased advice you can trust.
Not only is Gartner research unbiased, it also contains key take-aways and recommendations for impactful next steps.
Our research practices and procedures distill large volumes of data into clear, precise recommendations.
We provide actionable, objective insight to help organizations make smarter, faster decisions to stay ahead of disruption and accelerate growth.
We offer one-on-one guidance tailored to your mission-critical priorities.
We work with you to select the best-fit providers and tools, so you avoid the costly repercussions of a poor decision.
Connect directly with peers to discuss common issues and initiatives and accelerate, validate and solidify your strategy.
Gartner clients can log in to access the full library.
Join your peers for the unveiling of the latest insights at Gartner conferences.
©2022 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form without Gartner’s prior written permission. It consists of the opinions of Gartner’s research organization, which should not be construed as statements of fact. While the information contained in this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such. Your access and use of this publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its research is produced independently by its research organization without input or influence from any third party. For further information, see Guiding Principles on Independence and Objectivity.
