Enterprise capabilities, essential for achieving strategic goals and operating requirements, are outlined in the Generative AI Architecture capability model. It includes six major categories, showcasing unique and supporting capabilities needed for effective generative AI implementation, with comprehensive documentation available in other architectures.
- GenAI Operations are the capabilities required to manage, deploy, and customize generative AI models for use within an enterprise. Included in this category are capabilities for training and tuning models, managing the lifecycle of models once deployed, and for managing models and datasets available to users within the enterprise.
- GenAI Application Development are the capabilities necessary to tune general foundation models for use in enterprise- and domain-specific solutions, and to develop full-feature generative AI applications. This includes capabilities necessary to create and deploy agentic AI applications, and test and tune prompts.
- GenAI Governance is a suite of capabilities required to effectively monitor and manage models deployed into production. These include capabilities to monitor models' continuing accurate and appropriate responses, capabilities to safeguard models from inappropriate and/or malicious inputs, and governance capabilities to manage enterprise risks and assist with both regulatory compliance and reporting requirements.
- GenAI Security Management focuses on the capabilities required to safeguard AI systems, ensuring their ethical and responsible deployment. This involves securing the entire AI stack, protecting both the models and their usage, as well as the data they rely on.
The remaining capability groups are supporting capabilities for generative AI. The capabilities are not unique to generative AI but must be present to support it as an enterprise capability. These groups are:
Data Management is a group of capabilities to store, manage, and transform data to forms that make it suitable for tuning and training of generative AI models. Also included in this category are capabilities to log and rate model responses for auditing purposes, and as input to further model tuning and refinement.
Supporting Capabilities is a catch-all grouping of application, integration, and IT operations capabilities required to successfully deploy and manage generative AI solutions with an enterprise.
GenAI Resources captures the hardware and platform capabilities necessary to efficiently and effectively develop, tune, deploy, and manage generative AI models and solutions.
Each capability category is made up of one or more capability groups. This section highlights groups and capabilities key to generative AI.
Model Hub capability group encapsulates the capabilities necessary to manage imported models as well as models tuned or trained by the enterprise. These capabilities enable enterprises to manage the models and data sets available for use within the enterprise, and to limit access to models and data sets to specific users or groups within the enterprise. Model importing and Data importing are key capabilities for enterprises to gate the intake of models from the growing number of public model repositories such as Hugging Face.
Model Hosting Model Hosting offers capabilities for deploying general and tuned models as API-enabled services within an enterprise, optimizing resource utilization, allowing independent refinement and replacement, and simplifying governance. Key to this is Model Access Policy Management, ensuring model access is restricted to authorized users and groups, preventing unauthorized usage.
Model Customization is a group of capabilities that enable an enterprise to tune and train generative AI models for specific business needs. Typically this capability will be realized using a cloud platform as the cloud's pay-as-you-go model is well-suited to the 'bursty' nature of tuning and training resource demands.
Model and Data Governance is a critical set of capabilities for an enterprise to make use of generative AI models on a wide scale. Specifically, these capabilities provide enterprises with the insights they need to monitor and manage model risks such as the introduction of bias in model responses, and to help address regulatory and compliance requirements for model transparency and fairness.
Model Monitoring is the operational analogue to Model Governance; where Model Governance deals with long-term model and risk management, the capabilities in Model Monitoring enable enterprises to monitor and management model operations in real time. Model Monitoring is comprised of several key capabilities, including:
- Bias Detection the ability to detect and flag when a model's responses deviate from established / ideal responses and begin to favor a set of outcomes over another.
- Hate, Abuse and Profanity (HAP) Detection is the ability to detect and filter hate, abuse, and profanity in both prompts submitted by users and in responses generated by the model. These are considered 'base' capabilities; enterprises will often choose to extend the list of filtered topics to include topics not appropriate to the business, eg. sexually suggestive topics in a lending office, or to accommodate the social norms of a target audience.
- Prompt Monitoring and Security is an emerging capability required to protect deployed models against attacks, such as prompt injection, designed to corrupt the model or to circumvent model controls established by the enterprise.
GenAI Compliance Management is a category of capabilities is about enabling the controls needed to “secure the usage” of AI through the application stack and “secure the applications” themselves. Adhering to ethical standards and guidelines to ensure that AI systems respect human values and rights.
- AI App Compliance. The capability to enable adherence of AI "applications" to established guidelines, regulations, and standards. Providing capabilities to enable controls such as: AI App Posture Management & Compliance, Conduct Trustworthy AI System Testing.
- AI Model Compliance is the capability to enable management and adherence of AI models to established guidelines, regulations, and standards. It enables key controls such as: Model Drift Tracking. Model Posture Management & Compliance.
- Legal and Compliance Management. Ensuring an organization stays current with the regulatory landscape (regulatory watch) and adheres to legal requirements, rules, and standards governing the development, deployment, monitoring, and use of AI (e.g. Ongoing Compliance & Regulatory Watch)
AI Application Security Management. This category of capabilities is about enabling the controls needed to “secure the usage” of AI through the application stack and “secure the applications” themselves. Adhering to ethical standards and guidelines to ensure that AI systems respect human values and rights.
- AI App Threat Management. and Protection is the ability to identify, assess, and mitigate potential risks and vulnerabilities that could impact the security, functionality, or reliability of an AI application and ensuring its ability to adapt, recover, and continue functioning effectively even in the face of unexpected events, failures, or disruptions. Also to ensure the robustness, security, and reliability of AI systems throughout their lifecycle.
- AI App Interaction Safety and Security is mechanisms to ensure that interactions between AI systems and their users, other systems, and the environment occur in a safe and secure manner. (e.g. Input manipulation, Restrict Excessive Queries, Prevent Toxic Result)
- AI App Safety. The capability and practices to ensure the safe and reliable operation of AI systems. It encompasses strategies to prevent unintended consequences, errors, and harm caused by AI applications (e.g. Document Trustworthy AI Effort, Fairness Assessment).
AI Model Security Management. This category of capabilities is about enabling the controls needed to “secure the model” layer as well as securing the usage of models. Implementing best practices for model training, validation, and evaluation to enhance performance and reliability.
- Prompt Safety & Security is the capability to ensure the input prompts provided to AI models are safe, non-malicious, and aligned with the intended behavior of the model and protect the model from unauthorized access, tampering, or attacks. Equally checking that response prompts are safe and secure in terms of content it may inadvertently reveal like PII (e.g. Prompt Injection Protection, Prevent Inference Attacks / Minimize Query Response, Prevent Toxic Results)
- AI Model Threat Detection is the ability to identify and mitigate potential risks or vulnerabilities that could compromise the integrity, security, or performance of the AI models and the interaction with those models (e.g. Model Security Testing, Model Vulnerability Mitigation)
- AI Model Access Management. Restricting access to Gen AI models, model parameters, training data, and APIs to prevent potential misuse and attack vectors. This capability establishes access controls around models that are exposed for enterprise usage, including for AI applications; secure internal model registries and limit internal access to production models. Such access controls should include contextual policies that factor in who, what, when, and from where.
AI Data Security Management. This category of capabilities enables controls to “secure the data” layer. Establishing clear guidelines for data collection, storage, and usage to ensure data quality and mitigate bias. While data security is not unique to GenAI, we'll focus only on areas where GenAI require particular attention from a data standpoint.
- Data Privacy and Confidentiality is the ability to safeguard sensitive information, ensuring it is handled appropriately and remains private & accessible only to authorized systems & users (e.g. Privacy Protections, Sensitive Data Security, Data Confidentiality)
Agentic AI is a group of capabilities required to create and deploy agentic AI applications. These include core capabilities like Routing and Orchestration, and Tool Management and Tool Calling.
GenAI Tuning is a group of capabilities necessary to 'customize' a general generative model to the needs of the enterprise. Models are trained on a broad base of knowledge and will lack knowledge of specific industry jargon and processes. Thus, most enterprises will need to make use of capabilities like Prompt Engineering, Prompt Tuning, and Model Fine-tuning to create a model that understands the terms and processes of the enterprise's business.
GenAI Application Capabilities enable enterprises to develop advanced generative AI applications. Capabilities include the ability to dynamically generate functions to respond to user queries; conversational memory, which enables generative AI applications to retain and reference prior interactions in a conversational manner; and model routing, which enables applications to dynamically route queries to a model best suited to respond.