Isaca AAIA ISACA Advanced in AI Audit (AAIA) Exam Practice Test

Data cleaning is a foundational task in the AI development lifecycle. The AAIA™ Study Guide identifies data quality—ensuring completeness, accuracy, consistency, and correctness—as critical to building effective and unbiased AI systems. Cleaning the data involves removing duplicates, correcting errors, addressing missing values, and standardizing formats.

“Data cleaning is a prerequisite for effective training and evaluation. Poor-quality data leads to inaccurate or misleading model outputs, increasing operational and ethical risks.”

While training (D) is essential, it must occur only after the data has been adequately prepared. Stratification (A) supports certain modeling approaches but is secondary to data integrity. Therefore, C is the most important task at the data-gathering stage.

Transparency in AI, especially in regulated sectors like finance, is best achieved by using explainability tools that interpret the internal workings and outputs of complex ML models. The AAIA™ Study Guide highlights model explainability as essential for both auditors and regulators to understand why certain decisions are made.

“Explainability tools allow auditors to trace predictions back to input features, helping verify fairness, logic, and compliance. These tools support transparency and trustworthiness in black-box models.”

While documentation (A) and reporting (C) are part of governance, D directly supports decision-level clarity. Dashboards (B) show results, not rationale.

The use of customer data in AI training presents a significant privacy risk, especially when the data is not properly anonymized or when consent has not been explicitly obtained. The AAIA™ Study Guide classifies the unauthorized or inadvertent disclosure of personally identifiable information (PII) as the most severe risk in such contexts.

“Training AI models on customer data without proper safeguards can result in the unintentional exposure of personal or sensitive information, leading to data breaches and compliance violations.”

While bias (B) and hallucinations (D) are important operational concerns, they do not pose the same regulatory and ethical severity as PII exposure. Therefore, A is the most critical risk.

The effectiveness of an AI-driven business process—such as categorizing customers for promotional campaigns—depends on how well it supports defined business objectives. The AAIA™ Study Guide recommends validating that AI methodology aligns with intended outcomes as part of performance auditing.

“Effectiveness is best measured by assessing whether the AI logic contributes meaningfully to business goals. Output alignment with organizational KPIs or campaign strategies provides clear evidence of functional success.”

Options A and D support operational resilience and quality assurance. Option C is a privacy technique, not directly tied to effectiveness validation. Thus, B is correct.

Using postal codes as a primary factor in credit scoring raises concerns of geographic and socioeconomic bias. Postal codes can serve as proxies for race, ethnicity, or income level—potentially violating fair lending laws and ethical guidelines.

“Auditors should flag models that rely on proxies for protected attributes. Postal codes are high-risk features that may inadvertently lead to redlining or other discriminatory practices.”

A, B, and C are more justifiable under fair lending guidelines. Thus, D presents the highest concern.

An AI acceptable use policy (AUP) defines how AI tools and technologies should be ethically and responsibly used within an organization. According to the AAIA™ Study Guide, the primary goal of an AUP is to prevent misuse and promote adherence to ethical, legal, and operational standards.

“An AI acceptable use policy provides governance over how AI tools may be used, especially regarding data handling, fairness, and prohibited uses. It aligns employee actions with organizational values and compliance requirements.”

Monitoring procedures (B), training (C), and taxonomy explanations (D) may be included in broader AI documentation, but the AUP’s core purpose is ethical usage governance.

When using generative AI tools during audit activities, the most critical concern is the reliability and appropriateness of the information being entered and processed. According to the AAIA™ Study Guide, auditors are accountable for ensuring that audit data is valid, confidential, and that generated outputs are factual and verifiable.

“IS auditors must evaluate whether the information entered into AI tools is reliable and appropriate for the audit context. Inputting sensitive or unverified data may lead to regulatory violations or audit inaccuracies.”

While hallucinations (C) and privacy notices (B) are important concerns, the primary auditor responsibility is to ensure that source data is accurate and suitable. Therefore, D is the correct response.

Ensuring that input data is appropriate and relevant (option D) is the most critical factor in preventing hallucinations—where generative models produce fabricated or misleading outputs. The AAIA™ Study Guide notes, “Generative models are highly sensitive to input data; inaccurate, irrelevant, or inappropriate inputs increase the likelihood of nonsensical or incorrect outputs.”

While bias detection, data quality audits, and anonymization are important, ensuring the relevance and suitability of input data is foundational for reliable generative AI performance.

The confusion matrix is a key performance evaluation tool in machine learning and AI auditing. According to the AAIA™ Study Guide, a confusion matrix presents detailed information about actual versus predicted classifications, allowing auditors to assess accuracy, precision, recall, and F1 scores.

“A confusion matrix reveals not just how often predictions are correct, but also the types of errors being made—false positives and false negatives—thereby providing a clear view of the model’s predictive reliability.”

Adversarial testing evaluates robustness, group analysis identifies bias across subgroups, and latency testing examines performance speed—not predictive accuracy. Thus, D is the most relevant for ensuring correct predictions.

The greatest challenge for IS auditors in evaluating the explainability of generative AI models is the changing nature of algorithms as AI continues to learn (option D). Generative AI models, especially those using advanced techniques like deep learning and reinforcement learning, often employ continuous or dynamic learning, which results in models that evolve over time. This adaptability can significantly hinder explainability because the logic, parameters, or decision pathways may shift with ongoing retraining or real-time learning.

The ISACA Advanced in AI Audit™ (AAIA™) Study Guide stresses that: “Continually learning AI systems present unique audit challenges, as their internal representations and reasoning can change after deployment, making it difficult to fully capture and explain the rationale for outputs at any given point.”

Other options, such as bias in input data or computational performance, are significant but do not pose as fundamental a challenge to explainability as a model whose internal workings can dynamically change.

Cross-validation testing is a statistical method used to assess how well a model generalizes to an independent data set. The AAIA™ Study Guide recommends this method as a best practice to fine-tune model accuracy and reduce both false positives and false negatives. It involves splitting the dataset into training and testing subsets multiple times to ensure model robustness.

“Cross-validation allows auditors and developers to identify overfitting and adjust model parameters to achieve better generalization and predictive accuracy, especially in fraud detection contexts.”

Regression testing (A) focuses on changes over time; substantive testing (C) is audit-specific but not model-focused. Benford’s Law (D) applies to numerical distributions but is not designed for optimizing ML models. Hence, B is the best approach.

The AAIA™ Study Guide defines the primary objective of AI governance as establishing structure and accountability for AI initiatives. This includes clearly assigning responsibilities across development, deployment, risk management, and auditing roles to ensure that AI is used responsibly and transparently.

“AI governance establishes the policies, roles, and oversight structures that guide the ethical and secure deployment of AI. Clear accountability helps prevent unauthorized use and ensures strategic alignment.”

Options A and C are essential components of governance but are not its core definition. Option D is a business outcome, not a governance goal. Thus, B is the most comprehensive and accurate objective.

Audit trails in AI systems serve to document the inputs, processes, and outputs of AI decisions, allowing stakeholders and auditors to trace how decisions were made. The AAIA™ Study Guide identifies transparency and traceability as the core functions of an AI audit trail.

“Maintaining audit logs is critical to explainability and accountability. It ensures that decisions made by AI systems can be reconstructed and assessed for accuracy, ethics, and legality.”

Although compliance (D) is an outcome and fairness (B) a goal, only A captures the foundational purpose of audit trails in AI systems.

Transparency in AI systems is a key requirement to ensure trust, accountability, and ethical compliance. According to the ISACA AAIA™ Study Guide under the "AI Governance and Risk Management" section, understanding the decision-making process of an AI system falls under the principle of explainability. Explainability refers to the degree to which an observer can understand the internal mechanics of an AI system and the rationale behind its outputs.

“Reviewing the explainability of AI outputs allows auditors and stakeholders to determine whether model decisions are interpretable and justifiable. High transparency means stakeholders can trace how and why a decision was made.”

While algorithm complexity and computational cost are technical considerations, they do not directly facilitate the audit of decision-making transparency. Similarly, training data diversity is essential for bias reduction but does not explain how decisions are derived. Therefore, option D is the most aligned with auditing transparency.

The AAIA™ Study Guide emphasizes that encoding categorical variables must preserve the semantic meaning and order of categories when relevant. The greatest risk occurs when ordinal data—such as customer rewards tiers—is treated as nominal through one-hot encoding, which removes the inherent order and may impair model learning.

“Improper encoding of ordinal variables as nominal can distort the model’s understanding of relationships, leading to inaccurate predictions or biased outcomes.”

Customer reward categories (economy < business < first class) have a natural order. One-hot encoding ignores this order, potentially degrading model accuracy. Other options represent nominal data and are appropriately encoded.

The AAIA™ Study Guide outlines the primary goal of auditing AI systems as ensuring that they operate ethically, transparently, and fairly. This includes identifying potential biases in decision-making and confirming that the rationale behind outcomes can be explained and understood.

“The key focus of an AI audit is to evaluate whether the system performs within the parameters of fairness, legality, and accountability. This involves testing for bias and ensuring decision-making transparency.”

Options B, C, and D refer to performance and usability concerns, which are secondary to the ethical and governance-focused purpose of AI audits.

The AAIA™ Study Guide emphasizes that the foundation of any high-performing and ethical AI system lies in the quality and integrity of its data. For high-risk AI systems, such as those used in healthcare, finance, or criminal justice, it is essential to base models on trustworthy data. This ensures reliable predictions, reduces bias, and mitigates risk.

“Trustworthy datasets are characterized by accuracy, completeness, consistency, and ethical sourcing. In high-risk AI applications, ensuring data quality at every stage is crucial to system reliability and compliance.”

While backups, user training, and MFA are important for security and operational resilience, they do not address the core challenge of ensuring model accuracy and fairness at the development and design phase. Therefore, option C is the most effective practice.

In predictive maintenance use cases—such as detecting turbine failure—the most critical concern is identifying as many actual failures as possible to prevent catastrophic events. The AAIA™ Study Guide emphasizes that in such high-risk scenarios, Recall is the most appropriate metric because it measures the proportion of true positives correctly identified.

“Recall is critical in scenarios where missing a positive instance (e.g., a failure) is costly or dangerous. It ensures that most real issues are caught by the model, even at the expense of some false positives.”

Precision measures correctness of positive predictions, specificity measures true negatives, and accuracy may be misleading if the data is imbalanced. Thus, D (Recall) is most appropriate.

Precision measures the proportion of true positives among all positive predictions. A low precision rate indicates a high rate of false positives. The AAIA™ Study Guide recommends using precision when the goal is to minimize incorrect positive alerts, which is especially relevant in fraud detection, cybersecurity, and classification models.

“Precision is the key metric when false positives have a significant operational cost. It provides insight into the model’s ability to avoid incorrect positive classifications.”

Accuracy and recall give broader insights, but only precision directly measures false positive risk. Completeness is not a standard ML metric.

Generative AI excels at synthesizing large datasets and technical documentation into understandable insights. The AAIA™ Study Guide recommends leveraging generative AI to identify domain-specific risks and control considerations by analyzing complex environments and correlating them with industry risk patterns.

“AI can assist auditors during planning by generating tailored risk profiles for technologies under review, helping prioritize audit focus and scoping.”

While summarizing interviews (D) and creating diagrams (B) are helpful, only C directly informs audit planning with actionable intelligence. A (separation of duties) is a later-stage control assessment.

Prompt engineering manipulation often involves disguising inappropriate questions in hypothetical or conditional formats, tricking the AI system into bypassing its filters. The AAIA™ Study Guide highlights this as a common attack vector in generative AI misuse.

“Adversaries may reframe queries as academic or theoretical scenarios to exploit AI models’ reasoning patterns, bypassing ethical constraints. Auditors should test for these circumvention techniques.”

Other methods (A, C, D) may work under specific conditions, but B is the most widely effective and documented prompt-engineering technique.

Including AI-specific disruption scenarios in the organization’s Disaster Recovery Plan (DRP) ensures preparedness for worst-case events affecting AI systems. The AAIA™ Study Guide recommends tailoring business continuity and DR strategies to cover model unavailability, data corruption, and AI-specific dependencies.

“AI disruptions can arise from unique causes such as model corruption, adversarial attacks, or drift. Integrating these into the DRP allows for effective, scenario-specific responses and minimizes downtime.”

Tabletop exercises (A) are valuable for preparedness but are less comprehensive than scenario planning. Kill chains (B) are security-specific. KRIs (C) aid in monitoring but don’t ensure recovery. Therefore, D offers the most robust mitigation.

Ensuring AI model resilience against external threats involves validating that the model is configured to resist attacks, such as adversarial inputs, data poisoning, or misuse. The AAIA™ Study Guide emphasizes configuration testing as a crucial control to simulate threat scenarios and assess robustness.

“Model configuration testing simulates real-world threat conditions to validate model resilience. This includes testing against adversarial attacks, input manipulation, and exposure of sensitive outputs.”

While access monitoring (C) and anonymization (A) reduce risks, they don’t actively validate model behavior under threat conditions. Therefore, D offers the most effective resilience measure.

Overfitting occurs when a model performs very well on training data but poorly on unseen data, indicating that the model has learned patterns specific to the training set rather than generalizing effectively. The AAIA™ Study Guide identifies overfitting as a common problem that impacts model reliability.

“Overfitting limits the model's applicability to real-world scenarios. It reflects excessive tailoring to the training data and poor performance on new, diverse inputs.”

Underfitting (A) would result in poor performance on both training and test data. Generalization (C) is the desired state, and bias (D) is a separate issue. Therefore, B is correct.

Allowing AI to autonomously generate code without human review introduces significant risks, including security vulnerabilities, logic errors, and noncompliance with organizational development standards. The AAIA™ Study Guide strongly advocates for human-in-the-loop oversight, particularly in automated development contexts.

“AI-assisted development must include manual code reviews to ensure functionality, compliance, and security. Autonomous code generation without validation increases the risk of introducing undetected flaws.”

While A, B, and C involve operational risks or inefficiencies, only D constitutes a direct breach of secure development life cycle principles.

K-means clustering is a widely used unsupervised learning algorithm. However, it is sensitive to outliers and assumes that features are on the same scale, which can distort clustering results if not properly normalized. According to the AAIA™ Study Guide, this sensitivity can impact model reliability and the meaningfulness of clusters.

“Auditors should assess whether proper data preprocessing (e.g., normalization, outlier removal) was applied in clustering models. K-means assumes Euclidean distances, making it prone to errors when features differ in scale or contain outliers.”

Therefore, C correctly identifies the key risk.

Unchecked AI outputs pose a significant risk because incorrect or biased results may propagate into decision-making processes. The AAIA™ Study Guide stresses that validating AI outputs for consistency, accuracy, and reasonableness is a critical responsibility of both operators and auditors.

“Auditors must verify that appropriate output validation mechanisms are in place. Failure to check outputs increases the likelihood of undetected errors influencing downstream processes or decisions.”

While cascading failures (C) are a consequence of unchecked outputs, B is the root issue. A and D present logistical and documentation risks but not the highest immediate impact.