Exam CT-AI Topic 3 Question 101 Discussion

Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularlyExploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data.
* Option A:"Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set."
* This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions.
* Option B:"Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model."
* Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias.
* Option C:"Acceptance testing should be used to make sure the algorithm is suitable for the customer.
The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present."
* Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias.
* Option D:"A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model."
* A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias.
* Bias Testing Methods:"AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias".
* Exploratory Data Analysis (EDA):"EDA helps uncover potential bias in training data through visualization and statistical analysis".
Analysis of the Answer Options:ISTQB CT-AI Syllabus References:Thus,Option A is the best choice for detecting bias in the loan applicant model.