A. INCORRECT: This answer contradicts section 1.1 by suggesting prototypes be nearly finished solutions early in the process.
B. INCORRECT: This answer contradicts section 1.1 by suggesting the process is not iterative.
C. INCORRECT: This answer contradicts section 1.1 by suggesting the design team does not need to engage the users. It is rarely the case where the team has the necessary experiences and other considerations to substitute for actual users.
D. CORRECT: Exploring a variety of solution alternatives is an attribute of the design thinking process as described in section 1.1.

A. CORRECT: This answer comes from the glossary.
B. INCORRECT: This is a fictional definition.
C. INCORRECT: This is a fictional definition has somewhat believable qualities to it, but it is inferior to the actual definition provided as an option to this question.
D. INCORRECT: This fictional definition is a contradiction to the correct definition.

A. INCORRECT: This is a business model example.
B. INCORRECT: This is an example of business strategy development.
C. INCORRECT: This is an example of commercial service development.
D. CORRECT: Although this is a product development project, the intended use is a social service

A. CORRECT: Section 1.3 mentions the number of stages among several methodologies and they are different.
B. INCORRECT: Section 1.3 mentions that there is a lack of research that would cause one to avoid a methodology. Limited resources and educational value are the primary reasons more methodologies were not mentioned.
C. INCORRECT: This is a non-difference. Any method not using divergent thinking is not a design thinking method as per section 1.3 and 1.1 (as referenced in 1.3).
D. INCORRECT: This is a non-difference. Any method not consulting with end-users is not a design thinking method as per section 1.3 and 1.1 (as referenced in 1.3).

A. INCORRECT: This example appears to show that the manager is imposing his/her will and is not being mindful by stepping out. This contradicts the need for mindfulness and team members being equals.
B. INCORRECT: Section 1.4 mentions that team members are flexible, brave, imaginative and expansive in their thinking. This example does not exhibit those attributes.
C. INCORRECT: This choice has two significant issues. The first is brainstorming over the phone. The other is verbal communication instead of visual. There is a lack of presence, which affects their mindfulness. This choice contradicts section 1.4.
D. CORRECT: In section 1.4, it is mentioned that design is an emotional process. This an example of the team wanting to know the emotional influence the prototype is having.

A. CORRECT: This answer is essentially the sentence in section 2.3.1 “With the world changing so quickly, an “ideal” methodology would not be relevant long enough to be established.”
B. INCORRECT: This contradicts the finding in section 2.3.1.
C. INCORRECT: Per section 2.3.1, design processes have been adapted to the business not the other way around.
D. INCORRECT: There is no such finding suggested in section 2.3.1. Design Council does not suggest that Double Diamond is the best approach to design thinking.

A. INCORRECT: Sociology definition for “social fact” as provided by sociologydictionary.org. Social norm is contained within social fact, and so it is broader in scope than social norm.
B. CORRECT: This definition comes from the glossary.
C. INCORRECT: Sociology definition for “social control” as provided by sociologydictionary.org. Relevant to social norms in terms of this concept relates to their enforcement.
D. INCORRECT: Sociology definition for “social action” as provided by sociologydictionary.org. Relevant to social norms in terms of this concept relates to a person being aware of social norms and seeking to conform to them.

A. INCORRECT: This choice is a combination of the importance of trust and incorrectly asserts that convergent thinking is needed for trust.
B. CORRECT: Section 2.1.1 discusses the importance of trust and the relationship between trust and empathy. This statement consolidates some of those ideas.
C. INCORRECT: This choice is a combination of the importance of trust and incorrectly asserts that visual thinking is needed for trust.
D. INCORRECT: This choice is a combination of the importance of trust and incorrectly asserts that assumption testing is needed for trust.

A. INCORRECT: This is an example of brainstorming and empathy. This particularly activity is not using ethnography.
B. INCORRECT: This is prototyping and empathy, but this activity does not involve ethnography.
C. INCORRECT: The data the team analyzes in part came from ethnographic research, but that research phase transitioned once the findings were established.
D. CORRECT: The process of interviewing a stakeholder is an ethnography exercise. Most opportunities to engage stakeholders for the purposes of understanding a problem and its viable solutions can be considered ethnography. See section 2.1.2.

A. INCORRECT: This choice suggests that ethnography is divergent thinking.
B. INCORRECT: This choice suggests that divergent thinking is convergent thinking.
C. CORRECT
D. INCORRECT: This choice associates the meaning of empathy with divergent thinking.

A. CORRECT: The process of reducing the number of prototypes will require analysis and possible blending of compatible features prior to establishing the final prototype. This is consistent with section 2.1.4.
B. INCORRECT: This is not an example of convergent thinking. This is an example of ethnography.
C. INCORRECT: This is an example of ethnography.
D. INCORRECT: This is an example of divergent thinking.

A. CORRECT: The team is using visual thinking to construct the storyboard visuals. This is consistent with section 2.1.5.
B. INCORRECT: Although the data may be tabulated and graphed the primary mode of communication is language and numerically based, since the analytical challenge is an assumption test.
C. INCORRECT: This choice associates an aspect of ethnography with visual thinking.
D. INCORRECT: Visual thinking may be used but is not a dominant activity in this scenario

A. CORRECT: This choice is consistent with section 2.1.6.
B. INCORRECT: This choice suggests that ignorance from the lack user-centered design practices is risky. Assumption testing requires that an unfounded notion exists prior to testing (testing known “knowns”). Key difference between this choice and the right one is to being uninformed vs. mistaken. In this context the team would be avoiding unknown unknowns, which requires ethnography among other tools to address.
C. INCORRECT: This choice is associating the purpose of convergent thinking with assumption testing.
D. INCORRECT: This choice associates the purpose of divergent thinking with assumption testing.

A. CORRECT: This answer is consistent with section 2.1.7.
B. INCORRECT: This choice associates the purpose of empathy with the term prototyping. Prototypes will elicit feelings, but it requires empathy to interpret them.
C. INCORRECT: This choice associates the purpose of assumption testing with prototyping. Prototypes enable assumptions to be tested, but they do not in themselves collect and analyze data relevant to determining if a hypothesis has been disproved or not.
D. INCORRECT: This choice associates a phase within ethnography with prototyping.

A. INCORRECT: This choice stops learning short and suggests that validation is achieved at the end of the design thinking process.
B. CORRECT: This is a paraphrase of section 2.1.8.
C. INCORRECT: This choice addresses learning properly, but the degree of validation achieved is over stated.
D. INCORRECT: This choice stops learning short and suggests that a soft launch will result in market validation, which is false.

A. INCORRECT: This choice is weaker because relative to the correct answer this person is not as committed to the effort nor is this person specifically creative or works well with other people.
B. CORRECT: This a paraphrase of a selection of sentences within 2.2.1 that describe good people to have on the team.
C. INCORRECT: This choice is weaker relative to the right answer because this person does not have particularly good people skills and doesn’t listen as well. One subtle weakness is the explicit lack of creativity. An expert in a field has likely developed analytical skills in order to achieve expert status, but creativity may not be cultivated within some fields.
D. INCORRECT: This choice is relatively weaker than the correct answer. This person lacks an area of expertise. Another person is needed or no members are experts. Familiarity with several fields is a bonus, but expertise may be necessary if the field is critical to the project. A good listener is important, but effective communication is needed to share ideas and work with stakeholders.

A. CORRECT: Design thinking involves a number of different styles of activity. A flexible space is more likely to accommodate the team’s current needs. Important artifacts should always be accessible in order to utilize them.
B. INCORRECT: The designation of zones structures the room, which limits flexibility. This structure could interfere with efforts. Having the important artifacts visible is good, but designating them to a “gallery” prevents them from being organic inputs to current efforts.
C. INCORRECT: This space has no where to sit and no place to set tools. This
space would be hard to work in for long periods of time. Some activates like
brainstorming would be uncomfortable to perform here.

C :INCORRECT: This bull pen serves conflicting needs. Team members needing time away from group activity or small groups for unstructured discussion will interfere with those not engaged. The personal workspaces will restrict space flexibility. Artifacts not present are out of sight out of mind.

A. CORRECT: This answer is consistent with section 2.2.3.
B. INCORRECT: Fun and exciting are nice. However, if they interfere with thought processes or the outputs are somehow valued beyond their actual contribution to the design process, then these tools and materials should only be used when a specific objective dictates.
C. INCORRECT: Approved requests are a necessity for every choice, but what isn’t clear is whether they are fit for purpose. This ambiguity weakens this answer.
D. INCORRECT: This choice positions style over substance.

A. CORRECT: This answer is a paraphrase of content in section 2.2.4.
B. INCORRECT: The section does not suggest a firm with cross-functional collaboration will swiftly reassign resources.
C. INCORRECT: The first sentence is fairly consistent with the syllabus. The second suggests the business will change to accommodate any result. This is not consistent with the syllabus.
D. INCORRECT: The syllabus does not suggest a firm as described would actively thwart design thinking. As a point of inconsistency, the second sentence suggests that the business is willing to change in order to accommodate any innovation. Even though the group which made it extremely difficult to function is willing to embrace any result?

A.CORRECT: This answer is consistent with section 2.3.2.

B. INCORRECT: This is the output of Deliver.

C. INCORRECT: This is an output from Designing for Growth.

D. INCORRECT: This is an output described by Lewrick et. al.

A. CORRECT: This answer is consistent with section 2.3.3.
B. INCORRECT: This choice contradicts their “Show don’t tell” recommendation.
C. INCORRECT: This choice contradicts their suggestion to act instead of analyze.
D. INCORRECT: This choice contradicts their suggestion that the team be mindful of the process and work towards the goal of the current stage instead the anarchy implied in the choice.

A. INCORRECT: Level of convergent thinking is less.
B. INCORRECT: Level of convergent thinking is less.
C. CORRECT: This answer is consistent with section 2.3.4. Intensity of convergent thinking is symmetric or consistent with the preceding divergent thinking effort. A lot of work is involved in processing all of the divergent thinking results into a manageable collection of enriched concepts.
D. INCORRECT: Level of convergent thinking is less.

A. CORRECT: This answer is consistent with section 2.3.5.

B. INCORRECT: This choice is wrong in two ways. Neither methodologies are linear. Design thinking tasks, durations and resources are only partly known in advance and much less so than in Agile software development.

C. INCORRECT: This choice is wrong in three ways. Neither method is cautious in pace nor would describing iterative progress as circular (helical, maybe). Design thinking is not less risky.

D. INCORRECT: This choice is wrong in two ways. The pace of the methodologies is not deliberate, and the processes are not linear.

A. INCORRECT: This answer contradicts the glossary definition by seeking vulnerability to as well as opportunity over the competition.
B. INCORRECT: This answer is essentially the opposite of the correct answer. This might be called “competitive disadvantage”.
C. CORRECT: This definition comes from the glossary.
D. INCORRECT: This answer replaces advantage over competitors with advantage over partners. Partners are not commonly considered competition.

A. CORRECT: This answer is consistent with section 3.2.1.
B. INCORRECT: Assumption testing is not performed in What if.
C. INCORRECT: Rapid prototyping is not performed in What if.
D. INCORRECT: Mind mapping and assumption testing are not performed in What if.

A. CORRECT: This answer is consistent with section 3.2.2.
B. INCORRECT: This answer flips the relationship described in section 3.2.2.
C. INCORRECT: This answer flips the relationship described in section 3.2.2.
D. INCORRECT: This answer flips the relationship described in section 3.2.2.

A. INCORRECT: This answer has too few stakeholders.
B. INCORRECT: This choice has too many participants.
C. INCORRECT: This choice has too few participants. It suggests the person helping with correcting the team’s understanding will be formally interviewed despite their previous help with providing feedback.
D. CORRECT: This answer is consistent with section 3.2.3

A. INCORRECT: This choice should not be referencing What works.
B. CORRECT: This answer is consistent with section 3.2.4.
C. INCORRECT: This choice should not be referencing What if.
D. INCORRECT: This choice should not be including What if.

A. INCORRECT: This tool requires folks beyond the core team. The tool is convergent not divergent.
B. INCORRECT: This answer suggests the consultants are a crucial element, but this is not suggested in the syllabus. The “design criteria” document is specified in 3.2.1 as a document not describing what is to be built. This document is the primary outcome of this tool.
C. INCORRECT: This answer suggests divergent thinking but describes a convergent outcome. The output of solution concepts is premature. This occurs after brainstorming in the What if stage.
D. CORRECT: This tool requires folks beyond the core team. The tool is convergent not divergent.

A. CORRECT: This answer is consistent with section 3.2.6.
B. INCORRECT: This answer suggests that the activity is unguided. This is not the case, in fact, considerable energy is spent prior to and during the activity to provide guidance. The size of the group is much too large.
C. INCORRECT: This answer suggests this activity is a form of convergent thinking and will yield prototypes instead of a large collection of ideas.
D. INCORRECT: This answer suggests that this activity is unguided. It also suggests only the core team is involved.

A. CORRECT: This answer is consistent with section 3.2.7.
B. INCORRECT: Too many solution concepts to construct and manage. There will not be enough time or money to pursue more than one concept at the end.
C. INCORRECT: Too many initial solution concepts to construct. Consolidating to one promising concept limits the team’s to explore other viable options with stakeholders. Stakeholders have no real influence over what solution concept will be pursued, which contradicts the notion that the collective stakeholder opinion counts most when making design commitments.
D. INCORRECT: Too few initial concepts to export at the beginning. Consolidating to one promising concept limits the team’s to explore other viable options with stakeholders. Stakeholders have no real influence over what solution concept will be pursued, which contradicts the notion that the collective stakeholder opinion counts most when making design commitments.

A. INCORRECT: This choice should not be referencing What if, and it is missing What works.

B. INCORRECT: This choice should not be referencing What if and is missing What wows.

C. CORRECT: This answer is consistent with section 3.2.8.

D. INCORRECT: This choice should not be including What if, and it is missing What wows and What works.

A. INCORRECT: Quality control is not the objective of rapid prototyping.
B. CORRECT: This answer is consistent with section 3.2.9.
C. INCORRECT: Manufacturing engineering is not the objective of rapid prototyping.
D. INCORRECT: Broad marketing the solution is likely contradictory given the sensitivity of innovation, but marketing to investors may be desirable. However, this is not the objective of rapid prototyping.

A. INCORRECT: Only one stakeholder should be involved. The level of documentation occurring is low given the number stakeholders and the challenges of facilitating them.

B. INCORRECT: Only one stakeholder should be involved. Documentation will be better having someone dedicated to it. Multiple unmanaged stakeholders may prove to challenging to document. The session may or may not accomplish its objectives without facilitation. Chances are high the objectives will not be accomplished.

C. INCORRECT: A stakeholder advocate is not documenting but acting as a buffer, interpreter and support for the stakeholder. This role should not be necessary in a cooperative explorative effort with able adults who speak the same language as the team members. Disabilities and age may require this role, but this should cause the group to grow to 4 people and not be introduced at the expense of documentation.

D. CORRECT: As section 3.2.10 describes, the team should be prepared to facilitate and document. Facilitating is an intense effort that can make accurate documentation difficult to perform. It is common to have a second person document

A. INCORRECT: The design criteria document is not relevant to the learning launch.
B. INCORRECT: Being of high fidelity is important, but if does not reflect stakeholder insights recent to the current iteration, the risk of value test failure is higher. The napkin pitch is not relevant to the learning launch.
C. INCORRECT: The quality of the prototype is too low for learning launch. The design brief is not relevant to the learning launch.
D. CORRECT: This answer is consistent with section 3.2.11.

A. CORRECT: This answer is consistent with sections 3.1.1and 3.2.5. The team is at the end of the What is stage. They need to use mind mapping to synthesis and hone in on key information they uncovered. The resulting design criteria will be a useful document going forward.
B. INCORRECT: What if is the wrong stage. “Value chain analysis” is the wrong tool.
C. INCORRECT: What wows is the wrong stage. Visualization is the wrong tool.
D. INCORRECT: What works is the wrong stage. Brainstorming is the wrong tool.

A. CORRECT: This answer is consistent with sections 3.1.2, 3.2.7 and 3.1.3. The team wanted to be in What wows, but got ahead of themselves. They did not perform concept development, which would have structured the ideas sufficiently for prototyping. They skipped assumption testing as well, but that is not worth doing until the concepts are formed. The team needs to go back to What if to complete concept development.

B. INCORRECT: The team wanted to be in What wows, but got ahead of themselves. They skipped assumption testing as well, but that is not worth doing until the concepts are formed. The team needs to go back to What if to complete concept development not staying in What wows.

C. INCORRECT: The team did not want to be in What if. They did not perform concept development, which would have structured the ideas sufficiently for prototyping. Mind mapping in this situation is not relevant. The team does not need to go to What is.

D. INCORRECT: The team did not want to be in What is. They did not perform concept development, which would have structured the ideas sufficiently for prototyping. Journey mapping in this situation is not relevant. The team does not need to go to What works.

A. INCORRECT: The stage where prototyping is performed is What wows not in What works. Section 3.1.2 mentions the sections of the napkin pitch. The word ‘method’ is a synonym for ‘approach.’ The students missed an approach entry while they were crafting the prototype. Needs is the wrong section.

B. INCORRECT: The stage where prototyping is performed is What wows. Section
3.1.2 mentions the sections of the napkin pitch. The word ‘method’ is a synonym for ‘approach.’ The students missed an approach entry while they were crafting the prototype. Benefit is the wrong section.

C. INCORRECT: The stage where prototyping is performed is What wows. Section 3.1.2 mentions the sections of the napkin pitch. The word ‘method’ is a synonym for ‘approach.’ The students missed an approach entry while they were crafting the prototype. Competition is the wrong section.

D. CORRECT: This answer is consistent with sections 3.1.2 and 3.1.3. The stage where prototyping is performed is What wows. Section 3.1.2 mentions the sections of the napkin pitch. The word ‘method’ is a synonym for ‘approach.’ The students missed an approach entry while they were crafting the prototype.

A. INCORRECT: The current stage is What works. The document essential for maintaining outstanding assumptions and hypotheses is the learning guide, not the design criteria. Despite how practical this exercise is, this is not a learning launch. Restaurant customers are involved but they have been enlisted to collaborate with the design effort.

B. CORRECT: This answer is consistent with sections 3.1.4 and 3.2.10. The current stage is What works. The document essential for maintaining outstanding assumptions and hypotheses is the learning guide. Despite how practical this exercise is, this is customer co-creation. Restaurant customers are involved but they have been enlisted to collaborate with the design effort.

C. INCORRECT: The current stage is not What wows. The document essential for maintaining outstanding assumptions and hypotheses is the learning guide, not the design criteria. Despite how practical this exercise is, this is customer co-creation. Restaurant customers are involved but they have been enlisted to collaborate with the design effort.

D. INCORRECT: The current stage is not What wows. The document essential for maintaining outstanding assumptions and hypotheses is the learning guide, not the napkin pitch. This exercise is not assumption testing.

A. INCORRECT: At least from the company’s perspective, this recommendation is based on an untested assumption. Bicycles and scooters are simply different. The differences are sufficient to question the validity of the analogy.

B. INCORRECT: Having a recognized brand will help with initial interest. However it will not address weak execution and poor value. Market advantage is only advantage if the differentiators are of value to the end user. Beating the competition in something no one wants is not much of a win.

C. INCORRECT: The effort showed that important assumptions were not reliable. This lesson is worth learning and avoiding going forward. What will the commercial design team build instead? Based on what understanding? What untested assumptions will the commercial design team make? Design is a constant skirmish between requirements and priorities. Giving up before the battle starts makes the design thinking effort rather pointless.

D. CORRECT: The recommendation is consistent with 3.3.1 paragraph 2. Personal Transport Devices management should seriously consider assumptions that were proven to be weak. They may be risks for the London effort. Those features, qualities and insights that are common and different should help product management spot where city specific design changes may be necessary.

A.Incorrect. This is the definition of an A/B test.

B.Correct. This is part of the Glossary definition of a cluster.

C.Incorrect. This is part of the definition of correlation.

D. Incorrect. This is the definition actuators

A. Incorrect. The perceptron was proposed in 1958.
B. Incorrect. The perceptron was proposed in 1958.
C. Incorrect. The perceptron was proposed in 1958.
D. Correct. The perceptron was proposed in 1958.

A.Incorrect. Fairness and discrimination are ethical issues in AI. See Section 1.6 of the syllabus.

B.Correct. This is a technical problem. See Section 1.6 of the syllabus.

C.Incorrect. Transparency is an ethical issue in AI. See Section 1.6 of the syllabus.

D.Incorrect. Reinforcement of existing bias is an ethical issue in AI. See Section 1.6 of the syllabus.

A.Incorrect. Measuring accuracy is not the goal of the Turing test.

B.Incorrect. The Turing test is limited to text message (not an oral conversation).

C.Correct. See syllabus – section 1.4.

D.Incorrect. Measuring the IQ is not the goal of the Turing test.

1. Supervised learning uses reward and punishment mechanisms to organize the learning.

FALSE – This is true for reinforcement learning as described in the syllabus (1.4.1).

2. In reinforcement learning, the machine learning model interacts with the environment.

TRUE – In reinforcement learning, the machine learning model interacts with the environment,

as per syllabus (1.4.1).

3. Unsupervised learning use labelled data during training.

FALSE – This is true for supervised learning as described in the syllabus (1.4.1).

4. Semi-supervised learning uses unlabeled data during training and labeled data during validation.FALSE – Semi-supervised Learning combines both supervised and unsupervised learning by using

   a set of labelled and unlabeled data for training (see syllabus 1.4.1).

   Thus, option B is CORRECT.

A.Incorrect. Fairness and discrimination are ethical issues in AI. See Section 1.6 of the syllabus.

B.Correct. Data understanding, preparation and cleaning are essential activities of ML projects: poor data lead to poor results. See Section 1.6 of the syllabus.

C. Incorrect. The configuration of hyperparameters is part of the implementation of the ML model so this is done in this case. See Section 1.6 of the syllabus.

D.Incorrect. Unit testing of the whole system is not the first priority here because the focus of the question is on the ML component. See Section 1.6 of the syllabus.

A.Incorrect. There are not quantified variables in propositional logic (see syllabus – section 1.3.1).

B.Correct. See syllabus – section 1.3.1.

C.Incorrect. Fuzzy logic uses an infinite number of truth values (see syllabus – section 1.3.1).

D.Incorrect. Predicate logic is generally undecidable (see syllabus – section 1.3.1).

D is CORRECT – see syllabus – section 1.4.3

A.Incorrect. This sentence is true (see syllabus – section 1.5.1).

B.Correct. Bayesian networks are trained with labelled data (supervised learning).

C.Incorrect. This sentence is true (see syllabus – section 1.5.1).

D.Incorrect. This sentence is true (see syllabus – section 1.5.1).

A.Correct. See syllabus – section 1.5.3.

B.Incorrect. This is correct for the Naive Bayes classifier not the SVM algorithm.

C.Incorrect. This is correct for the perceptron not the SVM algorithm.

D.Incorrect. This is correct for the K-Means algorithm not the SVM algorithm.

Using the Naïve Bayes algorithm, we can compute:

 P(Mary’s BMI category is 18.5-24.9) ∝ 50% * 30% * 60% = 9 %

 P(Mary’s BMI category is 25 – 29.9) ∝ 30% *40% * 30% = 3.6 %

 P(Mary’s BMI category is 30 – 34.9) ∝ 20% * 30% * 10% = 0.6 %

where ∝ denotes proportionality.

Thus, option D is correct.

For vector A, w0 + x1*w1 + x2*w2 = 10 + -10*2 + 10*3 = 20, so y=1

For vector B, w0 + x1*w1 + x2*w2 = 10 + -10*1 + 10*0 = 0, so y=1

Thus, option A is correct.

A.Incorrect. Per section 2.4, these are sensors.

B.Correct. These are actuators per section 2.4

C.Incorrect. Per section 2.4, these are sensors.

D.Incorrect. Per section 2.4, this is knowledge.

A.Incorrect. This would simply be a limited software system. See Section 2.1 of the syllabus.

B.Incorrect. This is a non-testable program per section 2.1.

C.Correct. This is the logical opposite of the definition of a deterministic system in section 2.1.

D.Incorrect. This is a deterministic system per section 2.1.

A.Correct. The act of training on data makes it less practical to determine the test oracle, i.e. expected results of the system, in advance. See Section 2.1 of the syllabus.

B.Correct. Probabilistic and non-deterministic systems have a weaker test oracle. See Section 2.1 of the syllabus.

C.Incorrect. The existing of multiple agents does not directly make it difficult to specify a test oracle. See Section 2.1 of the syllabus.

D.Incorrect. Testing machine learning does not require an understanding of the mathematics of a specific model. In the same way that it is not necessary to understand software code to test it. See Section 2.2 of the syllabus.

E.Incorrect. The bias/variance trade-off does not directly make it difficult to design tests. See Section

2.1 of the syllabus.

A.Incorrect. Whilst it is difficult to justify defects without a strong test oracle, it does not affect the reporter’s ability to identify the correct component. See Section 2.1 of the syllabus.

B.Incorrect. A weak test basis or oracle does not normally make it difficult to understand test levels. See Section 2.1 of the syllabus.

C.Incorrect. A weak test basis or oracle does not directly cause architectural complexity. See Section 2.1 of the syllabus.

D.Correct. It is difficult to complete test design without being able to specify the expected result. See Section 2.1 of the syllabus.

A.Correct. This is the definition of probabilistic system as described in section 2.1.

B.Incorrect. Whilst machine learning systems are often probabilistic, not all probabilistic systems use machine learning. See Section 2.1 of the syllabus.

C.Incorrect. Probabilistic refers to our ability to describe the expected behavior of a system, not how it works. See Section 2.1 of the syllabus.

D.Incorrect. Probabilistic refers to our ability to describe the expected behavior of a system, not how it works. See Section 2.1 of the syllabus

A.Incorrect. The data selected to train the models can affect the accuracy. See Section 2.2 of the syllabus.

B.Incorrect. If the relationship between the inputs and outputs changes since the model was training, this can lead to defects, or incorrect behavior. See Section 2.2 of the syllabus.

C.Incorrect. Data that is labelled incorrectly will lead to inaccurate models. See Section 2.2 of the syllabus.

D. Correct. Self-optimization occurs after training. See Section 2.2 of the syllabus

A.Incorrect. Labeling data with predictions is a supervised machine learning technique as explained in Section 2.3.

B.Incorrect. Human interaction is a reinforcement learning technique as explained in Section 2.3.

C.Correct. Rule-based knowledge is symbolic AI rather than machine learning as explained in Section 1.3.

D.Incorrect. Historical data is usually used to label and train machine learning models as explained in Section 2.3.

A.Incorrect. This is a useful measurement of the coverage of a model training process as described in Section 2.2, however it is not a useful statement for system acceptance criteria.

B.Correct. This is a well-structured criterion include false positive and negative outcomes, and ranges. This is in line with Section 2.4.

C.Incorrect. This is a useful measurement of the coverage of a testing process as described in Section 2.5 (Statistical Significance).

D.Incorrect. This criterion does not allow for False Negatives. See the syllabus Section 2.4.

A.Incorrect. Metamorphic testing is a type of pseudo-oracle as explained in Section 2.4, and therefore could be used.

B. Incorrect. Expert panels can be used where specialist knowledge is required to determine the test result, as explained in Section 2.4.

C. Incorrect. A/B is useful for situations with oracle problems. See section 2.4.

D.Correct. Fuzz testing is not useful for solving a testing problem where the correct answer cannot be easily determined. See section 3.2.

A.Incorrect. Variance in real user behavior is a good reason to conduct acceptance testing. See Section 2.4.

B. Incorrect. Comparing human to system behavior is a good reason to conduct acceptance testing. See Section 2.4.

C.Correct. Monitoring ongoing production behavior is not generally acceptance testing. See Section 2.4.

D. Incorrect. Measuring automation bias is a good reason to conduct acceptance testing. See Section 2.4.

A.Correct. If it is impractical to consume the output it will be difficult to determine if the system is passing tests. As described in section 2.1 of the syllabus.

B. Incorrect. This is a non-deterministic system, which is not necessarily non-testable. See section 2.1 of the syllabus.

C.Incorrect. Machine Learning algorithms are testable if the expected results can be determined. See section 2.1 of the syllabus.

D.Incorrect. The difficulties of coordinating 100 users do not make the system non-testable. See section 2.1 of the syllabus.

E. Correct. An event in the future that cannot be reasonably predicted, cannot be testable, at least untilthe future time has passed! See section 2.1 of the syllabus.

A.Correct. Drift can be caused by changes in the relationship between inputs and outputs. See section 2.1 of the syllabus.

B. Incorrect. The absence of a test oracle does not cause drift. See section 2.1 of the syllabus.

C.Incorrect. Incorrect training data will result in an inaccurate model, but this is not known as drift. See section 2.1 of the syllabus.

D. Incorrect. The act of testing a model which self-optimizes can cause drift, but not the act of reporting. See section 2.1 of the syllabus.

A.Incorrect. Test oracles are not part of the D-SOAKED model described in section 2.3.

B.Correct. Knowledge is part of the D-SOAKED model described in section 2.3.

C.Incorrect. Overfitting is not part of the environment as described in section 2.3.

D.Incorrect. Supervised learning is not part of the environment as described in section 2.3.

A.Correct. Metamorphic testing must involve multiple executions. See section 2.4.

B.Incorrect. It is not called metamorphic testing because it changes the system under test. See Section 2.4.

C.Incorrect. Metamorphic testing can be used for many applications. See section 2.4.

D.Incorrect. Metamorphic testing does not rely on the environment. See section 2.4.

A.Correct. A/B testing allows for field testing recommendation systems across different groups of users to compare their effectiveness. See section 2.4.

B.Incorrect. Model training is where the majority of the work is performed to verify the accuracy of recommendations, but acceptance testing requires the involvement of users or stakeholders, whichis not usually the case in model training. See section 2.4.

C.Incorrect. Metamorphic testing is rarely relevant for acceptance testing. See section 2.4.

D.Incorrect. Component testing is rarely relevant for acceptance testing. See section 2.4.

A.Incorrect. Per section 3.1.1, a test oracle is related to the outcome of a test, not its input.

B.Incorrect. Per section 3.1.1, a test oracle is needed for any test and is not AI-specific.

C.Correct. Per section 3.1.1, a test oracle is a source to determine expected results to compare with the actual result of the system under test.

D.Incorrect. Similar naming seems to be purely incidental, although probably inspired by the same mythological roots.

A.Incorrect. As stated in 3.4.1, monkey testing CAN be combined with AI approaches.

B.Incorrect. As stated in 3.4.1, monkey testing only makes sense using the implicit test oracle.

C.Correct. See section 3.4.1.

D.Incorrect. This is the definition of a deterministic system.

A.Correct (see section 3.3.1).

B.This is not a problem with component level tests, but an advantage of system level tests (see section 3.3.2).

C.This is not a problem with component level tests, but an advantage of system level tests (see section 3.3.2).

D.This is the definition of external validity.

According to section 3.1.2, test oracles can be specified, derived or implicit. So, the answer is B. What it means for an oracle to be well-defined or not, was not discussed. Manual testing still needs some means to identify the outcome of a test as correct or not, so there is no such thing as a manual test oracle in its own right.

All three except A are given in section 3.2.3. The output of classification and labeling algorithms is usually not well-suited as test input.

The correct is answer D. As stated in section 3.4.3, an unsupervised learning method can always be used for test selection, even in the absence of labeled training data (i.e., past test performance). A supervised learning method on the other hand requires that kind of information. Code coverage can be used as additional input. But Monkey testing cannot be used for test selection.

The correct answer is B, according to 3.3.2: The correct initial state is created by the system itself, the input consists of primitive types, implicit test oracles can be used and the output format can be much less formal.

A.Incorrect. Code coverage does not help with identifier selection.

B.Correct. The screenshot or “looks” of elements can be used as input to identifier selection in the case of image recognition, see section 3.4.4.

C.Incorrect. A confusion matrix is a table that summarizes how successful a classification model’s predictions were.

D. Incorrect. The confidence in identifiers its output, not its input.

A.Incorrect. As mentioned in section 3.2.2, AI cannot be used to suggest improvements to the user experience and user interface.

B. Incorrect. Visual test automation does not aim to test functional or business aspects of the software (see section 3.4.6).

C. Incorrect. Many accessibility aspects are not visual.

D.Correct. Visual test automation aims solely at identifying visual regressions or visual differences on different platforms (cross-platform and cross-device testing), see section 3.4.6.

A.Incorrect. Defect clustering occurs on the code level and is not affected by testing techniques.

B.Correct. After some time, the pesticide paradox will then decrease the prediction efficiency (see section 3.5.1).

C.Incorrect. Absence-of-errors fallacy is not affected by AI techniques.

D.Correct. The self-fulfilling prophecy shows more errors in more intensely scrutinized areas of the code (see section 3.5.1).

E. Incorrect. The No Free Lunch theorem is not a paradox and not emphasized by AI-based testing.

The correct answer is B. According to section 3.4.2, genetic algorithms and neural networks can be used for test generation, and even be combined. K-means clustering and principal component analysis as well as support-vector machines do not produce results applicable to test generation

A.Correct (see section 3.4.5).

B.Incorrect. Genetic algorithms produce cannot be used to work with images.

C.Incorrect. Cross-platform testing is not an AI approach.

D.Correct (see section 3.4.5).

E.Incorrect. Unsupervised learning algorithms cannot be used, as either elements or visual artifacts (defects) need to be identified, and thus labeled.

The correct answer is A. Test cases should cover business use cases and associated risks (section 3.5.6). Revealed defects should have a high expected customer impact (section 3.5.9). There should be counter measures to test case explosion in place (at least a metric how many test cases are sensible, see section 3.5.7). And test case maintenance should be monitored in terms of effort and benefit (section 3.5.8).