CRISP-DM, which phase and tasks relate to User Acceptance Testing?

Question

EDIT: The project is about creating an algorithm for Procedural Content Generation for a videogame. The actual game is secondary to the algorithm. It must be something basic to test the Procedural Content Generation. I'm approaching this as a single developer.

Thanks to @ThomasOwens I realize this question is more about how to work combining CRISP (the datamining process model that will help me create the algorithm) with a project management methodology (preferably something agile such as XP).

---

I'm working on a school project involving an interactive genetic algorithm where the program must create some content in real time based on the user's preferences. Near the end of the project, in order to validate the algorithm and creating the necessary reports, it must be tested by actual users.

My tutor wants me to use CRISP-DM as the methodology for the project, and I'm wondering in which phase will the user tests go. Please don't advice me to change methodology. And also, my tutor isn't available this week, but I want to make some advances in the planning.

So my question is: Regarding the CRISP-DM process model, which phase and tasks relate to User Acceptance Testing?

---

You can skip the following paragraphs if you have the answer. I'll just try to explain my confusion a little better. The main problem here is my lack of knowledge on CRISP-DM and there's a liiiitle bit of a language barrier. To my udertanding the User Testing would be on the Evaluation phase, but I just want to be 100% sure, so I'm seeking expert advice.

Yes, my project doesn't involve reading data from a database, but it's a data mining problem because my algorithm will have to extract information from some data and analyze it and transform it into something else. This is how I've related the CRISP phases to my project so far:

• Business Understanding will be all about defining a problem that would benefit on the user for finding it's solutions.
• Data Understanding and Data Preparation will be merged on a single phase that involves figuring out how the data will be treated.
• Now, on the Modeling phase, there's the Generate test design task, and this test is performed on the Assess model model task. But this tests don't involve User Acceptance Testing, only Unit Testing or Functionality Testing. I'm right here, right?
• That leaves me with the Evaluation phase, in which it's task Evaluate results says:

This step assesses the degree to which the model meets the business objectives, and seeks to determine if there is some business reason why this model is deficient. Another option is to test the model(s) on test applications in the real application, if time and budget constraints permit.

Is this where it goes? Should I adapt this task to my User Tests? Or would it be better to add another task before this one that specifically refers to executing the user tests?

Answer 1

On project management and being a solo developer.

Most development methodologies are built to support a team working on a project, not with an individual developer in mind. There's quite a bit of information on Software Engineering Stack Exchange about working as a solo developer. There's a common theme found when talking about working alone - many of the more defined development methodologies come with various practices. These practices can be useful, even if you are working alone.

From a software development perspective, there are plenty of processes and frameworks to draw upon. As a whole, the lean and agile family of methods gives us practices around a highly iterative and incremental approach. Extreme Programming embraces ideas such as test-driven development, continuous integration, small releases, a sustainable pace. Scrum provides regular reflection and improvement and very strict timeboxing on events and activities. Kanban encourages visualization of work and managing flow. Lean Software Development teaches us about various types of waste and finding ways to reduce them, including limiting work-in-progress (partially done work).

Since your project is also about data mining, you can also look for good practices from processes from that field. CRISP-DM is one such process model that you can look at to help you identify and organize part of the work that needs to be done. However, do be aware that it may be built around considerations for a team and not an individual working alone, so some aspects of it may not be appropriate.

On User Acceptance Testing in CRISP-DM.

CRISP-DM doesn't have considerations for user acceptance testing. CRISP-DM contains an Evaluation phase that is used for evaluating data mining models that are to be used. This is the equivalent of unit testing. After you evaluate (read "unit test") your data mining models, you would integrate them into a larger software system, perform integration and system testing, and then submit for user acceptance testing.

In XP, your user acceptance testing strategies would be part of your release and iteration planning. In Scrum, you want your potentially shippable increment at the end of every Sprint to meet the needs of the user and be ready for deployment if it's deemed to be appropriate. The guidance from these project management methods would inform when and how you did the activities associated with the Evaluation and Deployment phases of CRISP-DM.

This is why you need to combine CRISP-DM with other, more suitable project management methods.

On combining CRISP-DM with other project management techniques and methods.

One of the activities in any systems development lifecycle is planning. When you plan, whether it's a more up-front project planning or planning for a particular iteration, you need to understand what work you need to do. The phases and steps from CRISP-DM can be used as an input into any kind of planning activity. This will help you to make sure that the things that you need to do to be successful at a data mining project are considered.

CRISP-DM is silent on a lot of other things that go into projects - how iterative and incremental vs plan driven you are, how often you deploy your product, how you keep track of work that you need to do or are currently doing, how to reflect on your work and improve your methods, how to keep your work visible to stakeholders, test methodologies and approaches. You'll have to turn to other sources for suggestions and practices for doing these things.