I'm interested in starting coding dojos at the company I work at (during or after working hours). I think it will help to spread knowledge on how to write clean, concise and consistent code. I've been part of coding dojos in the past and it's really helped me be a better software engineer. I especially think that this could be useful for engineers who are switching to a new language e.g. Java to Golang.

What I'm not sure of is how to measure its success or failure? Is it worth taking 30-60 minutes of the company's time per week to allow their engineers to hone their skills?

I could send out a simple questionnaire after every session, and try to measure how much people have learned. Or I can try and measure the quality of code in PRs? That would be really interesting to do!

Has anyone done such a thing? If so what did you do?

  • maybe belts? and battles to defeat the next rank up?
    – Ewan
    Feb 24, 2020 at 14:04
  • @Ewan If you want belts can I interest you in six sigma process improvement? Where traditionally you start off as a Green belt (and the next level is Black belt!)
    – Peter M
    Feb 24, 2020 at 14:10
  • The dojos are a success if your colleagues keep coming to them. Whether they have a beneficial impact on any software development metrics, that'll be much more difficult to figure out. Feb 26, 2020 at 16:39

4 Answers 4


If you work out how to measure whether someone is a good programmer or not then you have solved an age old problem and will be hailed as a genius by project managers the world over.

Instead measure whether people are enjoying the dojos. I suggest beer and pizza rather than in depth critiques of naming conventions.

  • Cool, measuring enjoyment is a good step forward. I guess engineers like to learn and if they're learning then they're having fun.
    – Ankur22
    Feb 24, 2020 at 14:42
  • Handing out beer shortly before people drive home could get you into trouble. Feb 25, 2020 at 11:04
  • #londonCentricAnswer
    – Ewan
    Feb 25, 2020 at 14:17

As part social scientist by training, I'm partial to the questionnaire idea. If your organization is large enough and management is willing to support your efforts, it may be possible to do a double blind experiment to see if a particular dojo helped the participants in that particular subject matter (As Ewan mentioned, we currently lack an objective method to determine how good someone is at programming overall).

  1. Determine the topic of your first dojo.
  2. Create a brief survey/quiz on the key concepts. Ideally the questions will be verifiable enough that the answers can be easily determined to be correct or incorrect, but difficult enough that most participants will not get all the answers correct without the dojo.
  3. Get as large of a group of volunteers/conscripts as possible to give yourself a lower margin of error.
  4. Randomly assign half to the control group, which will not initially attend the dojo. Give this group the survey right away. Allow anonymous results to be submitted, all you need to keep track of is if the participant is in the control or not (Two different Google Forms etc. with the same content might be an easy way to set this up).
  5. Give the first dojo lesson to the other half. Either at the end of the session or the next day, give them the survey.

This will give you the data to do a statistical analysis on the two groups. If the group that attended the dojo saw a statistically significant improvement over the control, we can conclude that the developers at minimum became more competent in the subject area in the short term.

As a bonus, if you would like to see if this information was retained longer term, create another quiz about the same topics, yet is different enough that remembering the answers to the first will not directly assist them. A few months later, give the new quiz to both groups. You can then compare the dojo group with the control to see if the results are still significant.

This would certainly be overkill to do for every dojo, but if it turns out that you can prove that one dojo has a positive impact on one area of programming, we can assume that others will as well.

  • 1
    Surveys? Quizzes? Yuk. You could take each class to be a group. First group does task A, dojo, task B. Next group task B, dojo, task A. Now every class is a control group. Just have to be sure to isolate the students from class to class. Feb 24, 2020 at 21:43
  • @candied_orange What exactly is a task in your example? A short programming sample/assignment? I do like pre and post checking during the class itself, but somehow you have to measure the participants knowledge, whether you call that a quiz, task, survey, assignment, example problem etc. Feb 24, 2020 at 21:47
  • You don’t have to measure knowledge if you measure performance. Rather than speed I’d measure quality. That is, readability. Feb 24, 2020 at 21:53
  • Thank you very much for the step-by-step on the questionnaire idea @Nathanael! I really like it a lot. @candied_orange I don't quite follow how i would measure the quality of the tasks if i switch the ordering of tasks from one group to another. What would your expectations be for a well devised "experiment"?
    – Ankur22
    Feb 25, 2020 at 12:48
  • 2
    @Ankur22 the two greatest achievements in software are (1) readable code that (2) performs correctly. Of those (1) is the most important because readable code can be fixed. After they do their tasks you can run tests that show how correct their code is. Then you can hand printouts of the code to programmers and ask them which tests will pass. The easyist code to predict test passage is the easist to read. So long as you have a mix of passing and failing tests this should give meaningful results. Not every test has to be given to those originally performing the coding task. Hold some back. Feb 25, 2020 at 18:36

Coding dojos should have "lightweight learning" for who is learning there. I believe PR in this context are not recommended once the intention of the dojo is not get fix or improve code. I had good experiences in rotative dojos once there was a laptop and the participants were intercalating between them, we used timer to give each participant 6 minutes to create a unit test in that case, given 30 minutes total + presentation and briefing of 15 minutes. A feedback sheet is a good ideia.


It's not how you measure success, but when that's important. Measurements made too early lead to premature judgment and false results. To know if something is working or not you must operate in larger time intervals or coordinate your measurements with others.

IOI (International Olympiad in Informatics) are organized every year. Just by looking at table (2019) I can say China is doing better than USA and Poland is doing better than Iran. But only with repetitive winning I can infer some underlying structure, a series "coding dojos" behind Poland being better than USA.

God knows I don't know what I'm talking about, but if it helps:

Do tests once a month and coding dojos once a week for a year and track the bug reports (there should be less bugs) and company's financial (there should be more money) returns and once you have enough data you make a judgment. If you conclude it does have a success on your business -- continue, if you conclude it does not -- stop.

  • 1
    Agreed that it will take a long time to measure if coding dojos are worth the company and SEs time, even then how can you tell that the reason for the improvement is due to better and consistent coding thanks to the dojos and not another reason?
    – Ankur22
    Feb 26, 2020 at 11:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.