The Stack Overflow podcast is back! Listen to an interview with our new CEO.
2 added 9 characters in body
source | link

So, my day job is in research data preservation and publication and preservation for the University of California system. A couple of folks have mentioned reproducibility, and I think that's really the core issue here: documenting your code the way you'd document anything else someone needs to reproduce your experiment, and, ideally, writing code that makes it straightforward for someone else both to reproduce your experiment and to check your results for sources of error.

But something that I haven't seen mentioned, that I think is important, is that funding agencies are increasingly looking at software publication as a part of data publication, and at making software publication a requirement for open science.

To that end, if you want something specific, targeted at researchers rather than general software developers, I can't recommend the Software Carpentry organization highly enough. If you can attend one of their workshops, great; if all you have time/access to do is read some of their papers on scientific computing best practices, that's good too. From the latter:

Scientists typically develop their own software for these purposes because doing so requires substantial domain-specific knowledge. As a result, recent studies have found that scientists typically spend 30% or more of their time developing software. However, 90% or more of them are primarily self-taught, and therefore lack exposure to basic software development practices such as writing maintainable code, using version control and issue trackers, code reviews, unit testing, and task automation.

We believe that software is just another kind of experimental apparatus and should be built, checked, and used as carefully as any physical apparatus. However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is. This can lead to serious errors impacting the central conclusions of published research. …

In addition, because software is often used for more than a single project, and is often reused by other scientists, computing errors can have disproportionate impacts on the scientific process. This type of cascading impact caused several prominent retractions when an error from another group's code was not discovered until after publication.

A high-level outline of the practices they recommend:

  1. Write programs for people, not computers
  2. Let the computer do the work
  3. Make incremental changes
  4. Don't repeat yourself (or others)
  5. Plan for mistakes
  6. Optimize software only after it works correctly
  7. Document design and purpose, not mechanics
  8. Collaborate

The paper goes into considerable detail on each of these points.

So, my day job is in data preservation and publication for the University of California system. A couple of folks have mentioned reproducibility, and I think that's really the core issue here: documenting your code the way you'd document anything else someone needs to reproduce your experiment, and, ideally, writing code that makes it straightforward for someone else both to reproduce your experiment and to check your results for sources of error.

But something that I haven't seen mentioned, that I think is important, is that funding agencies are increasingly looking at software publication as a part of data publication, and at making software publication a requirement for open science.

To that end, if you want something specific, targeted at researchers rather than general software developers, I can't recommend the Software Carpentry organization highly enough. If you can attend one of their workshops, great; if all you have time/access to do is read some of their papers on scientific computing best practices, that's good too. From the latter:

Scientists typically develop their own software for these purposes because doing so requires substantial domain-specific knowledge. As a result, recent studies have found that scientists typically spend 30% or more of their time developing software. However, 90% or more of them are primarily self-taught, and therefore lack exposure to basic software development practices such as writing maintainable code, using version control and issue trackers, code reviews, unit testing, and task automation.

We believe that software is just another kind of experimental apparatus and should be built, checked, and used as carefully as any physical apparatus. However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is. This can lead to serious errors impacting the central conclusions of published research. …

In addition, because software is often used for more than a single project, and is often reused by other scientists, computing errors can have disproportionate impacts on the scientific process. This type of cascading impact caused several prominent retractions when an error from another group's code was not discovered until after publication.

A high-level outline of the practices they recommend:

  1. Write programs for people, not computers
  2. Let the computer do the work
  3. Make incremental changes
  4. Don't repeat yourself (or others)
  5. Plan for mistakes
  6. Optimize software only after it works correctly
  7. Document design and purpose, not mechanics
  8. Collaborate

The paper goes into considerable detail on each of these points.

So, my day job is in research data publication and preservation for the University of California system. A couple of folks have mentioned reproducibility, and I think that's really the core issue here: documenting your code the way you'd document anything else someone needs to reproduce your experiment, and, ideally, writing code that makes it straightforward for someone else both to reproduce your experiment and to check your results for sources of error.

But something that I haven't seen mentioned, that I think is important, is that funding agencies are increasingly looking at software publication as a part of data publication, and at making software publication a requirement for open science.

To that end, if you want something specific, targeted at researchers rather than general software developers, I can't recommend the Software Carpentry organization highly enough. If you can attend one of their workshops, great; if all you have time/access to do is read some of their papers on scientific computing best practices, that's good too. From the latter:

Scientists typically develop their own software for these purposes because doing so requires substantial domain-specific knowledge. As a result, recent studies have found that scientists typically spend 30% or more of their time developing software. However, 90% or more of them are primarily self-taught, and therefore lack exposure to basic software development practices such as writing maintainable code, using version control and issue trackers, code reviews, unit testing, and task automation.

We believe that software is just another kind of experimental apparatus and should be built, checked, and used as carefully as any physical apparatus. However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is. This can lead to serious errors impacting the central conclusions of published research. …

In addition, because software is often used for more than a single project, and is often reused by other scientists, computing errors can have disproportionate impacts on the scientific process. This type of cascading impact caused several prominent retractions when an error from another group's code was not discovered until after publication.

A high-level outline of the practices they recommend:

  1. Write programs for people, not computers
  2. Let the computer do the work
  3. Make incremental changes
  4. Don't repeat yourself (or others)
  5. Plan for mistakes
  6. Optimize software only after it works correctly
  7. Document design and purpose, not mechanics
  8. Collaborate

The paper goes into considerable detail on each of these points.

1
source | link

So, my day job is in data preservation and publication for the University of California system. A couple of folks have mentioned reproducibility, and I think that's really the core issue here: documenting your code the way you'd document anything else someone needs to reproduce your experiment, and, ideally, writing code that makes it straightforward for someone else both to reproduce your experiment and to check your results for sources of error.

But something that I haven't seen mentioned, that I think is important, is that funding agencies are increasingly looking at software publication as a part of data publication, and at making software publication a requirement for open science.

To that end, if you want something specific, targeted at researchers rather than general software developers, I can't recommend the Software Carpentry organization highly enough. If you can attend one of their workshops, great; if all you have time/access to do is read some of their papers on scientific computing best practices, that's good too. From the latter:

Scientists typically develop their own software for these purposes because doing so requires substantial domain-specific knowledge. As a result, recent studies have found that scientists typically spend 30% or more of their time developing software. However, 90% or more of them are primarily self-taught, and therefore lack exposure to basic software development practices such as writing maintainable code, using version control and issue trackers, code reviews, unit testing, and task automation.

We believe that software is just another kind of experimental apparatus and should be built, checked, and used as carefully as any physical apparatus. However, while most scientists are careful to validate their laboratory and field equipment, most do not know how reliable their software is. This can lead to serious errors impacting the central conclusions of published research. …

In addition, because software is often used for more than a single project, and is often reused by other scientists, computing errors can have disproportionate impacts on the scientific process. This type of cascading impact caused several prominent retractions when an error from another group's code was not discovered until after publication.

A high-level outline of the practices they recommend:

  1. Write programs for people, not computers
  2. Let the computer do the work
  3. Make incremental changes
  4. Don't repeat yourself (or others)
  5. Plan for mistakes
  6. Optimize software only after it works correctly
  7. Document design and purpose, not mechanics
  8. Collaborate

The paper goes into considerable detail on each of these points.