Figure out how we can measure "understandability" of code

[benjamingr]

Hey,

After the last docs WG meeting we figured out we need a better way to measure how understandable the docs are and what newcomers find confusing.

I'd like to toss around ideas for how we can measure how understandable the docs are.

[Qard]

It might work well to partner with nodeschool on this to collect feedback from nodeschool students.

[techjeffharris]

This inspired me to suggest a Target Audience index for Guides and Topics to help steer users to docs appropriate for their skill levels in the roadmap. #59 (comment)

Any thoughts?

[eljefedelrodeodeljefe]

+1 on nodeschool hook. Maybe also discuss nodeschool's roadmap on this.

[benjamingr]

Hi, I contacted Dror Feitelson from the Hebrew University who researches the understandability of code. I'm posting here his remarks on this issue:

great question.
I'd follow Gilb on this. Gilb's law states that "Anything you need to quantify can be measured in some way that is superior to not measuring it at all". specifically he suggests a methodology of components, scales and metrics:

1. identify components of the thing you want to quantify
2. for each component find the scale with which it can be measured
3. and then find mechanisms to actually measure it in this scale

in your case of software documentation understandability, possible components that come to mind are:

• find which function to use
• how to use a certain function
• etc.

A scale can be the time needed by a to perform a certain . This is parameterized by developer type being a novice, a casual contributor, a core member, etc., and tasks being correcting a usage bug, using the function in new code, etc.

the mechanism can be to actually test this at least 5 times with different users and actually measure the time it takes them.

[chrisdickinson]

/cc'ing @ashleygwilliams and @feross on this — I suspect the Foundation may be able to help out here. We'd likely want to run polls and possibly run recorded UX interviews (e.g., an interviewer asking a developer "Show me how you'd find out how to do XYZ with Node")

[feross]

@chrisdickinson Nice. Thoughts on how to conduct these polls and UX interviews? If there's a concrete proposal from the WG then I think we can take it to the board. Also, if possible, we should try to seek funding/volunteers from the community first, since more community ownership is always better IMO :)

[Trott]

Closing as this repository is dormant and likely to be archived soon. If this is still an issue, feel free to open it as an issue on the main node repository.