A few weeks ago, I did a root-cause analysis of the defects we fixed in a recent release of our software. The last few posts have been to share what I learned, but that is only for our code base.
Yours will have different types of defects. Maybe you’ve already put practices in place to avoid the causes of our defects, and vice versa. It’s important that you do your own root-cause analysis.
With that caveat, I’d like to share a checklist I’ve devised for our developers, which I plan to put in place for our next release. (I’ll share what difference it made on this blog. Stay tuned!)
This is only an excerpt. The real checklist is about 100 items long, but I hope this gives you the flavor of it.
1) Should some users’ access to the new feature be restricted?
a) What authorization permissions did you use?
b) Have you tested the app’s behavior for non-privileged users?
2) Have you implemented any new caching of server data on the client?
a) Have you guarded against PUTs being processed slower than GETs?
b) Have you ensured that there is only one source for the cached data, including where it may be in the form of child records of other data?
3) Does the new feature use any cached data, including caches that might have been in the code before you programmed this feature?
a) Have you tested the app’s behavior when you add something to the cache and then attempt to access it elsewhere in the app?
4) Have you added any controls to the user interface?
a) Are the new control(s) pleasingly aligned with others on the page?
b) Is there a pleasing amount of space around the control(s)?
c) Have you checked the appearance when the browser window is at a variety of sizes?
d) Have you checked the color appearance in all themes?
e) Have you done all of this in Chrome?
f) In Internet Explorer, too?
5) Have you added any controls that the user can key into?
a) Have you validated the user input as to content and, if applicable, as to length?
b) Have you tested the application’s behavior when the user enters properly formed but invalid data?
c) Have you tested the application’s behavior when the input includes any of these characters? \/:<>
d) Have you tested the application’s behavior when the user leaves the input empty?
e) Have you tested the downstream effects when the data item is at its maximum length?
6) Have you added any dropdown lists or combo boxes?
a) Have you initialized the control with a sensible sort order?
b) If the control is loaded with data from the server, have you put a spinner in the control but allowed the rest of the page to function?
c) Have you tested the performance with a very large number of rows in both Internet Explorer and Chrome?
7) Have you added any CSS classes?
a) Is it true that there was no way to express what you wanted without adding an additional class?
b) Are your new classes semantic (not, for example, margin-left-10)?
c) Have you used LESS to minimize duplication of the CSS?
8) Does your code use Promises?
a) Have you ensured that all callers wait for the Promise to resolve?
b) Have you unit-tested the “reject” path?
9) Have you created any new modules?
a) Have you truly used Test-Driven Development, writing unit tests first?
b) What percent of your code is covered by unit tests, as reported by our tools?
c) Did you double-check your code to ensure that you followed the Single Responsibility Principle?
10) Did you double-check your code for missing abstractions?
11) Did you double-check your code to verify that you have not introduced any new responsibilities to existing functions or classes?