Project Hygiene

Overview

• “Eat your veggies!”
• “Exercise multiple times a week!”
• “Brush your teeth and floss daily!”

Such are the exhortations that every child has heard (many times!) and grown to loathe. However, these are not practices designed solely to make one suffer: They are encouragements to help one develop and maintain good hygiene. Dictionary.com defines hygiene as “A condition or practice conducive to the preservation of health, as cleanliness.” As in, “If you do these things consistently, you’ll reduce the chance of bad things happening to your health in the future.“ However much having to keep brushing teeth daily is a pain, having to get the dentist to pull said teeth due to a neglect of this care is going to be much more painful!

This is a concept that can be easily applied to software engineering as well. Software projects have their own maintenance aspects outside of the main code development tasks: documentation, dependency management, deployment, and so on. Supporting these aspects might not be the most exciting or intellectually-stimulating work, but just like human hygiene, such is the nature of supporting a project’s hygiene: a lack of doing so could cause major pain for the developer in the future. Thus, when developing the practices and activities for a software development team, try making it a practice to develop a plan for maintaining the project’s hygiene via regularly scheduled activities that can be incorporated into the project’s development plan.

Concepts

Dependencies

Virtually all software projects nowadays are built using external software libraries: from the third-party frameworks like Spring or Django to the programming language in which the code for the project is written (Java, C++, Python, and so on). These external libraries, naturally, are software projects themselves; in the majority of cases, the developers behind these projects are actively maintaining them and publishing new versions of the code at varying points in time.

Points to Ponder

• How will the project team know when a new version has been released for any of the project’s software dependencies? Manually checking for new versions can be an arduous process; automated detection tools like Dependabot can reduce or eliminate this type of toil.
• Once a new version of a software dependency is released, what will the protocol be for evaluating when to incorporate it into the project? Save for unresolvable breaking changes, a new version of a software dependency should be incorporated as soon as possible for a project. While it’s not delivering immediate value to the project, it enforces the habit of keeping the project’s code up-to-date in conformance with the dependencies that it uses. Sticking to an older version of a dependency because “it just works” is fine — until it isn’t. One day, the announcement from the internet may arrive that it is absolutely necessary to upgrade; suddenly, it could be necessary to jump several years’ worth of dependency versions. What could have been a series of small changes to the project’s code every so often is now one large set of unplanned work that needs to be done yesterday.

Deprecations

As mentioned above, software is frequently in a constant state of improvement and refinement; sometimes, this does not mean solely adding new code and functionality, but also taking code and functionality away. Ideally, software developers will indicate code and functionality that they eventually intend to remove by marking it as deprecated.

Points to Ponder

• When code in an external dependency has been deprecated, what will the protocol be for replacing it? Think of deprecation flags as a good-will gesture: the developers of the external dependency have announced that they will be making a code change that will require changes in the code that uses the dependency, but they will give external users the benefit of time to prepare their own code for the change. Take advantage of that courtesy and plan likewise for how and when to make the proper adjustments, as eventually, a new version of the external dependency will be published in which that deprecated code will no longer exist.
• Once a software dependency has been deprecated, what will the protocol be for replacing it? Sometimes, deprecation occurs not only to specific parts of a software project’s code, but to the entire project itself. An example of this was Netflix’s Hystrix library, for which the company announced in 2018 that it would cease active development. When cases like this occur, it is advisable to devote time to investigating how to replace its usage in a project’s code, mostly for the same reasons as with code dependencies: Eventually, some event like an announced security vulnerability might compel the upgrade, at which point the work would become necessary but also would not have been planned for.

Workflow

The software development life cycle contains many steps besides the software development itself: testing, code linting, deployment, and so on. Frequently, what were once steps that required manual action in order to be completed have fallen under the purview of software programs to be configured and then executed as part of an automated process. Indeed, the concept of automating the various parts of the software development life cycle — now known as DevOps — has grown rapidly in the past decade into its own career path.

Points to Ponder

• How will the team determine whether any processes in the development workflow could be automated? Continuous Integration/Deployment systems like CircleCI and Jenkins already provide a multitude of capabilities for automating various parts of the development life cycle; however, sometimes there exist opportunities for creating further automation in a project outside of CI/CD environments. For example, if a project includes the development of multiple microservices, it might be worthwhile to devote time to investigating whether the creation of a home-grown equivalent to a programmatic initializer service like Spring Initializer might be worthwhile.
• How will the team evaluate whether any steps of the workflow could be re-arranged? Even if a verification step like code linting might already be a defined part of the automated CI/CD process, moving that step into an earlier part of the development process could reduce the turnaround time between the developer creating changes and finding out whether those changes could be improved. Git provides hooks wherein it will execute a script at a before or after a step of the Git process; in this scenario, the code linting step could be executed in the pre-commit hook so that the developer is notified about a potential code smell immediately instead of eventually receiving a message from the CI/CD at a later point.
• If any parts of the development workflow have been disabled, when will the team re-examine whether they could be re-enabled? Sometimes, code or development life cycle steps need to be disabled for a variety of reasons, e.g. an external test dependency that has suddenly become unreliable. This should, however, be an expressly-temporary measure. If left unresolved, the reason why the measure was intended to be temporary could eventually be forgotten, and the measure will ossify and become just another “that’s just how things are” that could potentially prevent other issues from being detected. It is worthwhile to schedule a periodic check-up of the project code base and development process to detect whether a “temporary” measure has indeed become de-facto permanent and what would need to be done to resolve the underlying issue.

Documentation

Ah, writing documentation — the software developer’s favorite task! /s However, while documentation work might not be the most pleasant of activities, the efforts will eventually bear fruit. Tribal knowledge (i.e., not having access to it) and winding up on the wrong side of the bus factor are two potential impediments to group productivity in a project, and the more documentation that is produced for the project, the more the risks of getting impeded can be mitigated, especially in cases where it’s not possible to increase the bus factor via “organic” means (i.e., assigning more people to the project).

Points to Ponder

• How and when will the team evaluate whether a developer with zero knowledge of the project can configure their environment solely by reading the project’s documentation? On projects that are in active development, the answer might always be “no” in the strictest sense of the term, as non-trivial software projects will have enough aspects that eventually *some* part of the documentation will fall out of date. However, it is a worthwhile effort to strive towards as much of a “yes” as possible, lest the mentality of “it’s not possible to keep fully up-to-date, so why bother” sets in and the documentation ages to eventual uselessness. A good exercise here is to provide an experienced developer on the project with a wiped-clean environment and have them attempt to set their environment up via the method described above; they will have the experience to know which gaps in the documentation need to be filled in and where.
• How and when will the team evaluate whether a developer with zero knowledge of the project can learn how to contribute to the project solely by reading the project’s documentation?Related to the point above, a project’s development process can vary from others in a multitude of ways:
  • What is the project’s branching strategy?
  • What is software development lifecycle for the project?
  • How does one create a task? What types of tasks are there?
  • How are code reviews conducted? Who participates in the code reviews?

And so on. While these might be considered innate knowledge for somebody immersed in the project, it will not be so for somebody completely new to the project. A walkthrough with somebody who’s knowledgeable about the project to “show the ropes” will help instill this knowledge in the newcomer, but that a) takes time away from the experienced developer that could be otherwise be used for development, and b) there’s no guarantee that the experienced developer will know precisely which subjects they need to cover. Again, the same exercise for the point above would help: have an experienced developer periodically check the project documentation to make sure that the development process information is up-to-date.

Conclusion

All of the points listed above are merely guidelines. The circumstances for each software project will be different and hence require different policies for maintaining the project hygiene. Likewise, it is possible that other aspects of project hygiene that have not been mentioned here will be relevant for a particular software project; these should naturally be considered as well. What matters most is the concept of incorporating periodic tasks that re-examine the overall state of the software project and evaluate what steps can be taken either to bring the project in good working order or to keep it there. Again, these will not be the most engaging of tasks, but this “boringness” will help prevent “exciting” catastrophes in the future, so get the veggies on the plate, and start checking that documentation!