Relying on Second-Hand Information

Recently, my toddler son visited his pediatrician for his twenty-four-month check-up. The event reminded me back to a prior visit we had for him, which ultimately inspired me for this article.

The visit was with another specialist a few months prior. After we finished talking with the doctor, one of his assistants returned to complete the remaining tasks for our visit. During that time, we had a question for the specialist. Instead of stepping away to ask the specialist our question, the assistant went ahead and provided an answer to us.

The answer provided by the assistant was prefaced with the fact that the specialist recommends the information that was provided to us. However, when my spouse pressed for additional details (through follow-up questions), the assistant decided to step away and bring the specialist back to talk with us.

This made me wonder how many times we rely on second-hand information to make decisions with our daily efforts.

The Real Formula for a Computation

Years ago, I worked for a real estate corporation that provided various levels of shopping facilities across North America and Europe. In their efforts to maximize revenue, there was a reliance on utilizing several data points to help drive what they called the "optimal rent" for a space in their facilities.

Based on the tier level of the property and the location of the space — especially in proximity to certain other areas of the property (food court, movie theater, an anchor store, etc.) — the price per square foot would be different than other locations in the same property. It was an impressive service offering, especially since this was years before the concepts of Data Warehouses, Data Marts and Big Data had started to gain momentum.

What was not impressive was when the "optimum rent" logic was needed in a new application. Our feature team utilized the analyst on the project to help us with the computations. The problem was, the analyst had started on that project late in the lifecycle and thought he had an understanding of the computations involved; however, his information was based upon discussions with others and not via review of the actual program logic.

When we released the new application to the general users, the results were very positive; however, when the leasing agents began to review the application, they became confused with the results of the values being presented to them. The second-hand knowledge about the computations was not correct.

The lesson learned here is that the original source code should have been validated instead of relying on second-hand knowledge.

But This Is Not Montana

I have mentioned working on a government project in prior articles, but this is one of my favorite lessons from that project.

During the project, I was asked to document the business rules that were employed in the source code for cases where someone receiving benefits were provided more compensation than they were allowed.

The reason for the ask is that this work was completed early in the project, by an off-shore team. This was at a time when the goal was to send the product to an off-shore team for complete conversion from Oracle forms to a web-based application utilizing Spring and JSP. It is also important to note that this product was actually purchased from another state (Montana) in order to cut down on the actual time required to build the system.

This type of work is something that I have always enjoyed — being able to work through the original source code to reverse engineer the requirements. As I worked through each class and method, I became familiar with the process. I packaged the documentation nicely and included a presentation to illustrate the flow.

A meeting was scheduled where I was to present the results of my analysis. With confidence, I was able to walk through the complete process — from happy paths to any alternate flows, I found in the source code.

When I finished, the group representing the business looked stunned. One finally broke the silence to state, "we have no idea why the code is functioning that way." It turned out, the original process was the exact process employed by the state of Montana — instead of the process for the state housing the government agency.

No one ever documented the process when the original conversion was attempted. Instead, the original developers relied on the processes in place with the purchased application and re-wrote them, as is.

Which Service Does Which Task?

While working on a multi-service project, the original design utilized one large code repository to house several services. There was a section of the project which housed classes and methods used across all of the services. As one might expect, value objects (VO) were in this section of the repository. As one might not expect, business logic related to only a single service was also stored in this section of code. In short, this section of code became a one-stop-shop for everything in the application.

The result of this design is that it became very easy for server-side functionality expected to be executed by one service to be actually executed by another service. In short, all the necessary code was available for all of the services to act as any other service.

When an issue was encountered during my time on the project, all signals indicated that the issue should be related to something happening with one of those single-purpose service nodes running the application. We increased the debug levels and analyzed the logs to figure out the cause of the issue. Looking at the code, we did not immediately see the root cause of the issue that needed to be resolved.

Then, taking a step back, I decided to bring up all the services locally in debug mode using the same code base. I also made sure to set the same breakpoint in each of the services, in hopes to figure out when the request left the primary server and when the request was routed to the focused service on the network.

To my surprise, the debugger for the code stopped on another service altogether. At that point, the reality became crystal clear. The original developer had programmed the request to be executed on another service. The reason was unknown and the developer was no longer working at the client.

In this case, we made the assumption that the focused service would only perform the role that it was designed to accomplish; however, the reality was another service was performing the action instead. Once the reality was known, fixing the issue became a trivial task.

Conclusion

In prior articles, I have mentioned having a passion for music in my high school years. I recall taking a road trip to a music store about 90 minutes from my home — to check out a new line of gear that only this particular store had in stock. Two members of the band I was playing in at the time joined me for the experience.

Since this was long before the days of Google maps and even mobile phones, we drove to the town and then decided to ask for directions on how to get to the music store. Stopping at a gas station in this modest-sized town, the following dialog took place:

me: Can you tell us how to get to the Anderson Music store?
gas attendant: (walks to the edge of the street and then walks back)
gas attendant: (then asks) What do you want to go to Anderson Music for?
bass player (in car with me): We want to get something to eat.
gas attendant: (gave a blank stare, then returned to the edge of the street)
gas attendant: (returned to our car and provided directions)

Turns out, the attendant's directions were not correct. Despite taking two long glances in the direction he felt was the right direction for us to travel. This caused delays in us being able to meet our goal to see the new gear. It also further demonstrates my point of the dangers of relying on second-hand information.

If you would have told me in 1984 that I would be telling this story to a purely technical audience for a publication that focuses on Information Technology 35 years later, I probably would have given you the same blank stare that the gas attendant gave us when he didn't quite get the bass player's joke.

Have a really great day!