Performance Issues (The Problem)

Today I want to talk about my experience working on solve performance issues in the systems. In several years working as a software developer I have faced performance issues several times, and I want to share some tips about this topic.

In my opinion, to solve a performance problem you need all your focus on THE PROBLEM, you need as much information about it as you can get. In this post, I want to share the way how I work to understand the problem. usually, I have spent around 70% of the time on understanding the problem and only 30% on making changes in the source code. 

I want to focus this post on how to get as much information about the problem as you need it to solve the problem.

Reproduce

To me, this is the first task to do when you start to work on performance issues, you have to be able to reproduce the issue any time you need it. and to all the sceneries as possible. In this way when you start to work on your changes, you will be able to validate them and check whether you are on the right way or not.

One tip in this topic "Be smart to test your chances", usually to reproduce the performance issues you need a lot of time or many users connected to the system concurrently. And of course, you want to reduce this complexity to work faster. For example, maybe you have a problem storing 1.000.000 records into your database (You expect to be able to store all those records successfully). But I am pretty sure you can reproduce the problem storing only 200.000 records. Then you can test your changes faster (Off curse is faster storage 200.000 records instead 1.000.000)

Understanding

You need to identify the type of problem, think about these questions.

1. Is it a problem related to the amount of data processed? (For example, there are millions of records to be processed).
2. Is it a problem related to concurrency? (For example, you need a high number of users connected to the system).
3. Is it a problem related to the velocity? (The system can complete the task but takes too much time).

Once you get your answer, you know what kind the problem you are facing. keep in mind this answer all the time, to work on the next questions, and to find the solution (remember At this point focus on THE PROBLEM).

Now lets to identify what resource has the problem. think about these questions.

1. Is it a problem related to memory?.
2. Is it a problem related to the database connection? (For example, you can not create or get a connection from the pool).
3. Is it a problem related to the file system? (For example, there is no space into the file system, or you are reading to writing too much data from a file).
4. Is it a problem related to network connection? (For example, you can not get a connection to an external resource, there is a lot of traffic into the network).
5. Is it a problem related to SQL statements?.
6. Is it a problem related to the algorithm? (For example, you can improve your process and reduce the statements. Or you can use a caching tool like Ehcache or store date into local storage inside the browser, etc).

Measuring

Ok, now you can reproduce the issues and you know the kind of problem you are working on. The next step you have to monitor resources inside your environment.

• Measure the memory of your application (Define the behavior of the memory when the process is running).
• Verify the number of sessions and connections of your database.
• Get the size of requests and responses (If you are working on web applications).
• Count the number of records into the database (you do not need to verify into all the tables, verify only the tables related to the service with the problem).
• How long time take your service to finish when there are several users.
• You need to define a baseline for the performance of your application
  • The number of users of concurrent users allowed by the system before fail.
  • The number of successful operations before fail.
  • The number of records stores into the database before fail.

At this point, you should have enough information about what is happening, how it is happening, and where is happening. you have a general vision and you are ready to start to think about how to solve the problem (Believe me, you are very very close to solving the problem).