RTMP Performance Testing With JMeter: Learn How

RTMP is one of the most commonly used protocols for real-time video streaming. In this article, we will look at ways to load test services that use this protocol, with Apache JMeter™. We will cover a few different options.

About RTMP

Real-Time Messaging Protocol is a proprietary TCP-based protocol developed for real-time streaming audio, video and data. The protocol can split video and data into fragments while simultaneously maintaining the ability to transmit bigger chunks of information, to ensure smooth delivery of video and audio streams. The size of the fragments used can be negotiated dynamically between the client and server, and even disabled completely if desired. The default fragment sizes are 128 bytes for video and most other data types, and 64 bytes for audio data.

RTMP Servers

Unfortunately, RTMP has known problems with licensing, for example, its official specification does not allow users to write a full-fledged RTMP server if you follow it thoroughly. However, there are several implementations:

• Red5 - one of the first RTMP-servers, written in Java
• Wowza - proprietary RTMP server, written in Java. In addition to RTMP, it also supports RTSP, Apple HLS and Smoothstreaming (Silverlight)
• Erlyvideo - open source RTMP server with separately distributed proprietary modules, written in Erlang
• NGINX-based Media Streaming Server

Because of the unclear specification, every developer had to add something to her/his server implementation, and developing a universal client is not such a simple task (if it is real).

For the demo, we will use Red5 Pro server. It has a trial access (30 days) and to start working with it you only need 2 clicks (really, and it’s adorable), also it has another free version with reduced functionality. You can download it from here and run .bat file for Windows or .sh for Linux users.

Note for Windows users: DO NOT USE ‘Run as Administrator’ option, just double click, otherwise you will get an error when trying to stream a video.

Now you can go to http://localhost:5080/live/broadcast.jsp and start broadcasting. After that, you can watch it using RTMP and 1935 port. That is all!

Creating Your JMeter Script

At the moment there are no JMeter supported plugins that allow you to read the stream or write to it. Two plugins were found: FlazrJmeter, which is based on the Flazr library and RtmpPluginForJmeter, which is based on the Red5 library, but the last commit for the first one was 2 years ago and for the second one it was 7 years ago. We will discuss FlazrJmeter below, but the RtmpPluginForJmeter is too old, so we will write our own sampler to work with RTMP using Red5 library.

Option 1: JAVA Request Sampler + Red5-client Library

We will not use the JSR223 sampler now due to the amount of code. Instead, we will use the Java Request Sampler. Besides, the JSR223 Sampler may work slower than the JAVA Sampler in some cases, because it compiles code during the test. We can both stream the video and watch it. A client who streams the video is called a publisher and the watcher is called a subscriber. The client is the element that interacts with the server.

To quickly learn how to implement Java class for the Java Request Sampler, read here. We got the following class for the subscriber: click here.

The parameters:

• url is the RTMP server address
• port is the RTMP port, the default is 1935
• app is the name of the application that broadcasts the video stream (for Red5 Pro, it is ‘live’ by default)
• streamName is the name of the stream
• timeout is the duration of the subscription in seconds, the time that the client will be subscribed to the stream. The stream must exist for the whole time of the subscription.

For the publisher, the class is similar, but we will add a new parameter for the setup path to the video that will be streamed.

Now let us proceed to the implementation of the subscriber and the publisher.

For the subscriber implementation, tests from the red5-client repository and the source code of the RTMP Bee tool have been observed.

The subscriber has the following flow:

1. The client establishes a connection using the client.connect() method.
2. The ConnectCallback processes the answers of the server.
3. If the handshake is successful, client.createStream() initiates StreamConnection.
4. If step 3 is successful, client.play() starts playing video stream until the timer expires.

For the publisher implementation, YouTube RTMP Client implementation and this topic have been observed.

The first three steps of the publisher’s flow are similar to the steps of the subscriber thread, but at the fourth step, we execute client.publish() to start broadcasting with the "recording" parameter instead of live, because we stream the flv file. Also, we need the flv file that will be streamed.

After downloading the code and potential corrections making, you need to package it and put into /lib/ext. Besides, all third-party libraries from the .pom file should be put into the folder. Thereafter you can choose these classes in the Java Sampler as shown in the pictures above.

Option 2: JAVA Sampler + Flazr Library

Another option for performance testing RTMP is with the same sampler as before, but with another library. We can use part of the FlazrJmeter plugin. In addition, we need the PullRequestTest.java and PushRequestTest.java files from the repository, which are classes that are ready for subscription/publishing, and the Java Request Sampler.

You need to compile them and then put them and the required libraries in the lib/ext folder as in the example above.

The parameters for PullRequestTest are the following:

• IP, port, app, stream - similar to the ones in the Red5 paragraph.
• Version - the version of the tested server, you can set 000000.
• Threads - the number of threads, you can regulate the number of threads using this field, but it will be more correct to set the value to 1 and set up the thread number using a thread group.
• Load - the number of clients. In our case, it is 1.

The parameters for PushRequestTest are the following:

• IP, port, app, stream similar to the ones in Red5 paragraph
• Video - the path to the video that will be streamed

Option 3: Other Libraries

Flazr and Red5 are the most commonly used open-source libraries, but there are others too:

• JUV RTMP library is a commercial library
• srs-librtmp library supports only audio, but it is a lightweight library

Option 4: Process OS Sampler

Another way to load test RTMP is to use ready-made console applications:

• RTMP Bee - console tool for subscription
• Flazr - console tool for subscription and publishing
• https://github.com/fillest/rtmp_load - console tool for subscription, collects perf metrics

You can use these tools from the console without JMeter, if they suit you, but if you test a use case that includes RTMP streaming, you cannot start them from the console, because it will not be connected to other actions from the use case. Therefore, the script for the scenario will be incorrect.

To add this call to the test plan, you can use the OS Process Sampler, which can be used to call 3rd party tools from JMeter. You can see work of the Sampler and the description of the interface here.

For example, the screenshot below shows the sampler with the command to subscribe to the Flazr tool.

The name of the executed file is in the Command field, the path to the file is in the Working directory and the Commandparameters are options for the executed file. The ResponseTab will show the entire log of the app, so you can use assertions and extractors, if necessary.

Read more about how to load test live streaming with JMeter from these blog posts:

• The New HLS Plugin for JMeter - The Complete Guide
• How to Load Test RTMPT Live Media Streaming with JMeter
• How to Load Test HTTP Live Media Streaming (HLS) with JMeter
• How to Load Test Podcasts with JMeter

Our free JMeter academy provides an advanced course where you can learn even more ways to create complex performance testing scenarios. Sign up here.

Running Your Performance Tests With BlazeMeter

After creating your RTMP testing script in any of the four ways we’ve shown here, you can then run it in JMeter. But, if you need to scale your test, run it in BlazeMeter. All you need to do is upload your file, configure the load and the global locations your test is running from, and run.

You will immediately get insightful results you can share with your managers and teammates. By the way, you can also collaborate on your tests with them! The results can be viewed in real-time, or you can store the data and look at trends over time.

To learn more, request a demo.