Load Test Scenarios

A load test scenario is a recipe for how Loadster’s bots should run your scripts simultaneously in a load test. Each bot represents one human user.

A scenario can have multiple groups of bots, with each group running a different script. Additionally, each group of bots has its own schedule for ramping up, continuing at peak load, and ramping down.

During the ramp-up phase, the number of running bots starts at 0, and adds more bots gradually until the target number is reached. During the peak phase, bots run the script and repeat as long as there is time remaining in the test. During the ramp-down phase, the bots gradually exit until all are finished. The scenario specifies the bot count and the duration of each of these phases.

Concurrent Users vs. Total Users

The number of bots you specify in your scenario is analogous to the number of concurrent users hitting your site. That is, it’s the number of simulated users who are active at any given moment. If you say you have 75 concurrent users, that’s roughly equivalent to 75 humans somewhere in the world browsing your site simultaneously.

Your concurrent users may be different from the total users visiting your site over an extended period of time. For example, you might have 50,000 unique users (total users), but only 1,000 are active at a time (concurrent users).

This distinction is important, because when creating Loadster scenarios, you want to think in terms of how many concurrent users you will be simulating.

In Loadster, when a bot finishes an iteration of the script, it starts another iteration of the same script, as long as there is time remaining in the test. Therefore a steady load of a thousand concurrent bots, repeating the script continuously over a period of time, can actually simulate many thousands of visits.

Designing Realistic Load Test Scenarios

A load test scenario should be realistic. That is, the mix of behaviors (scripts) should be similar to what you can expect from your real users. If you go to the trouble of load testing, but simulate unrealistic behaviors or traffic patterns, the test results might be kind of useless.

For example, at any given moment, your online store might have:

• 300 users browsing the online store
• 250 users adding items to cart and checking out
• 150 users viewing their account and past orders

It is up to you, and your partners and stakeholders, to determine what user behaviors are most realistic and relevant for your site.

Once you’ve made this determination, you can create Loadster scripts to replicate each user behavior, and build a scenario with groups of bots running those scripts.

Editing Load Test Scenarios

A scenario consists of one or more Bot Groups. Each group can run its own script, or you can have multiple groups running the same script. Each group can have its own schedule and configuration as well.

Adding and removing bot groups

Every scenario needs at least one bot group in order to run. To create a bot group, click the Add a group button in the scenario editor.

You can run with just a single group, or add multiple groups to a scenario. Each group has its own configuration and schedule. When you run the load test, these groups will execute in parallel.

Setting the bot count

Adjust the number of bots in the group by clicking on the number and typing in a different number.

Each group can have anywhere from 1 to 9999 bots.

Selecting a script

Each bot group is configured to run a script. To select a different script, click on the script name and choose a different one.

You can use any script in the same project as your scenario.

Ramping up and down

Not all bots start at the same exact second. Instead, they start gradually according to your ramp-up schedule. The number of running bots gradually increases, continues at peak load for a while, and then gradually decreases.

Each of these phases (the ramp-up phase, the peak phase, and the ramp-down phase) has its own duration in minutes. You can also select a ramp-up formula and ramp-down formula to determine at what rate the load should be ramped up or down.

Ramp-up formulas:

• Linear - Bots start at a constant rate until the peak load is reached. This is the simplest formula.
• Natural - Bots start gradually at first, accelerate towards the middle of the phase, and then taper off. A real world example of this would be users gradually coming online during a shopping day.
• Aggressive - Bots mostly start right away and then taper off. A real world example of this might be a distributed online class where many users sign in to take an exam at the same time, or tickets going on sale for a popular concert.

Ramp-down formulas:

• Linear - Bots exit at a constant rate until all have exited.
• Natural - Bots exit gradually at first, with more and more exiting towards the middle of the window, and then it tapers off. A real world example of this would be business gradually slowing down at the end of a shopping day.
• Delayed - Bots mostly keep running longer, with the majority of them exiting towards the end of the phase. A real world example of this might be an auction or voting system in which activity happens right up to the deadline.

If you are trying to simulate realistic production load, we recommend selecting ramp-up and ramp-down formulas that match your real usage patterns. Sometimes, systems can perform well when load increases gradually, but may fail when a burst of traffic arrives all at once.

Selecting a load engine or cloud region

Each group of bots runs on either a self-hosted Loadster Engine, or one of the Loadster Cloud regions. Select the engine or region for this group by clicking on it.

If the site you’re testing is publicly reachable, we recommend selecting one of the cloud regions.

If you need to generate load from a private network, read more about Testing with Loadster Engine.

Additional Options

There are also options to limit the total number of iterations per group, simulate a slower connection speed, or override DNS entries.

Limiting the total number of iterations

There are times when you only want to run a certain number of iterations of your script. For example, you may have a requirement like “the system must process 1000 shipments within a 10 minute period”.

In these cases, Loadster can limit the total number of iterations. This limit is the cumulative limit across all bots in the group.

Once the group has reached this number of iterations, bots will begin to exit, even if there is time remaining in your test. If the number of bots in your group exceeds the number of total iterations, some bots may not even start.

Simulating network bandwidth

Normally, bots use all the network bandwidth available to them. If you want to simulate the impact of slower connections, you can configure the group to throttle bandwidth.

Slower clients will experience longer response times. Also, slower clients often cause the TCP connections with your server to stay active longer, which may cause it to hit a max connections bottleneck or other problems, that may not happen when all your clients are on fast connections.

Overriding DNS

When testing against multiple environments of your application, you may wish to direct traffic to certain hosts. Loadster can accomplish this by overriding DNS name resolution.

This is similar to editing your /etc/hosts file, but it automatically propagates out to all your engines as well as Loadster Cloud when you run the test. Only the bot group(s) with the override are affected.

Additional Playback Options

There are a few global options that can affect how a load test runs. You can change them on the Settings page.

Launching a Load Test

You’re ready to run a test! To start a load test, click the Launch Test button in the scenario editor.

Read Running Load Tests to learn more about monitoring and observing your load test.