August 2021 - Traces

One of the challenges of load testing is diagnosing errors that happen during the test. You played a script in the editor with a single bot and everything worked great, but now when running with lots of bots at scale, your test throws a bunch of errors. What now?

When you’re load testing, it would be nice to have screenshots of every step like you get in the script editor. When things go right, you’d have screenshots to prove the bots interacted with your site the way you wanted them to. When things go wrong, you’d have screenshots to help you figure out what broke.

Unfortunately due to the large amount of data being processed in a load test, it hasn’t been feasible to capture screenshots of every single interaction from every single bot. We’d have to charge more and it also raises other architectural concerns.

The compromise (which we think is a pretty good one) is Loadster’s new Tracing feature, available now for all load tests.

Traces show up on the Traces tab of your live test, and in the Traces section of the test report.

Error Traces

Every time a Browser Bot encounters an error, it will generate a trace with:

  • A screenshot of what the browser looked like at that moment
  • A waterfall chart of resource timings from the page
  • Detailed error messages from the browser automation library when available
  • The bot number, step number, and URL where the error occurred

These traces will be labelled ERROR, and you’ll wonder how you ever load tested without them.

Info Traces

Additionally, the first Browser Bot in each group (Bot 0) will generate traces of every step, even if the step was successful. These traces are labelled INFO. Info traces have all the same diagnostic information as error traces, except the error, because there wasn’t one.

Even when the test is running well, it’s a good idea to spot-check some of the info traces to make sure the bots are seeing what you expected them to see as they navigate your site.

Trace Expiration

Traces are pretty heavy, so they only stick around for about a week after your test finishes. Older test reports will still have aggregate information in the graphs and tables about the types and frequency of errors, but you won’t be able to click them to view screenshots or expanded error details after the traces expire.