This simple alternative to Xray might be exactly what you’re looking for

7 minutes read

You’re searching for a test management app for Jira, and your first thought is Xray. It’s a solution that is popular and has a well-established position on the Atlassian Marketplace. But is it really your only choice? Are there no alternatives? It might be beneficial to ask yourself these questions before making your decision.

In this article we will explain why you don’t necessarily need Xray to manage your tests, and how a more cost-efficient option – QAlity Plus – is sufficient for most of your needs.

Learn more about QAlity Plus on the Atlassian Marketplace

Save half the money

Let us start with the most significant difference between Xray and QAlity Plus, and that is price. It’s not a secret that money plays an important role in the decision-making process, and the less expensive an app is, the better. Of course, it shouldn’t happen at the cost of the app’s functionality. The dream combination is, naturally, low price and high quality.

However, since quality often automatically means a bigger investment, less pricey options tend to get overlooked. Accustomed to expensive solutions being the only ones worth time and money, people often don’t realize that there are valuable alternatives waiting within their reach. It’s a trick of human perception. If something is expensive, it must be better than something that is cheap, right? But it’s not always true, and can sometimes stop us from exploring other options, and make us miss the opportunity to change our perspective on the topic.

Here’s a short cloud pricing comparison of QAlity Plus and Xray. As you can see, Xray is about two times more expensive than QAlity Plus.

QAlity Plus

FREE

USD 150

USD 370

USD 570

USD 1 770

USD 2 770

USD 3 770

Users

Up to 10 (Flat fee)

100

500

1 000

5 000

10 000

20 000

XRAY

USD 10

USD 500

USD 1 250

USD 2 000

USD 4 200

USD 5 700

USD 8 700

Pricing for monthly cloud payments based on Vendor’s Marketplace Pricing Pages.

This contrast in pricing undoubtedly makes you wonder – how are these two apps different? Is the less expensive one somehow lacking in comparison to the other?

These questions cannot be easily answered, not without acknowledging your needs and expectations, and confronting them with each app’s offer.

Below, you will find an overview of the most basic features that every test management app should have, sectioned into four categories: test writing, test cycles, test executions, and reporting. We will compare the features available for each of these categories in both QAlity Plus and Xray.

Hopefully, at the end of this article, you will be able to determine whether QAlity Plus can meet all of your needs, and whether you can or cannot save money by choosing it over Xray.

Test writing

There is no testing without the tests themselves. QAlity Plus and Xray offer a similar list of features related to test writing. In both apps you can:

All the test writing basics are covered in both apps. There are, of course, some differences as well. In QAlity Plus you can also create tests directly in an existing issue, a feature not present in Xray. On the other hand, Xray offers an option to add new test steps in between the existing ones, as well as manually set the position for each step.

These minor differences are not impacting the general functionality of either of the two apps. Despite QAlity Plus being less expensive than Xray, it allows you to achieve similar results in terms of test writing.

Test cycles/Test plans

When you have your tests already written, you’d probably want to organize them somehow, making it easier for testers to go through them according to your plans. QAlity Plus and Xray both offer test organizing features, however, they approach the topic differently.

In both cases, you can organize your tests in groups. In QAlity Plus these groups are called Test Cycles, whereas in Xray there are Test Plans. You can reorder tests within a Test Cycle/Test Plan by dragging and dropping them, or by manually setting their position (Xray only). In both apps you can access the overview of your Test Cycle/Test Plan, where all the crucial information on test executions and their statuses is gathered.

The difference between the apps is that QAlity Plus’ test grouping features are simpler and more straightforward, while Xray offers a more advanced and complex system of test organization. Apart from Test Plans, there are two other methods of organizing tests in Xray – Test Sets and Test Repository. It might be useful in more complicated testing operations.

However, if simply organizing your tests into the Test Cycles is enough for you, QAlity Plus will definitely suffice to meet your needs.

Test execution

Both apps offer various test execution features, allowing the user to perform tests, change their statuses, and monitor the overall execution status.

There is a similar range of test execution statuses available in Qality Plus and Xray, even though they come under different names:

QAlity Plus:

Xray

When it comes to your overall execution status, it changes automatically, along with test execution statuses. In QAlity Plus, if all tests are passed, overall execution status will automatically be set to Passed as well. In Xray, overall execution status is automatically adjusted to the current status of test executions.

Adding attachments to your test steps is also available in both apps. Additionally, in Xray you can add attachments to the whole test execution as well.

Overally, Xray is equipped with more flexible test execution options, such as custom statuses. However, both apps allow the user to successfully and comfortably execute tests, which is the primal function of every test management app.

Reporting

Without reporting there would be no efficient method of monitoring the testing process and its results. Reports are a crucial part of every test management app, and should definitely be considered when choosing a solution suitable for your team.

QAlity Plus and Xray offer various reporting options. The most fundamental types of reports – Test Execution Report, Test Coverage Report, and Traceability Report, are all available in both apps.

Test Execution Report

In QAlity Plus this report serves as a simple overview of all test executions for a chosen period of time. In Xray, it is split into multiple reports, built-in specific pages. In both cases reports can be exported, in QAlity Plus to an .XLSX file, in Xray to a CSV file.

Try it yourself. Check Test Execution Report on our demo server

Test Coverage & Traceability Reports

In QAlity Plus these two elements are combined into one comprehensive report, while Xray provides them separately. Reports can be exported to a CSV file (Xray only).

Try it yourself. Check Traceability Report on our demo server

QAlity Plus

Functionality

XRAY

✅

Free up to 10 users

⛔️

✅

Available for the Jira Cloud and the Jira Data Center

✅

Possibility to group executions

✅

Past executions history accessible from the execution page

⛔️

✅

Quickly raise and link bugs to the execution with automatically filled reproduction steps

⛔️

✅

Requirements and bugs traceability Report

✅

Test Execution Reporting

✅

Summary

While there certainly are benefits from using Xray, since in some areas it is more advanced and complex than QAlity Plus, not every user will need these additional features, for their testing to be comfortable and successful. In fact, the simplicity of QAlity Plus makes it more user-friendly, and can be seen as a benefit as well.

Nevertheless, every user needs to make the calculations for themselves, and decide whether the additional features that come with Xray are necessary for their testing, and worth spending two times the price of QAlity Plus.