
How QA Engineers Approach Non-Functional Testing Beyond the Basics

Key Takeaways
QA teams get reliable non-functional testing by giving each quality attribute a measurable target, matching it to a test that can fail, and running cheap checks on every merge.
- Vague expectations need rewriting as numbers a test can pass or fail, such as checkout under 2 seconds at forecast peak.
- Performance testing is the umbrella term. Load testing confirms capacity at expected and peak traffic, and stress testing pushes beyond peak to find the breaking point and recovery time.
- Most products eventually need baseline, load and stress tests, usually in this order: baselines during development, load tests before launch, stress tests before known spikes.
- OWASP's 2025 Top 10 ranks Broken Access Control at A01 and Software Supply Chain Failures at A03, so access-control tests and dependency scans come early.
- Test plans written against PCI DSS v3.2.1 are out of date, because the PCI Security Standards Council retired it on 31 March 2024.
- In CI/CD, quick checks such as SAST, dependency scans and short k6 smoke tests run on every pull request. Full load, stress and soak tests run before release.
Start with three or four attributes that would hurt most, and automate the cheapest check first.
A checkout flow can pass every functional test and still fail on sale day, when traffic spikes and the payment service starts timing out. Non-functional testing is how QA teams find that kind of failure before customers do. It checks how well software works (speed, security, recovery, usability, compliance), while functional testing checks what it does.
In practice, the approach that works is to give each quality a measurable target, choose a test type and tool that can fail against that target, and run the cheap checks on every merge. This guide covers that approach, with a worked e-commerce example and a comparison of performance, load and stress testing.
What is non-functional testing in QA?
Non-functional testing checks the quality attributes of software, such as response time under load, resistance to attack, recovery after failure, ease of use and legal compliance. Functional testing, through unit, integration, system and acceptance tests, confirms that each feature does what its requirement says. Picture a product search that returns the right items in nine seconds. It passes the functional test but fails the non-functional one.
Many teams still treat non-functional testing as one load test just before release. That leaves gaps. A single run against a quiet staging server won't catch a memory leak that appears after six hours, and it won't catch a vulnerable library someone added last sprint.
How do QA engineers turn business needs into non-functional requirements?
They start from the product's business context and turn each expectation into a number that a test can pass or fail.
The right numbers depend on the product. A travel search site needs results pages that load quickly on every device. A trading app has to stay secure and responsive under heavy load. A multiplayer game needs low latency and smooth rendering.
QA engineers agree these targets with business analysts, product managers and solution architects before anyone writes a test. Typical rewrites look like this:
| Vague expectation | Testable requirement | Test type |
|---|---|---|
| Checkout should be fast | Checkout response time stays under 2 seconds at forecast peak | Load |
| The site must survive sale day | Error rate stays below 1% at twice the forecast peak | Stress |
| It should run all weekend | Memory use stays flat over an 8-hour run at average load | Soak |
| Payments must be secure | No open high-severity findings on payment endpoints before release | Security |
| Everyone can use it | Key journeys pass automated WCAG checks and a screen-reader walkthrough | Accessibility |
Each number should come from something real, such as analytics for peak traffic or a contractual service-level agreement. Teams tend to ignore an arbitrary target the first time a test fails against it.
Non-functional testing types, metrics and tools
Each non-functional testing type measures a different attribute and uses its own tools. The table maps the main types, so you can check a test plan for gaps.
| Type | What it checks | Typical metric | Common tools |
|---|---|---|---|
| Performance | Speed and stability in normal use | p95 response time, throughput | JMeter, Lighthouse |
| Load | Behaviour at expected and peak traffic | Error rate and latency at peak | k6, Gatling |
| Stress | Breaking point and recovery | Failure threshold, recovery time | k6, Locust |
| Soak (endurance) | Degradation over many hours | Memory growth, error drift | k6, JMeter |
| Security | Vulnerabilities in code and the running app | Open findings by severity | OWASP ZAP, Burp Suite, Snyk |
| Usability | How easily users finish tasks | Task completion, steps per task | Persona walkthroughs, A/B tests |
| Accessibility | Use by people with disabilities | WCAG violations | Axe, WAVE, Lighthouse |
| Reliability | Failover and recovery from faults | Recovery time | Gremlin, Chaos Monkey |
| Compliance | GDPR, CCPA and PCI DSS controls | Controls passed | Checklists plus security tools |
Arbisoft's performance and security testing service also covers scalability, stability and volume testing, for systems that run 24/7 or process large data sets. Our guide to the top performance testing tools compares JMeter, BlazeMeter, Gatling, k6 and Lighthouse side by side.
Performance testing vs load testing vs stress testing
Performance testing is the umbrella term. Its baseline run measures speed and stability under normal use. Load testing is the type of performance test that applies expected and peak traffic to confirm capacity. Stress testing pushes beyond that peak to find where the system breaks and how quickly it recovers.
| Test | Goal | Load profile | Key metric | Example tool | Best for |
|---|---|---|---|---|---|
| Performance (baseline) | Meet speed benchmarks | Normal, steady traffic | Response time, throughput | JMeter | Every release, from early development |
| Load | Confirm capacity at peak | Ramp to forecast peak and hold | Error rate at peak | k6 | Pre-launch, sales and enrolment periods |
| Stress | Find the limit and recovery | Beyond peak, often until failure | Failure threshold, recovery time | Locust | Launches, campaigns, capacity planning |
The traffic pattern decides which test matters most. For a banking app, a baseline test might check that account balances load within two seconds during typical usage. For an online learning platform, a load test before enrolment opens checks that thousands of students can register at once without slowdowns. For a gaming platform launching a new title, stress testing at double the predicted traffic checks that servers recover quickly if they become overloaded.
Most products eventually need all three, usually in this order: baselines during development, load tests before launch and stress tests before known spikes.
Worked example of performance testing an e-commerce platform
The earlier version of this article described a performance test that our QA engineers ran for a large e-commerce platform (the client stays unnamed) before peak-hour traffic. The team set three targets first:
- Response time under 2 seconds.
- Support for 50,000 transactions per minute.
- Error rate below 1%.
- Define user journeys from real data, including browsing categories, adding and removing cart items, entering shipping details and completing purchases.
- Script the journeys. JMeter tested backend APIs such as search and checkout, k6 simulated user actions at scale, and Gatling checked that pages kept loading smoothly at peak.
- Turn the targets into pass/fail thresholds, so a run fails automatically when it misses one.
- Analyse database queries, server logs and frontend timings to find where the time goes.
- Fix the bottlenecks, retest, then monitor daily.
The bottlenecks were familiar ones. Database queries were slow because of poor indexing or missing caching. The payment gateway returned 500 errors when it failed. Some pages loaded slowly because of high-resolution images or too many HTTP requests.
The Grafana k6 documentation describes thresholds as "the pass/fail criteria that you define for your test metrics." We write "under 2 seconds" as a 95th-percentile target because an average hides the slowest requests:
export const options = {
thresholds: {
http_req_duration: ['p(95)<2000'], // 95% of requests under 2 s
http_req_failed: ['rate<0.01'], // error rate under 1%
},
};Watch the test data. If every virtual user requests the same product with the same cart, caches answer almost everything and the results look better than reality. Spread requests across many products, accounts and payment methods so the database has real work to do.
Security testing approaches in non-functional testing
QA engineers test security at every layer, from source code to the running application, and they share the work with dedicated penetration testers. On the e-commerce platform above, QA built security checks into the development lifecycle to find vulnerabilities early. The main approaches are:
- Static application security testing (SAST) scans source code for unsafe patterns before deployment.
- DAST (testing of the running application) attacks a running build, usually in staging, and focuses on areas such as login and payment.
- API security tests check authentication, authorisation and data integrity on each endpoint.
- Role-based access control (RBAC) tests confirm that admins, vendors and customers can reach only what their role allows.
- Token and transport checks confirm that OAuth and JWT are implemented correctly and that traffic uses TLS.
OWASP ZAP, Burp Suite and Snyk are common vulnerability scanners for this work. The OWASP Top 10 is a common checklist. Its 2025 edition ranks Broken Access Control at A01 and Software Supply Chain Failures at A03, which makes access-control tests and dependency scans early priorities.
Where usability and accessibility testing fit
Usability testing measures how easily people complete tasks. Accessibility testing checks that people with visual, hearing and motor impairments can complete the same tasks.
Usability bugs are different from design bugs. Alignment problems and colour inconsistencies are design bugs that affect how the interface looks. A search that takes five steps when two would do is a usability bug, even if every screen looks right.
QA engineers test usability by walking key journeys as different personas, such as a first-time buyer and a returning customer, and by comparing variants in A/B tests. Axe, WAVE and Lighthouse flag many WCAG violations automatically, and cross-device testing checks that layouts work on different screen sizes. Our guides to accessibility testing and usability testing in Agile development explain these methods in depth.
Reliability testing with failover, chaos and disaster recovery
Reliability testing checks that software keeps working, or recovers fast, when part of it fails. On an e-commerce platform, a payment gateway outage during a sale loses orders and damages customer trust.
- Failover testing checks that the system switches to backup servers and recovers quickly when a component fails, such as the payment gateway or the inventory service.
- Chaos testing injects disruptions, for example cutting database connections, to check resilience before peak sales such as Black Friday. Gremlin and Chaos Monkey can simulate these faults.
- Disaster recovery testing checks backup and restore after a simulated server crash or data loss.
- Stability and endurance testing run the system for long periods to find memory leaks and resource exhaustion.
Run your first chaos experiments in staging and limit each one to a single instance or dependency.
Compliance testing for GDPR, CCPA and PCI DSS
Compliance testing confirms that software meets the data protection and payment security rules that apply to it. For an online store, that usually means the GDPR, the CCPA and PCI DSS. Typical QA checks:
- Personal data is encrypted, and anonymised where required.
- Deletion requests actually remove a user's data.
- Cookie consent behaves correctly in each region where the store operates.
- Payment flows meet PCI DSS controls.
PCI DSS changed recently. The PCI Security Standards Council retired PCI DSS v3.2.1 on 31 March 2024, and 51 of the 64 new requirements in v4.x became effective on 31 March 2025. Update any test plan still written against the old version. These are legal obligations, so confirm the exact scope with your compliance or legal team.
How to run non-functional tests in CI/CD
Put each non-functional test at the pipeline stage where it runs fast enough to be useful. Quick checks run on every pull request and slow ones before a release.
| Pipeline stage | Non-functional checks |
|---|---|
| Every pull request | SAST, dependency scan, automated accessibility checks, short k6 smoke test with thresholds |
| Nightly | DAST scan against staging, load test at expected peak |
| Before release | Full load, stress and soak tests, manual usability and accessibility review |
| Scheduled in staging | Chaos and failover experiments, backup restore drills |
Thresholds turn these runs into gates. The Grafana k6 documentation says that when a threshold fails, k6 exits with a non-zero exit code, and CI systems treat a non-zero exit as a failed job. The e-commerce platform above added its non-functional tests to its CI/CD pipelines for continuous monitoring and early detection of issues. Shift-left testing is built on the same idea, and here it is applied to quality attributes.
Frequently asked questions
Can non-functional testing be fully automated?
Most of it can. Performance thresholds, security scans and many accessibility checks run unattended in CI/CD. Usability testing and parts of accessibility testing still need people, because a script cannot tell whether checkout feels confusing or whether a screen reader reads content in a sensible order.
Who owns non-functional testing on a software team?
QA engineers usually own test design and automation. Product managers and business analysts set the targets, architects confirm the targets are feasible, and developers fix what the tests find.
How often should load tests run?
A short load test can run nightly against staging. Full load and stress tests usually run before each major release and before known traffic spikes such as sales or launches. Teams that release several times a week often add a scheduled weekly full run, which catches gradual slowdowns that no single change caused.
What is the difference between soak testing and stress testing?
Stress testing applies more load than the system expects, usually for a short time, to find where it breaks. Soak testing applies ordinary load for hours to expose slow problems such as memory leaks or a growing error rate.
Where to start with non-functional testing
Pick the three or four quality attributes that would hurt your product most if they failed, and write a measurable target for each. Then automate the cheapest check first, usually a k6 threshold or a SAST scan on every pull request. Add load, stress and chaos runs once release dates and traffic forecasts justify the time they take. If your team can't cover performance and security testing alongside feature work, Arbisoft's software testing and QA services include performance, security, accessibility and automated testing.
...Loading Related Blogs









