This lab testing methodology explains how every product on SpywareInfoForum is tested. Every review follows the same process. No product gets a score without completing every step. Furthermore, the methodology is fixed before testing begins. Therefore scores reflect measurements, not opinions. Moreover, the same lab testing methodology applies to every category — VPNs, password managers, hardware keys, and network devices. Because results use identical test conditions, scores are directly comparable. In addition, every failure point found during testing is published. However uncomfortable, failures are never hidden. For independent standards see NIST Cybersecurity Framework and Cure53 audit methodology.
Contents
Lab Testing Methodology
How Nolan Voss Tests Every Product
Every review on SpywareInfoForum starts with a real lab test. No marketing claims. No vendor benchmarks. No single-device speed tests. Real hardware. Real measurements. Real failure documentation.
Lab Testing Methodology — The Full Process
Every product goes through the same lab testing methodology before a score is assigned. Nothing ships without completing every step below.
01 — VPN Lab Testing Methodology
iperf3 on dedicated 1Gbps VLAN. 3 runs averaged. Both WireGuard and OpenVPN protocols tested. CPU usage monitored via Proxmox dashboard simultaneously.
1000 pings to nearest exit node. Minimum, maximum, and average recorded in milliseconds. Compared against 4ms baseline LAN latency.
500 iterations on dnsleak.com and dnsleaktest.com. Pi-hole query log checked simultaneously. Any query resolving outside the tunnel = automatic fail.
Physical WAN uplink unplugged on Protectli pfSense appliance. Time from interface down to confirmed traffic termination measured in Wireshark. Any packet leaking after failover = fail.
IPv6 connectivity checked via test-ipv6.com and ipleak.net while tunnel is active. Any IPv6 address visible = automatic disqualification regardless of other scores.
Full packet capture during 30 minute idle session. Traffic analyzed for unexpected outbound connections to third-party analytics or telemetry endpoints not disclosed in privacy policy.
Rated Full, Partial, or None. Full = native WireGuard config files available. Partial = workaround required. None = no supported integration path.
Every VPN runs for a minimum of 7 consecutive days before scoring. Connection stability, reconnection behavior, and edge cases documented throughout.
02 — Password Manager Lab Testing Methodology
YubiKey 5 NFC tested via FIDO2 on Linux, Windows, and macOS. Registration, authentication, and recovery flow all tested. Backup key registration verified. Any platform failure documented.
Command line credential retrieval tested on Ubuntu 22.04 and Debian 12. Ability to inject credentials into shell scripts and retrieve SSH keys programmatically verified.
Self-hosted option deployed in Proxmox LXC container where available. SSL configuration, backup procedures, and mobile sync all tested. Deployment time recorded.
Breach monitoring verified against Have I Been Pwned database. Alert time measured from breach publication to in-app notification. Manual vs automatic monitoring noted.
Wireshark capture during vault sync operations. Traffic analyzed for plaintext credential exposure. Encryption protocol and key derivation function verified against published documentation.
Emergency access and account recovery flow tested end to end. Time from lockout to credential recovery measured. Any gap in recovery options documented as a failure point.
03 — Hardware and Network Device Lab Testing Methodology
iperf3 between pfSense and test node on dedicated VLAN. Measured at idle, 50% load, and 100% load. CPU usage on device under test recorded at each load level.
Simulated port scan and known malware signature trigger from isolated test VLAN. Alert detection time measured in seconds. False positive rate documented over 7 day period.
Cold boot to fully operational measured in seconds. Relevant for failover scenarios where device must recover quickly after power loss or firmware update.
802.1Q VLAN tagging tested with minimum 3 VLANs — management, trusted LAN, and isolated test VLAN. Inter-VLAN routing rules verified via firewall log analysis.
Lab Testing Methodology — Scoring System
Every product receives a score from 1.0 to 10.0 based on the lab testing methodology results above. Furthermore, the scoring is weighted by category — a VPN that fails the kill switch test cannot score above 7.0 regardless of speed. Moreover, a password manager that fails YubiKey integration cannot score above 7.5. However, partial failures are noted specifically rather than treated as binary pass/fail where the failure is minor.
| Score Range | Meaning | Recommendation |
|---|---|---|
| 9.0 — 10.0 | Passed all tests. Minor failure points only. Best in class for home lab use. | ✅ Strongly recommended |
| 8.0 — 8.9 | Passed core tests. One notable failure point. Good for most home lab users. | ✅ Recommended with caveats |
| 7.0 — 7.9 | Passed most tests. Multiple failure points. Acceptable for casual use only. | ⚠️ Conditional recommendation |
| Below 7.0 | Failed one or more critical tests. DNS leak, IPv6 leak, or kill switch failure. | ❌ Not recommended |
Lab Testing Methodology — Editorial Independence
SpywareInfoForum participates in affiliate programs with VPN providers and security software companies. Therefore it is important to state clearly how affiliate relationships affect — and do not affect — the lab testing methodology and scoring on this site.
Affiliate relationships do not influence scores. Furthermore, products are not ranked based on commission rates. In addition, vendors cannot pay for guaranteed positive coverage or specific placement in comparison tables. Because the lab testing methodology is fixed before testing begins, scores reflect measured performance rather than editorial preference.
Failures are always published. However uncomfortable, every genuine failure point found during lab testing is documented in the final review. Moreover, products that fail critical tests receive scores below 7.0 regardless of any affiliate relationship.
For more information on our affiliate relationships see our Affiliate Disclosure page. For the cybersecurity framework we reference during testing see NIST Cybersecurity Framework and independent audit results from Cure53.
Lab Testing Methodology — Related Pages
Best VPN for Home Lab 2026
The lab testing methodology applied to 14 VPNs. Kill switch timing, DNS leak results, pfSense integration ratings — all measured using the process described on this page.
See the results →Best Password Manager 2026
The lab testing methodology applied to 10 password managers. YubiKey integration, Linux CLI, and self-hosted deployment all tested and scored.
See the results →About Nolan Voss
12 years in enterprise IT security. Former penetration tester. The person behind every test on this site and the lab setup described on this page.
Read more →Browse All Reviews
Every product reviewed using this lab testing methodology. VPNs, password managers, hardware security keys, network devices, and privacy tools.
Browse reviews →Home Lab Security Setup Guide
6-layer security implementation for Proxmox home labs — pfSense, VLANs, Pi-hole, WireGuard kill switch, and YubiKey MFA tested in Austin TX.
Read the guide →