What sample size for SIM batch acceptance testing?

IEC 2859-1 General Inspection Level II. For a 10,000 SIM batch: 200 SIMs. Verify ICCID matches manifest, IMSI ranges correct, applet version matches approved firmware, and basic attach succeeds. Cost: roughly EUR 500-1,000. Cost of skipping this step: EUR 50-150 per device in rework if a personalization error is found after assembly.

Why do SIMs that pass lab tests fail in the field?

Lab tests use reference boards with optimal antennas in controlled RF environments. Field failures are caused by: antenna mismatch (different gain, different placement), enclosure attenuation (metal housing, underground installation), network rejection codes from visited carriers (PLMN not allowed, GPRS services not allowed), multi-IMSI applet timing incompatibility with specific modem firmware, and APN configuration differences between home and visited networks. None of these appear on a lab bench.

What is the most commonly skipped test in IoT SIM qualification?

Multi-IMSI switch timing measurement. Most teams test the first IMSI, confirm it works, and skip testing IMSIs 2 through N. When IMSI 4 fails in the field — because its home carrier deactivated that IMSI range 6 months after deployment — the applet cycles through all IMSIs trying to find one that works, and the worst-case cycle time (10-15 minutes) was never measured because it was never tested.

IoT SIM Testing: How to Qualify a SIM Before Deploying 10,000 Units

Q: How long should an IoT SIM pilot deployment run?

60-90 days minimum. The first 30 days catch configuration errors (wrong APN, wrong IMSI, activation failures). The second 30-60 days catch environmental degradation — antenna corrosion, SIM contact oxidation, firmware memory leaks, and carrier network changes. A 90-day pilot with zero anomalies across 50 devices is the gold standard for production sign-off.

2025-06-07 · 7 min read · Technical Whitepapers

A SIM that passes lab tests on a reference board in the office fails in the field 30-60% of the time. The difference is not the SIM — it is the antenna, the enclosure, the APN configuration, the network rejection policy in the visited country, and the 3GPP Release version of the carrier core network your device is attaching to. A structured qualification process — lab bench, field pilot, batch acceptance — catches these failures before they become truck rolls. This is the procurement-grade testing framework: what to test, in what order, with what pass/fail criteria, for how long.

The standard IoT SIM qualification process in most enterprises consists of ordering 5 test SIMs, inserting one into a development board on a lab bench, confirming it attaches, and signing off. A SIM that passes this test has been validated in exactly one RF environment, on one network, with one module, at one temperature, with zero mobility, and with the default APN configuration that happened to work on that carrier. The failure rate when this SIM reaches field deployment across 20 countries and 5 module variants is 30-60% — not because the SIM is defective, but because the qualification process never tested the conditions that actually cause failures: network rejection codes, multi-IMSI rotation delays, APN mismatch on visited networks, and modem firmware incompatibility with the SIM applet.

Phase 1: Lab Bench — 3 Days, 10 Tests

The lab bench phase answers one question: does the SIM-modem combination work at the protocol level? Equipment needed: the target cellular module on a development board (not a reference design — the actual module that will ship), AT command terminal, and the production SIM (not a test SIM from a different batch). Test 1: Basic attach. Insert SIM, power on, verify IMSI attach and GPRS attach complete within 10 seconds. Pass/fail: attach time under 10 seconds on first attempt. Test 2: PDP context activation. Initiate a PDP context to the production APN. Verify IP address assignment, DNS resolution, and a 1 KB ping to the cloud endpoint. Pass/fail: under 500ms RTT, under 1% packet loss over 100 pings. Test 3: APN error handling. Configure an intentionally wrong APN. Verify the module receives the appropriate SM cause code (typically #33 — Requested service option not subscribed) and does not continuously retry. Pass/fail: back-off behavior observed, no more than 3 retries in 60 seconds. Test 4: SIM removal and reinsertion. Remove SIM while PDP context is active. Verify the module detects removal within 5 seconds and does not attempt further data transmission. Reinsert. Verify clean re-attach. Test 5: Network reject simulation. If the test environment includes a network simulator (Rohde & Schwarz CMW500 or similar), simulate GMM reject causes #11 (PLMN not allowed), #14 (GPRS services not allowed in this PLMN), and #7 (GPRS services not allowed). Verify the modem behavior for each — particularly FPLMN list management after #11.

Phase 2: Multi-Network and Multi-IMSI — 2 Days

This phase validates the SIM in the actual radio environment of the deployment country — not a lab simulator. Order 20 test SIMs from the production batch. Divide them across 5-10 physical locations that represent the deployment profile: indoor deep, outdoor urban, outdoor rural, mobile (vehicle-mounted), and basement/underground. At each location, for each of at least 3 carriers, measure: RSRP (Reference Signal Received Power — should be above -115 dBm for LTE-M, above -120 dBm for NB-IoT), RSRQ (Reference Signal Received Quality — above -15 dB), attach time, PDP activation time, ping RTT, and data throughput. Run each test at 4 time points across 24 hours to capture diurnal network load variation. For multi-IMSI SIMs, this phase also validates the steering logic. Force an IMSI switch — the most reliable method is to configure the device with the wrong initial IMSI and observe whether the applet detects the rejection, cycles to the next IMSI, clears the FPLMN list, and achieves attach within the spec window (40 seconds to 5 minutes for most commercial applets). Measure and log the actual switching time. If it exceeds 5 minutes consistently, the applet STATUS message polling interval may need adjustment — this requires OTA reconfiguration from the SIM provider.

Phase 3: Batch Acceptance Testing — 1 Day per Batch

When SIMs arrive from the personalization bureau, test a statistical sample before provisioning. Sample size: IEC 2859-1 General Inspection Level II. For a batch of 10,000 SIMs, that is 200 SIMs. For each sampled SIM: verify the ICCID matches the manifest, verify the IMSI(s) match the assigned ranges, verify the MSISDN (if pre-assigned) is correct, perform a basic attach test (can be automated with a scripted modem), and verify the SIM applet version matches the approved firmware version. This catches personalization bureau errors — wrong IMSI ranges, wrong applet versions, wrong MSISDN assignments — before 10,000 devices are assembled with defective SIMs. The cost of this test is roughly EUR 500-1,000 in lab time and automated test hardware. The cost of discovering a personalization error after device assembly is EUR 50-150 per device in rework.

Phase 4: Pilot Deployment — 60-90 Days, 20-50 Devices

Deploy 20-50 production devices with production SIMs to actual deployment sites. Monitor via CMP for 60-90 days. Track: attach success rate (target above 99.5%), PDP activation success rate (above 99%), data session drop rate (under 0.5% per device per day), average attach time, IMSI switch count per device per week, and data consumption vs plan allocation (verify a 50-100% buffer above measured consumption). The pilot answers the question no lab test can: what happens when the device sits in a metal enclosure for 8 weeks through temperature cycles, firmware updates, and network maintenance windows? A pilot of 50 devices for 90 days that shows zero anomalies is a stronger signal than 1,000 lab tests.

Procurement Checklist for SIM Qualification

1. Test with production SIMs from the actual batch — not engineering samples from a different personalization run. 2. Test on the production module and production antenna — not a reference design board. 3. For multi-IMSI: measure actual IMSI switch time across all loaded IMSIs, not just the first two. 4. For roaming: test in every target country, not just the home country. A SIM that works in Germany may fail in Brazil due to permanent roaming restrictions. 5. Batch acceptance: sample per IEC 2859-1 Level II. 6. Pilot duration: minimum 60 days. The first 30 days catch configuration errors. The second 30 days catch environmental degradation — antenna corrosion, SIM contact oxidation, firmware memory leaks. 7. CMP monitoring: configure alerts for attach failure rate above 1%, IMSI switch rate above 5 per device per day, and data consumption deviation above 50% of plan.