RTEMS known failures¶
This page enumerates the RTEMS tests that do not PASS on Lince, with a root-cause classification for each. It is reference material: a fixed snapshot, not a live result table. For the machine-generated per-ELF outcomes see Test results; for the harness layout and scoring rules see the testing index.
Snapshot scope
As of 2026-06-03. Guest: RTEMS 5.3 built with RCC 1.3.2.
Configs: gr712rc_uniprocessor_config() (N=1) and gr740_config()
(N=4) for the uniprocessor (sp*) and SMP (smp*) suites
respectively. Every classification below is against the Gaisler/RTEMS
SIS reference simulator as the oracle.
How a test is scored¶
Lince scores RTEMS runs exactly as the upstream rtems-tools
rt/report.py scores a SIS run (see
tests/integration/rtems_csv_harness.hpp):
| Console contents | Outcome | Counts as a failure? |
|---|---|---|
*** BEGIN OF TEST … and *** END OF TEST … |
PASS | — |
*** BEGIN OF TEST … but no *** END OF TEST … (or a *** TEST CASE FAILED … marker) |
FAIL | Yes |
No *** BEGIN OF TEST … at all |
INVALID | No — excluded from scoring |
An INVALID run is one that fatals before the BSP debug UART is
initialised, so the BEGIN banner never reaches the console. On real LEON3
silicon, and on SIS, the early printk output goes to a 32-byte pre-UART
RAM ring instead — this is correct behaviour, not an emulator defect, and
the official scorer drops these from the denominator.
Classification summary¶
| Category | Meaning | Emulator defect? |
|---|---|---|
| External | The reference SIS fails them too (RTEMS 5.3 / RCC build issue) | No |
| Invalid | Fatals before the UART is up; no BEGIN banner; excluded from scoring | No |
| Flake | Timing-sensitive; passes on a cool host at governor=performance |
No |
| Recently fixed | Was a real emulator gap, now resolved (SIS agrees) | Done |
External (also fail on SIS)¶
These reach the BEGIN banner but never print END on both Lince and
SIS. The SIS N=4 runs were confirmed with
sis -leon3 -m 4 -freq 50 reaching BEGIN and never END, matching Lince.
They are RTEMS 5.3 / RCC build issues, not emulator bugs.
| Test | Suite(s) |
|---|---|
sp18 |
sptests (N=1) — GR712RC and GR740 |
spmrsp01 |
sptests (N=1) — GR712RC and GR740 |
sptimecounter03 |
sptests (N=1) — both SIS and Lince abort at init.c:56 (a bsd_binuptime monotonicity assertion) and enter error mode. Lince passed it before the GPTIMER IP fix only because the stuck-IP bit constantly offset the timecount; with IP correct Lince now matches SIS. |
smpmigration02 |
smptests (N=2 and N=4) |
smpmrsp01 |
smptests (N=2 and N=4) |
smpmutex02 |
smptests (N=2 and N=4) |
smpscheduler04 |
smptests (N=2 and N=4) |
smpwakeafter01 |
smptests (N=2 and N=4) |
Invalid by official scoring¶
These fatal before the debug UART is initialised, so the BEGIN banner
never prints; printk lands in the pre-UART RAM ring. SIS behaves
identically. The official scorer excludes them from the pass/fail
denominator — they are not failures.
| Test | Suite |
|---|---|
spfatal09 |
sptests (N=1) |
spfatal12 |
sptests (N=1) |
spinternalerror01 |
sptests (N=1) |
sptimecounter01 |
sptests (N=1) |
smpfatal09 |
smptests |
Flake (timing-sensitive)¶
| Test | Suite | Note |
|---|---|---|
sp04 |
sptests (N=1) | Machine-dependent. Passes on a cool host with CPU governor=performance; can FAIL under heavy parallel build load or thermal throttling. Not a real defect. |
Recently fixed¶
| Test | Suite | Fix |
|---|---|---|
spnsext01 |
sptests (N=1) | Resolved. The GPTIMER IP (interrupt-pending) control bit was modelled as write-0-to-clear; the GRLIB hardware and the SIS oracle make it write-1-to-clear. The inverted model left IP stuck set, so rtems_timecounter_simple's pending check over-compensated and rtems_clock_get_uptime() ran backwards across a tick boundary. |
A comprehensive Lince-vs-SIS sweep with scripts/oracle_compare.py (2026-06-05,
after the run_until slicing fix) scores 242 / 244 (99.2 %) agreement with
the reference across the sptests and the smptests-n2-mt profile (see
Test results → SIS oracle).
Of the two divergences, sp04 is the timing flake above; the single remaining
test the reference SIS passes and Lince fails is:
| Test | Suite | Divergence |
|---|---|---|
smplock01 |
smptests (N=2, MultiThread) | The MrsP/ticket-lock stop-timer does not fire under MultiThread, so the test never reaches END OF TEST while SIS does. The instrumentation rules out MCS-lock/memory-ordering causes; it is a timer-under-MT defect, the lone open Lince↔SIS gap. |
Everything else non-passing is in the external, invalid, or flake
categories above — smplock01 is the achievable ceiling's one real exception.
Background — recent recoveries¶
Two fixes in this work recovered previously-failing tests, both by matching the Gaisler SIS reference exactly:
WRPSRapplied immediately (was modelling the SPARC V8 3-instruction delay; SIS applies it at once). Recovered the GR740 N=4 SMP testssmpschededf03,smpschededf02,smpthreadpin01, andsmpload01; the GR740 N=4 smptests suite now stands at 48 / 54, its ceiling given the external failures above.- GPTIMER
IPwrite-1-to-clear (seespnsext01above).