In 233722, Lang removed our existing tests for the small code model with MCJIT because they were entirely non-deterministic. We should resurrect these tests to ensure we have test coverage of the small code model as used through MCJIT/OrcJIT, but do so, we need to fix the non-determinism issue first. Lang explained the issue in his submission comment as: "These regression tests are supposed to test small code model support, but have been XFAIL'd because we don't have an in-tree memory manager that can guarantee a small-code-model compatible memory layout. Unfortunately, they can occasionally pass if they get lucky with memory allocation, causing unexpected passes on the bots. That's not very helpful. I'm going to remove these until we have the infrastructure (small-code-model compatible memory manager) to run them properly." In email, he expanded: "If we add a simple custom memory manager to llvm-rtdyld that's compatible with the small memory model (maybe just reserve ~1Mb up front? We'll never need more than that for regression tests). Then we can add a flag to chose that memory manager and tests specifically for small-code-model relocations." Having a pluggable memory manager with a fixed allocation area for small test cases seems like a generally useful construct.
This doesn't sound impossible to do. Do we expect any conceptual issues? IIUC this is mostly a memory management issue. The old tests have been removed with this commit: https://reviews.llvm.org/rL233722 I remember reading about OrcV1 incompatibilities with the small code model, but cannot recall the details. Are there extra show-stoppers in the particular JIT engines? (MCJIT, OrcV1, OrcV2) Same question for the linkers/loaders: does this affect RuntimeDyld the same way as JITLink? Can we roughly summarize the steps it would take to bring back small code model tests? Thanks!
I've got a better handle on this now than I used to. Let me see if I can summarize the state of things. Background: Relocations in small code model can be broken into two groups. (1) Those that *require* that symbols be allocated within constrained ranges. For example, the MachO SUBTRACTOR relocations can express the 32-bit displacement between two global symbols: A: .long B - A The result of a 32-bit subtractor is only guaranteed to fit the storage if A and B are guaranteed to be allocated within +/- 2Gb of one another. (2) Relocations that can be made to work even if the target is allocated at arbitrary address. For example, a 32-bit branch relocation to an external symbol: If the target is out of range a jump-stub can be used to reach the target anywhere in the address space. Relocations of type (1) are equally "supported" in both RuntimeDyld and JITLink: They will work if the user-supplied allocator can guarantee that symbols remain in-range, and may fail otherwise. Relocations of type (2) *can* be supported in RuntimeDyld, but I would argue that JITLink is better at it (by design): RuntimeDyld's limited notion of "stubs" makes it difficult to mix, for instance, GOTs and PLTs. JITLink's notion of an extensible AtomGraph makes it easy to add both. So: In general we want to support (and test) relocations of both kinds. Relocations of type (2) come up everywhere (and are more or less unavoidable), relocations of type (1) are rarer, but do come up in use-cases that we care about (e.g. Swift metadata). We can write relocation (non-execution) tests (i.e. rtdyld-check, jitlink-check tests) already: The respective testers simulate allocation starting contiguously from low memory (>4k) by default. So we can write (and in JITLink, have already written) tests for each of the small code model relocations for the target that verify that bits are fixed up as expected. If we want execution tests too, we have two options, (A) and (B): (A) Test using object files. This has the advantage of allowing us to specify precise, minimal test cases, since we're not relying on codegen to produce the relocations that we are interested in. (i) Test using llvm-rtdyld: Davide Italiano already wrote a small-code-model compatible allocator for llvm-rtdyld. One problem we will encounter is that for several formats/architectures (e.g. MachO/x86-64) RuntimeDyld *doesn't* fully support the small code model yet, and would require some work to bring up to speed. (ii) Test using llvm-jitlink: Davide's allocator could be ported to JITLink, at which point (at least notionally) we can run arbitrary execution tests, since the small code model is already supported. The downside is that (so far) JITLink is MachO/x86-64 only. Neither (i) and (ii) would allow us to turn the LLVM IR test cases back on. (B) Test using IR. If Davide's allocator were ported to lli we could notionally turn the old, small-code-model tests back on. However, lli is using RuntimeDyld by default so we will hit the same limitation described above: RuntimeDyld does not fully support the small code model on all platforms, and getting it to support the small code model may take some work. Since the old tests must (presumably) have passed at some point, I guess they must not hit the small code model relocations that RuntimeDyld doesn't not support. If turning the old tests back on were a priority I would take path (B) and hope for the best. This approach seems clunky though. As things stand, my focus is going to be getting JITLink up and running for more targets, and I think approach (A.ii) is the best one.
Minor update: llvm-jitlink now has a -slab-allocate to support small code model. The MachO/x86-64 relocation test tests all relocations including those used in the small code model. I'm going to leave this open for now to track other formats/architectures and re-enabling of the lli small code model tests.
From what I see in JITLink, small code model should work with the Macho/arm64 backend as well. I just ran an ad-hoc test on TOT that supports the assumption. However, it's quite hacky as I don't have a proper hardware at hand. @Lang can you confirm that? Anything I should take care of? Thanks
JITLink supports small code model for both arm64 and x86-64