Created attachment 17900 [details] main.cc Consider the following c++ program: $ cat main.cc #include <iostream> int main(int argc, const char *argv[]) { std::cout << "Hello" << std::endl; } And the following build script: $ cat build.sh #!/bin/bash set -ue export CFLAGS="-g -c -flto -fsanitize=cfi -fsanitize-cfi-cross-dso -fno-sanitize-trap=all -fvisibility=default" export LDFLAGS="-flto -fuse-ld=gold -fno-sanitize-trap=all -flto -fuse-ld=gold -fsanitize=cfi -fsanitize-cfi-cross-dso -fvisibility=default -static-libstdc++" clang++ -O0 -o main_0.o main.cc ${CFLAGS} clang++ -o main_0 main_0.o ${LDFLAGS} echo "Running a binary built with -O0:" ./main_0 clang++ -Os -o main_s.o main.cc ${CFLAGS} clang++ -o main_s main_s.o ${LDFLAGS} echo "Running a binary built with -Os:" ./main_s When running the script it outputs: $ ./build.sh Running a binary built with -O0: Hello Running a binary built with -Os: /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ostream:113:9: runtime error: control flow integrity check for type 'std::basic_ostream<char> &(std::basic_ostream<char> &)' failed during indirect function call (/usr/local/google/home/krasin/play/cfi-inline/main_s+0x71b70): note: std::basic_ostream<char, std::char_traits<char> >& std::endl<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&) defined here SUMMARY: CFI: undefined-behavior /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/ostream:113:9 in There are few unclear moments in the report: 1. Why is it only happening at -Os, but not -O0. The answer is likely related to inlining, but the exact cause is still not found. 2. Why isn't the address symbolized (main_s+0x71b70) and what does it point to? 3. Given that the main linkage unit has CFI and non-CFI together, it's an unsupported combination. Do we want to do anything in order to support it? It has been the reason for a major loss of productivity for a user.
Created attachment 17901 [details] build.sh
Per Eugenii Stepanov: Regular CFI requires that all code is built with CFI, i.e. anything that is not built with it can not be a valid virtual call target. It also requires that the target is in the same DSO as the call site. These requirements are relaxed in the Cross-DSO CFI mode (-fsanitize-cfi-cross-dso): calls can go across DSO boundary, and, if a library does not contain any code built CFI, all calls into it are implicitly allowed. What still does not work is mixing CFI and non-CFI code in the same DSO. This is what the example does by statically linking libstdc++ into the executable. We've discussed a few ways to support this, but they seem complex and error-prone.
Linking a CFI-enabled object with a non-CFI object or a non-CFI static library is an error. The current linker behavior is to link these objects together into a binary. No warnings or errors are emitted. Would it be possible to mark CFI-enabled objects (perhaps via some metadata field?) to indicate they are incompatible with non-CFI objects, and emit a linker error (or warning) when linking these incompatible objects? I suspect there are other situations (perhaps the other sanitizers?) where similar incompatibility may exist.
This could be useful, but there are exceptions: for example, the cfi runtime library itself is a static library which is not built with cfi. In general, uninstrumented object files or static libraries that are not called indirectly are perfectly fine. Also, this check would need to be implemented in the linker, which is always a problem to deploy.