1

When compiling Rust, various additional checks are made for correctness. These include bounds checking, borrow checking for multithreading and memory ownership, and the like. Once compiled, these checks aren't made again: it's not verified safe to load code dynamically, for example a plug-in library.

In practice, how difficult would it be to extend this?

I'm not sure what's going on under the hood entirely, but it seems obvious rustc doesn't read and re-read the text of all of your source and try to work out from local semantics what's happening, again, and again, and again. It's more likely it identifies specific dangerous actions and tokenizes them, then applies some kind of mathematical solver to show that contractual guarantees are equivalent to the interactions between the various bodies of code, kind of like formal verification.

That means things like adding two mutable local variables, comparing values, and so forth aren't unsafe and don't need checking. A function foo() that receives ownership of memory can do unsafe things with that memory, and the borrow checker ensures that foo() receives ownership when it is called, so if foo() repeatedly does things that are only safe if it owns a certain passed piece of memory then we only really need to treat this as one check: is that memory passed to foo() safely?

If bar() is in the same body of code, it's checked that bar() passes memory to foo() in a way that gives foo() ownership. If it's not, then dynamically linking bar() into the program loses this checking.

I've been told it's materially impossible to store the information used to do this checking in the header of the ELF or PE binaries for the program and libraries. Like, not that it's complicated, but that it's not possible. Another dispute I've had over this is that you can't guarantee that the library you're loading is actually providing honest information, i.e. you can't know that the code you're loading is safe because it could be from a malicious source; but that's irrelevant, because you can say that about the program that's trying to dynamically load the code in the first place (did you download firefox.exe from mozilla.org, or did you pick it up from 2warez.totallylegit?), or you could just accuse the rust compiler of possibly being malicious.

Is it really all that complicated?

8
  • 2
    The linking part is easy. Rust uses an "rlib" file for that, similar to header files in C or C++. The difficult part is that Rust doesn't have a stable ABI. You can do dynamic linking, as long as all involved binaries were compiled with the exact same dependency versions and the exact same compiler. Calling conventions and type layout details are largely UB and may change without notice. Standard library types may change their internals between versions. ABI stability is a really hard problem, so you must use the C ABI if you want stable interop. But see lib.rs/crates/stabby
    – amon
    Commented May 27 at 5:42
  • It's not the linking part that's the issue, it's the part about the borrow checker and everything doing their job when dynamic linking without the source code.
    – John Moser
    Commented May 28 at 11:37
  • The rlib files contain all the type metadata of a crate/library (including generics and lifetime parameters) and is sufficient for typechecking. That typechecking happens when dependent crates/libraries are compiled, not at link-time. This works exactly like header files in C/C++ libraries, just automated. The linker is not aware of Rust or C++, it just sees symbols. But yes, this has the opportunity to be unsafe, which relates to the ABI stability problem.
    – amon
    Commented May 28 at 19:22
  • Maybe an aside but my understanding is that bounds checking in Rust is not confined to compilation.
    – JimmyJames
    Commented May 28 at 19:22
  • I'm a little confused by the assertion you are making here: "but it seems obvious rustc doesn't read and re-read the text of all of your source and try to work out from local semantics what's happening". If I understand what you are saying, that's not how Rust works, AFAIK. The language and compiler are designed to limit your actions to those that are known to be safe. If it was possible (or feasible) to prove or disprove the same things about C (compiled or otherwise), I would expect that someone would have already done that. A new restrictive language would not be needed.
    – JimmyJames
    Commented May 28 at 19:30

1 Answer 1

3

In short

Once compiled, these checks aren't made again.

Correct.

it's not verified safe to load code dynamically, for example a plug-in library.

Rust supports dynamic linking safely, so as long as:

  • The plug-in a native Rust interface.
  • The plug-in is never unloaded.

Then it'll be just like an "optionally" loaded dynamic library in a Rust process, and thus safe.

In depth

The Rust language has been carefully designed with separate compilation in mind, and therefore the compiler never needs to look inside a function body to determine whether it's safe to call it, or not.

If you ever wondered why function signatures can be so verbose in Rust, that's why: they encode all those properties.

The situation is more complicated on data, as there the compiler peeks in to determine variance of generic parameters, auto-traits such as Send and Sync, etc... But then again, for two pieces of software to communicate, they must also agreed on memory layout, etc... which requires an exact match, so it's no different.

In order to ensure an exact match, the symbols exported in Rust libraries are adorned with a cryptograhic hash which essentially fingerprints the world: compiler version, compiler flags, dependency versions, dependency features, etc... a slight version (bump a library version, disable a feature) and the fingerprint is different, leading to an error:

  • While linking, for a static library.
  • While loading, for a dynamic library.

It works!

What's about a C API then?

When talking plugins, you'll often find a recommendation to use a C API which is wildly unsafe of course.

The reason is that whole "fingerprint the world" issue. The only way, currently, to get the correct fingerprint, is to use the exact same compiler and dependencies.

Using the exact same compilers is easy. Using the exact same dependencies is not too complicated. It essentially means the application which wishes to have a plug-in should provide a plug-in API crate with the API to implement: types & traits.

But that's not really how plug-ins are used in the wild, is it?

Plug-ins are regularly distributed (in binary form) that work with versions of a software released years ago... and still work today, using backward compatibility practices, etc...

And that Rust is not capable of providing natively & safely today:

  • Either you need a C API, which is unsafe.
  • Or you need to adopt a 3rd-party library which will do all the heavy-lifting of forward/backward compatibility & ABI stability for you... and even then there may be limitations.

And that 3rd-party library will use unsafe code, of course.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.