[Computer Security] Retrofitting Fine Grain Isolation in the Firefox Renderer

Retrofitting Fine Grain Isolation in the Firefox Renderer

Firefox and other major browsers rely on dozens of third-party libraries to render media content, but these libraries are a frequent source of vulnerabilities. To mitigate this threat, the authors migrate Firefox to an architecture that isolates these libraries in lightweight sandboxes.

Why and how sandbox the renderer?

• Media content such as images and video are rich attack vectors, as web applications allow them to be shared pervasively.
• Content libraries can be effectively sandboxed, as they require little privilege to operate.
• The existing library-renderer interface provides a natural place to partition code.

Isolation strategies：per <renderer, library, content-origin, content-type>

• Per-renderer sandboxing prevents attacks across tabs and browsing profiles.
• Per-library ensures that a weakness in one library does not impact any other library.
• Per-content-origin sandboxing prevents cross origin (and thus cross site) attacks on content.
• Per-content-type sandboxing addresses the problem of passive content influencing active content.

Attacker Model: They assume a web attacker that serves malicious (but passive) content—from an origin they control or by uploading the content to a trusted origin—which leads to code execution (e.g., via a memory safety vulnerability) in a RLBox sandbox.
RLBox ensures that such an attacker can only affect (corrupt the rendering of and potentially leak) content of the same type, from the same origin.

Pitfalls of retrofitting protection

Changing the threat model to assume libraries are untrusted and modify the renderer-library interface accordingly.

Insecure data flow

• Failing to sanitize data: Data received from untrusted libraries (return values, callback parameters) is dangerous, especially for pointers, which are easy to be manipulated by the attackers as gadgets.
• Missing pointer swizzles: Some sandboxing techniques use different ways of representing pointers, e.g., 32-bit to 64-bit. “Swizzling” is the process of converting between these representations. If you forget to swizzle a pointer, it becomes invalid, and the renderer might access the wrong memory.
• Leaking pointers: If the renderer gives a pointer to its own internal data to an untrusted library, the library might be able to figure out the layout of the renderer’s memory.
• Double fetch bugs: The shared memory between renderer and untrusted libraries of RLBox may be modified by attackers. The attackers can modify the data after check and before the value is fetched again and used.

Insecure control flow

• Corrupted callback state: The library might store some state related to the callback. If the library can corrupt this state, it might be able to trick the renderer into calling the wrong function.
• Unexpected callback invocation: Even if the library calls the correct callback function, it might call it at the wrong time.
• Callback state exchange attacks: In a multithreaded renderer, multiple threads might be using the same library. If the library mixes up the state between these threads, it can lead to data races and use-after-free vulnerability.

RLBox: automating secure sandboxing

RLBox aims to automate security checks, avoid changes to third-party libraries, and simplify the migration process of incorporating sandboxing.

• Automate security checks: RLBox automatically handles tasks like swizzling pointers, checking if pointers are valid, and identifying data that needs to be validated. This reduces the amount of manual work and the risk of errors.
• Minimize renderer changes: RLBox is designed to minimize the amount of code in the renderer that needs to be changed. Validation is only enforced when necessary, allowing for a more gradual migration.
• Efficiently share data structures: RLBox helps the renderer and the sandboxed library share data efficiently. It ensures that shared data is in a safe memory area and avoids unnecessary copying of data.
• Assist with code migration: RLBox provides helpful error messages during the process of adding sandboxing. These messages guide developers on what code needs to be changed for security.
• Bridge machine models: Sandboxing mechanisms and the main application might use different ways of representing data. RLBox can automatically handle the translation between these.

Data flow safety

• Tainting of sandbox data: Data coming from the untrusted sandbox is marked as “tainted” to indicate that it might be dangerous. The only way to remove a taint is through explicit validation.
• Data flow into the renderer:
  sandbox_invoke(): Functions called in the sandbox return tainted data.
  sandbox_callback(): Parameters of functions called by the sandbox into the renderer are tainted.
  Tainted pointers: Pointers to data in the sandbox are also tainted.
• Data flow into the sandbox: Data passed to the sandbox must be either simple numbers or tainted. Untainted pointers (pointers to the renderer’s memory) are not allowed.
• Benefits of tainted pointers:
  • Prevents passing untainted pointers to the sandbox.
  • Automatic pointer swizzling/unswizzling.
  • Automatic checks to ensure pointers point to sandbox memory.

Data validation

A validation method takes a closure (C++ lambda) that unwraps the tainted type by performing necessary safety checks and returning the untainted result.

verify(): Used to validate simple data types that have been copied into the renderer’s memory.
copyAndVerify(): Copies data from the sandbox into the renderer’s memory before validating it, to prevent double-fetch bugs.
unsafeUnverified(): Removes the taint without any checks; should be used with caution.

Validation in the presence of double fetches:
freeze(): Creates a copy of a tainted variable and ensures the original hasn’t changed.
unfreeze(): Re-enables writes to a frozen variable.
To prevent accidental misuse of freezable variables, RLBox disallows the renderer from reading freezable variables and fields until they are frozen.

Writing validators:
Maintaining application invariants: Validating data based on what the renderer expects.
Checking library invariants: Validating data based on what the library guarantees.

Control flow safety

Control transfers via callbacks:
sandbox_callback(): Function used to explicitly register callbacks from the sandbox.
unregister(): Method to unregister callbacks.
RAII semantics: Automatic unregistration of callbacks.
Thread local storage: Storage for callback state to prevent leaks.

Non-local control transfers:
Protecting control flow via setjmp()/longjmp() (used for exception handling).
Storing jmp_buf in renderer’s thread local storage.
longjmp() trampoline function.

Simplifying migration

• Step 1 (creating the sandbox): This step involves creating a sandbox for the library using the “None” sandboxing architecture.
  A “None” sandbox doesn’t provide isolation but redirects function calls to the existing, unsandboxed library.
  RLBox’s type-level guarantees (like taint tracking) are still enforced.
• Step 2 (splitting data and control flow): This step focuses on modifying function calls to use the sandbox_invoke() API.
  RLBox flags calls that pass pointers to the sandbox as compile-time errors.
  Objects passed to the library are allocated using sandbox_malloc() to resolve these errors.
  An “unsafe alias pattern” can be used to temporarily bypass taint checking and allow the code to compile.
• Step 3 (hardening the boundary): This step involves removing the “unsafe alias pattern” and ensuring that all data from the sandbox is handled safely.
  The “unsafe alias pattern” is gradually moved downwards in the code.
  Code that was using the alias is converted to use tainted values and validation functions.
  By the end of this step, all tainted values are validated, and no instances of unsafeUnverified() remain.
• Step 4 (enabling enforcement): This final step replaces the “None” sandbox with one that enforces strong isolation.
  The statically-linked library is removed, and the sandbox type is changed to None_DynLib (which dynamically loads the library).
  Any remaining calls to the library that don’t use sandbox_invoke() will result in a compile-time error.
  Finally, the sandbox type is changed to a sandbox type that enforces isolation.