Final project for CS 378H: Network Security with Anirudh Goyal

We discovered and exploited a vulnerability in SpiderMonkey, Firefox's JavaScript engine. We identified a flaw in the JIT compiler's bounds check elimination optimization that could be weaponized to achieve arbitrary memory read/write capabilities.

The Vulnerability

The bug existed in the IonMonkey JIT compiler's Global Value Numbering (GVN) optimization, specifically in bounds check elimination (BCE). A bounds check is considered redundant if it's dominated by another bounds check with the same length and the indices differ by only a constant amount.

The vulnerability occurred because the congruentTo() function in MIR.h failed to verify whether two bounds check nodes had identical fallibility status. This allowed the compiler to incorrectly eliminate a fallible bounds check by treating it as congruent to an infallible one, removing necessary safety validations.

Exploitation Process

Initial Challenges

Our first strategy—using out-of-bounds reads on strings to locate objects in memory—proved impractical. Objects allocated in the tenured heap (rather than the nursery) were scattered across memory with distances of ~500KB, making precise address location impossible with only ~30 usable malicious accesses before bailout.

Breakthrough Strategy

We shifted focus to TypedArrays and ArrayBuffers, leveraging several key insights:

• ArrayBuffer headers contain metadata about underlying memory chunks, located 64 bytes before the data region
• Consecutive allocation of small ArrayBuffers (<96 bytes) creates predictable, back-to-back memory layouts
• Int32Array views store a data pointer at offset 56 bytes (the 7th qword) that controls which memory region they access

By allocating 50,000 consecutive ArrayBuffers, we could reliably leak addresses and construct a memory-manipulation chain.

The Exploit Chain

1. Created three consecutive Int32Views (v89, v90, v91) on sequential ArrayBuffers
2. Used the bounds check bug to read out-of-bounds from v89, leaking the memory addresses of v90 and v91
3. Overwrote v91's data pointer to point at v90's metadata using out-of-bounds writes
4. Modified v90's data pointer via v91 to target arbitrary memory locations
5. Achieved primitive read/write operations through v90 array access

Key Technical Insights

We employed several sophisticated techniques:

• NaN boxing awareness: Understanding how JavaScript engines box/unbox 64-bit values with metadata tags
• Type inference gaming: Heating functions with generic objects to prevent TypeBarrier bailouts that would otherwise terminate exploitation
• GVN optimization mechanics: Recognizing how constant offsets between array indices trigger redundant check elimination

Tools and Debugging

We used:

• GDB for memory inspection and breakpoint analysis
• Iongraph (GraphViz visualizer) for analyzing IonMonkey intermediate representations when standard GDB debugging failed
• Custom Int64 helper functions for address arithmetic on 64-bit systems

Outcome

The exploit successfully demonstrated a complete read/write memory primitive, showing how a seemingly small optimization bug can cascade into critical security vulnerabilities. This finding parallels Google's decision to disable bounds check elimination entirely in V8, underscoring verification challenges in JIT compiler optimizations.

🍅

← Back to all posts