Shrinking Test Failures in Rust Fuzzing
When libFuzzer or a property-testing framework finds a failing input, the raw bytes are often megabytes long and opaque. Shrinking is the process of automatically reducing that input to the smallest, simplest example that still fails. A well-shrunk failure is invaluable: it narrows the problem scope and makes debugging faster. This article teaches you to understand shrinking algorithms, verify shrunk examples by hand, and use framework-specific features to optimize shrinking performance.
What Is Shrinking?
Shrinking is a mutation strategy: take a failing input and repeatedly try removing or simplifying parts, checking if the simplified version still fails. If so, keep the simpler version and repeat. Stop when you can't simplify further. The result is a minimal example that reproduces the bug.
Example: suppose a JSON parser crashes on the input {"a": [1, 2, 3, 4, 5]} (128 bytes). Shrinking might produce:
{"a": [1, 2]} (failure)
{"a": [1]} (failure)
{"a": []} (failure? no, skip)
{"a": [1]} (stop; can't simplify more)
The final shrunk input is just {"a": [1]}, which you can reason about much more easily than the original.
Shrinking Algorithms
Different frameworks use different strategies. All involve:
- Candidate generation: Generate simpler versions of the failing input (remove bytes, reduce numbers, truncate strings).
- Test: Run the property/fuzz target on the candidate.
- Accept/reject: If it still fails, make it the new baseline. If it passes, try a different simplification.
- Repeat: Continue until no further simplification is possible.
quickcheck's Recursive Shrinking
quickcheck shrinks by recursively calling shrink() on each component:
// Pseudo-code; simplified
fn shrink_string(s: String) -> Vec<String> {
let mut results = vec![];
for i in 0..s.len() {
// Try removing each character
results.push(s[..i].to_string() + &s[i+1..]);
}
// Also try prefixes, empty, etc.
results
}
quickcheck generates all shrinks, tests them, picks the first that fails, and recurses. This is exhaustive but slow: a 1 KB failing string might generate tens of thousands of shrink candidates.
proptest's Guided Shrinking
proptest uses a smarter approach: each strategy knows how to shrink itself. For example:
// String strategy; shrinks by removing characters from the end
proptest::string::string_regex("[a-z]+")
// Shrinks toward ""
.prop_shrink_self_to(String::new())
proptest's shrinking is directional (toward a target value) and lazy (generates one candidate at a time, not all possible shrinks). This is 5–100× faster than quickcheck for large inputs.
Verifying a Shrunk Failure
After the framework reports a shrunk failure, verify it by hand. Create a simple test file:
#[test]
fn test_shrunk_failure() {
// The minimal failing input the framework reported
let input = r#"{"a": [1]}"#;
// Call the function that should panic or fail
assert!(parse_json(input).is_ok());
// If this test passes, the "failure" wasn't real (framework bug).
// If it panics, you've confirmed the minimal example.
}
Run it with cargo test -- --nocapture:
$ cargo test -- --nocapture
thread 'test_shrunk_failure' panicked at 'index out of bounds'
This confirms the minimal failure case. Now debug it.
Real Example: Shrinking a Regex Failure
Suppose your regex engine panics on a{100000}x (a followed by 100,000 repetitions, then x). libFuzzer or proptest shrinks it:
a{100000}x → panic
a{50000}x → panic
a{25000}x → panic
...
a{16}x → panic
a{15}x → ok (skip)
a{16}x → panic (keep)
After removing the x:
a{16} → ok
a{17} → panic
a{17} → panic (final)
The minimal failure is a{17} (17 as), much easier to understand than the original.
Shrinking in Practice: proptest Regression Storage
proptest saves shrunk failures to proptest-regressions/. Look at an example:
proptest-regressions/
└── my_module.txt
Content:
# shrinks to the minimal failing example
# you run `cargo test` and this is automatically loaded
regex::prop_invalid_escape = "[PERSISTED]\\x00[PERSISTED]"
This means proptest found a failure and shrunk it to a minimal case. If you re-run tests and this input still fails, proptest considers it a regression—it's checking that past bugs stay fixed.
Optimizing Shrinking Performance
Shrinking can take time (seconds to minutes for large inputs). Speed it up:
1. Use proptest, not quickcheck, for large inputs
proptest shrinks a{100000} in milliseconds; quickcheck takes 30+ seconds.
2. Narrow your fuzz target
Instead of:
fuzz_target!(|data: &[u8]| {
let _ = parse_json(data);
});
Try:
fuzz_target!(|s: String| {
let _ = parse_json(&s);
});
Fuzzing strings instead of bytes means libFuzzer generates valid UTF-8, shrinking toward simpler characters (', ", \n) rather than arbitrary bytes.
3. Reduce input size limits
In proptest, limit vector sizes:
proptest! {
#[test]
fn prop_parse(
vec in prop::collection::vec(0i32..100, 0..1000)
// ^^^^^^ max 1000 items
) {
// ...
}
}
Smaller max sizes shrink faster.
4. Use seeds for reproducibility
Once you've shrunk a failure and confirmed the minimal case, save it as a seed input:
# Re-run the exact same shrinking sequence
LIBFUZZER_SEED=path/to/failing/input cargo +nightly fuzz run fuzz_target_1
This deterministically reproduces the same failure without re-shrinking.
Understanding Shrinking Limits
Shrinking always finds a local minimum, not necessarily the global minimum. For example, if "abc" and "xyz" both fail, shrinking from "abcxyz" might reach "abc" (local min) and never explore "xyz" (global min). This is fine—either minimal example is useful.
Key Takeaways
- Shrinking reduces failures to minimal examples: A 1 MB crashing input becomes 10 bytes.
- proptest shrinks faster than quickcheck: If shrinking time matters, use proptest.
- Verify shrunk failures by hand: Confirm the reported minimal case actually fails.
- Regression storage helps: Save shrunk failures so they don't reoccur.
Frequently Asked Questions
Why does my shrinking take so long?
Your fuzz target or property test is slow (takes > 1 ms per iteration). Optimize the code being tested, use smaller input sizes, or switch to proptest for faster shrinking.
Can I disable shrinking?
Yes, but why? In proptest, set max_shrink_iters: 0 in your proptest_config!. In quickcheck, you can't disable it directly, but you can set very low iteration counts to minimize shrinking impact.
What if shrinking finds a "false positive"?
E.g., the shrunk input doesn't fail when you run it manually. This suggests non-determinism in your code (e.g., using randomness without a seed). Fix the non-determinism, then re-shrink.
How do I compare shrinking speed between frameworks?
Use PROPTEST_VERBOSE=1 or quickcheck's -vvv flag to print each shrink attempt. Count iterations and time them.
Can I customize shrinking for my types?
In proptest, yes: implement Strategy with a custom tree_shrink() method. In quickcheck, implement Shrink trait. This is advanced; usually the default shrinking is sufficient.