Package com.code_intelligence.jazzer.junit

Annotation Interface FuzzTest

@Target({METHOD,ANNOTATION_TYPE}) @Retention(RUNTIME) @ArgumentsSource(com.code_intelligence.jazzer.junit.SeedArgumentsProvider.class) @ExtendWith(com.code_intelligence.jazzer.junit.FuzzTestExtensions.class) @ParameterizedTest(name="{0}") @Tag("jazzer") public @interface FuzzTest
A parameterized test with parameters generated automatically by the Java fuzzer Jazzer.

Test parameters

Methods annotated with FuzzTest can take either of the following types of parameters:

byte[]
Raw byte input mutated by the fuzzer. Use this signature when your fuzz test naturally handles raw bytes (e.g. when fuzzing a binary format parser). This is the most efficient, but also the least convenient way to write a fuzz test.
FuzzedDataProvider
Provides convenience methods that generate instances of commonly used Java types from the raw fuzzer input. This is generally the best way to write fuzz tests.
any non-zero number of parameters of any type
In this case, Jazzer will rely on reflection and class path scanning to instantiate concrete arguments. While convenient and a good way to get started, fuzz tests using this feature will generally be less efficient than fuzz tests using any of the other possible signatures. Due to the reliance on class path scanning, any change to the class path may also render previous findings unreproducible.

The FuzzTest annotation can also be applied to another annotations as a meta-annotation and then applies to all methods annotated with that annotation. This can be used to create reusable custom annotations for fuzz tests combined with other JUnit annotations such as Timeout or Tag.

Test modes

A fuzz test can be run in two modes: fuzzing and regression testing.

Fuzzing

When the environment variable JAZZER_FUZZ is set to any non-empty value, fuzz tests run in "fuzzing" mode. In this mode, the method annotated with FuzzTest are invoked repeatedly with inputs that Jazzer generates and mutates based on feedback obtained from instrumentation it applies to the test and every class loaded by it.

When an assertion in the test fails, an exception is thrown but not caught, or Jazzer's instrumentation detects a security issue (e.g. SQL injection or insecure deserialization), the fuzz test is reported as failed and the input is collected in the inputs directory for the test class (see "Regression testing" for details).

When no issue has been found after the configured maxDuration(), the test passes.

In fuzzing mode, only a single fuzz test per test run will be executed. All other fuzz tests will be skipped.

Regression testing

By default, a fuzz test is executed as a regular JUnit ParameterizedTest running on a fixed set of inputs. It can be run together with regular unit tests and used to verify that past findings remain fixed. In IDEs with JUnit 5 integration, it can also be used to conveniently debug individual findings.

Fuzz tests are always executed on the empty input as well as all input files contained in the resource directory called <TestClassName>Inputs in the current package. For example, all fuzz tests contained in the class com.example.MyFuzzTests would run on all files under src/test/resources/com/example/MyFuzzTestsInputs.

  • Optional Element Summary

    Optional Elements
    Modifier and Type
    Optional Element
    Description
    Lifecycle
    lifecycle
    Controls the JUnit lifecycle of fuzz tests during fuzzing.
    String
    maxDuration
    A duration string such as "1h 2m 30s" indicating for how long the fuzz test should be executed during fuzzing.
    long
    maxExecutions
    If set to a positive number, the fuzz test function will be executed at most this many times during fuzzing.

  • Element Details

    • maxDuration

      String maxDuration
      A duration string such as "1h 2m 30s" indicating for how long the fuzz test should be executed during fuzzing.

      To remove the default limit of 5 minutes, set this element to "".

      This option has no effect during regression testing.

      Default:
      "5m"

    • maxExecutions

      long maxExecutions
      If set to a positive number, the fuzz test function will be executed at most this many times during fuzzing. Otherwise (default), there is no bound on the number of executions.

      Prefer this element over maxDuration() if you want to ensure comparable levels of fuzzing across machine's with different performance characteristics.

      This option has no effect during regression testing.

      Default:
      0L

    • lifecycle

      Lifecycle lifecycle
      Controls the JUnit lifecycle of fuzz tests during fuzzing.

      During regression testing, fuzz tests always go through the full JUnit lifecycle for every execution regardless of the value of this option.

      Default:
      PER_TEST