THE GARDEN

Integer Overflow (Signedness Bypass)

Updated 13 Sep 2026

• •

  What it is

  • •

    Inconsistent signedness or overflowing size arithmetic can turn a checked value into an unsafe allocation or read length. Distinguish signed-to-unsigned conversion from addition or multiplication wraparound.

• •

  How it works

  • •

    Signedness: a negative signed size can pass an upper-bound check, then convert to a large unsigned length. The result depends on the destination type width.

    • •

      -1 converted to uint32_t → 4294967295
      -1 converted to 64-bit size_t → 18446744073709551615
      

    • •

      Signed comparisons use jl/jle/jg/jge; unsigned comparisons use jb/jbe/ja/jae. Inspect the complete validation and conversion path, not only the jump mnemonic.

  • •

    Addition: unsigned size + 1 may wrap to zero, allocating too little memory if the later write uses an unwrapped size.

  • •

    Multiplication: count * item_size may wrap in a narrow unsigned type. The later allocation or read must be checked for a mismatched size calculation.

• •

  When to use it

  • •

    Look for a signed input size followed by an upper-bound-only check and conversion to an unsigned length.

  • •

    Check for a missing lower bound and the later conversion; a signed comparison is not inherently unsafe.

  • •

    Inspect additions and multiplications before allocation. Check widths, conversions, and whether overflow is detected.

  • •

    Confirm unusual input behavior in normal execution as well as under GDB; account for environment and input differences.

  • •

    A huge requested read may fail, return short, or wait for input. Inspect the actual return value before assuming it caused an overflow.

  • •

    When input controls both count and item size, check the product for overflow even if each input looks reasonable.

  • •

    A printed size and the value used by a bounds check may differ in width. Trace both calculations.

  • •

    The declared size and transmitted payload length may differ; determine what the input loop actually consumes.

  • •

    If the input loop waits for more bytes, an EOF may let it finish. Verify the protocol before using p.shutdown("send").

• •

  Examples

  • •

    A signed check size <= MAX accepts -1; converting it to unsigned size_t produces a huge length.

  • •

    In 32-bit unsigned arithmetic, 65536 * 65536 wraps to 0, potentially defeating a check on the product.

• •

  pwntools

  • •

    p.sendline(b'-1')  # Only for a verified signedness bug in the lab.
    p.send(payload)
    

• •

  Debugging

  • •

    disas challenge  # Inspect size checks, widths, and conversions.
    

• •

  Related

  • •

    Buffer Overflow

  • •

    Unavailable reference

• •

  Cards

  • •

    Why does sending -1 as a "size" ever get more data accepted, not less?

    • •

      A missing lower-bound check can admit a negative signed value, which converts to a large unsigned length at the call site.

  • •

    What should you check besides whether a size comparison is signed or unsigned?

    • •

      Check lower bounds, arithmetic width, conversions, and the size actually passed to the consuming operation.

  • •

    How does an arithmetic-wraparound bug differ from a sign-mixup bug, even though both stem from integer type issues?

    • •

      Signedness changes the interpretation or conversion of a value. Wraparound changes an arithmetic result; either can make validation disagree with use.

  • •

    Why should exploit testing avoid relying solely on behavior observed inside GDB for this bug class?

    • •

      Debugger and standalone runs can differ in environment and input handling. Inspect return values and reproduce the exact conditions.

  • •

    Why can two individually-unremarkable numbers combine into a dangerous integer overflow?

    • •

      Because the check validates their product, not each factor — two moderate-looking values can multiply past the integer type's width and wrap around to a small (or zero) value that trivially passes a "too large" check.

◌ Explore connections in Graph view