Linux Stack Overflow Protection: Canaries, NX, ASLR, PIE, and RELRO

Linux binary hardening is not a contest to collect compiler flags. Each mechanism cuts one part of an exploit chain: a Canary detects some stack-frame corruption, NX blocks instruction execution from the stack, PIE lets ASLR move the main executable, and RELRO narrows writable relocation state.

This article focuses on modern GNU/Linux user-space programs. The goal is not to memorize names. It is to build one sample and verify which protections reached the final ELF file.

Start with the boundary of each protection

These layers belong together, but none replaces bounds checks, memory-safe APIs, fuzzing, or code review.

Prepare a reproducible sample

Save this as overflow.c:

#include <stdio.h>
#include <string.h>

static void copy_input(const char *input) {
    char buffer[32];
    strcpy(buffer, input);
    puts(buffer);
}

int main(int argc, char **argv) {
    if (argc != 2) {
        fprintf(stderr, "usage: %s <text>\n", argv[0]);
        return 2;
    }

    copy_input(argv[1]);
    return 0;
}

The strcpy call is intentionally unsafe. Build and run this sample only in an isolated lab; never ship it in a service.

Build a comparison binary with common protections disabled:

gcc -O0 -g \
  -fno-stack-protector \
  -U_FORTIFY_SOURCE \
  -fno-pie -no-pie \
  -Wl,-z,norelro,-z,execstack \
  overflow.c -o overflow-weak

Build an explicitly hardened binary:

gcc -O2 -g \
  -fstack-protector-strong \
  -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 \
  -fstack-clash-protection \
  -fPIE -pie \
  -Wl,-z,relro,-z,now,-z,noexecstack \
  overflow.c -o overflow-hardened

Recent GCC versions also provide -fhardened on GNU/Linux to enable a production-oriented set of hardening options:

gcc --help=hardened
gcc -O2 -g -fhardened overflow.c -o overflow-auto

The exact -fhardened option set can change with GCC releases. Record the compiler version and full link command; “hardened enabled” is not enough for a release audit.

1. Stack Canary: check the frame before return

-fstack-protector-strong covers functions with risk signals such as local arrays or references to local frame addresses. It does not mean “every function with a local variable.” Use -fstack-protector-all only when every function must be instrumented.

Inspect the result:

readelf -Ws ./overflow-hardened | grep __stack_chk_fail
objdump -d ./overflow-hardened | sed -n '/<copy_input>/,+30p'

TLS registers and offsets differ by architecture. Do not treat %gs:0x14 or %fs:0x28 as a portable rule. Stable evidence is the function prologue saving a Canary, the epilogue comparing it, and failure calling __stack_chk_fail.

Test the Canary in isolation:

gcc -O0 -g \
  -fstack-protector-strong \
  -U_FORTIFY_SOURCE \
  -fno-pie -no-pie \
  overflow.c -o overflow-canary

set +e
./overflow-canary "$(printf 'A%.0s' {1..128})"
test $? -ne 0

A non-zero exit proves detection, not remediation. An attacker may still continue after leaking the Canary or corrupting other control data before the function returns.

2. _FORTIFY_SOURCE: use object sizes known by the compiler

With optimization enabled, glibc Fortify wrappers can transform some strcpy, memcpy, sprintf, and similar calls into checked variants. Inspect the dynamic symbols:

readelf -Ws ./overflow-hardened | grep -E '__.*_chk|__fortify_fail'

Whether a _chk call appears depends on optimization and whether the object size can be derived. No matching symbol does not automatically prove a configuration failure; inspect compiler diagnostics and disassembly as well.

_FORTIFY_SOURCE is not general memory safety. It may not help when the compiler cannot determine the destination size, code bypasses libc, or corruption occurs inside a custom data structure.

3. NX and Stack Clash Protection solve different problems

The linker describes stack permissions through PT_GNU_STACK:

readelf -lW ./overflow-weak | grep GNU_STACK
readelf -lW ./overflow-hardened | grep GNU_STACK

The hardened binary should show a readable and writable, but non-executable, stack—typically RW, not RWE. NX blocks direct execution of stack-resident shellcode; it does not stop attackers from reusing existing executable code.

-fstack-clash-protection addresses a separate issue. It probes large stack allocations page by page so an allocation cannot jump over the kernel Guard Page. It does not check an ordinary overrun such as buffer[32], and it does not replace a Canary.

4. ASLR and PIE: system policy and binary layout both matter

Common Linux randomize_va_space values are:

• 0: disable address-space randomization.
• 1: randomize the mmap base, stack, VDSO, and other regions.
• 2: add heap randomization to level 1; this is commonly the default.

Check the system setting:

cat /proc/sys/kernel/randomize_va_space

Then check whether the main executable is PIE:

file ./overflow-hardened
readelf -hW ./overflow-hardened | grep 'Type:'
readelf -lW ./overflow-hardened | grep INTERP

PIE commonly appears as ET_DYN, but shared libraries are also ET_DYN. Do not conclude “PIE” from Type: DYN alone; combine file, PT_INTERP, and the executable entry point.

5. RELRO: require both GNU_RELRO and BIND_NOW

Full RELRO needs two conditions: a GNU_RELRO Segment, plus dynamic binding during initialization.

readelf -lW ./overflow-hardened | grep GNU_RELRO
readelf -dW ./overflow-hardened | grep -E 'BIND_NOW|FLAGS.*NOW'

• GNU_RELRO alone usually proves only Partial RELRO.
• GNU_RELRO plus BIND_NOW or NOW is the common evidence for Full RELRO.
• dlopen and dlsym still work; Full RELRO does not ban runtime library loading.

Immediate binding can increase startup work and expose missing symbols earlier. Treat that as a release test, not as a default reason to disable Full RELRO.

6. CET and Shadow Stack: build properties are not runtime proof

x86 GNU/Linux can use -fcf-protection to emit control-flow protection instructions and ELF properties:

gcc -O2 -fcf-protection=full overflow.c -o overflow-cet
readelf -nW ./overflow-cet | grep -E 'IBT|SHSTK'

An IBT or SHSTK property only shows that the build declares capability. Execution still depends on the CPU, kernel, dynamic loader, and compatibility of loaded shared objects.

Linux runtime checks include:

grep -w user_shstk /proc/cpuinfo
grep '^x86_Thread_features' /proc/$$/status
grep '^x86_Thread_features_locked' /proc/$$/status

Do not infer CET status from a CPU release year or cloud instance name. Inspect the target node and target process.

7. A five-minute artifact check

binary=${1:?usage: check-hardening.sh /path/to/binary}

file "$binary"
readelf -hW "$binary" | grep 'Type:'
readelf -lW "$binary" | grep -E 'INTERP|GNU_STACK|GNU_RELRO'
readelf -dW "$binary" | grep -E 'NEEDED|BIND_NOW|FLAGS.*NOW'
readelf -Ws "$binary" | grep -E '__stack_chk_fail|__.*_chk' || true
readelf -nW "$binary" | grep -E 'IBT|SHSTK' || true

These commands are an audit entry point, not a complete scanner. Static linking, stripped symbols, LTO, and different libc implementations can change the output. When evidence is ambiguous, return to the link command, linker Map file, and disassembly.

Production build checklist

1. Prefer -fhardened when supported, and record the output of gcc --help=hardened.
2. Otherwise enable -fstack-protector-strong, PIE, Full RELRO, No-exec stack, and Stack Clash Protection explicitly.
3. Use -Wformat -Werror=format-security in CI to reject statically detectable format-string mistakes.
4. Use AddressSanitizer, UndefinedBehaviorSanitizer, and fuzzing in test builds; do not treat Sanitizers as production hardening.
5. Inspect the final binary inside the release image. A Makefile or base-image claim is not artifact evidence.
6. Fix the out-of-bounds write. Hardening lowers exploit reliability and narrows the failure window; it does not make memory corruption acceptable.

References

• GCC Instrumentation Options
• glibc Source Fortification
• GNU ld Options
• Linux randomize_va_space
• Linux x86 Shadow Stack