Skip to main content

Debug Protected Software

Troubleshoot RunSafe Protect​

If you have been provided a build where Runsafe Protect is allowed to read environment variables there are a few that may be useful to be aware of.

Environment Variables​

  • LFR_DEBUG_LEVEL - Set to a numeric value to see the debug output of RunSafe's protection. This is largely useful to share logs with RunSafe support but can also be used to verify that protections are applied.
  • LFR_CHACHA_KEY - Set a specific seed. The format is 10 base 16 numbers. For example "0 1 2 3 4 5 6 7 8 9". This can be useful if encountering a problem with a given random seed or to have debug sessions that use the same key.
  • LFR_skip_rando - Skips all RunSafe Protect protections and runs the application normally. Only use when attempting to debug an issue and want no protections in place.

RunSafe Protect GDB Plugin​

LFR GDB provides python support for debugging RunSafe Protect randomized binaries in gdb. This plugin supports debugging normal executables, protected executables and applications with mixed protected and unprotected code (common with dynamically linked executables).

Target Platform(s)​

LFR GDB currently targets:

  • gdb 14.2 or higher.
  • DWARF 2, 4, 5
  • Python 3.7 or higher
  • x86_64 host - Can leverage gdb support for remote targets of other architectures.

Installation​

If your RunSafe Protect distribution has not already pre-installed the python plugin into your gdb installation, follow the instructions below to install it manually.

Install Python Wheel​

To install the python wheel lfr for python3 it is recommended to install it directly into your gdb python package directory:

sudo python3 -m pip install --target=/usr/local/share/gdb/python/ --no-index --force-reinstall -U --no-deps --find-links /usr/share/alkemist/lfr/dist/ lfr

Note: If you do not have pip you can unzip the /usr/share/alkemist/lfr/dist/lfr-1.0.0-cp36-abi3-linux_x86_64.whl into the gdb python directory for the same effect. The LFR GDB plugin has no external Python dependencies and can be extracted as a normal zip file.

sudo unzip /usr/share/alkemist/lfr/dist/lfr-1.0.0-cp36-abi3-linux_x86_64.whl -d /usr/local/share/gdb/python/

Building A Custom GDB Release​

The targeted version (14.2) of gdb can be acquired at GDB Downloads and is generally compilable from source on any system.

Example gdb Compilation and Installation​

This is an example and may require additional configuration flags depending on your system. Please refer to the GDB provided documentation if this doesn't match your build host configuration.

wget https://ftp.gnu.org/gnu/gdb/gdb-14.2.tar.gz
tar xf gdb-14.2.tar.gz
cd gdb-14.2
./configure --with-python=`which python3`
make -j`nproc`
make install

Changes to GDB Debugging Flow​

In order to enable LFR GDB during a gdb session, run the following command in gdb.

If you prefer you can place this in a .gdbinit script, however due to the intrusive nature of the plugin it's recommended you load it as needed to avoid complications in a non-protected debug session.

pi import lfr.gdb

Normal gdb commands will continue to work as expected during a normal debugging process.

Please keep in mind that the expectation is that the targeted executable will have debug symbols in DWARF format (commonly produced in C/C++ applications by compiling with -g). While we make the best effort for our plugin to work without these symbols, your debugging experience will be greatly reduced without them and may not work effectively.

New Functionality​

New Parameters​

File Based Parameters​
  • lfr-symbol-file - The value that mirrors what would normally be stored in symbol-file.
  • lfr-exec-file - The value that mirrors what would normally be the target executable.

These two parameters are used in the event that you need a custom path to debug symbols or exec file that cannot be determined from GDB directly. They will normally be the same values that GDB will indicate when looking at symbol-files or the current target.

If LFR GDB cannot find your symbols, check that lfr-symbol-file is correctly set.

LFR Annotations​
  • lfr-annotate - Enabled produces additional annotations that are visible during backtraces.

If you do not want visible annotations in gdb, set lfr-annotate off.

  • lfr applied indicates that the code is randomized
  • lfr skipped indicates that the code could be randomized but was skipped.
  • lfr trampoline indicates that you are currently inside an LFR trampoline.

Example:

(gdb) backtrace
frame1 (lfr applied) () at stack.c:23
frame0 (lfr applied) (depth=0) at stack.c:30
Stack Frame Unwinding​

LFR GDB has a special mechanism in place for unwinding randomized stack frames. Due to limitations in GDB, GDB always references the executable file for unwind information on disk, rather than in-memory for a running process. This is only an issue when the executable has been shuffled at runtime and the two instances no longer match each other.

We provide a GDB unwinder that always uses the in-memory unwind information for randomized regions of memory.

To disable this behavior, disable unwinder lfr_unwind. This is not recommended.

Troubleshooting​

If you encounter issues that are not resolved by the following troubleshooting steps, please contact support@runsafesecurity.com.

Troubleshooting Tools​

The following tools provide debug information about the state of the plugin and may be useful to provide to support in the event of debugging issues.

Setting the environment variable LFR_GDB_DEBUG_LEVEL to warn, error, log, or trace (default level is error) when invoking gdb will enable output of additional logging information.

lfr info symbols: Shows the symbols files generated by the plugin. These are the symbol files that represent randomized code regions.

lfr info meta : Shows all metadata being stored by liblfr.so to facilitate debugging.

lfr diag: Provides a "diagnostic" view of a lot of the plugin internal information and tries to point out areas that may be causing problems with plugin operation. Use on core files or debugged process.

The plugin can't find debug symbols.​

Occasionally the plugin can become out of sync with the target location that gdb is debugging. Try checking the parameter lfr-symbol-file.

show lfr-symbol-file

If that does not match the location of your symbols run:

set lfr-symbol-file ${SYMBOL_LOCATION}

Then reset the state of the plugin by invoking pi lfr.gdb.symbol_reset().

The debug session addresses no longer appear to be correct or stack unwinding provides invalid stack frames.​

Occasionally the plugin may stop loading symbols after becoming out of sync with the target application. To reset the state of the plugin invoke pi lfr.gdb.symbol_reset().

Stack unwinding does not appear correct.​

Note: Some statically compiled executables do not have unwind info preventing LFR GDB from correctly unwinding the stack.

You can check if the binary has unwind info by running:

readelf -x .eh_frame ${EXECUTABLE}

GDB breakpoints at function entrance rather than post-prologue​

Unfortunately, GDB heuristics don't always identify randomized sections of code and sometimes will place breakpoints at the first instruction of a function rather than after the stack has been initialized. This manifests in arguments to a given function appearing malformed.

You can workaround this issue by running the stepi command until you have reached the end of the function prologue and view the correct function arguments.

stepi walks into a function trampoline instead of function​

The gdb stepi instruction always moves to the next actual instruction, rather than jumping over otherwise "ignored" code. RunSafe Protect creates a section of memory named .textramp that is simply a set of trampolines to the randomized code. Normally, we have set up the GDB plugin to ignore this region of memory so you shouldn't see if you run step or next and it traverses it.

The easiest workaround to get out of this region is run the stepi command until you are no longer in the trampoline and instead are in the target function.