7 Things I Learned From Porting a C Crypto Library to Rust

https://sharpend.io/7-things-I-learned-from-porting-a-c-crypto-library-to-rust/

Rust has always been the programming language that reminds me the most of my game hacking days, and for good reasons. Rust is a natural fit for embedded systems like video game consoles – or rather emulators thereof. The compiler supports a high number of platforms and lets you drop down to C or assembly if necessary. Plus, the language lends itself well to implementing (reverse-engineered) cryptographic algorithms and other low-level shenanigans.

Given these benefits, it’s no surprise that I keep coming back to Rust. This time, I decided to revisit a pull request from five years ago, which has been lingering in my mind ever since. The ambiguous goal of the pull request is to port cb2util, one of my old crypto tools for PlayStation 2, from C to pure Rust.

Among other things, cb2util allows you to decrypt all cheat codes for the notorious CodeBreaker cheat device, making it possible to scrutinize them in their raw form and use them with other devices.

To give you an idea, the following code might make your character invincible by constantly writing the byte value 99 (0x63) to memory address 0x0096F5B8:

Infinite health
0096F5B8 00000063

The very same code but encrypted:

Infinite health
81E1C95B 9764DA87

Back in 2006, it took me weeks to reverse-engineer the proprietary encryption scheme from MIPS 5900 assembly and convert everything to C in a piecemeal fashion. Naturally, I learned a ton in the process. Trying to port those same crypto routines from C to Rust now, 14 years later, sounded like another promising learning opportunity.

Luckily, I wasn’t disappointed. In fact, I went so far as to extract and publish everything that I did as a Rust crate. It’s called, well, codebreaker.

Here are seven things I picked up while working on this little side project – some of them specific to cryptography, others more general in nature.

Soft migration via FFIPermalink

Rust provides a foreign function interface (FFI) to talk to other programming languages with ease. This interface allowed me to gradually port one C function at the time until there was no foreign function call left. Since the crypto code involves a state machine, I also had to temporarily export a few global C variables in order to access them from Rust. A large set of existing integration tests ensured that I didn’t break anything along the way, which was very helpful (thanks, past me!).

Wrapping operationsPermalink

In C, unsigned integer arithmetic is defined to be modulus a power of two, which is almost always what you want when it comes to cryptography. To achieve the same in Rust without running into any overflow errors, you have to use dedicated wrapping operations for modular addition (wrapping_add), subtraction (wrapping_sub), multiplication (wrapping_mul), etc.

Here’s an example for 32-bit unsigned integers:

assert_eq!(200u32.wrapping_add(55), 255);
assert_eq!(200u32.wrapping_add(u32::MAX), 199);

Transmuting typesPermalink

Due to the nature of the encryption scheme, it’s sometimes necessary to read or write the same data in two different ways. For example, I defined an RC4 key as a fixed-size array [u32; 5], which is convenient most of the time. That is, until the key itself needs to be encrypted as a slice of type &[u8].

Doing that in C via pointer casting is easy enough, but it took me a while to figure out how to implement it in Rust. I found one (albeit unsafe) solution in the byteorder crate for transmuting types – reinterpreting the bits of a value of one type as another type:

unsafe fn slice_to_u8_mut<T: Copy>(slice: &mut [T]) -> &mut [u8] {
    use std::mem::size_of;

    let len = size_of::<T>() * slice.len();
    slice::from_raw_parts_mut(slice.as_mut_ptr() as *mut u8, len)
}

RSA with num-bigintPermalink

In addition to RC4, I needed a simple RSA implementation for small 64-bit keys (the size of one cheat code). I used to rely on libbig_int, a portable C library to calculate integers of arbitrary length. Now I needed something similar for Rust. That something turned out to be the superb num-bigint crate.

fn rsa_crypt(addr: &mut u32, val: &mut u32, rsakey: u64, modulus: u64) {
    let code = BigUint::from_slice(&[*val, *addr]);
    let m = BigUint::from(modulus);

    if code < m {
        let digits = code.modpow(&BigUint::from(rsakey), &m).to_u32_digits();
        *addr = digits[1];
        *val = digits[0];
    }
}

No stdlibPermalink

Speaking of num-bigint, I studied its sources intensively and learned how to write code that can also be used for embedded environments such as game consoles (another CodeBreaker clone, anyone?). Said code must not depend on Rust’s standard library. This exercise showed me how straightforward conditional compilation and optional dependencies are in Rust.

mod_inverse using Newton’s methodPermalink

As the saying goes, “make it work, make it right, make it fast”. After porting all crypto routines to Rust, I did some local optimizations. One of them involved replacing a messy reverse-engineered function with something much more mathematically sound. As luck would have it, I stumbled upon a blog post by Daniel Lemire that describes an elegant method for computing the multiplicative inverse of odd integers. I proudly present the diff:

Automated testing with actions-rsPermalink

Being the diligent engineer that I am, I devoted some time to unit tests and doctests (executable examples in the documentation). I also became friends with Clippy, Rust’s amazing linter, which helped me write code that’s more correct, more readable, and more idiomatic. Finally, I brought everything together in a handy CI pipeline using actions-rs, the GitHub Actions toolkit for Rust.

---

All in all, I can confidently say that the Rust port is better than the original in many ways. But see for yourself.

If you have any additions or questions, feel free to ping me on Twitter (@mlafeldt). I’m always eager to learn something new! ✌️

 Updated: July 01, 2020