The Rust allocator API requires a Layout struct to be passed into each of its functions. For example, to reallocate or free memory, you need to pass the pointer to the existing memory block and the layout used during its allocation.

This is a problem when interacting with C API which does not provide any means of storing additional allocation data.

The Cassandra C++ driver’s memory allocation functions is the example of such API, e.g.

unsafe extern "C" fn realloc(ptr: *mut c_void, size: usize) -> *mut c_void

See? The memory reallocation function has the same signature as realloc from the standard C library and takes a pointer and the new size of the block.

Now compare this to the Rust memory allocation API:

unsafe fn realloc(ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8

This function has one additional parameter: the layout, I was talking about earlier. This must be the same layout used during the memory allocation.

And here comes the problem: we have no information about the layout in the C API and thus cannot pass it to Rust.

Oh, well, back to the old good memory tricks we used to use in C in the old days.

We need to store the layout used to allocate a memory block in the block itself, i.e. we need to allocate extra bytes, write the layout at the beginning of the block, and return the pointer to the first byte that follows the layout data. For example, when the driver requests 60 bytes of memory we allocate the memory block as follows:


                     requested amount of bytes (60)
                                    |
   size_of::<usize> bytes (8)       |
                |                   |
                v                   v
             +----+----------------------------------+
     +------>| 68 |0000000000000000000000000000000000|
     |       +----^----------------------------------+
block size        |
                  |
   pointer we return to the caller

Then, for Rust realloc and dealloc functions, we take the pointer, subtract the size of the allocation information, read the size of the block and reconstruct the layout structure. We don’t need to store the alignment, as we always use the same alignment (C API does not provide alignment parameters at all anyway).

The implementation could be found here.