I'm confused. You're essentially saying "if kernel only gives out addresses between 1 .. 2^48 then it's safe to set ADDRESS_BITS to a value lower than 48".

It seems to me that the opposite is true and the opposite is what you're in fact doing in your change. What am I missing?

I should probably improve the wording if it's confusing. If ADDRESS_BITS < POINTER_BITS, the code assumes the high bits of memory addresses can be ignored (i.e. upper 16 bits) when indexing into the radix tree memory map.

The comment is explaining why it might be safe to set ADDRESS_BITS to 48. I'm changing the default to 64 because that's the safer default assumption, even though it has a bit more overhead (mostly as virtual memory usage).

@jrtc27 has suggested that machines certainly exist which violate that assumption that makes ADDRESS_BITS=48 safe. Setting ADDRESS_BITS to 64 has some small additional overhead vs using 48. I don't like the idea of Python randomly crashing.

@ambv, is this comment better:

/* High bits of memory addresses that will be ignored when indexing into the
 * radix tree.  Setting this to zero is the safe default.  For most 64-bit
 * machines, setting this to 16 would be safe.  The kernel would not give
 * user-space virtual memory addresses that have significant information in
 * those high bits.  The main advantage to setting IGNORE_BITS > 0 is that less
 * virtual memory will be used for the top and middle radix tree arrays.  Those
 * arrays are allocated in the BSS segment and so will typically consume real
 * memory only if actually accessed.
 */
#define IGNORE_BITS 0

I would replace ADDRESS_BITS with POINTER_BITS-IGNORE_BITS. The name ADDRESS_BITS is confusing since the number of physical address bits on the machine is not really the key thing.

That is muuuch clearer, thanks!

Well, not quite. You care about the number of bits in a virtual address, which is a 32-bit or 64-bit quantity that just happens to have some of the bits required to be copies of the others. On traditional architectures, POINTER_BITS is the number of bits in a virtual address, but this is not true for CHERI, and thus Arm's experimental Morello prototype; here, C language pointers are represented with hardware capabilities, which are unforgeable and consist of a standard integer address, plus bounds and other metadata (exactly what isn't important). This provides fine-grained spatial memory safety for C/C++ (and temporal with some tricks), but means that sizeof(void *) is 16 on 64-bit architectures, whilst only 8 of those bytes are the actual integer address, and are the only bits you're interested in here. So you do mean ADDRESS_BITS, not POINTER_BITS.