Following the same fix for unpack.
With this patch alignment is done relative to the start of the buffer that is being unpacked, not the raw pointer value, as per CPython. Fixes issue #3314.