All of the code was there except the linker was failing to clear the bss section because I added too many .zeros. The should have only been the exported globals that start with _ like _szero = .. Fixing that and turn on the usb transmit fixed everything.
Everything works fine without USB being plugged in but faults (I think) when USB is plugged in. This is switched away from the USB code from the bootloader onto the USB code thats generated by Atmel Studio using the high level classes from ASF.
Scott Shawcroft