by Lionel Debroux » 26 Aug 2018, 18:47
On bootloader.img, I've spent a significant amount of time:
* triaging ARM code / Thumb code / strings / data / BSS;
* marking known functions: most yaffs functions, some FreeRTOS functions;
* annotating MRC, MCR and companion (mostly ANDS, ORRS, BICS) instructions.
I wonder whether the trick used by ExtendeD, back in the day, to decrypt the Nspire's OS could be pulled here, in order to help RE the hash checking code in bootloader.img.
BTW, there's also a software RSA implementation in bootloader.img. I just didn't happen to stumble on it at first. There are plenty of references to a rsa.cpp file, which seems to have many lines, judging by the assert messages.
For instance, 0x8001B56C contains such assert messages. It's called by 0x8001A848 (and 0x8001A98E, but that is called only by 0x8001A848), itself called only by 0x8001A3F0, called only by 0x8001A566 - which also calls the SHA256 functions, as mentioned in my above post.
0x8001A3F0 receives from 0x8001A566 a pointer to an interesting data structure, which it passes directly to 0x8001A848. The structure contains the following 32-bit values:
0x800 (2048)
0x100 (256)
0x8003AFA0
0x800417E8
0x0
0x0
0x0
0x0
0x8003AFA0 reads:
* a 32-bit value, 0x40:
* 256 bytes starting at 0x8003AFA4:
0x31, 0x7A, 0x84, 0x92, 0x43, 0x32, 0xA9, 0xD4,
0x78, 0x7C, 0xAF, 0xD3, 0x9C, 0x2E, 0xD0, 0x99,
0x08, 0x09, 0x48, 0x22, 0x5C, 0x59, 0x63, 0xAD,
0xBB, 0xB3, 0xF6, 0x35, 0x3C, 0x9C, 0x51, 0x2E,
0x9A, 0x01, 0xAD, 0xAA, 0x7A, 0x9D, 0x8D, 0x01,
0xE1, 0xA7, 0xC8, 0x66, 0x35, 0xF3, 0x16, 0x43,
0x03, 0xFD, 0xFA, 0x89, 0xEE, 0xA2, 0x77, 0xE8,
0x68, 0xDC, 0x72, 0x48, 0x5F, 0xC5, 0x00, 0x23,
0x33, 0x90, 0x4C, 0xAC, 0xA9, 0x54, 0x74, 0x76,
0x01, 0xBA, 0xDC, 0x33, 0xCF, 0xA8, 0x2B, 0x51,
0xE9, 0x81, 0x44, 0x38, 0x57, 0xD1, 0xC1, 0x53,
0xDA, 0x65, 0x33, 0xA3, 0x53, 0x79, 0xE6, 0xD7,
0x5F, 0xD5, 0x43, 0x70, 0x5C, 0x12, 0xEE, 0x5E,
0x43, 0xAD, 0xF9, 0xA8, 0x3F, 0x3B, 0xAF, 0xA4,
0x80, 0x3C, 0x26, 0xFD, 0xE5, 0xB3, 0x66, 0x85,
0x70, 0xD0, 0xA0, 0x4B, 0x81, 0x86, 0x06, 0xEC,
0xBC, 0xA8, 0xF3, 0xF7, 0xA8, 0xF2, 0xD8, 0xD2,
0x89, 0x8B, 0x32, 0x56, 0x95, 0x7C, 0x5A, 0xC9,
0x19, 0x21, 0x40, 0xE5, 0x56, 0x65, 0x33, 0x4F,
0x07, 0xDD, 0xFE, 0xAC, 0x28, 0xC0, 0xBD, 0xA1,
0x47, 0xDD, 0xAB, 0x91, 0x4A, 0x6C, 0x1C, 0xA9,
0xC2, 0xD8, 0x80, 0x4F, 0x15, 0xDD, 0xCF, 0x7B,
0x3D, 0x6D, 0xAC, 0xFF, 0x5E, 0xE0, 0xA4, 0x8F,
0xBC, 0xEF, 0x8C, 0xCD, 0x5B, 0xF9, 0xBA, 0x8B,
0xD9, 0xBE, 0x0E, 0x7F, 0x0C, 0xE8, 0x7D, 0x03,
0xE1, 0xEA, 0x0B, 0x7A, 0x91, 0xC4, 0x22, 0x7B,
0x1D, 0x68, 0x97, 0x3B, 0xA5, 0xF0, 0x55, 0x43,
0x9C, 0x84, 0x34, 0x9C, 0x46, 0x85, 0xB6, 0x6D,
0x34, 0xB9, 0xAE, 0x16, 0x57, 0x53, 0x75, 0x4A,
0xC4, 0xA1, 0x95, 0xB1, 0xDF, 0x5C, 0x0E, 0xA4,
0xB1, 0x91, 0x35, 0x2B, 0x31, 0x45, 0xDB, 0x16,
0x78, 0x87, 0xCC, 0xC8, 0xD9, 0xD3, 0xA8, 0x92
That may be a size in words, and a 2048-bit RSA key. Again, a wild guess.