NM-NAM reconnaissance

Setup of the module is reasonably straight forwards - the module takes power from the router through the big backplane connector, and can communicate with the router via an internal serial and internal ethernet connection. The module also has an external ethernet port, and two USB 1.0 ports that are usually hidden behind the front panel, and a CompactFlash card slot which is the second IDE bus.

The "host" router needs some config - you need to set an IP address on the "AnalysisModule" interface, and some ACL tweaks for the serial line to allow you to manage the module. (You may need to s/AnalysisModule/ContentEngine/ etc. depending on which card you get.)

version 12.4
aaa authentication login nm-nam none
!
interface Loopback0
 ip address 10.1.14.1 255.255.255.255
!
interface Analysis-Module1/0
 ip address 10.1.14.65 255.255.255.248
 hold-queue 60 out
!
line 33
 login authentication nm-nam
 no activation-character
 no exec
 transport preferred none
 transport input all
 transport output all
!
end

The IP address above (10.1.14.65) is the router's IP on that "link". The module's OS must use an IP within that range (so 10.1.14.66 in this case) and use the above address as the default gateway.

Once you've got the config in place, you can issue a service-module analysis-Module 1/0 reset, to hard-reset the module. Then issue a service-module analysis-Module 1/0 session to view the boot log.

Initializing memory.  Please wait.
256 MB SDRAM detected
BIOS Version: SM 02.00
BIOS Build date: 09/17/02
System Now Booting ...

Booting from flash..., please wait.

[BOOT-ASM]
7

Please enter '***' to change boot configuration:
Filesystem type is ext2fs, partition type 0x83
kf: a1 : (hd0,0)/bzImage root=/dev/hda1 ro plat=nm
kf: a2 : (hd0,0)/bzImage root=/dev/hda1 ro plat=nm
in grub_open: (hd0,0)/bzImage root=/dev/hda1 ro plat=nm
in grub_open1: /bzImage root=/dev/hda1 ro plat=nm
in grub_open2: /bzImage root=/dev/hda1 ro plat=nm
in grub_open3: /bzImage root=/dev/hda1 ro plat=nm 0
in grub_open: (hd0,0)/bzImage root=/dev/hda1 ro plat=nm
in grub_open1: /bzImage root=/dev/hda1 ro plat=nm
in grub_open2: /bzImage root=/dev/hda1 ro plat=nm
in grub_open3: /bzImage root=/dev/hda1 ro plat=nm 0
In verify_kernel_sig
Chksum: final image size: 900958
plat: 1
Debug: bl_sz: 109024
After: buf_len: 2048
After KEY_InitMem
reading key: 0
reading key: 1
reading key: 2
reading key: 3
reading key: 4
reading key: 5
After karr
After 2: buf_len: 2048
sig len : 172
in verifysignature_md5, MD5 hash generated now:02x02x02x02x02x02x02x02x02x02x02
x02x02x02x02x02x
in verifysignature_md5, MD5 hash generated now, str format:hexmd5:d5cba1e2b4c5e
a87857acfdedc5313a7
calling RSA_eay decrypt now
got value i:33
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7
Kernel signature verified successfully
In load_imagea1
In load_imagea2
Dbg ********* filemax/data_len/SECSIZ: 900958/2560/512
 [Linux-bzImage, setup=0xa00, size=0xdb35e]
kernel_func: kt: 3
in boot func: kt: 3
Linux version 2.4.21 (root@namlab-pc6.cisco.com) (gcc version 2.96 20000731 (Red Hat Linux 7.2 2.96-112.7.1)) #9 SMP Wed Jul 30 16:30:10 PDT 2003
Jetfire detected
setup.c: handling flash window at [15MB..16MB)

The second thing the kernel prints is "Jetfire detected" - I believe this to be the codename for this platform, so that's what I'll use to refer to these modules.

Once the OS finish booting up, you'll get dropped at a typical login prompt. Logging in with the default root:root credentials gets you dropped into an iOS-like shell. That's easy enough to fix - just attach the disk to another machine, and update /etc/passwd:

root:x:0:0:root:/root:/bin/bash

(default has /usr/local/nam/bin/cli)

Once the system boots up again, you can poke around more. Of particular interest is the flash device:

cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "Bootloader"
mtd1: 00020000 00020000 "Boot Parameters"
mtd2: 00020000 00020000 "Signatures"
mtd3: 00020000 00020000 "SYS DB"
mtd4: 00c00000 00020000 "MP"
mtd5: 00100000 00020000 "Diagnostics image"

• mtd0 is indeed the bootloader, more on that in a bit
• mtd1 looks to be the bootloader configuration
• mtd2 and mtd3 is probably used by the bootloader
• mtd4 is a "helper image"; a linux kernel with embedded initrd to help re-flash the module
• mtd5 I haven't looked

lspci doesn't show anything too crazy:

00:00.0 Host bridge: Intel Corporation 440BX/ZX - 82443BX/ZX Host bridge (AGP disabled) (rev 03)
00:07.0 ISA bridge: Intel Corporation 82371AB PIIX4 ISA (rev 02)
00:07.1 IDE interface: Intel Corporation 82371AB PIIX4 IDE (rev 01)
00:07.2 USB Controller: Intel Corporation 82371AB PIIX4 USB (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB PIIX4 ACPI (rev 02)
00:0d.0 Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 08)
00:0e.0 Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 08)

In verify_kernel_sig
Chksum: final image size: 18126516
plat: 1
Debug: bl_sz: 109024
After: buf_len: 2048
After KEY_InitMem
reading key: 0
reading key: 1
reading key: 2
reading key: 3
reading key: 4
reading key: 5
After karr
After 2: buf_len: 2048
sig len : 172
in verifysignature_md5, MD5 hash generated now:02x02x02x02x02x02x02x02x02x02x02
x02x02x02x02x02x
in verifysignature_md5, MD5 hash generated now, str format:hexmd5:ffce7129487a7
4b3e6f1651ff8b34311
calling RSA_eay decrypt now
got value i:33
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7
signatures do not match...
Invalid Kernel Signature !!!
Rebooting�

Breaking the bars

I was able to make enough sense of the code that the boot flow is roughly:

• BIOS, "Booting from flash"
• 16-bit bootloader init code (@ 7600h)
• 32-bit bootloader code (@ 8000h)
  • PCI Init, scans for a particular string on the chipset to detect if it's running on an AIM- or NM- module
  • Reads the bootloader config from flash (mtd1 perhaps?)
  • Presents the boot prompt ("Please enter '***' to change boot configuration:"), handles the menu etc.
  • Builds a GRUB (-style?) command list, and runs functions to handle them (possibly just GRUB linked into the binary?)
    • root (hd0,0)
    • kernel (hd0,0)/bzImage root=/dev/hda1 ro plat=nm
      • Detect and mount the file system, load the kernel and verify it
    • boot
      • Does some voodoo to jump into the kernel

After a lot of fumbling around, naming functions and variables, I found the part we'd need to change:

kern_image_size = verifysignature_md5(buf + 1,0x10,(char *)keyring,uVar4,kern_image_size);
if (kern_image_size == 0) {
  printf("Invalid Kernel Signature !!!\n",uVar3,pbVar2,pbVar5);
  printf("Rebooting\n");
  open_file_anyFS?(extraout_ECX_00);
  ??reset_82c3();
}
else {
  printf("Kernel signature verified successfully\n",uVar3,pbVar2,pbVar5);
}

Here's what that looked like in the code listing (disassembly) window:

000090e2 e8 2f b4        CALL       verifysignature_md5                              int verifysignature_md5(byte * h
         00 00
000090e7 89 c3           MOV        EBX,buf_len
000090e9 83 c4 20        ADD        ESP,0x20
000090ec 85 db           TEST       EBX,EBX
000090ee 74 12           JZ         LAB_00009102
000090f0 83 ec 0c        SUB        ESP,0xc
000090f3 68 e0 f1        PUSH       s_Kernel_signature_verified_succes_0001f1e0      = "Kernel signature verified suc
         01 00
000090f8 e8 7f 1f        CALL       printf                                           void printf(char * format, ...)
         00 00
000090fd 83 c4 10        ADD        ESP,0x10
00009100 eb 24           JMP        LAB_00009126
                     LAB_00009102                                    XREF[1]:     000090ee(j)
00009102 83 ec 0c        SUB        ESP,0xc
00009105 68 08 f2        PUSH       s_Invalid_Kernel_Signature_!!!_0001f208          = "Invalid Kernel Signature !!!\n"
         01 00
0000910a e8 6d 1f        CALL       printf                                           void printf(char * format, ...)
         00 00

Knowing a small amount of Z80 machine code / assembly, which is a cousin of x86, here's what's going on:

• Call (jump to and come back here on a RET instruction) verifysignature_md5
• Move the contents of buf_len into register EBX
• Add 0x20 to the stack pointer
• Bitwise AND EBX & EBX and set the Zero Flag if the result is zero
  • This is how you do "if (EBX >= 1)" in x80-style assembly
• Jump, if the zero flag is set, +0x12 bytes forward, to LAB_00009102

By only changing one instruction we can always succeed - if we convert the JZ instruction to a no-op (NOP), it won't jump to the failure part of the code and it should carry on regardless.

We could perhaps disable the check entirely, but it's harder to know if the process of doing the check loads the kernel into memory, or sets some global states that we don't understand.

I used Ghidra's "Patch Instruction" tool on the JZ line. You first have to clear the value (the destination in this example) but then you can type in NOP in the left section, and Ghidra will pick the right bytes to put in.

We end up with this:

000090e7 89 c3           MOV        EBX,buf_len
000090e9 83 c4 20        ADD        ESP,0x20
000090ec 85 db           TEST       EBX,EBX
000090ee 66 90           NOP
000090f0 83 ec 0c        SUB        ESP,0xc
000090f3 68 e0 f1        PUSH       s_Kernel_signature_verified_succes_0001f1e0      = "Kernel signature verified suc
         01 00
000090f8 e8 7f 1f        CALL       printf                                           void printf(char * format, ...)
         00 00
000090fd 83 c4 10        ADD        ESP,0x10
00009100 eb 24           JMP        LAB_00009126

Clearly only the 000090ee line has changed, but the disassembly now looks like this:

  verifysignature_md5(buf + 1,0x10,(char *)keyring,uVar5,kern_image_size);
  printf("Kernel signature verified successfully\n");
}

Now I need to test it. I didn't have any luck emulating the bootloader, so I edited the raw bootloader binary, and imported it into Ghidra as a new file to make sure I hadn't broken it. It looked fine.

As the firmware was on /dev/mtd0, this meant I couldn't just dd if=firmware.bin of=/dev/mtd0. Linux's MTD code abstracts away most of the nuances of talking to Memory Technology Devices (e.g. flash) when there's no controller. There is still one main limitation in that you have to erase a block before you write it.

Thankfully, the original OS image already had a tool to do that. /usr/local/bin/eraseall /dev/mtd0 simply erased all of the blocks in mtd0, ready for writing.

Now I could dd if=firmware.bin of=/dev/mtd0, and reboot.

With my fingers crossed, I rebooted the card from the router, just in case a normal OS reboot wasn't enough.

The system booted fine!

I shuffled the symlink at /bzImage around to point to my cisco.sux kernel, and rebooted once again...

In verify_kernel_sig
Chksum: final image size: 900958
plat: 1
Debug: bl_sz: 109024
After: buf_len: 2048
After KEY_InitMem
reading key: 0
reading key: 1
reading key: 2
reading key: 3
reading key: 4
reading key: 5
After karr
After 2: buf_len: 2048
sig len : 172
in verifysignature_md5, MD5 hash generated now:02x02x02x02x02x02x02x02x02x02x02
x02x02x02x02x02x
in verifysignature_md5, MD5 hash generated now, str format:hexmd5:2dc51fa8e3008
edb9edc8df75b56f99b
calling RSA_eay decrypt now
got value i:33
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7calling RSA_eay decrypt now
got value i:-1
verifysignature_md5, Orig MD5 hash generated during encryption:d5cba1e2b4c5ea87
857acfdedc5313a7
psignatures do not match...
Kernel signature verified successfully
In load_imagea1
In load_imagea2
Dbg ********* filemax/data_len/SECSIZ: 900958/2560/512
   [Linux-bzImage, setup=0xa00, size=0xdb35e]
 kernel_func: kt: 3
in boot func: kt: 3
Linux version 2.4.21 (root@namlab-pc6.cisco.com) (gcc version 2.96 20000731 (Red Hat Linux 7.2 2.96-112.7.1)) #9 SMP Wed Jul 30 16:30:10 PDT 2003

Success! The system was now booting a modified kernel.

The issue was however, the bootloader could only load Linux 2.4 and very very early 2.6 kernels. Anything newer would end up trapping into the BIOS debugger, which is a hilarious failure state.

Breaking in to break out

I was able to locate an update package for the NM-CUE that shipped with a 2.6.11-ish kernel. I flashed it and was able to confirm that the original kernel from the NM-NAM and the new NM-CUE kernel booted.

I now had to "port" my patch to the new version. Problem is that the code that Ghidra could understand no longer did the verification, but the kernel was still being verified..?

Long story short, Cisco moved the code around, so now the process is something like this:

• BIOS, "Booting from flash"
• 16-bit bootloader init code (@ 7600h)
• 32-bit bootloader code (@ 8000h)
  • PCI Init, scans for a particular string on the chipset to detect if it's running on an AIM- or NM- module
  • Reads the bootloader config from flash (mtd1 perhaps?)
  • Presents the boot prompt ("Please enter '***' to change boot configuration:"), handles the menu etc.
  • Builds a GRUB (-style?) command list, and runs functions to handle them (possibly just GRUB linked into the binary?)
    • root (hd0,0)
    • kernel (hd0,0)/bzImage root=/dev/hda1 ro plat=nm
      • Detect and mount the file system, load the kernel into memory
    • boot
      • Does some voodoo, loads an embedded ELF and jumps into it
• ELF #1
  • Verifies the kernel image, jumps into ELF #2
• ELF #2
  • Some weird voodoo (moving the bzImage sections around for the Linux boot code to work propery?)

Ghidra didn't like the idea of the embedded ELFs, so I used binwalk to pull them out (before I knew they were ELFs):

binwalk nm+aim_bl_2.1.14

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
1026          0x402           LZMA compressed data, properties: 0x9A, dictionary size: 0 bytes, uncompressed size: 207 bytes
94032         0x16F50         ELF, 32-bit LSB executable, Intel 80386, version 1 (SYSV)
152681        0x25469         ELF, 32-bit LSB executable, Intel 80386, version 1 (SYSV)

file _nm+aim_bl_2.1.14.extracted/*
_nm+aim_bl_2.1.14.extracted/16F50:  ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
_nm+aim_bl_2.1.14.extracted/25469:  ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
_nm+aim_bl_2.1.14.extracted/402:    data
_nm+aim_bl_2.1.14.extracted/402.7z: OpenPGP Public Key

Wait...

ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped

sickos.jpg

Thanks Cisco!

That makes figuring out the jailbreak a tonne easier (that and an ELF is much easier for Ghidra to understand than a raw flash image). Loading it up, I didn't have to change anything, I could just poke around and find it:

printf("sig len : %d\n",iVar3 + -1);
iVar3 = VerifySignature_MD5(param_1 + 3,0x10,&swv_key_buf,iVar3 + -1,uVar1);
if (iVar3 != 0) {
  printf("Kernel signature verified successfully\n");
  return iVar3;
}
printf("Invalid Kernel Signature !!!\n");
return 0;

Applying the same patch...

01001486 85 c0           TEST       EAX,EAX
01001488 89 c3           MOV        EBX,EAX
0100148a 74 16           JZ         LAB_010014a2
0100148c c7 04 24        MOV        dword ptr [ESP]=>local_2c,s_Kernel_signature_v   = "Kernel signature verified suc
         88 b6 00 01
01001493 e8 38 ec        CALL       printf                                           int printf(char * __format, ...)
         ff ff

to

01001486 85 c0           TEST       EAX,EAX
01001488 89 c3           MOV        EBX,EAX
0100148a 66 90           NOP
0100148c c7 04 24        MOV        dword ptr [ESP]=>local_2c,s_Kernel_signature_v   = "Kernel signature verified suc
         88 b6 00 01
01001493 e8 38 ec        CALL       printf                                           int printf(char * __format, ...)
         ff ff

I flashed it the same way as before, and the system still booted. I swapped in my patched kernel again, and it too worked fine!