Following the initial analysis of the IP camera, we continue with the analysis of the software. After gaining access to the underlying control system of the device (removing a few screws) we were able to access the device using either the UART pins or the unsecure and open Telnet port. When we gained access to the device through either of these methods, we were shown a Linux console.
Finding a Linux distribution running on the camera isn’t all that surprising. Many IoT devices out there are powerful enough to handle a stripped down Linux operating system. Generally running an OS on the device makes it easier for the designers since all of the basics are included in the OS, and the developers can focus more on the software to control the video and audio on the device without worrying about booting, partition, memory paging etc.
What was really nice about finding a full Linux distribution on the the device is we were able to do analysis like we would do for any typical Linux system. We could follow our standard analysis methodologies. In addition, we had access to all of the hardware of the device and since we logged in as root, we could manipulate as many things as we wish.
The first thing we wanted to look out for is how the device is partitioned on the hardware. Usually the OS and the required software are written to a read-only flash memory, while a small writeable flash block is allocated for configuration settings.
The IP camera’s info shows the following device flash
dev: size erasesize name
mtd0: 00040000 00010000 “boot”
mtd1: 000c0000 00010000 “config”
mtd2: 00480000 00010000 “rootfs”
mtd3: 00a80000 00010000 “app”
And the following device mount points
rootfs on / type rootfs (rw)
/dev/root on / type cramfs (ro,relatime)
devtmpfs on /dev type devtmpfs (rw,relatime,size=15968k,nr_inodes=3992,mode=755)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)
tmpfs on /tmp type tmpfs (rw,relatime)
/dev/mtdblock3 on /app type jffs2 (rw,relatime)
/dev/mtdblock1 on /config type jffs2 (rw,relatime)
tmfs on /system type tmpfs (rw,relatime)
The root and boot mount points were pretty typical and are read only memory, along with a writable configuration. This is typical, but what was interesting was that the ‘app’ memory was mounted as writable. The app directory listing is show below;
-rwxr–r– 1 527 527 2396620 Oct 27 2015 App3518
-rwxr–r– 1 527 527 598 Mar 31 03:38 app.sh
drwxr-xr-x 3 527 527 0 Sep 24 2015 dvr
drwxr-xr-x 2 527 527 0 Sep 24 2015 fontfile
drwxr-xr-x 3 527 527 0 Sep 24 2015 hi3518
-rwxr–r– 1 527 527 8584 Sep 24 2015 message
-rwxr–r– 1 527 527 128 Sep 24 2015 softwareversion
-rwxr–r– 1 527 527 11434 Sep 24 2015 upgrade
drwxr-xr-x 2 527 527 0 Sep 24 2015 voice
drwxr-xr-x 4 527 527 0 Oct 27 2015 wifi
More interestingly, the custom software that is present on the device was in writable space as opposed to read-only memory. This means that we could utilize this partition and even modify the custom software itself. We could replace the software with our own version, giving us even greater access and control over the device. Essentially, we could write our own customer firmware and replace that on the device.
The /app directory has some important pieces to note. First, the ‘wifi’ directory, which contains all of the wifi drivers and wifi related utilities for the device. It also has a voice folder which contains the audio files of various sound clips used on the device. The hi3518 directory has all of the device drivers for the on board hardware of the device. There is also a startup script called app.sh that runs at boot time and is used to manage loading all of the device drivers and starting up the main software.
The most important piece of software on this device is in the hi3518 directory and it is the App3518 program. After a little bit of investigation, and clues from the name of the program itself, we determined that this program controls the majority of the functionality of the system and has several running threads.
In order to more thoroughly understand the app program, we needed to evaluate the open ports. The app program controls three main ports, 80, 8000 and 9000. A custom web server is running on port 80. Using a local browser as a client and browsing to the IP, gave us access to the camera and configuration controls. Essentially, we had complete control over the device configuration and utilization of the camera when were were on the same local network as the IP camera.
Port 8000 and 9000 are more interesting as these are not a reserved port like port 80. We needed to do more analysis of the App3518 to figure out what these ports were specifically being used for. To start this process we had to analyze all of the various threads that the program is running and look for the threads that are handling the ports. We did this by using the ‘strace’ program and looked for threads hanging on an ‘accept’ call. Once we found one of these, we connected to the target port using netcat. We knew that we have found the thread/port combination that we were looking for when the thread’s accept triggers.
Okay, so we found the thread, now what? The goal was to figure out what the program does once it has a connection. During this stage, using strace really won’t help us any more as it only shows us what system calls have been made. We needed to look at the actual code itself to try to figure out what it is doing. We did not have any source code, so we resorted to trusty old GDB. GDB itself is a fairly large program, was was too big for the limited flash memory supplied on the device. To overcome this, we used GDB Server, a small server component of GDB. We attached GDB server to the target program and had a full GDB implementation to debug the software remotely.
Once we attached the GDB server to the thread and connected our remote debugging session, we could insert breakpoints into the program. We set a breakpoints in the code and waited for an incoming connection. We stepped through each system command of the software, tracing everything it does. While this was great for research, stepping through code line by line is time consuming, especially since the program was stripped of all debugging symbols.
In addition to our GDB debugging, we analyzed the program using other software such as IDA Pro or Radare2. Using a combination of all these tools we were able to find some interesting functionality and we were able to determine the functionality of the mystery port 8000 and 9000. Port 9000 is uses a custom protocol developed by Zmodo, which allows remote manipulation of the camera itself. It gives access to functions such as moving, zooming and focusing of the camera. Port 8000 on the other hand is much more interesting. It is also running a custom protocol, but after some code analysis and trial and error, we were able to crack the protocol.
The main purpose of port 8000 during normal operation is to gather system information about the device such as the version of the software running. This is usually used in conjunction with the web server running on port 80. The web server also has an update feature, that will update the device. It does this by downloading the current update files from a remote server, and then sending this update image to port 8000 with the appropriate headers. What’s interesting about this update method, is that the update file is just an JFFS2 image. The JFFS2 is the Journalling Flash File System version 2, a file management system. This is a common format found on small flash chips like the ones found on the device. If this image is mounted as a director on a laptop, you can see the contents of the entire /app directory of the device.
This means that the device updates by simply rewriting an entire partition on the flash memory. And it will write anything to the flash memory, anything. This is a fantastic way to gain a remote presence on the device. Basically, just send it JFFS2 image, with an remote access program running on there. This is possible if you were able to understand the propriety update format.
Using the same tactics we used on port 9000, we were able to figure it out. So we build a JFFS2 directory, using a copy of all the files currently on the flash device, and installed our own remote program. This program was small and would call out to our server located on a different network. Then to insure that the program was run at startup we modified the app.sh script and added a line to run our program. With a little bit of help from python we are able to remotely gain access to the device without any credentials.
Even better than that, the device supports UPnP port forwarding. So if we simply enabled UPnP on our internet facing router, it exposes holes to the three ports (80, 8000, 9000) on the device. This means we can gain access from outside the network. Then giving us full local access to the local network. Since it also forwards port 80 as well, we were able to see everything the camera could see from outside network.
What is convenient for malicious users is that unlike a laptop or mobile devices, the user wouldn’t have any knowledge of the camera being compromised. If someone was going to attack a laptop, they need to spend time developing software that is undetectable and goes unnoticed on machine. But since no user would ever be seeing the Linux interface on the device, we can be as loud as we want.