Intel SGX CPU (staring from Skylake) has been there for while. The good news is that there is still no known exploitation against SGX self yet, though there are some exploitations in the enclave code and Intel SGX SDK. In general, SGX is still believed to provide strong security guarantee for the data/code in the enclave. If there is really something messed up in SGX, it has to be the CPU logic/mirocode. This post tries to peek into a specific bug reported by Intel for its SGX CPU implementation. Moving forward, we invesitgate the possible mitigations and add new features into the well-known Intel platform security tool chipsec. Cheers.

1. SKL012

In the spec update [1] released by Intel on Sep 2016, there are 6 CPU bugs related with SGX in general [2]. None of them is treated seriously by Intel thus no fix is planed for any of it. Among them, we especially look at SKL012:

My interpretation for this SKL012 bug is that SGX was designed to not allow SMSW instruction in an enclave, though the instuction could be executed by ring-0 and ring-3. Unfortunately, due to this bug, SMSW “may” be executed in the enclave.

2. So what the fuss?

SMWS is one of the secuirty sensitive instructions, which can be run by ring-3 and reveal some security sensitive information of the platform and the kernel to the user space. These instructions are usually leveraged by malware to detect the VM enviornment or exploit the system/kernel configuation [3][4]. Similar with SMWS, other security sensitive instructions include SGDT, SLDT, SIDT, and STR. All these instructions could be run by ring-3 code without any problem. And what about these instructions for SGX besides the SMWS mentioned in SKL012.

3. Verify the bug(s)

Here we implement a tool called sgxbug [7] based on the enclave creation sample code provided by Intel SGX SDK for Linux [8] by adding the SMWS instruction into the enclave and retrieving the results from the application. We also covered other 4 security sensitive instructions mentioned above. For implementation detailes, please check out the commit (https://github.com/daveti/sgxbug/commit/373c9e6f3d0d70a172cf5315dce15151fa4271e1). For normal ring-3 testing code without SGX, one could refer to [6] for details.

Here is the result of running sgxbug app:

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/sgxbug# ./app 0
Got senstive instruction idx: 0
Checksum(0x0x7ffe4ba35c30, 100) = 0xfffd4143
Info: executing thread synchronization, please wait…
Start sensitive instruction testing…
GDT: limit=0127, base=ffff880273c49000
Sensitive instruction testing done…
Info: SampleEnclave successfully returned.
Enter a character before exit …

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/sgxbug# ./app 1
Got senstive instruction idx: 1
Checksum(0x0x7ffd677a11f0, 100) = 0xfffd4143
Info: executing thread synchronization, please wait…
Start sensitive instruction testing…
IDT: limit=4095, base=ffffffffff578000
Sensitive instruction testing done…
Info: SampleEnclave successfully returned.
Enter a character before exit …

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/sgxbug# ./app 2
Got senstive instruction idx: 2
Checksum(0x0x7fff399040a0, 100) = 0xfffd4143
Info: executing thread synchronization, please wait…
Start sensitive instruction testing…
LDT: ffffffffffff0000
Sensitive instruction testing done…
Info: SampleEnclave successfully returned.
Enter a character before exit …

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/sgxbug# ./app 3
Got senstive instruction idx: 3
Checksum(0x0x7fffb1b31300, 100) = 0xfffd4143
Info: executing thread synchronization, please wait…
Start sensitive instruction testing…
MSW: ffffffffffff0033
Sensitive instruction testing done…
Info: SampleEnclave successfully returned.
Enter a character before exit …

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/sgxbug# ./app 4
Got senstive instruction idx: 4
Checksum(0x0x7ffcfe43bee0, 100) = 0xfffd4143
Info: executing thread synchronization, please wait…
Start sensitive instruction testing…
TR: ffffffffffff0040
Sensitive instruction testing done…
Info: SampleEnclave successfully returned.
Enter a character before exit …

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/sgxbug#

(Maybe not) Surprisingly, only SMWS is working smoothly but also SGDT, SLDT, SIDT, and STR. In summary, there is no any limitation for code in the enclave to execute these security sensitive instrucitons.

4. UMIP

Now what? It turns out the latest Intel CPU has a thing called UMIP – User Mode Instruction Prevention [5], which as its name implied can block some security sensitive instructions running at ring-3. The blocked instructions currently include all the 5 instructions mentioned above. That means, once UMIP is enabled, the user-space application (or malware) is not able to run these instructions anymore. This is really good in my opinion and I would recommend enabling this if possible to reduce the attack surface from the user space. Due to this reason, we add some new features to the CHIPSEC tool [9], detecting the UMIP feature, checking if the feature enabled, and enabling the UMIP for all cores if possible. For implementation details, please refer to the commit (https://github.com/daveti/chipsec/commit/0da5abf7e41b7fd9750a1a559db8b2f427745f88) and the commit (https://github.com/daveti/chipsec/commit/3c9b04903a616040e93e59260f313e1ff018eed6).

5. What if…

With the knowledge of UMIP, we can start thinking what would happen for SKL012 when UMIP is enabled. We have verified that without UMIP, all these instructions work in the enclave. Will they work again when UMIP is enabled? Unfortunately, my SGX CPU is apparently not “latest” enough to have such a feature:

root@sgx2-HP-ENVY-x360-m6-Convertible:~/git/chipsec# chipsec_util cpu umip detect

#################################################################
### ##
### CHIPSEC: Platform Hardware Security Assessment Framework ##
### ##
#################################################################
[CHIPSEC] Version 1.2.5
****** Chipsec Linux Kernel module is licensed under GPL 2.0
[CHIPSEC] API mode: using CHIPSEC kernel module API
[CHIPSEC] Executing command ‘cpu’ with args [‘umip’, ‘detect’]

[cpu] CPUID out: EAX=0x00000000, EBX=0x029C67AF, ECX=0x00000000, EDX=0x00000000
[CHIPSEC] UMIP available for CPU: False
[CHIPSEC] (cpu) time elapsed 0.000

Here is what I imagined: Because of SKL012 bug in SGX, it is possible that UMIP could not prevent the execution of these security sensitive instructions within the enclave. If this is true, with UMIP enabled, malware would need to use SGX to guarantee the exeution of these (e.g., VM detection).

Reference:

[1]http://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/desktop-6th-gen-core-family-spec-update.pdf
[2]https://www.blackhat.com/docs/us-16/materials/us-16-Aumasson-SGX-Secure-Enclaves-In-Practice-Security-And-Crypto-Review.pdf
[3]https://www.usenix.org/legacy/publications/library/proceedings/sec2000/full_papers/robin/robin_html/index.html
[4]http://hypervsir.blogspot.com/2014/10/kernel-information-leak-with.html
[5]https://lkml.org/lkml/2016/11/8/68
[6]http://charette.no-ip.com:81/programming/2009-12-30_Virtualization/2009-12-30_Virtualization.c
[7]https://github.com/daveti/sgxbug
[8]https://github.com/01org/linux-sgx
[9]https://github.com/daveti/chipsec