Skip to content

API Security Layers

Jump to bottom Edit New page
Stefan Thomas edited this page Jul 26, 2014 · 1 revision

Introduction

Codius is a very flexible architecture, which allows different APIs to co-exist. But we need to decide which APIs to build first. This page is designed to explain what the choices are, the design goals and the constraints.

Definitions

The API layer is the thing we standardize on. For example, if we have a JavaScript API for Codius contracts, then it is up to the host as to what JavaScript engine they want to use, how it is sandboxed, etc. so long as their solution ultimately exposes the exact classes and methods that are part of the API.

The security layer(s) is where the sandboxing occurs. We want to run people's untrusted code, but it shouldn't be able to access any resources beyond the APIs explicitly given to it. We can stack multiple security layers. For example, we can run code in the Native Client sandbox, inside of a Docker container, with an AppArmor policy inside of a one-time-use VM. Breaking out of one layer does not necessarily compromise security, so it can make sense to stack multiple layers.

Goals

  • Security: We must be extremely confident that contracts cannot break out of their sandbox.
    • Attack surface: A smaller attack surface means less code to review and less chance for bugs.
    • Battle-tested: How many people have tried to attack the system and how motivated were they?
    • Secure By Design: If a system has been designed to be secure, it's more likely to be than a system which you are trying to make secure by patching holes.
  • Flexibility: Allow contract developers to use different tools for different jobs.
  • Ease-of-use: It should be easy for developers to get started developing contracts.

Constraints

Before I start getting into the meat of this post, some observations:

  1. The security layer(s) and the API layer are distinct. For example, if I use JavaScript as the API layer, I have a choice of various security layers (JavaScript, Native Client, OS-based security, VMs, ...)
  2. The security layer(s) must be higher or equal to the API layer. For example if I use VMs as the API layer, that means users can upload arbitrary VMs. Obviously I can no longer use JavaScript as the security layer, since the contract may not use JavaScript at all.
  3. The API layer must be standardized, security layers do not need to be. The contract is written to use a certain API, but the host can choose whatever security layer it wants. However, it will have to convince users that whatever it chose is secure enough.

The trade-off, simply stated, is:

A higher API layer means contracts are easier to implement. A lower API layer means more security.

We should note as well that we don't want to tie Codius to a single choice of API layer. A lot of stuff like hashing contracts, billing, finding hosts, etc. is independent of the execution method, so we want to make it possible for people out there to define their own API layers. If you can get hosts to support them, great!

Current roadmap: We have a JS sandbox implementation and are working on the NaCl sandbox (for C/C++/Go and any other language we don't yet support). Anything beyond that is up for debate and of course community members are free to work on

Layers

The available layers are (top to bottom):

  • Physical layer
  • Hardware Virtualization (VT-d etc.)
  • Protection-domain based (i.e. OS-based)
    • AppArmor/SELinux
    • Containers/jails (OpenVZ, Docker)
    • Process isolation
  • Enclaves (Intel SGX)
  • Static native code validation (Native Client)
  • Interpretation (Qemu, JVM, .NET, V8)
  • Static virtual code validation
  • Compilation

Note that this list is somewhat x86-centric, but such is the world we live in. :)

Examples

Now to go through some of the options we discussed while working on Codius:

API layer: JavaScript Interpreter; Security layers: JavaScript Interpreter, LSM (e.g. AppArmor)

This one's the first we've implemented and it's secure enough for demonstration purposes. JavaScript VMs use capability-based security and the language was designed to run untrusted code from day one. Even so, JavaScript VMs are extremely complex pieces of software and there are exploits from time to time. Our current experimental Codius implementation is using this setup. AppArmor can be used to add security via the Linux kernel, but the Linux kernel also has exploits against it from time to time.

API layer: JavaScript Interpreter; Security layers: JavaScript Interpreter, Native Client, AppArmor

The next step for us will be to add Native Client as an additional security measure in case the JavaScript interpreter ends up getting exploited. Brandon is working full-time on this right now. The merits of this particular stack are debatable, but it paves the way for:

API layer: Native Client x86-32 or -64; Security layers: Native Client, seccomp-bpf, LSM (e.g. AppArmor)

Once we have JavaScript running inside of Native Client, it’s fairly trivial to open it up and allow any language interpreter.

API layer: Portable Native Client; Security layers: Native Client, seccomp-bpf, LSM (e.g. AppArmor)

Portable Native Client is a well-defined application binary interface (ABI) based on LLVM bytecode (Intermediate Representation - IR). It’s designed as a future-proof, portable byte-code. It’s an ideal target for compiled languages. For interpreted languages it isn’t (yet), since just-in-time (JIT) compilation is difficulty to do on PNaCl.

API layer: Java bytecode and standard library; Security layers: Java HotSpot VM, (Native Client), seccomp-bpf, LSM (e.g. AppArmor)

(TODO)

API layer: Hardware virtualization; Security layer: Hardware virtualization

The benefit of this choice is that it requires perhaps the least amount of work in terms of porting existing software to be a contract. There is an interesting implementation of this using AWS as the root of trust.

Choosing an API layer this high up the stack leaves very little flexibility on the side of the contract host, but it’s still an interesting option to be explored. If anyone reading this has experience with hypervisors, deterministically building VM images and such, we’d be very interested in your thoughts.

Difficulties include designing the API (how does the VM interact with the Codius host) and the VM building process (which is a lot more complex than uploading a snippet of JavaScript.) It also means that contracts take a while to boot up and have to run continuously if they want to be able to handle requests, so it would make them more expensive to run.

API layer: Docker x86-64-Linux-3.13; Security layer: Containers (libcontainer)

Containers are more lightweight than VMs and remove the requirement to boot up an OS first. The security is weaker however, since kernel bugs are potentially exploitable from within the sandbox. It also means that users have to standardize on a specific OS version and architecture, contracts would not be portable in this case.

Links

Add a custom footer

Clone this wiki locally