Security: Restrict internal access by third-party software

Question

I have a Java application in which third-party "plugins" can be loaded by users to enhance the user experience. An API exists for use by these plugins, but the third-party software should be restricted from access to internal application classes for purpose of security. The restricted package to plugins would be "com.example" and the allowed would be "com.example.api". The API classes do make calls to the internal, obfuscated classes.

After researching this, I came across a couple methods of SecurityManager: checkMemberAccess(Class, int) and checkPackageAccess(String), which both seemed to be viable paths to my goal. However, after doing some tests and further research, I have found that checkMemberAccess only applies to reflection calls, and checkPackageAccess is only called when a class loader invokes loadClass.

What is a reasonable way to restrict access to a package (com.example, but not com.example.api)?

Answer 1

I suggest writing a custom class loader for the plugins, which hides the existence of the com.example package from classes loaded using that classloader. Usually class loaders delegate to their parent, but there are several implementations out in the wild which will do so only in part or not at all. I believe e.g. ant uses this technique. When loaded with such a class loader, any class linked against forbidden functinality would fail to load. Or if the implementation used lazy linking, and it did load successfully, it would still fail during execution of the forbidden code.

Having denied your plugins link-time access to the forbidden package, you can then use a SecurityManager to deny runtime access via reflection, and also to deny creation of a new class loader which might be used to circumvent yours.

class RestrictingClassLoader extends URLClassLoader {
  @Override
  public Class<?> loadClass(String name) throws ClassNotFoundException {
    if (!name.startsWith("com.example.") || name.startsWith("com.example.api."))
      return super.loadClass(name);
    return findClass(name);
  }
}

class RestrictingSecurityManager extends SecurityManager {
  private boolean isRestricted() {
    for (Class<?> cls: getClassContext())
      if (cls.getClassLoader() instanceof RestrictingClassLoader)
        return true;
    return false;
  }
  // Implement other checks based on isRestricted().
}