How to Secure Apache Ignite From Scratch
Currently, Apache Ignite doesn't provide a security implementation out-of-the-box. So, I'm going to show you how to create an Apache Ignite security plugin from scratch.
Join the DZone community and get the full member experience.
Join For FreeTo use Apache Ignite securely, you need an implementation of GridSecurityProcessor, a security plugin. Currently, Apache Ignite doesn't provide this implementation out-of-the-box. So, I'm going to show you how to create an Apache Ignite security plugin from scratch.
To create and test our plugin, we will use Apache Ignite 2.9.1, which, at this time, is the most recent version of Ignite.
The plug-in that we're going to create will be able to do the following:
- Authenticate an Apache Ignite node that is joining to a cluster.
- Authorize Ignite operations.
- Authenticate a thin client.
- Run a user-defined code with restrictions.
The code that is used in this article is available on GitHub.
Implementation of Apache Ignite Security Interfaces
Let's start with the pom.xml
file. All we need do is define the following dependencies:
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-core</artifactId>
<version>2.9.1</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-spring</artifactId>
<version>2.9.1</version>
</dependency>
Now, we are ready to create implementations of the security interfaces.
The first implementation is SecuritySubject.
x
package security;
import java.net.InetSocketAddress;
import java.util.UUID;
import org.apache.ignite.plugin.security.SecurityPermissionSet;
import org.apache.ignite.plugin.security.SecuritySubject;
import org.apache.ignite.plugin.security.SecuritySubjectType;
public class SecuritySubjectImpl implements SecuritySubject {
private static final long serialVersionUID = 0L;
private UUID id;
private SecuritySubjectType type;
private Object login;
private InetSocketAddress address;
private SecurityPermissionSet permissions;
public UUID id() {
return id;
}
public SecuritySubjectImpl id(UUID id) {
this.id = id;
return this;
}
public SecuritySubjectType type() {
return type;
}
public SecuritySubjectImpl type(SecuritySubjectType type) {
this.type = type;
return this;
}
public Object login() {
return login;
}
public SecuritySubjectImpl login(Object login) {
this.login = login;
return this;
}
public InetSocketAddress address() {
return address;
}
public SecuritySubjectImpl address(InetSocketAddress address) {
this.address = address;
return this;
}
public SecurityPermissionSet permissions() {
return permissions;
}
public SecuritySubjectImpl permissions(SecurityPermissionSet permissions) {
this.permissions = permissions;
return this;
}
/** {@inheritDoc} */
public String toString() {
return "TestSecuritySubject{" +
"login=" + login +
'}';
}
}
The SecurityContext interface can be implemented in the following way:
x
package security;
import java.io.Serializable;
import java.util.Collection;
import org.apache.ignite.internal.processors.security.SecurityContext;
import org.apache.ignite.internal.util.typedef.F;
import org.apache.ignite.plugin.security.SecurityPermission;
import org.apache.ignite.plugin.security.SecuritySubject;
public class SecurityContextImpl implements SecurityContext, Serializable {
private static final long serialVersionUID = 0L;
private final SecuritySubject subject;
public SecurityContextImpl(SecuritySubject subject) {
this.subject = subject;
}
public SecuritySubject subject() {
return subject;
}
public boolean taskOperationAllowed(String taskClsName, SecurityPermission perm) {
return hasPermission(subject.permissions().taskPermissions().get(taskClsName), perm);
}
public boolean cacheOperationAllowed(String cacheName, SecurityPermission perm) {
return hasPermission(subject.permissions().cachePermissions().get(cacheName), perm);
}
public boolean serviceOperationAllowed(String srvcName, SecurityPermission perm) {
return hasPermission(subject.permissions().servicePermissions().get(srvcName), perm);
}
public boolean systemOperationAllowed(SecurityPermission perm) {
Collection<SecurityPermission> perms = subject.permissions().systemPermissions();
if (F.isEmpty(perms))
return subject.permissions().defaultAllowAll();
return perms.stream().anyMatch(p -> perm == p);
}
public boolean operationAllowed(String opName, SecurityPermission perm) {
switch (perm) {
case CACHE_CREATE:
case CACHE_DESTROY:
return systemOperationAllowed(perm) || cacheOperationAllowed(opName, perm);
case CACHE_PUT:
case CACHE_READ:
case CACHE_REMOVE:
return cacheOperationAllowed(opName, perm);
case TASK_CANCEL:
case TASK_EXECUTE:
return taskOperationAllowed(opName, perm);
case SERVICE_DEPLOY:
case SERVICE_INVOKE:
case SERVICE_CANCEL:
return serviceOperationAllowed(opName, perm);
case EVENTS_DISABLE:
case EVENTS_ENABLE:
case ADMIN_VIEW:
case ADMIN_CACHE:
case ADMIN_QUERY:
case ADMIN_OPS:
case JOIN_AS_SERVER:
return systemOperationAllowed(perm);
// You should decide what is a proper reaction when unknown permission is getting
default:
throw new IllegalArgumentException("Unknown security permission: " + perm);
}
}
private boolean hasPermission(Collection<SecurityPermission> perms, SecurityPermission perm) {
if (perms == null)
return subject.permissions().defaultAllowAll();
return perms.stream().anyMatch(p -> perm == p);
}
}
There are a few tricky points. The SecurityContext interface has to extend the Serializable interface; so, don't forget to add Serializable
to the list of implementing interfaces.
Permissions are divided into four groups:
- system operations:
CACHE_CREATE
,CACHE_DESTROY
,EVENTS_DISABLE
,EVENTS_ENABLE
,ADMIN_VIEW
,ADMIN_CACHE
,ADMIN_QUERY
,ADMIN_OPS
,JOIN_AS_SERVER
- service operations:
SERVICE_DEPLOY
,SERVICE_INVOKE
,SERVICE_CANCEL
- cache operations:
CACHE_CREATE
,CACHE_DESTROY
,CACHE_PUT
,CACHE_READ
,CACHE_REMOVE
- task operations:
TASK_EXECUTE
,TASK_CANCEL
To determine whether a security subject has been given permission, you have to know what the permission group is. The operationAllowed()
method shows how you can identify the permission group.
Notice that CACHE_CREATE
and CACHE_DESTROY
are included in two groups (system operations and cache operations). When CACHE_CREATE
(CACHE_DESTROY
) is treated as a system permission, it applies to all caches. In other cases, when CACHE_CREATE
(CACHE_DESTROY
) is treated as cache permission, permission checking is executed with the account of the cache name.
In the future, the SecurityPermission enum can be extended by new constants. So, when you get unknown permissions, you need to decide on an appropriate reaction.
The GridSecurityProcessor is the central interface for Ignite security. We will improve our implementation of GridSecurityProcessor
step-by-step by adding the capabilities that are described in the introduction to this article.
x
package security;
import java.net.InetSocketAddress;
import java.util.Collection;
import java.util.UUID;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.cluster.ClusterNode;
import org.apache.ignite.internal.GridKernalContext;
import org.apache.ignite.internal.IgniteNodeAttributes;
import org.apache.ignite.internal.processors.GridProcessorAdapter;
import org.apache.ignite.internal.processors.security.GridSecurityProcessor;
import org.apache.ignite.internal.processors.security.SecurityContext;
import org.apache.ignite.internal.util.typedef.F;
import org.apache.ignite.plugin.security.AuthenticationContext;
import org.apache.ignite.plugin.security.SecurityCredentials;
import org.apache.ignite.plugin.security.SecurityException;
import org.apache.ignite.plugin.security.SecurityPermission;
import org.apache.ignite.plugin.security.SecurityPermissionSetBuilder;
import org.apache.ignite.plugin.security.SecuritySubject;
import org.apache.ignite.plugin.security.SecuritySubjectType;
public class GridSecurityProcessorImpl extends GridProcessorAdapter implements GridSecurityProcessor {
private final SecurityCredentials localNodeCredentials;
public GridSecurityProcessorImpl(GridKernalContext ctx, SecurityCredentials cred) {
super(ctx);
localNodeCredentials = cred;
}
public void start() throws IgniteCheckedException {
U.quiet(false, "[GridSecurityProcessorImpl] Start; localNode=" + ctx.localNodeId()
+ ", login=" + localNodeCredentials.getLogin());
ctx.addNodeAttribute(IgniteNodeAttributes.ATTR_SECURITY_CREDENTIALS, localNodeCredentials);
super.start();
}
public SecurityContext authenticateNode(ClusterNode node, SecurityCredentials credentials) {
// This is the place to check the credentials of the joining node.
SecuritySubject subject = new SecuritySubjectImpl()
.id(node.id())
.login(credentials.getLogin())
.address(new InetSocketAddress(F.first(node.addresses()), 0))
.type(SecuritySubjectType.REMOTE_NODE)
.permissions(
SecurityPermissionSetBuilder
.create()
.appendSystemPermissions(SecurityPermission.JOIN_AS_SERVER)
.build()
);
U.quiet(false, "[GridSecurityProcessorImpl] Authenticate node; " +
"localNode=" + ctx.localNodeId() +
", authenticatedNode=" + node.id() +
", login=" + credentials.getLogin());
return new SecurityContextImpl(subject);
}
public boolean enabled() {
return true;
}
public boolean isGlobalNodeAuthentication() {
return false;
}
// other no-op methods of GridSecurityProcessor
}
You can use the GridProcessorAdapter to create your processor. This abstract class enables you to override only the methods that you need to override and to hide boilerplate code. In our case, we want to override the start()
method and, thus, add the credentials to the node's attributes. One or more cluster nodes will use the credentials to authenticate the node that is joined to the cluster in the authenticateNode()
method. Before you create a SecuritySubject
instance, make sure that the credentials are valid.
The code that we wrote for GridSecurityProcessor
enables nodes to join a cluster.
Let's try it!
Start Ignite Nodes With a Security Plugin
To start Ignite nodes with our GridSecurityProcessor
, we must define the processor as an Ignite plugin. You can read about Ignite plugins in the Plugins document, but when you create an Ignite processor plugin, you need to be aware of a few idiosyncrasies.
x
package security;
import java.io.Serializable;
import java.util.UUID;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.cluster.ClusterNode;
import org.apache.ignite.internal.IgniteEx;
import org.apache.ignite.internal.processors.security.GridSecurityProcessor;
import org.apache.ignite.plugin.CachePluginContext;
import org.apache.ignite.plugin.CachePluginProvider;
import org.apache.ignite.plugin.ExtensionRegistry;
import org.apache.ignite.plugin.IgnitePlugin;
import org.apache.ignite.plugin.PluginContext;
import org.apache.ignite.plugin.PluginProvider;
import org.apache.ignite.plugin.PluginValidationException;
import org.apache.ignite.plugin.security.SecurityCredentials;
public class SecurityPluginProvider implements PluginProvider {
private final SecurityCredentials localNodeCredentials;
public SecurityPluginProvider(SecurityCredentials cred) {
localNodeCredentials = cred;
}
public Object createComponent(PluginContext ctx, Class cls) {
if (cls.isAssignableFrom(GridSecurityProcessor.class))
return new GridSecurityProcessorImpl(((IgniteEx)ctx.grid()).context(), localNodeCredentials);
return null;
}
public String name() {
return "SecurityPluginProvider";
}
public String version() {
return "1.0.0";
}
public String copyright() {
return "for the article";
}
public IgnitePlugin plugin() {
return new IgnitePlugin() {
};
}
// other no-op methods of PluginProvider
}
When Ignite's node-starting routine requires a security processor, the createComponent()
method returns an instance of GridSecurityProcessor
. We don't need an IgnitePlugin
object, but we cannot return null
, so we created this instance of IgnitePlugin
that does nothing.
Now, we are ready to use our plugin to start nodes. Using ignite.bat
and Spring configuration, I started a node:
x
<bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="pluginProviders">
<array>
<bean class="security.SecurityPluginProvider">
<constructor-arg ref="credentials"/>
</bean>
</array>
</property>
</bean>
<bean id="credentials" class="org.apache.ignite.plugin.security.SecurityCredentials">
<constructor-arg value="firstSubject"/>
<constructor-arg value=""/>
</bean>
And, the second node was started from the following Java code:
x
Ignition.start(
new IgniteConfiguration()
.setPluginProviders(
new SecurityPluginProvider(new SecurityCredentials("secondSubject", null))
)
);
Let's look at the console output on the joining node:
We see that the security plugin is in place and started, and authentication is on.
The first node that started is the coordinator. In our case, the coordinator processes authentication of the node that is joining. The coordinator displays the following text:
xxxxxxxxxx
[15:30:55] [GridSecurityProcessorImpl] Authenticate node; localNode=c060fe0a-0b95-4903-8612-3e773e702eb4, authenticatedNode=0958bfd8-f2fe-4a17-9e58-ded8bdaa4c7e, login=secondSubject
For all nodes in the cluster, the same security plugin must be configured and security must be enabled. Otherwise, when the node starts, you receive an error.
Authorize Ignite Operations
To authorize Ignite operations, we need to implement the authorize()
method, which can look like the following:
x
public void authorize(String name, SecurityPermission perm, SecurityContext secCtx)
throws SecurityException {
if (!((SecurityContextImpl)secCtx).operationAllowed(name, perm))
throw new SecurityException("Authorization failed [perm=" + perm +
", name=" + name +
", subject=" + secCtx.subject() + ']');
}
The authorize()
method throws a SecurityException if the requested access, defined by the name and the permission, is not permitted based on passed SecurityContext
.
Let's try to authorize cache operations.
Create two caches by adding a definition of cache configurations to the Ignite configuration file:
x
<bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="pluginProviders">
<array>
<bean class="security.SecurityPluginProvider">
<constructor-arg ref="credentials"/>
</bean>
</array>
</property>
<property name="cacheConfiguration">
<array>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<constructor-arg value="common_cache"/>
</bean>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<constructor-arg value="secret_cache"/>
</bean>
</array>
</property>
</bean>
Add to the implementation of GridSecurityProcessor
, the method that emulates a security poliсy:
xxxxxxxxxx
private SecurityPermissionSet getPermissionSet(Object login) {
if(login.equals("secondSubject"))
return new SecurityPermissionSetBuilder()
.appendSystemPermissions(SecurityPermission.JOIN_AS_SERVER)
.appendCachePermissions("common_cache",
SecurityPermission.CACHE_PUT,
SecurityPermission.CACHE_READ)
.build();
return SecurityPermissionSetBuilder.ALLOW_ALL;
}
The subject with the secondSubject
login can write to and read from only the cache that is named common_cache
. The process of node authentication uses the following method to get the subject's permissions:
x
SecuritySubject subject = new SecuritySubjectImpl()
subject.permissions(getPermissionSet(credentials.getLogin()));
// sets other fields like described early
Now, we are ready to start the cache-operations
example:
x
Ignite ignite = Ignition.start(
new IgniteConfiguration()
.setPluginProviders(
new SecurityPluginProvider(new SecurityCredentials("secondSubject", null))
)
);
try {
ignite.cache("secret_cache").put("key", "value");
}
catch (SecurityException e){
System.out.println("The try to put to 'secret_cache': " + e.getMessage());
}
IgniteCache<String, String> cache = ignite.cache("common_cache");
cache.put("key", "some_value");
System.out.println("Cache " + cache.getName() + " key=" + cache.get("key"));
try {
cache.remove("key");
}
catch (SecurityException e){
System.out.println("The try to remove from 'common_cache': " + e.getMessage());
}
The output looks like the following:
xxxxxxxxxx
The try to put to 'secret_cache': Authorization failed [perm=CACHE_PUT, name=secret_cache, subject=TestSecuritySubject{login=secondSubject}]
Cache common_cache key=some_value
The try to remove from 'common_cache': Authorization failed [perm=CACHE_REMOVE, name=common_cache, subject=TestSecuritySubject{login=secondSubject}]
In regard to the put
operation, the output indicates that the subject that has the secondSubject
login doesn't have permission to manipulate the cache that is named secret_cache
. The lack of permission is due to the fact that we didn't mention the name secret_cache
in our security policy. However, the subject can perform put
and read
operations, but not remove operations, on the cache that is named common_cache
. This behavior adheres to the permissions that we defined in the security policy method.
Thin Client Authentication
A thin client is a lightweight Ignite client that establishes a socket connection to a standard Ignite node. The node performs authentication and creates a security context associated with the thin client. The GridSecurityProcessor
provides the interface to get a thin client's security context on every node in the cluster. But how to implement this feature is the developers' decision. I'm going to use Ignite's cache to make a thin client's security context accessible throughout the cluster.
Define the following methods in the GridSecurityProcessor
implementation:
xxxxxxxxxx
public SecurityContext authenticate(AuthenticationContext context) {
// This is the place to check the credentials of the thin client.
SecuritySubject subject = new SecuritySubjectImpl()
.id(context.subjectId())
.login(context.credentials().getLogin())
.type(SecuritySubjectType.REMOTE_CLIENT)
.permissions(getPermissionSet(context.credentials().getLogin()));
SecurityContext res = new SecurityContextImpl(subject);
ctx.grid().getOrCreateCache("thin_clients").put(subject.id(), res);
U.quiet(false, "[GridSecurityProcessorImpl] Authenticate thin client subject; " +
" login=" + subject.login());
return res;
}
public SecurityContext securityContext(UUID subjId) {
return (SecurityContext)ctx.grid().getOrCreateCache("thin_clients").get(subjId);
}
public void onSessionExpired(UUID subjId) {
ctx.grid().getOrCreateCache("thin_clients").remove(subjId);
}
We use the thin_clients
cache to implement transmission of a client's security context between nodes.
Now, we can start a thin client:
xxxxxxxxxx
ClientConfiguration config = new ClientConfiguration()
.setAddresses("127.0.0.1:10800")
.setUserName("secondSubject")
.setUserPassword("");
try (IgniteClient client = Ignition.startClient(config)) {
ClientCache<String, String> cache = client.cache("common_cache");
cache.put("key", "some_value");
System.out.println("Cache " + cache.getName() + " key=" + cache.get("key"));
cache.remove("key");
}
Now, we remember that the secondSubject
subject has put
and read
permissions for the common_cache
cache but that the subject cannot remove anything from the cache. Therefore, the output looks like the following:
xxxxxxxxxx
Cache common_cache key=some_value
Exception in thread "main" org.apache.ignite.client.ClientAuthorizationException: User is not authorized to perform this operation
Run User-Defined Code With Restrictions
User-defined code contains custom logic via various APIs, including compute tasks, event filters, and message listeners. In some cases, you might want to restrict the code's capabilities on the nodes that execute the code. For this purpose, you can use the Ignite Sandbox feature.
Let's consider changes that have been made in the security interface implementations that allow code to be run inside Sandbox.
x
public class GridSecurityProcessorImpl extends GridProcessorAdapter implements GridSecurityProcessor {
public boolean sandboxEnabled() {
return true;
}
private PermissionCollection getSandboxPermissions(Object login) {
PermissionCollection res = new Permissions();
if (login.equals("sandboxSubject"))
res.add(new PropertyPermission("java.version", "read"));
else
res.add(new AllPermission());
return res;
}
public SecurityContext authenticateNode(ClusterNode node, SecurityCredentials credentials) {
// This is the place to check the credentials of the joining node.
SecuritySubject subject = new SecuritySubjectImpl()
.id(node.id())
.login(credentials.getLogin())
.address(new InetSocketAddress(F.first(node.addresses()), 0))
.type(SecuritySubjectType.REMOTE_NODE)
.permissions(getPermissionSet(credentials.getLogin()))
.sandboxPermissions(getSandboxPermissions(credentials.getLogin()));
U.quiet(false, "[GridSecurityProcessorImpl] Authenticate node; " +
"localNode=" + ctx.localNodeId() +
", authenticatedNode=" + node.id() +
", login=" + credentials.getLogin());
return new SecurityContextImpl(subject);
}
// other fields and methods
}
The sandboxEnable()
method enables you to switch off Ignite Sandbox when Java's SecurityManager
is in place. By default, Ignite Sandbox is off, so we will override this method. Where a node passes authentication, we use the getSandboxPermissions()
method to emulate a security policy inside Sandbox. A security subject with the sandboxSubject
login has only read access to the system property that is named java.version
.
xxxxxxxxxx
public class SecuritySubjectImpl implements SecuritySubject {
private PermissionCollection sandboxPermissions;
public PermissionCollection sandboxPermissions() {
return sandboxPermissions;
}
public SecuritySubjectImpl sandboxPermissions(PermissionCollection sandboxPermissions) {
this.sandboxPermissions = sandboxPermissions;
return this;
}
// other fields and methods
}
The implementation of SecuritySubject
contains the field and the methods that are required to pass permissions to Sandbox.
Ignite needs enough permissions to work correctly. I'm going to use the most straightforward way to grant all permissions to Ignite. I will use the {IGNITE-HOME}\security\test.policy
file:
xxxxxxxxxx
grant codeBase "file:{$IGNITE-HOME}\\libs\\-" {
permission java.security.AllPermission;
};
To start an Ignite node with SecurityManager
and a specific security policy, we can modify the ignite.bat
file by adding the following text into the ADD YOUR/CHANGE ADDITIONAL OPTIONS HERE
section:
x
set JVM_OPTS=%JVM_OPTS% -Djava.security.manager
set JVM_OPTS=%JVM_OPTS% -Djava.security.policy="file:%IGNITE_HOME%\security\test.policy"
Now, we are ready to do our final test. When we start an Ignite node, we see the following output:
xxxxxxxxxx
[15:30:56] Security status [authentication=on, sandbox=on, tls/ssl=off]
Notice that Sandbox is on.
The test example looks like the following:
xxxxxxxxxx
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCompute;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.lang.IgniteCallable;
import org.apache.ignite.plugin.security.SecurityCredentials;
import security.SecurityPluginProvider;
public class SandboxTest {
public static void main(String[] args) {
Ignite ignite = Ignition.start(
new IgniteConfiguration()
.setPeerClassLoadingEnabled(true)
.setPluginProviders(
new SecurityPluginProvider(new SecurityCredentials("sandboxSubject", null))
)
);
try {
IgniteCompute compute = ignite.compute(ignite.cluster().forRemotes());
System.out.println("Java version: " + compute.call(new PropertyReader("java.version")));
System.out.println("Java home: " + compute.call(new PropertyReader("java.home")));
}
finally {
Ignition.stop(true);
}
}
private static class PropertyReader implements IgniteCallable<String>{
private final String propertyName;
public PropertyReader(String propertyName) {
this.propertyName = propertyName;
}
public String call() {
return System.getProperty(propertyName);
}
}
}
For the first calling, we get the Java version that is used on the remote node. But, for the reading java.home
property, we get the following output:
xxxxxxxxxx
java.security.AccessControlException: access denied ("java.util.PropertyPermission" "java.home" "read")
Peer class loading should be enabled on every node.
It is hard to find all the information that you need about how to implement an Ignite security plugin in one place, so I hope that this tutorial will be useful to you. If you find the tutorial useful, please like this blog post.
If you have a question or comment, please enter it below.
Opinions expressed by DZone contributors are their own.
Comments