How we use OPA at Cyral

How we use OPA at Cyral

Bottom

The Cyral service evaluates policies to monitor, protect, and control access to data. A key engineering requirement that we had to address while building Cyral is the system’s ability to evaluate these policies in the context of dynamically evolving systems. Perhaps one of the simplest examples of such decision making is the authorization task, for example, answering questions like “Does any user have access to any resource?”. This is quite simple when it comes to web APIs, for example, as various authorization solutions and protocols (OAuth2, etc.) can be used, many of which are supported directly by the web frameworks in use. However, at Cyral, we have often found ourselves needing to make similar decisions in the context of systems and domains that are not as straightforward as the simple API authorization example. Some examples include:

  • Enforce user and group access to different databases consistently.
  • Classify the content of random samples of data from databases.
  • Evaluation of network allow and deny lists based on the user’s IP address.
  • Extraction and evaluation of user identification information from database connection data.

While it would be possible to encapsulate all decision-making logic in standard application code, we quickly realized that such an approach is inflexible.

Cyber ​​Security Live - Boston

When designing the Cyral platform, we also realized that we needed a consistent approach to handling policy decisions across our entire stack. As mentioned above, our policies themselves vary greatly in substance and domains. Additionally, we needed something that would fit well into a mesh of disparate microservices.

We finally decided to adopt Open Policy Agent (OPA) in our stack. OPA is an open source, general-purpose policy engine that allows you to specify policies as code declaratively, and then use those policies to make decisions dynamically. OPA policies are defined using Rego, a data-driven, declarative Domain Specific Language (DSL).

Limitations and Complexities of Policy Evaluation in Microservices Architectures

Traditional monolithic software systems had a simpler problem to solve when it came to policy evaluation, since it was usually contained within a single part of the system. As the industry adopted microservices as a standard in software architecture, policy enforcement became more complex.

  • Decision making often required unique behavior across multiple independent services.
  • Services are often written in different languages ​​and use different technologies, adding further complexity.
  • The code to express policies can be complex and difficult to read and understand.
  • As services are updated, they may eventually become inconsistent with the policies associated with them.

Furthermore, while traditional policy frameworks can handle simpler authorization use cases (such as APIs and user authorization) well, they are not flexible enough to model more generalized systems and decision making.

Why OPA (and Rego)?

OPA is a general purpose policy engine, which makes it quite powerful and flexible. Some of its main benefits include:

  • OPA is a general purpose policy engine and can be used to make decisions at any layer of the stack, as long as the domain and its state can be modeled as JSON (or YAML). This can be harnessed to make decisions about any domain imaginable..
  • Rego is declarative and readable — Policies express what the result should be, not how to achieve that result..
  • Rego policies are decoupled and can be managed separately as code and injected into OPA at any time during their lifecycle.
  • OPA is based on data. External data can be injected into policy and used to make decisions, even when the data changes.
  • OPA is lightweight and can be integrated into the stack as a service or even as a simple library.

A quick introduction to Rego

Rego policies use two pieces of JSON (or YAML) data to make policy decisions, called input and data Input represents a query per decision, such as the data associated with a single HTTP request or a database query, etc. On the other hand, data represents external data that can be used in decision-making, inserted into OPAs ahead of time through a variety of means (APIs, etc.).

The OPA itself does not understand what the domain state data (or the schema for that data) means in any context, nor does it need to: it merely provides an interface to interact with the data.

Consider the example canonical policy: a policy to allow/deny requests to a REST API, given some input query containing a user, a path (eg. /balance/{username} ), and a method (for example GET, POSTetc.) (try Rego’s playground) :

# By default, the allow rule is set to false
default allow = false

# allow is true if...
allow = true {
    # The input method is 'GET', and...
    input.method == "GET"
    
    # ...the first element of the input path is 'balance', and...
    input.path[0] == "balance"
    
    # ...the second element of the input path is equal to the user 
    # making the request, and...
    input.path[1] == input.user
    
    # ...the input path is not restricted as defined by the
    # external data.
    not data.paths[input.path[0]].restricted
}

The policy prior to both uses input to make policy decisions, as well as cross references with data. Both input Y data they are just arbitrary JSON. The input data may look something like the following, representing a single GET request to an API by the user beto and on the way /balance/bob:

{
  "method": "GET",
  "path": ["balance", "bob"],
  "user": "bob"
}

As mentioned above, data it’s also just arbitrary JSON. For example, imagine we might have a set of API routes that we decide should be restricted. These routes could be defined as external JSON data (in this case, we will restrict the balance route):

{
  "paths": {
    "balance": {
      "restricted": false
    }
  }
}

Now, regardless of the input query, if the balance the route is set to restricted in the external data, the policy will evaluate to false.

Note that the data is just arbitrary JSON. In the example above, we modeled a simple deny list of API endpoints. Actually, your imagination is the limit. If you can model a domain and its state as JSON, you can take advantage of it in a Rego policy.

In addition, OPA provides mechanisms to update a policy’s external data dynamically and in real time, often as requirements and/or domain status change. This flexibility can be harnessed to completely alter how policy decisions are evaluated, without requiring underlying changes to the policies themselves. Quite simply, as the system and your data change, OPA can adapt its decision making. This type of data-driven behavior is extremely powerful, but rare in other policy engines (especially those that are custom built)

How Cyral OPA uses

Once we adopt OPA, we immediately start using it across our stack to make “traditional” policy decisions (such as allow/deny access approval, etc.) and more.”not“traditional” political decisions, taking advantage of the flexible and agnostic nature of the OPA domain.

Traditional Policy Decisions Non-traditional policy decisions
Generally implies authorization, eg “Can this user do X?” It can involve any kind of general decision, for example, “Is this data sensitive?”
The result of a decision is usually a single true/false (for example, allow/disallow). The output of the decision is arbitrary: it can be any structured or unstructured data.
Policies are usually evaluated only on input data (eg a user token, etc.) External data can be cross-referenced with input data.

For example, we use OPA and Rego to automatically classify sensitive data within a sample dataset. Given some sample data key/value pairs, we can classify them based on the content of that data. The following example policy shows a policy that can classify an arbitrary key/value input data as ADDRESS if it meets some defined criteria (try on playground rego):

package classifier

# The input data (a key/value pair of strings) is classified as ADDRESS if...
classify(key, val) = "ADDRESS" {
   	 # Any of the following statements are true...
    	any(
   		 [
            	# The lowercase key is equal to "state"...
       		 lower(key) == "state",
            	# ...or, the lowercase key is equal to "zip"...
            	lower(key) == "zip",
            	# ...or, the lowercase key is equal to "zipcode"...
            	lower(key) == "zipcode",
            	# ...or, the lowercase key contains the word "address"...
            	re_match(`\A.*address.*\z`, lower(key)),
            	# ...or, the lowercase key contains the word "street"
         		 re_match(`\Astreet.*\z`, lower(key)),
        	]
    	)
} else = "UNLABELED" {
    	true
}

# Classify each input k/v pair, and put the results in the "output" variable.
# Note we're using a comprehension here: https://www.openpolicyagent.org/docs/latest/policy-language/#object-comprehensions
output := {k: v |
	v := classify(k, input[k])
}

Given some key/value pair input:

{
  "address": "123 N. Example St.",
  "foo": "bar",
  "state": "NY",
  "zip": 12345
}

The result of the policy can look like this:

{
  "output": {
    "address": "ADDRESS",
    "foo": "UNLABELED",
    "state": "ADDRESS",
    "zip": "ADDRESS"
  }
}

Also, because the output of OPA policies are just JSON documents, we can also use OPA policies to return arbitrary information in response to some input data. For example, we can define an OPA policy that returns something like a set of rules on how to act on some data. The rule set is just data that can contain instructions on how the app interacts with other data (eg deserialization logic, etc.). An application can query the policy, pass the data and any other metadata as input, and use the response to understand how to process the data later. This set of rules is defined directly in the policy, and the application querying the policy only knows the source of the data and the actual data itself. The policy can use this input data, as well as external data, to return a set of rules to the application that tells it how to process its data.

Here’s a simple example policy that returns a “rule set” on how to process database connection data, for a specific set of database types (try on playground rego):

package dbruleset
import data.params

# Given some connection data, infer a ruleset about how to process said data.
inferRuleset(conn) = {
  "appName": params.appName,
  "encoding": "base64",
  "charset": "utf-8",
  "serde": {
"type": "json",
"usernameField": "un",
"passwordField": "pw",
"hostField": "host",
"portField": "port"
  }
} {
  # Only return the above details if the db type is one of the following...
  applicableDbTypes := ["postgresql", "redshift", "snowflake"]
  conn.db.type == applicableDbTypes[_]
} else = {
  "status": "unknown",
}

# Return the ruleset inferred by the input
ruleset := inferRuleset(input)

These policies can then be verified in source code control and managed externally from the application code. In addition, external data (the params in the example above) can change as needed and sync with OPA dynamically.

While the example above may seem a bit contrived, it should still serve as an example of the flexible and powerful nature of OPA. In fact, we use an even more advanced form of this technique in Cyral to instruct applications how to extract various connection details for tools and database platforms at runtime. We can dynamically evolve these policies by adjusting them or the external data they reference to change runtime behavior without the need to recompile or update our application.

Resume

OPA and Rego provide a lightweight, declarative, data-driven, readable framework for dynamic policy evaluation and decision making across the technology stack. Compared to other policy engines, OPA offers clear advantages and benefits. In addition to traditional policy use cases such as API authorization, OPA’s flexible architecture facilitates decision making for a wide range of simple to complex systems. Its data-driven approach to policy evaluation allows the state of systems to dynamically evolve to impact the outcome of decisions, while the policies themselves can remain unchanged. At Cyral, we found that our unique requirements and use cases for policy evaluation, distributed across a mesh of microservices, required something different than what legacy policy engines could provide. We have found that OPA fits very well into our stack and resolves these issues consistently and reliably.

The charge How we use OPA at Cyral first appeared in Ciral.

*** This is a syndicated Security Bloggers Network blog from Blog Archive – Cyral written by Chris Field. Read the original post at: https://cyral.com/blog/how-we-use-opa-in-cyral/

Leave a Comment