Skip to content

Config

Introduction

The config module is used to configure PSEngine and control integration behavior. It does not interact with any Recorded Future datasets, and its use is optional, primarily serving as a convenience when a configuration file is needed.

Configuration values can be provided in several ways:

  • Supported file formats: .toml, .json, or .env
  • Environment variables
  • Directly via parameters to the init method

When loading configuration, PSEngine follows a strict priority:

  1. Values passed directly to the init method
  2. Values from environment variables
  3. Values from configuration files

If an environment variable is set, it will override the corresponding value in the config file.

The Config class is a singleton, meaning its values are immutable once initialized and accessible from any module. You initialize the configuration using the init method, and retrieve its data with the get_config method.

By default, the Config class manages a ConfigModel, which is a pydantic.BaseSettings class containing common attributes such as proxy settings and HTTP timeout.

The variables pre-defined by the ConfigModel are:

  • platform_id -> str
  • app_id -> str
  • rf_token -> RFToken
  • http_proxy -> str
  • https_proxy -> str
  • client_ssl_verify -> bool
  • client_basic_auth -> (str, str)
  • client_cert -> str or (str, str)
  • client_timeout -> int
  • client_retries -> int
  • client_backoff_factor -> int
  • client_status_forcelist -> list of int
  • client_pool_max_size -> int

Warning

Define the Config before initializing the manager in your integration entry point. Once that is done, you can reference the Config from anywhere. See the example below.

See the API Reference for internal details of the module.

Examples

Warning

The following examples demonstrate how to use this module. Be sure to add appropriate error handling as needed; all possible errors for each method or function are listed in the API Reference page.

Additionally, you must configure the RF_TOKEN environment variable before getting started. For instructions, see Learn.

1: Read a Config from config.toml

To run this example, create a config.toml file with the following content:

1
my_value=5

Initialize the Config object with init, passing the path (absolute or relative) of the config file you intend to use. This creates an object but does not return it.

Since you want to print the value of my_value, use the get_config method to return the ConfigModel instance.

1
2
3
4
5
6
7
8
9
from pathlib import Path

from psengine.config import Config, get_config

CONFIG_PATH = Path(__file__).parent / 'config.toml'

Config.init(config_path=CONFIG_PATH)
config = get_config()
print(config.my_value)

This will print 5.

2: Configure a Config from environment variables

You can read only environment variables that are statically defined in the ConfigModel. They need to be prefixed with RF_ and must be of the type specified in the model as described in the Introduction section. For example, to set the app_id and platform_id variables:

1
2
export RF_APP_ID=example/1.0.0
export RF_PLATFORM_ID=Splunk/10.0.0

Then read the config:

1
2
3
4
5
6
from psengine.config import Config, get_config

Config.init()
config = get_config()
print(config.app_id)
print(config.platform_id)

The sample code will print the values defined above.

3: Configure a Config from Python

You can initialize your config from the init method directly:

1
2
3
4
5
from psengine.config import Config, get_config

Config.init(my_value=5)
config = get_config()
print(config.my_value)

This will print 5.

4: Define your own config

If you want to define your own config in an integration, you can. The steps are:

  1. Define your model (IntegrationModel in the example). It has to inherit from psengine.ConfigModel.
  2. Change Config.init to use the config_class and assign it to the IntegrationConfig model you just created.
  3. Use the config_class with IntegrationConfig in the get_config function as well.
  4. Keep doing everything else as usual.

To replicate this example, first create a custom_config.toml file with the following content:

1
2
3
4
5
simple_value=5

[complex_value]
data = [ "a", "list" ]
value = [ 1, 2, 3]

Place this in the same directory as the example Python code. Once the file configuration is created, the sample code will create the custom config in the IntegrationConfig class. Then call the init method, passing the custom configuration model as config_class and the usual TOML path.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
from pathlib import Path

from psengine.config import Config, ConfigModel, get_config
from pydantic import BaseModel

CONFIG_PATH = Path(__file__).parent / 'custom_config.toml'


class ComplexValue(BaseModel):
    """Model to define the `complex_value` table."""

    data: list[str]
    value: list[int]


class IntegrationConfig(ConfigModel):
    """The class of my integration config."""

    simple_value: int
    complex_value: ComplexValue


Config.init(
    config_class=IntegrationConfig, config_path=CONFIG_PATH
)
config = get_config()

print(config)
print(config.simple_value)
print(config.complex_value.data)
print(config.complex_value.value)

Each property can be accessed using dot notation, for example, config.complex_value.data.

5: Real example

Assume you are developing an integration that needs to fetch playbook alerts. The current requirements for the alerts to be ingested are:

  • Domain Abuse
  • New status
  • High priority
  • No older than yesterday

Each domain that triggered this alert has to be enriched with the links field.

You can opt for a quick script using free variables around the code or use the config.

Code 1 without the config:

In this example, you hardcode the values that you need for fetching the alert and enriching the IOCs. This is perfectly fine; however, in larger applications, it might be challenging to maintain if the requirements change.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from psengine.enrich import LookupMgr
from psengine.playbook_alerts import PlaybookAlertMgr
from psengine.playbook_alerts.pa_category import PACategory

pba_mgr = PlaybookAlertMgr()
enrich_mgr = LookupMgr()

alerts = pba_mgr.fetch_bulk(
    category=PACategory.DOMAIN_ABUSE,
    statuses=['New'],
    priority='High',
    created_from='-1d',
)

domains = [
    alert.panel_status.entity_name for alert in alerts
]
enriched_domains = enrich_mgr.lookup_bulk(
    domains, 'domain', fields=['links']
)

for enriched in enriched_domains:
    print(enriched)

An alternative is to save the requirements to a config file and use them instead of the hardcoded values.

Code 2 with the config:

With the int_config.toml file:

1
2
3
4
5
6
7
8
[pba]
category="domain_abuse"
statuses=["New"]
priority="High"
lookback="-1d"

[enrich]
fields=["links"]

The script can be rewritten as below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
from pathlib import Path

from psengine.config import Config, ConfigModel, get_config
from psengine.enrich import LookupMgr
from psengine.playbook_alerts import PlaybookAlertMgr
from pydantic import BaseModel

CONFIG_PATH = Path(__file__).parent / 'int_config.toml'


class PBAConfig(BaseModel):
    """Config for playbook alerts."""

    category: str
    statuses: list[str]
    priority: str
    lookback: str


class EnrichConfig(BaseModel):
    """Config for IOC enrichment."""

    fields: list[str]


class IntegrationConfig(ConfigModel):
    """The main integration config."""

    pba: PBAConfig
    enrich: EnrichConfig


Config.init(
    config_class=IntegrationConfig, config_path=CONFIG_PATH
)
config = get_config()

pba_mgr = PlaybookAlertMgr()
enrich_mgr = LookupMgr()

alerts = pba_mgr.fetch_bulk(
    category=config.pba.category,
    statuses=config.pba.statuses,
    priority=config.pba.priority,
    created_from=config.pba.lookback,
)

domains = [
    alert.panel_status.entity_name for alert in alerts
]
enriched_domains = enrich_mgr.lookup_bulk(
    domains, 'domain', fields=config.enrich.fields
)

for enriched in enriched_domains:
    print(enriched)

The code itself is longer; however, you gain maintainability since a person without development experience or inner understanding of the application can change the config to meet new requirements.

6: Using a proxy

In this example, a proxy is configured for the LookupMgr to use when connecting to the Internet. While client_ssl_verify is optional, it is included here to allow the example to work with a proxy that does not have a certificate.

As with previous examples, you first set up the Config, then initialize the manager. The https_proxy argument specifies the proxy URL, and the manager automatically uses this configuration during initialization.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
from psengine.config import Config
from psengine.enrich import LookupMgr

Config.init(
    https_proxy='https://localhost:8080',
    client_ssl_verify=False,
)
mgr = LookupMgr()

data = mgr.lookup('8.8.8.8', 'ip')
print(data)