The goal of data classification is to ensure Loggers are only logging appropriate information, and not exposing sensitive information to 3rd parties such as analytics.

The solution that’s been adopted is to create privacy schemas which will annotate every plugin’s payload properties with a level of privacy. Below are the standard data classification level used at SKYRIN.

• PUBLIC : Public information, and can be openly shared with 3rd parties, discussed in public and with anyone. Public information as the name implies, is public, and does not require any additional controls when used. For example, AppName,Country,Region
• INTERNAL : This type is used when data includes information that is not openly shared with 3rd parties but only shared internally within SKYRIN. For example, policies, apiName
• RESTRICTED : This is used when data is confidential business or personal information. Unauthorised disclosure of this information could have a serious adverse impact on the application. For example, CustomerName, CustomerID
• HIGHLY_RESTRICTED : This case used to hide the highly sensitive information. This might be business or personal information that is required to be strictly protected. This type of data MUST be PROTECTED. For example, SessionToken, Encryption Key.

The schemas looks fairly similar to a standard message schema, see an example below.

{
  "filename": "TestPluginA.Receive.Query.ExampleQueryA.OutputPayload.PrivacySchema.json",
  "privacySchema": {
    "type": "object",
    "privacy": "PUBLIC",
    "properties": {
      "exampleQueryRequestA": {
        "type": "String",
        "privacy": "INTERNAL"
      }
    }
  },
  "mode": {}
}

if "mode" is {} will automatically deserialize to Merge Mode (more info below)*

• Each property and interns objects nested properties are described with the “privacy” property which will state the level of privacy of that property.

• A plugin should define these all in the assets/privacySchema folder to get picked up correctly upon initialisation, however no further work is required from the plugin. As a bonus if you are using Auto-generation these privacy schemas will be generated for you automatically if you use the appropriate annotations in your plugin’s messages object.

@Receive
object ExampleQueryA : PolicyMessage.Query() {
    data class RequestB(@Privacy(RESTRICTED) val exampleQueryRequestA: String?) : Request()
    data class ResponseB(@Privacy(INTERNAL) val exampleQueryRequestA: String?) : Response()
}

ResponseB is represented by the privacy schema above, as one can see simply adding a single annotation to it’s single property saves the labour of having to write all of the above. This is even more true for more complex structures.

• Loggers must set a value call “loggerLevel” to one of Public, Internal, Restricted and Highly Restricted and this will denote the highest privacy level of a property this logger is allowed to log.

• For example a Logger that only runs in debug can set it’s privacy level to Public to log the entire payload for development reasons.

• While an Analytics logger may only be allowed to log Highly Restricted values as these will be transmitted to a 3rd party which sensitive data should not be shared with.

• Loggers can also define privacy schemas for messages in the exact same fashion as plugins, the only difference is that Loggers are providing privacy schemas for messages which are owned by plugins.

• This is done to facilitate a Plugin changing the privacy level of a particular message.

• When one or more loggers defines a privacy schema for a message, a merge of the these schemas is required.

• By default the highest privacy level for each property is chosen as new privacy level that all loggers will require an equal or greater logger level to log that particular property.

• In other words the privacy level can only be increased with this default behaviour which ensures properties are only further restricted, and all loggers will receive the same merged schema when filtering is required of the payload before it is logged. Merging and filtering of the payload is all handled transparently under the hood so no Logger is required to implement these functions.

• As an alternative Loggers also have the option to define privacy schemas in an override mode which will allow the schema to change the privacy level of payload’s property for only that Logger.

• Schemas of this type do not get merged with any other privacy schema and cannot affect the logging of other plugins.

This override has two levels of privileges to denote its behaviour.

• Strengthen Only

  The first level is called Strengthen Only, this will enable the Logger to increase the privacy level of properties just for this particular plugin. This can be really useful if there are particular values that are needed to be omitted from a particular logger without affecting other Loggers. In this vein there is an additional privacy level called Omitted which will ensure that the property is omitted for the Logger regardless of loggerLevel (Highly Restricted being the highest).

• Strengthen and Weaken

  The second level is called Strengthen and Weaken and is a dangerous privilege which should ideally be tracked on the CI level. This level will allow a plugin to bypass any privacy level set by the plugin and weaken the privacy access for any property.

  For example a Logger can override the privacy level of a highly restricted property and make it public. While this should not be abused its very feasible to be a good solution to force certain properties to be logged if the Plugin has set to restricted privacy levels for the Loggers use case.

  The implementation for merging and filtering payloads is included by default in the MultiLogger. Simply extend the MultiLogger and pass in a list of loggers in the **MultiLoggers constructor **.

  Then each of the loggers past into the list of loggers in the MultiLogger will have its Log function called which will already have the payload filter according all the merged privacy schemas, and its overridden privacy schemas if they exists.

Payload Types In AutoGeneration

• There are various types of payloads for different message types such as Command, Query, Publish SubscribeOnStartup, and SubscribeOnDemand. Let's understand each one separately.

Command:

• The Command message type only has an InputPayload for sending and receiving data.

• For instance, here TestPluginA sending data via payload to TestPluginB then you can create object or data class in TestPluginA.

@Send("TestPluginB")
data class ShowBalanceForCommand(val exampleCommandA: String) : PolicyMessage.Command()

• As we are sending data to TestPluginB, file name will be TestPluginB.Send.Command.ShowBalanceForCommand.InputPayload.PrivacySchema.json

@Receive
data class ShowBalanceForCommand(val exampleCommandA: String) : PolicyMessage.Command()

• As we are receiving data in TestPluginB, we should use @Receive annotation and privacyFile name will be TestPluginB.Receive.Command.ShowBalanceForCommand.InputPayload.PrivacySchema

:::note

In Command messageType, for Send and Receive will have InputPayload only.

:::

Query:

• The Query message type has both an InputPayload and an OutputPayload.

• In the case of Auto-generation, the Query message type has two subtypes: Request and Response.

  Request: This is the payload that is sent by a consuming plugin and it serves as an InputPayload.

  Response: This is the payload that is received in response by the same consuming plugin and it serves as an OutputPayload.

• For instance, TestPluginA sending some data via payload to TestPluginB, then we should use Request() type which is an InputPayload.

@Send("TestPluginB")
object AutoGenQueryMessage : PolicyMessage.Query() {
data class ExampleQuery(@Privacy(Privacy.Privacy.INTERNAL) val queryMessage: String) : Request()
}

• Example of TestPluginA will create below privacySchemas :

• TestPluginB.Send.Query.AutoGenQueryMessage.InputPayload.PrivacySchema

• Explanation of above privacySchemas :

• The payload the TestPluginA sends query with the ExampleQuery messageName with property queryMessage to TestPluginB.

• Now in TestPluginB as we are receiving the data, we should use Response() which is considered as an OutputPayload.

@Receive
object AutoGenQueryMessage : PolicyMessage.Query() {
data class ExampleQuery(@Privacy(Privacy.Privacy.INTERNAL) val queryMessage: String) : Response()
}

• Example of TestPluginA will create below privacySchemas :
• TestPluginB.Receive.Query.AutoGenQueryMessage.OutputPayload.PrivacySchema
• Explanation of above privacySchemas :
• The payload that TestPluginB accepts when TestPluginA is sending its ExampleQuery with property queryMessage.

EventStream:

• The EventStream system functions by transmitting payloads among various plugins and managing both input and output payload types.

• The key elements of the EventStream system include:

• EventStreamPayload(): Serves as an InputPayload.

• InEvent() and OutEvent(): Operate as OutputPayloads.

• EventStream manages data sent as OutputPayload and data received as InputPayload. For instance, in the scenario involving TestPluginA and TestPluginB:

Within TestPluginA

@Subscribe
data class StopTransactions(val debitedAmount:String) : PolicyMessage.EventStream() {
data class StopTransactions(val transactionAmount:String): InEvent()
}

• Example of TestPluginA will create below privacySchemas :

1. TestPluginB.Subscribe.EventStream.StopTransactions.InputPayload.PrivacySchema.json
2. TestPluginB.Subscribe.EventStream.StopTransactions.OutputPayload.PrivacySchema

• Explanation of above privacySchemas :

1. This payload will be specified in eventStream of SubscribeOnDemand takes as an input payload - "payload: transactionAmount"
2. The payload the TestPluginA expects from the TestPluginB publisher emitting values for StopTransactions which contains property in payload - "debitedAmount".

Within TestPluginB

@Publish
data class StopTransactions(val transactionAmount: String) : PolicyMessage.EventStream() {
  data class StopTransactions(val debitedAmount:String): OutEvent()
}

• Example of TestPluginB will create below privacySchemas :

1. TestPluginB.Publish.EventStream.StopTransactions.InputPayload.PrivacySchema.json
2. TestPluginB.Publish.EventStream.StopTransactions.OutputPayload.PrivacySchema.json

• Explanation of above privacySchemas :

1. The payload that TestPluginB sends when emitting values via the publisher for the StopTransactions eventStream which contains property value transactionAmount .
2. The payload that TestPluginB accepts when emitting values via the publisher for the StopTransactions eventStream which contains property "debitedAmount".

Note : In SubscribeOnStartup,for Send and Receive will have OutputPayload only.

:::note

This payload will be specified as null since SubscribeOnStartup doesn't take any input payloads

:::

:::note

Inside asset folder,folder name should be privacySchemas and file name should be match with regex ["${pluginName}(\.\w+)*\.${messageName}\.${payloadType.name}.PrivacySchema.json]

For example, it should be like ExamplePlugin.Send.Command.CommandMessage.InputPayload.PrivacySchema.json

:::

Data classification of ActionPayload Filtration :

The actionPayload filtration in the LogInfo.InternalPluginAction block ensures that sensitive or unnecessary data is filtered out before being logged.

The payload is processed using the filterPayload method, which applies privacy and logging rules based on the messageName, pluginName, and the logger's configuration. Additionally, the actionPayload undergoes a separate filtration process using the filterLoggingPayload method of the logger, which further refines the data to meet specific logging requirements.

This dual-layered approach ensures that both the primary payload and the action-specific payload are appropriately filtered, maintaining data privacy and compliance with logging policies.

We have implemented this in the MultiLogger class, which is responsible for managing multiple loggers and ensuring that the payloads & actionPayloads are filtered according to the defined privacy levels with privacySchema and kotlin reflection approach.

Here is a link of actionPayload filtration example - sample-app

:::note

You don't need to include privacySchemas for ActionPayload filtration. However, for Payload filtration, it is necessary to include privacySchemas.

:::

Here is an example of data class to filtered the data with ActionPayload.

@Serializable
internal data class Account(
    @Privacy(PUBLIC)
    val accountName: String
    @PrivacyLevel(PrivacySchemaModel.Privacy.PUBLIC)
    val accountNumber: String
    val accountDescription: String
    @PrivacyLevel(PrivacySchemaModel.Privacy.HIGHLY_RESTRICTED)
    val balance: Double?
    @PrivacyLevel(PrivacySchemaModel.Privacy.INTERNAL)
    val isPrimary: Boolean = false,
    val isCredit: Boolean = false,
    @PrivacyLevel(PrivacySchemaModel.Privacy.RESTRICTED)
    val isError: Boolean = false,
    @PrivacyLevel(PrivacySchemaModel.Privacy.INTERNAL)
    val errorCode: ErrorCode? = ErrorCode.BALANCE_ERROR,
    @PrivacyLevel(PrivacySchemaModel.Privacy.PUBLIC)
    val user: User = User()
)

@Serializable
internal data class User(
    @PrivacyLevel(PrivacySchemaModel.Privacy.PUBLIC)
    val name: String
    @PrivacyLevel(PrivacySchemaModel.Privacy.RESTRICTED)
    val company: String
    @PrivacyLevel(PrivacySchemaModel.Privacy.PUBLIC)
    val employeeList: List<PersonalDetails> = listOf(PersonalDetails())
)

@Serializable
internal data class PersonalDetails(
    @PrivacyLevel(PrivacySchemaModel.Privacy.HIGHLY_RESTRICTED)
    val employeeName: String
    @PrivacyLevel(PrivacySchemaModel.Privacy.PUBLIC)
    val employeeID: String
)

ActionPayload filtration supports all data types including JSONObject, HashMap,List, nested objects, etc. This allows for flexible and comprehensive data handling, ensuring that all relevant information is captured while adhering to privacy standards.