PlatformHub Auto-generation

Introduction

PlatformHub Autogenerator exists as an extension of platform-hub responsible for generating plugin related source code.

Setup

To use this autogeneration library add the following dependency:

platformhubVersion = "2.2.1"
// ksp Autogen
implementation "com.skyrin.mobilebanking:platformhub-annotation:$platformhubVersion"
ksp "com.skyrin.mobilebanking:platformhub-autogenerator:$platformhubVersion"

Drawbacks of JSON communication

One of PlatformHub's greatest benefits is how applications can be broken down into several components called plugins, effectively creating an ecosystem of decoupled micro frontends. These plugins do not expose any of its classes or methods meaning plugins are not coupled to each other via dependencies. Plugins instead communicate using json encoded messages which are effectively just strings. This however raises the biggest drawback to such an approach, json encoded payloads are not typesafe In short anytime a message is sent and received over PlatformHub there is a potential for the json payload to be malformed. This will always lead to runtime errors, which will either result in the app crashing or acting unexpectedly. With 100s if not 1000s of messages being transmitted in a production level app, this will inevitably lead to some critical errors. An additional drawback is that its now non obvious what the API of each plugin is as there are no type definitions to refer to. Lastly the boilerplate required to pack and unpack json creates additional overhead when working with plugins and PlatformHub.

How Auto-generation solves this

It is possible to bring back type safety without having to share interfaces or class definitions like a traditional library. This can be done in the following three phases.

1) Introducing a typed structure to Policies

Here the four types of message that PlatformHub recognises are defined in a sealed class.

1. Command
2. NavigationQuery
3. Query
4. EventStream

This will ensure that message models are created to inherit from one of these four types, which prevents new message types from being arbitrarily created.

Notice how the Query has a Request and Response sub class, this is to distinguish between the payload that is being sent by a consuming plugin, and the payload that will be received in response by same consuming plugin.

EventStream also has an Event sub class, this is to distinguish between the initial EventStream payload sent via PlatformHub and the subsequent event payloads that are sent via flows.

This file is autogenerated and placed in a build folder where a Plugin can inherit from it.

abstract class AutoGeneratedPlugin(assetsLoader: AssetsLoader) : Plugin(assetsLoader) {
    override val policyFilename = "test-plugin-a-policy.g.json"

    open class Policy {

        sealed class PolicyMessage {

            sealed class Command : PolicyMessage()

            sealed class NavigationQuery : PolicyMessage()

            sealed class Query : PolicyMessage() {
                open class Request
                open class Response 
            }
            sealed class EventStream : PolicyMessage() {

                sealed class Event

                sealed class InEvent : Event()

                sealed class OutEvent : Event() 
            }
        }
    }
}

How plugins define policies and schemas of messages

TestPluginA is inheriting from the AutoGeneratedPlugin which contains the class definitions for the message types. Here a Messages object is created which takes the form of a policy file that is written in code. This file is taking advantage of custom annotations such as @PluginPolicy and @Receive which add extra meta data that will be used later.

For example the @Send("TestPluginB") is explaining that message NavigationQueryB is to be sent to TestPluginB. The @Receive and @Publish annotations don't have plugin names for parameters as it can be inferred that the plugin name is TestPluginA. This policy written in code contains all the information a typical JSON policy would contain with the added benefit for declaring the models for the messages also.

@PluginPolicy("test-plugin-a")
@Singleton
class TestPluginA @Inject constructor(assetsLoader: AssetsLoader) : AutoGeneratedPlugin(assetsLoader) {
            object Messages : Policy() {

                @Receive
                data class NavigationQueryA(val exampleNavigationMap: String) : NavigationQuery()

                @Send("TestPluginB")
                data class NavigationQueryB(val exampleNavigationQueryB: String) : NavigationQuery()

                @Receive
                data class ExampleCommandA(val exampleCommandA: String) : Command()

                @Send("TestPluginB")
                data class ExampleCommandB(val exampleCommandB: String) : Command()

                @Receive
                object ExampleQueryA : Query() {
                    data class RequestB(val exampleQueryRequestA: String) : Request()
                    data class ResponseB(val exampleQueryRequestA: String) : Response()
                }

                @Send("TestPluginB")
                object ExampleQueryB : Query() {
                    data class RequestB(val exampleQueryRequestB: String) : Request()
                    data class ResponseB(val exampleQueryRequestB: String) : Response()
                }

                @Subscribe("TestPluginB")
                data class ExampleEventStreamB(val exampleEventStreamStringB: String) : EventStream() {
                    data class ExampleEventB(val exampleEventString: String) : Event()
                }

                @Publish
                data class ExampleEventStreamA(val exampleEventStreamStringA: String) : EventStream() {
                    data class ExampleEventA(val exampleEventStringA: String) : Event()
                }

                @Publish
                data class ExampleEventStreamA2(val exampleEventStreamStringA: String) : EventStream() {
                    data class ExampleEventA(val exampleEventStringA: String, val string2: String) : Event()
                }
            }


    override fun init(context: Context, configurationFiles: Map<String, InputStream>) {
        TODO("Not yet implemented")
    }
}

:::note

In @PluginPolicy(), you must specify the module name you given to your module, If moduleName will be incorrect it will not generate policy.g.json file.

:::

2) Generate Sending and Receiving Bindings

Restricting what plugins can send

Restrict what a plugin is capable of sending over PlatformHub. Meaning a plugin must only send messages that are first in its policy and secondly the structure to each type of message should be consistent. For example PluginA, sends a message called MessageA with the following json payload to PluginB.

{
    "exampleMessageA": "This is an example of a message",
    "exampleMessageB": "This is another example of a message",
}

If sometime in the future PluginA sends another MessageA with the following json payload to PluginB there will be problems.

{
    "exampleMessageAA": [1,2,3,4]
}

To begin with the property name has changed from exampleMessageA to exampleMessageAA, which will cause a crash in PluginB. Even if PluginB coded defensively and handled the potential crash and then somehow accessed exampleMessageAA, the type is now in a format it didn't expect also. Coding defensively is not only time consuming but also error prone. Instead we want a way to ensure the plugin can only send a correct MessageA payload. This can be done by forcing the plugin to create a model of MessageA, and only let that Plugin to send a json representation of that model over PlatformHub.

data class MessageA(val exampleMessageA: String, val exampleMessageB: String)

If PluginA creates a model for every message that it is planning to send then we can read these models and auto generate bindings that will convert these models back to json to be sent over PlatformHub. This will effectively limit what a plugin can send over PlatformHub to only the json representation of the models it has defined for its messages.

Restricting what plugins can receive

In the same way we can restrict what plugins can send, we can also restrict what plugins can receive using the same trick. If PluginB was expecting to receive MessageA in the following json format, this would be perfectly fine.

{
    "exampleMessageA": "This is an example of a message",
}

While this is not identical to what PluginA may send, it most importantly contains the field of interest that will be accessed. If PluginB always has the expectation to receive the payload in this format it can define a model as follows.

data class MessageA(val exampleMessageA: String)

Note how this model does not include exampleMessageB, and it doesn't need to if that property is not of interest. Then bindings can be auto generated to receive the json and convert it into our MessageA model ready for consumption.

An example of Plugin by extending AutoGeneratedPlugin would look like after reading the messages and annotations from the TestPluginA class at compile time. A json policy is generated and automatically bound to, also all the methods for sending and receiving messages have been created. This means the developer can now only use these methods to send messages over PlatformHub. They can no longer send any random JSON. NOTE: Much of the AutoGeneratedPlugin's implementation has been omitted as they are implementation details.

@Suppress("MaxLineLength")
@PluginPolicy("test-plugin-a")
@Singleton
class TestPluginA @Inject constructor(assetsLoader: AssetsLoader) :
    AutoGeneratedPlugin(assetsLoader) {
    
    private lateinit var context: Context

    object Messages : Policy() {

        @Send("TestPluginB")
        object ShowBalanceForCommand : PolicyMessage.Command()

        @Receive
        object ShowFragmentAutoGenEvent : PolicyMessage.NavigationQuery()

        @Send("TestPluginB")
        object ShowTestPluginB : PolicyMessage.NavigationQuery()

        @Receive
        object ShowAutoGenFragmentQuery : PolicyMessage.NavigationQuery()

        @Receive
        object ShowFragmentWithEvent : PolicyMessage.NavigationQuery()

        @Receive
        object ShowAutoGenFragmentCommand : PolicyMessage.NavigationQuery()

        @Receive
        object ShowAutoGenFragmentEventOnDemand : PolicyMessage.NavigationQuery()

        @Send("TestPluginB")
        object AutoGenQueryMessage : PolicyMessage.Query()

        @Subscribe("TestPluginB")
        object StopTransactions : PolicyMessage.EventStream()

        @SubscribeOnStartup("TestPluginB")
        object TestPluginaCommandResponse : PolicyMessage.EventStream()

        @Publish
        object TestPluginaEventStream : PolicyMessage.EventStream()
    }

    override fun init(
        context: Context,
        configurationFiles: Map<String, InputStream>
    ) {
        this.context = context
    }

    override fun subscribeOnStartup(
        subscription: SharedFlow<EventStreamOutput>,
        eventStream: BaseMessage.EventStream) {}

    override fun publish(eventStream: BaseMessage.EventStream): SharedFlow<EventStreamOutput> {}
}

Data Classification Example with AutogeneratedPlugin

Here we have created ExampleQueryA and extended PolicyMessage.Query() to object class. You can create data classes which will be considered as a properties in privacy schemas and to set privacy level for the properties, you can use @Privacy() annotation to set privacy level.

object ExampleQueryA : PolicyMessage.Query() {
            data class RequestB(@Privacy(RESTRICTED) val exampleQueryRequestA: String?) : Request()
            data class ResponseB(@Privacy(INTERNAL) val exampleQueryRequestA: String?) : Response()
        }

When the project is rebuilt, it will generate privacySchemas and messages with properties in the privacySchema.json file. To check data classification working, please click here.

3) Cross Validate Models in Host App

Cross Validating that Sending and Receiving messages are compatible

Finally now we have concrete definitions of what each plugin is expecting to send and receive, a 3rd party can run a check to ensure that these expectations are compatible.

Even though models are not being shared, at compile time a 3rd party script will have access to read the files that contain the models for each plugin in an application. If all models are compatible then the check will pass. If there is an incompatibility then the build will fail and this will prevent the developer from deploying code that will surely result in a runtime MalformedJsonException.

In affect even though we are legitimately only sending json over PlatformHub, due to our ability to constrain and then validate what is being sent and received, we can ensure we have compile time type safety

How AutoGen Works?

Auto-generation uses Kotlin Symbol Processing (KSP) library and hooks compile time annotations and generate relevant policy files and plugins. Unlikely manual plugin generation, developers are able to use concrete classes by defining payload types and library generates code to translate these definitions into json payload serializations.

Here are the steps how auto-generation handles annotation processing step-by-step:

1) Extracting module information

Auto-generation library first looks for a set of annotations to see if any AutoGeneratedPlugin definition and mandatory properties exist. If it is the case, module info is added into checkpoint.json files for next tasks. checkpoint.json contains module information.

After rebuild,to clear checkpoint.json, please add below code in build.gradle of your plugin.

tasks.register("resetCheckout") {
    gradle.allprojects {
        if (it != rootProject) {
            def checkpointFile = it.layout.projectDirectory.file("checkpoint.json").getAsFile()
            if (checkpointFile.exists()) {
                checkpointFile.write("{}")
                println("reseting")
                println(it.layout.projectDirectory)
            }
        }
    }
}

2) Generating a AutoGeneratedPlugin from module info

Once module info exits, next task is to generate plugin code. This task creates a plugin file snippet. Snippet contains policy class definition as well as concrete model classes for high-level programming abstraction. As mentioned above, this allows skipping json payload serialisation and directly working with data/model classes.

3) Modifying generated plugin class snippet

This step is not mandatory and is to modify existing file and to refactor current plugin. It can make partial updates by adding and removing class lines.

4 ) Message Generation

This step basically generates entire structure tree which represent all messages, all payload information and all the annotations. This representation is similar to schemas however structure is completely irrelevant.

5 ) Generating policy from artifact

As we have pass through all previous steps, now auto-generation library is ready to build plugin policy by using generated artifact info. This step simply builds a PolicyModel from the info generated in previous steps.

6 ) Generating policy schema from artifact

Generating schema from artifact is similar process as generating policy from artifact. As we have all information available, library build a schema model and exports it as schema. Schema model is built using recursion by visiting complex types, until all primitive types are visited, and then kotlin types are converted to schema types accordingly. Schema, PrivacySchema generation comes free when auto-gen is adopted by a plugin.

7 ) Generating source files

This step renders generated module info into a file.

For further in-depth details of how auto-gen library works, please see the video.

Also visit com.skyrin.mobilebanking.platformhub.autogenerator.AutoGenerator for implementation details.

Here is a autogen sample app which can serve as a reference to understand the functionality of auto-generation.

Final Considerations

During initial development Auto generation and Schema validation were initially coupled and designed as a single complete process to ensure type safety over PlatformHub. Now Auto Generation and Schema Validation should be treated as two distinct processes that when brought together create type safety in PlatformHub, but can be ran and used independently. The Benefits of Auto Generation are the generated bindings, redux and ui code that significantly reduces a developers work load. While the Benefits of Schema Validation is exposing an API for plugins that document the expected structure of messages they will send and receive, and making sure these expectations are consistent across plugins.