Journey Platform: A low-code instrument for creating interactive person workflows | by Arjun Raman | The Airbnb Tech Weblog | Might, 2023

Journey Platform: Low-code notification workflow platform that permits technical and non-technical customers to create complicated workflows by means of a easy drag and drop person interface.

By: Arjun Raman, Ken Snyder, Mengting Li

Efficient communication hinges on delivering the best message, to the best viewers, on the proper time. At Airbnb, our purpose is to have interaction our customers — each friends and hosts — by delivering inspirational and informational notifications by means of numerous channels, akin to electronic mail or in-app messages.

Traditionally at Airbnb, complicated notification workflows have been solely managed by engineering groups, with every workflow requiring the deployment of code. As our platform developed, we acknowledged the necessity for a low-code or no-code resolution to streamline the creation of those intricate notification workflows. In response, the Advertising and marketing Know-how crew developed the Journey Platform, a strong instrument that permits non-technical customers to construct and ship customized notifications primarily based on our customers’ engagement with Airbnb.

The targets of the Journey Platform are:

1. Empower customers to simply create event-driven notification workflows utilizing an intuitive drag and drop interface.
2. Allow real-time execution of those notification workflows for well timed and related communication.
3. Supply a unified interface for managing transaction notifications, akin to upcoming journey reminders and promotional notifications.
4. Assure Service Degree Agreements (SLAs) for processing numerous kinds of notification workflows, together with transactional and promotional communications.
5. Cut back the time required to develop complicated notification workflows.

Journey Platform permits customers to iterate quicker by permitting self-serve workflow creation. It has lowered the time taken to help a brand new use-case from 1–2 months to simply 1–2 weeks.

The important thing elements of the Journey Platform are:

1. Journey Platform UI: WYSIWYG instrument permits customers to tug and drop elements and create a workflow. The workflow definition is then transformed to a customized DSL (Domain-specific language) which will be interpreted and executed by the workflow orchestrator.
2. Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
3. Platform Retailer:

• Occasion Retailer: Pre-configured catalog of Kafka occasions which Journey Platform can hearken to and set off new executions of a workflow or go occasions to current workflow execution.
• Motion retailer: Repository of predefined, specific-purpose capabilities permits customers to carry out numerous duties, akin to sending emails, push notifications, or emitting Kafka occasions. Customized actions will be outlined and built-in into the instrument, making them accessible to all Journey Platform customers.
• Attribute retailer: Central repository for important knowledge, akin to person metadata (e.g. person’s geolocation, Airbnb search historical past, and so forth.) and contextual info. It helps decision-making in workflow branching processes by exposing these knowledge as a parameter to set situations upon by means of the parameter supervisor.
• Customized shops: Means to create customized motion or attribute shops which aren’t already outlined within the platform.
• Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.

When crafting the UI for the workflow automation system, we aimed to create a well-known and intuitive expertise. Drawing inspiration from movement charts, productiveness instruments with “inspector panels,” and incorporating drag and drop performance, we needed a platform the place customers might begin instantly with out consulting the guide.

We additionally had a purpose of utilizing progressive disclosure to incrementally allow the complete depth of the platform capabilities, whereas maintaining it easy for customers who solely want a small subset of the options. Through the use of wise defaults, and shifting extra complicated options into tabs and sub-screens, our superior customers might create distinctive options, going past the pre-planned use instances.

To edit the graph, we leveraged React Flow, an open-source library. This enabled us to show the graph, in addition to present primary operations like zooming, panning, shifting, and connecting nodes. On prime of this basis, we added our customized node and edge elements, together with drag and drop performance for including new nodes and an inspector panel for modifying current ones.

To create the types displayed within the inspector panel, we carried out a schema-based type system. This method gives a excessive degree of flexibility, permitting us to declaratively specify the UI for particular node enter/output fields as a part of their sort definitions. The system is inbuilt a type-safe method, making use of Thrift annotations and Java reflection. Primarily based on the schema info and UI-specific annotations, the interface shows the suitable type fields, assist textual content, and validation, guaranteeing our UI is mechanically up-to-date with the platform’s capabilities.

Area-specific language

DSL gives a excessive diploma of flexibility and customization, permitting us to outline the construction and habits of the workflow. As an alternative of getting to hardcode a workflow within the workflow engine, we as an alternative have a generic workflow outlined that may execute any DSL-based workflow. Nodes and edges make up a workflow, with nodes representing particular person actions or duties and edges defining the dependencies and relationships between them.

The nodes and edges embrace all the required info to outline a workflow akin to inputs, outputs, and parameters handed between nodes. The DSL generated by the UI is handed to the workflow orchestrator, the place the DSL parser executes it.

Journey Shops

The occasions, attributes, and actions shops are an integral a part of the backend, as they permit listening to occasions to start out workflow executions, filter customers, and execute duties within the journey. All these elements work collectively seamlessly to create a versatile and customizable backend that may be tailor-made to the particular wants of the platform.

Occasion Retailer

Journey Platform helps listening to completely different Kafka occasions and utilizing them to set off new executions of a workflow, or use the occasion to go indicators to a operating execution. For instance, begin a brand new execution when a visitor books a keep, go a sign to a operating execution when a person receives a push notification, and so forth. Much like the motion retailer, as soon as an occasion is on-boarded, all of the groups at Airbnb can use it.

Attribute Retailer

The attribute retailer capabilities as a central repository for fetching all crucial knowledge, akin to contextual knowledge, person preferences, and system info, which can be utilized to counterpoint the workflow branching course of and enhance decision-making capabilities. These shops are supported by a knowledge storage system that manages numerous attributes or traits of entities.

Think about you’ve a brand new person who simply signed up for Airbnb, and also you’re considering figuring out whether or not they’ve performed any itemizing searches on the platform. If the reply is sure, you’ll ship a customized message primarily based on their search historical past, and if it’s no, you’ll ship a static message.

It is a concrete instance of how the Airbnb Journey Platform leverages attributes, akin to “itemizing search historical past,” to reinforce the person expertise. These attributes are extracted and outlined as parameters, which can be utilized for numerous functions. Every workflow execution has its personal parameter knowledge assortment, which will be accessed within the parameter supervisor. Extra details about parameters shall be mentioned within the parameter supervisor.

Motion Retailer

The motion retailer is used to execute numerous duties, akin to sending an electronic mail or updating a database report, when a person reaches a selected level within the journey. It’s a frequent library the place every perform will be shared and reused by completely different customers of their workflow.

Every motion implements a typical interface, together with its metadata required for the UI schema-based types talked about above, and its habits through the precise workflow execution.

Parameter Supervisor

Managing a posh workflow that includes a number of steps with various inputs and outputs generally is a difficult process, particularly if the enter and output parameters change steadily or are completely different for every person. For example, you may want conditional branching in your workflow or customized communication content material primarily based on person search. That is the place parameterized workflows and parameter managers can show to be invaluable elements.

By specifying inputs and outputs (of attribute node / occasion node / customized node) as parameters, you’ll be able to reuse them all through your complete workflow execution. A parameter supervisor is a vital element that may retailer and handle your workflow parameters, streamlining the method of making, storing, retrieving, and modifying them.

Along with offering an environment friendly parameter administration system, a parameter supervisor additionally gives a spread of options akin to parameter creation, storage, retrieval, modification, versioning, entry management, and auditing. These options make sure that your workflow is executed reliably and constantly whereas additionally correctly managing and storing your parameters all through your complete workflow.

The Workflow Orchestrator executes workflows by deciphering the which means of every DSL node and performing the corresponding actions. It manages low-level capabilities akin to storing state, interacting with the motion retailer to carry out an motion, listening for callbacks by means of the occasion retailer, and permitting builders to focus on workflow logic quite than technical particulars. Journey Platform makes use of Temporal because the underlying workflow engine for state upkeep and orchestration. Temporal helps orchestrate workflows by means of Temporal Workers.

Builders can incorporate customized performance akin to new nodes or edges to broaden platform capabilities, making it less complicated to create workflows that fulfill the platform’s and customers’ distinctive necessities. Moreover, it helps superior options like parallel execution and automated retries and enhancing platform reliability and efficiency.

Guaranteeing SLA for processing several types of workflows (i.e. transactional and promotional) is vital at scale. Transactional notifications initiated by person motion (e.g. reserving affirmation, visitor/Host messaging, and so forth.) have a strict SLA and require greater precedence when in comparison with promotional notifications. To realize this, now we have carried out the next at completely different components of the system:

Occasion pre-processing:

• Pre-filter: As an alternative of passing all of the occasions on to the Workflow Handler, the occasion processor filters out occasions that don’t match the factors. e.g. solely go if the occasion sort is reservation_complete and filter out for all different reservation occasions. This drastically reduces the QPS feeding into the system.
• Mixture excessive QPS occasions: Occasions like searches have a excessive QPS. As an alternative of immediately processing, we batch and combination them over a time window. This reduces the QPS by a minimum of a number of orders of magnitude.

Devoted lanes:

• We’ve devoted lanes for various classes of workflows by means of the system. The occasion listener has completely different client teams with built-in throttling. The workflow handler has devoted Temporal namespaces for every class and strict limits on the processing QPS, max QPS to the database, and so forth.

The Journey Platform empowers non-technical and technical customers to create complicated stateful workflows by means of a easy drag and drop interface. By leveraging a generic workflow definition DSL, together with motion retailer, occasion retailer, and attribute retailer, the platform facilitates the creation of workflows that reply to real-time occasions, streamlining communication, and enhancing person experiences.

Excited by working at Airbnb? Take a look at these open roles.

Due to Balaji Kalaimani, Davis Wamola, Iris Feng, Jesse Garrison, John Bernardo, Kumar Arjunan, Michael Endelman, Priyank Singhal, Steve Krulewitz, Tej Sudha, Victoria Gryn, Xin Tu, and Zhentao Solar for his or her contributions in constructing Journey Platform.

Due to Sagar Naik and Michael Kinoti for his or her management and supporting us on this Journey.

All product names, logos, and types are property of their respective homeowners. All firm, product and repair names used on this web site are for identification functions solely. Use of those names, logos, and types doesn’t suggest endorsement.