Signals Tech #1: Breaking down the Signals Platform

We received so many requests for a walkthrough of Signals’ more technical aspects that we have launched a new series of articles which dives deep into the fundamentals. While all you see is our sleek web or mobile applications, there are many important things happening on the backend. So if you’re curious about what’s happening behind the scenes while you code your strategy, or after clicking the deploy button, this should provide some insight.

We will discuss what roles our microservices play in the Signals Platform, what data they handle and what kind of APIs they provide. We’re big advocates of the phrase, “a picture is worth a thousand words”, so we will start with a nice diagram displaying all of them together.

API, Routing and Authentication

As you can see, there are quite a few services used as the building blocks of our platform. They use protocol buffers with a messaging broker for lightning fast service-to-service asynchronous communication and GraphQL API as an interface for client facing applications. We use Apollo Server as a backend-for-frontend solution that merges multiple GraphQL APIs from our microservices into one big schema. This way, it provides a unified entry point for web and mobile applications accessing our backend services.

We have leveraged IdentityServer functionality to implement single sign-on, identity management, authorization, and API security. Together with Ocelot Gateway, it ensures that only authenticated requests are routed to our microservices endpoints.

Strategies and Signals

There are several services serving data and pushing real-time updates to the client applications via API. The main one is the Strategies Service, which handles strategies data, ensures proper strategies versioning and holds metadata about backtests and real-time strategies. It stores information about supported data streams, technical indicators, and integrated exchanges. It also consumes strategy logs and signals from the messaging broker, pushing them to GraphQL subscriptions, so that you can see strategy execution progress in real-time.

Another service which clients consume directly is Signals Service. Signals service persists signals released by strategies. It also updates and calculates all the strategy performance statistics within each received signal or market price update. These data are then provided to our frontend, so you can see how your strategy is performing.

Market Data

The last of the main services forming our GraphQL API is the MarketData Service, which persists, aggregates and provides all sorts of market data that boost our beautifully designed trading charts. The MarketData Service connects to our universal storage for market data. It doesn’t contain any exchange-specific logic and it receives and stores data in our standardized format. For that reason, we have implemented MarketDataConnector Services, which connect to concrete exchanges, apply data transformations and push them to message broker. The connector service instance consumes either websockets, or it can use API polling if no real-time updates are available on the given exchange. The connector service is also responsible for exchange’s connection monitoring, identification of gaps in the received market data and additional filling of those gaps.

This is how we deconstructed the functionality responsible for handling strategies data, signals, performance statistics and market data.

Signals Framework and Strategy Compilation

Last but not least, we are going to expose more details about the most important feature of our platform — strategy execution. All the services involved in execution depend on Signals Framework, which is our core trading library, containing definitions of methods and classes needed for implementing strategy code.

Strategies, as well as the Signals Framework, are implemented in C# language, which needs to be compiled into assemblies. For that reason, when a user clicks on the backtest or deploy button, the latest version of strategy code from the editor is serialized and sent to the Strategy Compilation Service. The Strategy Compilation Service deserializes and loads the strategy’s source code together with referenced libraries and produces compiled assembly. The assembly is then consumed by the Assembly Provider Service, which persists its bytes together with links to referenced assemblies. The Assembly Provider Service holds and also provides assemblies with the implementation of technical indicators, datastreams and Signals Framework itself.

All of these assemblies are used also within Language Server Service, which enhances our browser-based Monaco code editor with IntelliSense and other IDE-related features.

Strategy Executors

The final step in the strategy execution workflow is creating an isolated service which will execute the strategy. This part of the process is covered by the Executors Supervisor Service, which creates a new instance of service responsible for strategy execution of each individual backtest or strategy deployment. The Supervisor Service monitors all these instances, controls their health and manages their lifetime from start to end.

The service responsible for the actual strategy execution is called the Strategy Executor Service. As mentioned before, this service is instantiated per strategy execution and it implements interfaces from Signals Framework library. To sum it up, on startup it loads assembly for the defined strategy version and starts pushing data to the strategy. All the logs and signals produced by strategy code are pushed to the messaging broker and consequently to GraphQL subscriptions.

Based on the strategy execution mode, we have two derivations of the service — Backtest and Live Strategy Executor. The Backtest executor loads historical data, in contrast to Live Executor, which subscribes to real-time market data updates. The data are pushed to the strategy until the strategy reaches the end of backtest period, or is undeployed. The executor is then terminated by the Executors Supervisor Service.

As you can see, there are a lot of moving parts powering our beautiful frontend and strategy execution. It took months of sleepless nights to implement them properly, so we hope you enjoyed gaining some insight into our solution. We’ll have more articles about the technology behind Signals in the near future, so keep checking in with us or subscribe to our newsletter.