Building the CartaX ATS with MarketGrid technology

Building the CartaX ATS with MarketGrid technology

Author: The Carta Team
Read time:  7 minutes
Published date:  February 18, 2022
With the strategic acquisition of MarketGrid Systems, Carta Capital Markets began building the platform that would become CartaX.

Private markets are breaking records. There’s more “dry powder”—money ready and waiting to be invested—in venture capital and private equity than there ever has been before. As this capital accumulates, investor demand for private company stock continues to rise. In 2019, Carta created a new subsidiary called Carta Capital Markets, LLC (CCMX) to facilitate trading in private company shares.

Unlike the public stock market, the market for private stock is limited to people recognized as “accredited investors,” who typically have more investing experience. This restriction is meant as a protection, because private companies aren’t subject to the same reporting requirements as publicly traded companies.

But it also poses a problem: Unless private companies organize a secondary liquidity event, it’s difficult for employees to sell. Carta tasked CCMX with designing a way to increase liquidity in private markets. The goal was to design new transaction types that companies at various stages of development could use to hold secondary liquidity events, which help employees and early investors unlock the value of the equity they hold.

Enter MarketGrid Systems

In February 2020, Carta acquired MarketGrid Systems Pty Limited, a small and innovative company based in Sydney, Australia. MarketGrid specialized in building exchange trading systems (ETS), which are platforms for trading specific commodities or equities. These systems trade a large range of products, including equities, bills, bonds, currencies, commodities, futures, and options. In the U.S., private equities can trade on a subset of ETS known as Alternative Trading Systems (ATS). MarketGrid Systems’s three founders collectively hold more than 100 years of experience in building trading systems for financial exchanges all over the world. The company’s core trading system product, MarketGrid, became the foundation for the CartaX ATS.

What is MarketGrid?

MarketGrid is a third-generation product that improved upon first- and second-generation trading platforms built and operated in the 1980s and 1990s. To provide new and enhanced capabilities, the developer team included new features to improve performance through lower latency and higher reliability.

MarketGrid Systems was in the process of deploying MarketGrid into new markets—including carbon credits, diamond exchanges, and cryptocurrencies—when it caught the attention of the leadership at CCMX. MarketGrid’s unique third-generation features made it the ideal base for the sort of innovative standalone platform that CCMX wanted to develop for trading equity in private companies.

MarketTech 1

MarketGrid components

MarketGrid was able to offer CCMX the foundational software infrastructure required to operate an ATS. This infrastructure includes a number of components, processes, and technologies:

  • Core Matching Engine (C++)

  • Trading and admin GUI and backend (Node.js, TypeScript, C++)

  • An internal communications grid (ZeroMQ)

  • A native FIX server (C++)

  • Independent market database (real-time and historical) (kdb+/Q)

  • Tools for developers

  • Tools for operations

  • A number of APIs and access methods, including:

    • Native MarketGrid API (MGAPI) over ZeroMQ

    • Websockets

    • REST

    • FIX

    • JSON over Kafka

    • Native MarketGrid API (MGAPI) over ZeroMQ

    • Websockets

    • REST

    • FIX

    • JSON over Kafka

From a technical perspective, the MarketGrid components and processes are illustrated below.

Market Tech 5

Matching engine

The core functionality of any ATS is the matching engine, which handles incoming orders, order books, and trade generation. A key feature of MarketGrid is that it does not use a database for this core functionality. Instead, MarketGrid is built around a set of custom internally shared-memory structures that are designed to provide the high performance and the low latency required for financial markets.

The internally shared-memory structures, as well as the code associated with managing and accessing them, are generated automatically during the build process and are not handwritten.

Fully-logged event-driven architecture

MarketGrid employs an event-driven architecture with features that provide high performance and a high level of resilience against all types of failures, thereby reducing the chances of lost orders, inaccurate trades, system outages and delays.

Architectural features:

  • The internally shared-memory structures that comprise the matching engine may only be updated or written by incoming events.

  • The design of the internally shared-memory structures allows for lockless read access by processes other than the matching engine (such as the UI server component).

  • All incoming events to the MarketGrid matching engine come via one of the MarketGrid APIs and are sequenced by a logging thread.

  • Incoming transactions are logged to an immutable transaction log file that allows for complete replay of the system to any previous state.

  • MarketGrid provides tools to examine internally shared-memory and transaction log files in real time.

  • The logging thread also handles replication of transactions to remote instances of the matching engine. This provides resilience through both hot and warm backup nodes.

  • MarketGrid provides checkpointing using cache files that are special cases of transaction log files. This facilitates persistence, garbage collection, and the fast restarting of the system.

Market Tech 2

Code generation

When we originally designed MarketGrid, one of our goals was to employ code generation to have as much of the system generated as part of the build process as possible. This was particularly important for infrastructure-related code, which refers to all code that is not directly related to a particular piece of business logic or functionality, such as database access, internally shared-memory handling, or API and message handling.

Infrastructure code is the foundation on which business logic is written and implemented: It is the base on which the rest of the platform is built. But while the foundation of a building is a more or less permanent structure, developers can update infrastructure code to implement new features.

A few key reasons motivated our goal of maximizing the generation of infrastructure code in the build process:

  • Many years of experience working in financial markets and building automated trading systems taught us that over time, large, complex systems like trading systems become increasingly difficult to maintain and enhance, and increasingly less reliable. Much of this is due to increased complexity and inconsistencies in the infrastructure-related code on which the system is built. These are introduced as the system is enhanced and changed to meet changing business requirements.

  • Since MarketGrid was designed as a system that can be deployed to a number of different customers—such as financial exchanges, banks, brokers—a key requirement was that it be easy to customize, maintain, and deploy different instances of the system for different customers and requirements. Employing code-generation for core infrastructure makes it much easier to maintain multiple instances of the system than would otherwise be the case.

  • By nature, infrastructure code tends to be highly repetitive and predictable, which makes it easy to generate and tedious to write by hand. Generating this code automatically as part of the build process allows developers to concentrate on delivering business functionality, which increases productivity and reduces the surface area for bugs.

  • MarketGrid was originally developed by a very small team of three developers. Our goal was to allow the company to grow while keeping the team small and efficient. By building a custom code-generation tool-set first, the developers were able to do this work once, then reap the benefits of ongoing, high-quality, and highly consistent generated infrastructure code on which the business logic could be built and maintained by a very small team.

Market Tech 3

How code generation works

Code generation in MarketGrid is based on taking a core data model, API definitions, and some other associated data (collectively known as the “data model”), and then using those to generate the core infrastructure code for all components of the system.

MarketGrid’s developers designed a bespoke code generation framework to build all generated code. Source for the data model is written in the TypeScript language, which provides all of the benefits of a strongly typed language, such as early bug detection, when editing and updating the data model.

When the system requires a change—for example, a new field to a table for a specific business requirement—the data model allows the change to be made once, in one place. Then, when the trading system is built, the change will be propagated to all components where it is required. This means that developers won’t have to implement the change in multiple places manually.

As illustrated in the following diagram, the data model generates the shared memory model, core C++ code, the shared memory interfaces, the UI back end (shown below as “nq”), persistence, migration, load and unload code, tools (such as internally shared memory, log file inspectors, and other inspection tools), a test framework, and documentation.

Market Tech 4

Data model example: sector table

The following code snippet shows how a table (in this case, the sector table) is defined in the MarketGrid data model. It is represented as TypeScript code.

Market Tech 7

Note that the definition of the fields in the table (in this case ShortName, Name, and Industry) define both a type and a number of decorators (of which there are many), such as .required (which is used to generate code that enforces a value for a required field) and comment. Comments are used during code generation to build the documentation for the system, which is presented as an automatically generated wiki.

All of the code necessary to deal with sectors, in all components in the system, is generated from this data model definition. Maintaining the definition in the data model is much simpler than maintaining the generated code that would otherwise have to be hand-written.

A fully deployed instance of MarketGrid comprises approximately 746,000 lines of source code across a number of languages, the main languages being C++ and TypeScript. For compiled languages, source code is not distributed but used in the build process. However, the number of handwritten lines of source code in MarketGrid is approximately 169,000. This means that 77% of the source code in the system is generated automatically during the build process.

Market Tech 6

Code generation statistics

Facilitating iteration in the financial markets

MarketGrid has been an ideal platform on which to build CartaX. In particular, the matching engine and associated infrastructure provide a high-performance platform with the reliability, resilience, and transparency that is required for a highly regulated trading platform.

MarketGrid’s code-generation framework provides the system with greater flexibility, maintainability, and reliability. Efficient code generation has also allowed the CCMX team to design and prototype unique and sometimes very complex trading models, which can then be rapidly built and implemented with a high level of confidence.

© Carta Capital Markets, LLC (“CCMX”) | All Rights Reserved | Terms and Conditions Apply | CCMX member  FINRA/SIPC | CCMX undertakes no obligation to update content herein | No business, investment, tax or legal advice is provided by CCMX | Potential investors are advised to conduct their own due diligence and consult with their tax, legal, and financial advisors with respect to any investment | All securities involve risk and may result in partial or total loss | Investments in private securities are speculative, illiquid, and involve a high degree of risk, including the possible loss of your entire investment | Access to third-party content from this site, including content of CCMX’s parent company, Carta, Inc., may be available through framed areas or hyperlinks. Access to such content is for informational purposes only and does not constitute an explicit or implicit endorsement by CCMX of such content as to accuracy, timeliness, completeness or usefulness | There is no guarantee that a private company will conduct an initial public offering or provide an alternative exit strategy for your invested capital | Images are illustrative and may differ from application experience

The Carta Team
While we believe in assigning ownership at Carta, this blog post belongs to all of us.