纳斯达克ITCH Order 说明

Working with Market Data: NASDAQ_TotalView-ITCH Order Book

	这篇文章介绍了纳斯达克的TotalView ITCH直接数据提取协议，该协议允许订阅者跟踪股票等金融工具的订单，从下单到执行或取消。通过该协议，可以重建订单簿，以跟踪特定证券或金融工具的活跃限价买入和卖出订单列表，从而揭示市场深度。文章还介绍了一些技术术语，如FIX协议、Passive investing、Net Order Imbalance Indicator等，并提到了解析二进制ITCH消息的过程。

	在解析二进制ITCH消息时，需要了解ITCH v5.0规范中的20多种消息类型，包括系统事件、股票特征、限价订单的下单和修改以及交易执行等。文章中提到了一个样例文件，并展示了该文件中最常见的消息类型及其频率。对于每个消息类型，规范中都规定了组件及其长度和数据类型。

	文章还提到了一些与订单簿相关的概念，如市场深度、流动性和大宗交易等。同时，作者还指出，这个例子需要大量的内存，建议使用16GB以上的内存。

While FIX has a dominant large market share, exchanges also offer native protocols. The Nasdaq offers a TotalView ITCH direct data-feed protocol that allows subscribers to track individual orders for equity instruments from placement to execution or cancellation.

As a result, it allows for the reconstruction of the order book that keeps track of the list of active-limit buy and sell orders for a specific security or financial instrument. The order book reveals the market depth throughout the day by listing the number of shares being bid or offered at each price point. It may also identify the market participant responsible for specific buy and sell orders unless it is placed anonymously. Market depth is a key indicator of liquidity and the potential price impact of sizable market orders.

In addition to matching market and limit orders, the Nasdaq also operates auctions or crosses that execute a large number of trades at market opening and closing. Crosses are becoming more important as passive investing continues to grow and traders look for opportunities to execute larger blocks of stock. TotalView also disseminates the Net Order Imbalance Indicator (NOII) for the Nasdaq opening and closing crosses and Nasdaq IPO/Halt cross.

This example requires plenty of memory, likely above 16GB (I’m using 64GB and have not yet checked for the minimum requirement). If you run into capacity constraints, keep in mind that it is not essential for anything else in this book that you are able to run the code. First of all, it aims to demonstrate what kind of data you would be working with in an institutional investment context where the systems would have been built to manage data much larger than this single-day example.

Parsing Binary ITCH Messages

The ITCH v5.0 specification declares over 20 message types related to system events, stock characteristics, the placement and modification of limit orders, and trade execution. It also contains information about the net order imbalance before the open and closing cross.

The Nasdaq offers samples of daily binary files for several months. The GitHub repository for this chapter contains a notebook, build_order_book.ipynb that illustrates how to parse a sample file of ITCH messages and reconstruct both the executed trades and the order book for any given tick.

The following table shows the frequency of the most common message types for the sample file used in the book (dated March 29, 2018). The code meanwhile updated to use a new sample from March 27, 2019.

Message type	Order book impact	Number of messages
A	New unattributed limit order	136,522,761
D	Order canceled	133,811,007
U	Order canceled and replaced	21,941,015
E	Full or partial execution; possibly multiple messages for the same original order	6,687,379
X	Modified after partial cancellation	5,088,959
F	Add attributed order	2,718,602
P	Trade Message (non-cross)	1,120,861
C	Executed in whole or in part at a price different from the initial display price	157,442
Q	Cross Trade Message	17,233

For each message, the specification lays out the components and their respective length and data types:

Name	Offset	Length	Value	Notes
Message Type	0	1	S	System Event Message
Stock Locate	1	2	Integer	Always 0
Tracking Number	3	2	Integer	Nasdaq internal tracking number
Timestamp	5	6	Integer	Nanoseconds since midnight
Order Reference Number	11	8	Integer	The unique reference number assigned to the new order at the time of receipt.
Buy/Sell Indicator	19	1	Alpha	The type of order being added. B = Buy Order. S = Sell Order.
Shares	20	4	Integer	The total number of shares associated with the order being added to the book.
Stock	24	8	Alpha	Stock symbol, right padded with spaces
Price	32	4	Price (4)	The display price of the new order. Refer to Data Types for field processing notes.
Attribution	36	4	Alpha	Nasdaq Market participant identifier associated with the entered order

The notebooks 01_build_itch_order_book, 02_rebuild_nasdaq_order_book and 03_normalize_tick_data contain the code to

• download NASDAQ Total View sample tick data,
• parse the messages from the binary source data
• reconstruct the order book for a given stock
• visualize order flow data
• normalize tick data

The code has been updated to use the latest NASDAQ sample file dated March 27, 2019.

Warning: the tick data is around 12GB in size and some processing steps can take several hours on a 4-core i7 CPU with 32GB RAM.

Regularizing tick data

The trade data is indexed by nanoseconds and is very noisy. The bid-ask bounce, for instance, causes the price to oscillate between the bid and ask prices when trade initiation alternates between buy and sell market orders. To improve the noise-signal ratio and improve the statistical properties, we need to resample and regularize the tick data by aggregating the trading activity.

We typically collect the open (first), low, high, and closing (last) price for the aggregated period, alongside the volume-weighted average price (VWAP), the number of shares traded, and the timestamp associated with the data.

The notebook 03_normalize_tick_data illustrates how to normalize noisy tick using time and volume bars that use different aggregation methods.