Decoding TradingView Alerts and Mastering ib_insync: A Comprehensive Guide
Introduction
Interactive Brokers does not offer back-testing or forward-testing capabilities. However, with the integration of TradingView's strategy or indicator tools, we can supply these informational components to Interactive Brokers.
TradingView has the capacity to send alert messages containing TradingView's trading strategy to a Trading Robot, like the TradingBoat (TBOT) we'll discuss in this article.
The Trading Robot, in turn, can dispatch orders to Interactive Brokers, creating an efficient and interactive trading system. This article serves as a follow-up to a previous piece that provided an overview of TBOT's architecture and purpose.
Overview of the TradingBoat
This discussion delves into the specific role of the TradingBoat (TBOT) application when it receives TradingView alert messages. TBOT functions as a crucial interpreter and facilitator within the TradingBoat platform, decoding alert messages and placing orders accordingly. The broader TradingBoat platform incorporates several key components, including Nginx/Ngrok, Flask, Redis, and the IB Gateway, all working in harmony to ensure optimal trading efficiency.
TBOT is often described as the 'brain' of TradingBoat due to its pivotal role in the system. It receives alert messages from TradingView via the Redis Pub/Sub or Redis Stream, interpreting these messages in real-time. Once decoded, TBOT then places the corresponding orders with Interactive Brokers through the ib_insync API.
The diagram below illustrates the structure and interaction of these components. TradingBoat, represented by the large rectangle, forms the broader system, with TBOT, highlighted in red, operating as its control center.
Architecture and Role of the TBOT Application
Choosing between TWS API and ib_insync
The development of the TBOT application had dual goals in mind. First, it would decode alert messages (webhook) from TradingView via Redis Pub/Sub or Redis Stream. Secondly, it would place orders into Interactive Brokers utilizing the ib_insync API, depicted in the red block of the below image. To make the most of its asyncio-based Observer Design Pattern, the ib_insync API was selected.
The idea of implementing the Observer Design Pattern arose from the need to utilize Redis Pub/Sub as the message broker to receive TradingView Alert messages. It involved polling Redis messages and updating all observers upon arrival of a message, as illustrated in the subsequent diagrams.
Thus, it became essential to review and compare two API sets: the official TWS API from Interactive Brokers or the third-party IB Insync API.
TWS API: This official API from Interactive Brokers is used to access real-time market data, manage accounts, and send orders. Though powerful, the TWS API is comparatively low-level, requiring substantial programming knowledge for effective usage. It typically involves the utilization of two threads.
Ib_insync: A third-party Python library that serves as a high-level interface to the TWS API. Ib_insync leverages Python's async and coroutine support to simplify the API's complexities, offering a more Pythonic way to interact with Interactive Brokers.
Both the TWS API and Ib_insync have their merits. The former, as the official Interactive Brokers API, is reliable. The latter offers a more Python-friendly approach. Given that Ib_insync is actively maintained, well-documented, and available on Github, it's as reliable as the official API for my requirements. The most crucial factor was choosing an interface that is compatible with the asyncio for the Observer Design Pattern implementation, making IB Insync the ideal choice.
Observer Implementation
In this step, observers are attached to the subjects, one of which is TBOTDecoder, leveraging the IB-Insync library. We ensure that TBOT checks its connection status whenever it receives a new alert message.
The alert messages, received in JSON format, are first validated using the Python jsonschema library before being dispatched for order placement. The main function looks as follows:
def main() -> int:
subject = TbotSubject()
observer_i = TBOTDecoder()
observer_w = WatchObserver()
observer_d = DiscordObserver()
observer_t = TelegramObserver()
try:
subject.attach(observer_i)
subject.attach(observer_w)
subject.attach(observer_d)
subject.attach(observer_t)
Connection Management with ib_insync in TBOT
When new alert messages arrive, TBOT validates its connection status. If a connection does not exist, TBOT promptly establishes one before forwarding the alert message. It uses the connect() function from ib_insync API to set up the connection, with parameters such as IP address, port, and clientId, defined through environment variables via the python-dotenv library.
def update(
self, caller: object = None, tbot_ts: str = "",
data_dict: Dict = None, **kwargs
):
if not self.is_connected():
self.connect()
def connect(self) -> bool:
try:
self.ibsyn.connect(
hared.ibkr_addr,
int(shared.ibkr_port),
clientId=int(shared.client_id),
)
Placing and Tracking Orders with TBOT
Order Validation and Placement
Upon receiving an alert message, the TBOT application verifies it for legitimacy using Python's JSONschema library. This validation step ensures the message contains appropriate properties and values.
After the alert message is validated, it's dispatched to determine the order type. TBOT utilizes various order types like market order, stop order, limit order, stop-limit order, bracket order, and attached order, with each type supporting actions like creation, update, and cancellation.
def update(
self, caller: object = None, tbot_ts: str = "",
data_dict: Dict = None, **kwargs
):
.....
if self.is_connected():
if data_dict:
self.ib_dispatch_order(tbot_ts, data_dict)
Classifying Orders in TBOT
Before placing orders with Interactive Brokers via the ib_insync API's placeOrder() function, it's crucial to recognize the order type indicated by a specific alert message from TradingView.
Within the TBOT application, orders can be categorized as follows:
- Market order
- Stop order
- Limit order
- Stop-Limit order
- Bracket Order
- Attached Order
Each order type can perform different operations such as creation, update, and cancellation. For example, a Bracket Order can be created, updated, and then cancelled.
This means the alert message must contain necessary information about the type of order and its respective operation for a particular contract (whether a stock or forex) and the quantity.
We use the Python NamedTuple, specifically the OrderTV class, to store this information after extracting it from an alert message.
Here's the definition of the OrderTV class:
class OrderTV(NamedTuple):
"""
Create NamedTuple for Orders which is needed to place orders onto IB/TWS
"""
uniqueKey: str
timestamp: str
contract: str
symbol: str
timeframe: str
action: str
qty: float
currency: str
entryLimit: float
entryStop: float
exitLimit: float
exitStop: float
# TradingView's close price of the bar
price: float
orderRef: str
tif: str
The OrderTV NamedTuple serves as a container for crucial information extracted from alert messages, enabling efficient order placement and management.
Identifying Order Types in TBOT
TBOT leverages four pieces of price information—entryLimit, entryStop, exitLimit, exitStop—from alert messages to distinguish the order type. These values are converted into a binary array, where 1 represents a non-zero price. This binary array is then used to determine the order type.
For example, a binary array could look like this:
[entry.limit, entry.stop, exit.limit, exit.stop] = [1, 0, 1, 0]
Here, entryLimit and entryStop are parameters for the PineScript function strategy.entry(), while exitLimit and exitStop are parameters for the PineScript function strategy.exit().
Given this, order types can be identified as follows:
- [0, 0, 0, 0] corresponds to a Market order
- [1, 0, 0, 0] corresponds to a Limit order
- [0, 0, 1, 1 ] corresponds to a Bracket Order (Market, ProfitTaker, StopLoss)
Placing Specific Orders
TBOT places orders according to the price information received. If a Market Order is identified, TBOT uses the ib_insync API's MarketOrder() and placeOrder() functions in the place_market_order method.
The method looks like this:
def place_market_order(self, t_ord: OrderTV) -> ErrorStates:
...
contract = Stock(t_ord.symbol, "SMART", t_ord.currency)
self.ibsyn.qualifyContracts(contract)
m_ord = MarketOrder(
t_ord.action, t_ord.qty, tif=t_ord.tif, orderRef=t_ord.orderRef
)
trade = self.ibsyn.placeOrder(contract, m_ord)
...
Order Tracking
Though placing orders in Interactive Brokers doesn't guarantee their fulfillment, tracking them is essential. An order might not be filled for various reasons, such as being placed outside of trading hours, reaching the maximum number of trades per stock, or not meeting specific price conditions. Therefore, TBOT offers a way to effectively monitor and manage these order statuses.
Two elements are crucial for tracking orders: monitoring ib_insync events by registering event callbacks and saving the transition from alert messages to orders and any subsequent changes in order status into a database.
TBOT uses a Python NamedTuple, specifically the OrderDBInfo class, as a container to track orders:
class OrderDBInfo(NamedTuple):
tvPrice: float
orderId: int
ticker: str
action: str
orderType: str
qty: float
avgfillprice: float
orderStatus: str
orderRef: str
parentOrderId: int = 0
lmtPrice: float = 0.0
auxPrice: float = 0.0
# Either number for filled orders Or number of positions for portfolio
position: int = 0
Using ib_insync Events in TBOT
In TBOT, five key events are installed for effective operations:
def install_event_hdlrs(self):
self.ibsyn.orderStatusEvent += self.on_order_status
self.ibsyn.orderModifyEvent += self.on_order_modify_event
self.ibsyn.cancelOrderEvent += self.on_cancel_order_event
self.ibsyn.openOrderEvent += self.on_open_order_event
self.ibsyn.updatePortfolioEvent += self.on_update_portfolio
self.ibsyn.errorEvent += self.on_error_event
Event orderStatusEvent:
The on_order_status() callback is registered for the orderStatusEvent. This event is crucial for tracking your order status: PendingSubmit, PreSubmitted, Submitted, or Filled. By registering the callback, you can update the order transition into the database.
def on_order_status(self, trade: Trade):
self.on_order_common_event(trade)
if trade.orderStatus.status == OrderStatus.Filled:
positions = self.ibsyn.positions()
for pos in positions:
if pos.contract.symbol == trade.contract.symbol:
self.on_order_status_ptf_position(trade.contract, pos.position)
break
Event orderModifyEvent (ib_insync event):
If you are using trailing profit take and trailing stop-loss and you continue to receive new prices from TradingView and set updated orders, this event is handy. It allows you to modify the order in the database.
Event cancelOrderEvent (ib_insync event):
If PineScript sends strategy.cancel(), the TBOT application places the cancel order using ib_insync.cancelOrder() and then updates the status of cancelled orders.
self.ibsyn.cancelOrder(trd.order)
Event openOrderEvent:
This event returns the current open orders. If you are setting up the database table for the first time or need to recover from database loss, this event is beneficial to populate the current ongoing open orders.
Event updatePortfolioEvent:
The need for updatePortfolioEvent arises when, for instance, TradingView PineScript (strategy.close, strategy.close_all) sends messages of closing the position for a specific order ID. To verify if you have a position coming from that order, you should keep updating your portfolio if any changes occur.
Event errorEvent:
This event captures errors reported by Interactive Brokers. It includes an error code which could display your network connection information, the failure reason why your order failed, and more. You can find these error codes and their meanings in the System Message Codes documentation provided by Interactive Brokers.
Database Management with TBOT
TBOT employs SQLite3 to manage three critical tables: ALERTS, ORDERS, and ERRORS. The ALERTS table records incoming alert messages from TradingView, the ORDERS table tracks order status reported by the ib_insync API, and the ERRORS table documents any errors encountered. Managing these components effectively ensures the smooth operation of the TradingBoat application.
Here's a closer look at these tables:
Alerts Table
The ALERTS table captures incoming alert messages from TradingView. Not all alerts can be converted into orders, making this table necessary.
For instance, an invalid attempt to place an order could occur if you're trying to close a position that doesn't exist or adjust a Take Profit or Stop-Loss order that's already been filled. These situations can arise because TradingView's alerts cannot synchronize with orders and/or positions to your broker or exchange. This is reflected in the alertstatus column when the alert message is converted to an order.
The uniquekey serves as the primary key for the ALERTS table, representing the timestamp at which Redis receives the alert message.
The ALERTS table can be set up as follows:
def setup_connection(self, db_path, host =None, port=None):
....
sql_query = """
CREATE TABLE IF NOT EXISTS TBOTALERTS (
timestamp DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
uniquekey,
tv_timestamp,
ticker,
direction,
timeframe,
qty,
orderref,
alertstatus,
entrylimit,
entrystop,
exitlimit,
exitstop,
tv_price
)
"""
self._exec(sql_query)
....
ORDERS Table
The ORDERS table, linked to the ALERTS table via the foreign key uniquekey, primarily tracks the order status reported by ib_insync API events. Here's how you can set it up:
def setup_connection(self, db_path: str, host=None, port=None):
...
sql_query = """
CREATE TABLE IF NOT EXISTS TBOTORDERS (
timestamp DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
uniquekey,
tv_price,
orderid,
ticker,
action,
ordertype,
lmtprice,
auxprice,
qty,
avgfillprice,
orderstatus,
orderref,
parentid,
position,
mrkvalue,
avgprice,
unrealizedpnl,
realizedpnl
)"""
self._exec(sql_query)
...
ERRORS Table
The ERRORS table is designed to record errors relayed by the Event errorEvent. Observers from messaging apps retrieve this information through SQL queries and subsequently relay it to communication platforms such as Discord and WhatsApp.
Here's an example of how the ERRORS table can be set up:
def setup_connection(self, db_path: str, host=None, port=None):
.....
sql_query = """
CREATE TABLE IF NOT EXISTS TBOTERRORS (
timestamp DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
reqid,
errcode,
symbol,
errstr
)
"""
Conclusion
In this discussion, we've explored the intricacies of the TBOT application, a comprehensive tool designed to manage TradingView Alert messages, coordinate order placements using the ib_insync API, and maintain a robust database management system.
The source code for TBOT is readily available on GitHub at GitHub - PlusGenie/tbot-tradingboat
You can find it by conducting a search for "TBOT on TRADINGBOAT" or "PlusGenie TBOT" on GitHub.
Moreover, for those delving into the ib_insync Python package and its potential with interactive brokers, a detailed guide on seamlessly integrating TBOT with TradingBoat and deploying it with Docker can be invaluable. For this purpose, I highly recommend referring to the comprehensive Udemy course linked Link to Udemy Course. This course offers extensive instructions and invaluable insights, enabling you to effectively leverage TBOT for your trading activities."