How it was built and how it works.
(An introduction on how this project was born, is available here)
Algotrading is a kind of framework, which allows users to create bots, capable of autonomous trading, develop and test their strategies. Mainly built for crypto markets, can also be modified to run in other markets.
How it works
Algotrading uses the same principle for the three operating modules — real-time, backtest and tick-by-tick*. It runs continuously, reading the input data and making decisions, based on that data.
Currently works synchronously, waiting for data to take action, it could be to buy, sell or do nothing, based on previously available data.
*I’ll explain them later.
What programming language do I use?
The chosen programming language was initially Python 2. I took this decision based on my professional experience, as I always worked with Python as my primary language, but also because of available libraries, that could be helpful for this project, like Pandas, Numpy and Matplotlib.
Go Lang was a possibility, once I was learning and enjoying it, and also because it is faster than Python and that could be relevant for backtesting, but felt Go, in this field, wasn’t mature yet. With only a few projects on Github and without some needed libraries available.
The speed is a relevant question but is not fundamental. For real-time trading, the bot will have a considerable period, which gives bots plenty of time to decide, based on data, if it is a legitimate call or not. The biggest issue is related to backtest mode, once we want to have the fastest processing speed possible, to reduce the testing time. Taking this into account, Algotrading was built using the multiprocessing library, which parallelizes backtests. Users can also improve their bots by developing the respective entry and exit modules** in C++ or Julia.
Meanwhile, Python 2 was replaced by Python 3, once Python 2 was losing its official support. The transition was easy and can bring new features to the project in the future.
**will also explain this later.
Data
The first necessary thing to do is to get data. Data must be reliable and consistent, so the best source will always be the brokers you want to work with. Nowadays, brokers have very comprehensive APIs, capable of providing reliable data, with generous frequency, but everything depends on what type of analysis and trades you want to make. If you pretend a day trading bot, and your strategies pass through making several trades per day, it is crucial to have data with smaller intervals. Otherwise, if your style is more of swing trading, a reduced refresh interval is not so important. For high-frequency trading, this will not be your best option, once the refresh rate needs to be high, and if you try one high, like less than one call per second, the most probable thing to happen is brokers blocking your accesses. If you are planning to scalp trading, this could be a reliable option, but you need to test your strategies first, to be sure.
Database / files
In terms of data, can be stored in files or a database. The framework allows both scenarios, but for most cases, the database will be the best option.
Saving data to files is a good alternative in scenarios where you have the DB in another machine, or/and you are backtesting your strategies in multiple computers, accessing the same data, having data in files, could be the best option. The same occurs if you are making too many accesses to DB or the queries are heavy, you will be overloading the machine, making tests running slower than expected.
Choosing a database was not easy, once the offer is too big. I decided to restrict my search to only time-series databases. At the time of implementation, the best options seemed to be KDB+ and InfluxDB. Besides the superiority of Kx database, KDB+, the choice was a bit obvious once KDB+ is not free, presenting just a 32bit free version, which didn’t fulfil my requirements.
My project dedicated to this database is also available on Github. Here!
Operating modes
The three different ways, Algotrading can be used.
Real-time
Realtime mode is the real operating bot. It allows users to trade in live conditions, entering and exiting trades live. Users in this mode can make market trades and also simulate trades, not buying or selling coins, but testing their entry and exit methods. In this mode, the data used came directly from broker API, avoiding the latency of the database.
Backtest
Backtest is an essential part of the process. It allows traders to understand if our strategies are profitable in the real world, testing them with past data. A positive backtest in past data is not a guarantee of success in future data but a very reliable indicator.
Tick-by-tick
The tick-by-tick method is a step by step backtest, where users can check entry and exit points in more depth. You can adjust the time interval between data updates, to understand if your bot actions are
Entry and exit modules
The principal part of the bot lies here. Entry and exit functions define when and where our bots will make the calls. In entry methods, we establish the conditions to enter a market, and in the exit methods, we define the rules to close the trades. For now, these methods are synchronous with the rest of the bot, which works well in the terms we use them. In the future, the idea is to give the possibility to also work with asynchronous calls, which could appear from the most different sources: human intervention, sentiment analysis, Twitter/Telegram channels, other bots, etc.
Some extra modules were built to protect users. They are stop loss and trailing stop loss, as the names indicate, the first one can implement a variable stop-loss rule, which protects users from abrupt market declines. The second one, as the name implies, implement a trailing stop loss, which is a stop loss based on the previous best market price. For example, if a coin goes up 5% and then starts to decline, and trailing stop loss is defined to 1%, the bot will sell the coin when it reaches minus 1% of the best price, making a profit of 4%.
More info about the Algotrading framework, like the tutorial and examples, is available here.
Future work
The priority is to allow asynchronous interactions. Another important question is related to speed and to providing a Cython version of this framework.
Adding more brokers is also relevant, once not all coins are in the same brokers.
Not so important but also in my mind is to improve plots, facilitate data analysis, and improve the reports.
