Getting Back Into It

Well as with most movies the sequel seems to not have lived up to the expectation set by the original.

Overlord is the name for my current research and trading platform created entirely using NodeJS. It took years to develop and even more to polish off. Having started the project while still in school there have been many lessons learned along the way, leaving much to be desired form the functional though rudimentary product. A total rewrite was just a matter of time. I took the opportunity to create my "dream system" utilizing a C# backend and an Angular front end, and Python scripts for driving the bot logic.

There were some initial hurtles with matching old and new behavior and back testing results. I made some changes to the system, specifically executing daily candles before hourly. This was a rabbit hole that took me way too long to get out of. The next big issue was speed. Using a web api to get the data from the simulated exchange proved to be a major bottle neck. I switched over to just sending raw bytes over a socket but the hoops I had to jump through to translate the data seemed no. Turns out parsing large volumes of strings into doubles takes time. This is where I ran head long into a wall…

Version 1 of overlord is one application where variables can be directly passed from object to object resulting in next to no latency. It takes about three minutes to back test six years of data where prices changes are recorded down to 15 seconds. On the new system it takes about ten minutes stepping the granularity back to five minutes. This is far from ideal, not to mention that the surrounding infrastructure; get data process, exchange API wrappers, and live trading, does not exist yet.

Shared memory is one yet unexplored option that I hope can solve my speed problem in the future. Apache arrows seems like it could provide me with an answer, but I am tabling the project for a while. I haven't done any meaningful work on my trading in months. I have several research reports I want to write up that I am hoping will yield a new strategy or two, I’ve added no pairs to my portfolio, stock trading is still not a thing and the improvements I wanted out of the new system could have been implanted on the old with all the time I took.

The lesson I've learned from my little adventure around the hamster wheel is to avoid the engineers trap when it comes to trading. There are plenty of already existing systems for automated trading that exist. There is no need to invent your own and there is 100% no need to reinvent your own to the detriment of your trading.

I was kind of out of it for a while, but I can safely say…

I’m back!

Research report coming soon.

Until then stay safe out there.