Loading...

Follow Mechanical Forex on Feedspot

Continue with Google
Continue with Facebook
or

Valid

One of the first strategies I ever developed in our currency trading community involved volatility breakout trading in the EUR/USD. This strategy – which I called Teyacanani – first saw the markets in 2009 and has been consistently profitable from then to present. Although the strategy did not survive for too long in its simplest initial form – as we improved it with time – the strategy has remained completely unmodified since 2013. This system remains exceptionally consistent and continues to be one of our most trusted strategies through the years. On this post I want to talk a little bit more about the past few years of trading for this strategy, how it has performed and how it might perform going forward.

The equity curve above shows you the results of this volatility breakout strategy, taking a risk of 0.5% per trade. The last time we optimized this strategy – red line – was right before the system went into a new historical maximum drawdown, completely unprecedented in over 28 years of historical trading results. At this point – close to January 2014 – we had lost faith in the strategy and believed it would not be working again as it had gone significantly above the maximum level of historical loss we had ever seen, as a matter of fact the maximum drawdown after 2014 is almost 50% higher than the maximum level of loss we had seen before. Since the risk seemed much smaller before this drawdown we were actually trading this strategy at a higher risk, so we saw loses in the neighborhood of 30-40% through this time (reason why many stopped trading the strategy at this point in time). At this point in time many people in the FX space had realized volatility breakouts were a good play – many of them thanks to our community – and many were trading even riskier or more highly leveraged versions of this strategy. Of course, this period caused volatility breakout to become relatively unpopular again.

However, after this drawdown phase ended, the strategy recovered swiftly and started to reach new equity highs relatively consistently up until this year, where it went into another significant drawdown period (currently 6.45%).  Remarkably the compounded returns of the strategy remained relatively unaffected after the loses in 2013 – around 8% in the above graph –  although the strategy did become significantly more volatile after this time, something that can be relatively easily attributed to the fact that the strategy’s alpha has decayed after it was optimized in 2013. Of course some people discovered the strategy after 2014 and many started trading versions of this strategy that contain mechanisms to avoid the losing period in 2013, of course, at the cost of additional curve-fitting bias in the mix. Not surprisingly some of this traders are currently facing drawdowns in the order of 20-30% as you would expect if you traded this strategy at a higher risk (since their fitting to avoid the 2013 drawdown really did nothing to prevent losses under current market conditions).

At the time we also made significant efforts to develop these volatility breakout strategies outside of the EURUSD but it turned out that these strategies were never as consistent in other pairs and seemed to require either much more complicated parameters or fell appart in forward testing after deployment. Some currencies even showed promise for the opposite of volatility breakouts – which we can call volatility mean-reversals – but these too did not seem to be very stable and ended up giving back a considerable portion of their alpha after deployment. Many people actually tried more fancy techniques – including walk forward analysis – to deal with these problems but as far as I can tell few were able to consistently tackle the issue of volatility breakouts across other pairs. I know that my friend and colleague Fernando Monera developed a strategy called RobinVol using this sort of principle – volatility breakouts and mean-reversals for different pairs – and achieved a good measure of success on live accounts up until at least 2017 (as you can see here). Sadly this flagship live account – which I believe was traded by a third party – is no longer being updated so we are unaware of how his setup has behaved under more recent market conditions. Fernando definitely studied volatility breakouts more deeply than we ever did at Asirikuy.

Other people have also had great success with volatility breakouts during the past couple of years, for example these traders at Darwinex have a strategy that is heavily correlated with Teyacanani and probably heavily uses similar forms of volatility breakout trading. They clearly use more currency pairs and probably some complimentary strategies, but the equity curve shows a steep drawdown during the past 4 months, in agreement with what you would expect from heavy exposure to volatility breakout trading on the Forex majors this year. They are currently right next to a historical maximum drawdown for their strategy portfolio and it wouldn’t be surprising that the real risk of their trading setup eventually exceeds the 50% level, given the magnitude of the returns they have been able to offer traders during the past several years. As I have commented in the past it is exceedingly rare for managers to beat 1:2 maximum drawdown to average compounding return ratios in the long term, so eventually either the drawdowns or the returns mean-revert to more realistic levels. This can be confirmed by analyzing long term currency traders, such as those included in the Barclay’s currency trader index (read this 2010 post for more info).

Volatility breakout continues to be a time-tested and reliable Forex trading technique which has been able to deliver returns consistently during the past 10 years. Although volatility breakout is by no means a holy grail – as evidenced by the steep drawdowns faced when people try to over-leverage or curve-fit it – it does remain a reliable investment strategy for those who wish to experience diversification from traditional bond and stock investments in the currency markets. Realistically volatility breakout strategies can offer a slightly higher than 1:1 average compounded return to maximum drawdown ratio and can therefore potentially deliver consistent returns for those who have realistic goals with adequate risk management in mind. In my view a 15% CAGR with a 15% maximum drawdown is extremely valuable given that these returns are likely to hold weak correlations to traditional investment vehicles. However the drawdowns will always inevitably arrive as over-crowding of the trade will tend to generate periods where the inefficiency temporarily decays, however the inefficiency is expected to recover after the market “shakes off” highly levered players through these periods.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Up until now I have mainly dedicated this blog to the development and discussion of algorithmic currency trading strategies. However I have always been interested in passive investing strategies in the stock/bond market but have never gone into that subject within this blog. Although I won’t do that – get into the depths of passive investing strategies here – I wanted to share with you some thoughts about a new book I have published about passive investing strategies. The book is called “Passive investing on steroids” and you can find it on this link. Within this post I will talk a little bit about why I wrote this book and why I think it is a worthy read for people who are interested in maximizing their returns and reducing their risk exposure relative to normal index based stock investments.

An important characteristic of currency trading strategies is that the currency markets follow a zero or negative sum game – depending on whether you consider broker commissions or not – reason why any profit you get from these markets depends fundamentally on the amount of alpha you can generate. This basically means that whenever you trade something like this there is no implicit guarantee of returns and anything you got up until now could stop working tomorrow. The common saying that “Past performance does not guarantee future results” is very well suited for this market as there is always a probability that a strategy that has worked – even for decades – will simply not work going forward anymore if some market characteristic changes. This is why any investment in currencies, especially using significant leverage, is generally considered high risk – regardless of the track record of the strategy being traded – and why I have always considered alternative investment vehicles where the probability of future and past returns to match is expected to be bigger. These are inherently safer for long term investing and larger amounts of capital.

The stock and bond markets have the advantage of being positive sum games – meaning that they have a tendency to create value and go up – so an investor that just buys and holds these investments is extremely likely to get a positive return in the long term. This is so true that making money in these markets within a 15 year period is pretty much guaranteed. Because of this it becomes interesting to develop strategies that use buy-and-hold mechanics to try to maximize risk adjusted returns as much as possible while making sure that any sources of statistical bias are minimized. When investing in bonds/stocks I prefer to think about strategies that avoid active management, since I want to minimize the probability of fooling myself into making wrong assumptions about returns because of curve-fitting, data-mining or other potential sources of bias.

While looking for these types of opportunities I soon realized that there are ways in which you can implement buy-and-hold strategies using bond and stock investments using leverage that are able to increase returns and reduce risk relative to traditional stock index investments (like blindly buying the S&P 500). This using a minimal set of assumptions that rely on very strong market relationships while keeping the investments as passive as possible. As with any investment portfolio that involves more than one instrument periodic re-balancing might be needed to keep allocation consistent, but beyond this the amount of effort required by investors is minimal. Since I couldn’t find any books covering the subject I decided to write a short book describing these strategies and detailing their historical simulations along with R code to reproduce all the quantitative results.

Although the strategies covered within the book are already used by many types of investors (including some hedge funds) and there are certainly several articles online that approach the subject from similar angles, I think this book provides a very concise summary of these strategies that could be very valuable for any investors who are looking to increase their long term returns without having to become legendary value investors or spend a ton of time learning how to trade. Contrary to becoming an algorithmic trader – which requires years of hard work and effort – this book is about the entire opposite: how to maximize returns while using the smallest possible set of assumptions and the least possible amount of time. If this topic interests you I would encourage you to buy the book and leave a review with your honest opinion. Also feel free to leave a comment on this post with any questions you might have about it.

You might also be wondering if I’m not trading currencies anymore – since this is the first post since December 2017! – but don’t worry, I still do, although many things have changed in my life during 2018, which have kept me very busy in the trading world and away from posting here. This will be the subject of another post!

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

With bitcoin recently reaching above the 11,000 USD per BTC mark and other cryptocurrencies such as ethereum gaining traction as well, it has now become common place for people to start developing trading systems for the cryptocurrency space. One issue with this however is that cryptocurrency data is by no means centralized – like regular Forex data – which makes it difficult for traders to get a good historical data source. Many of the data repositories that are available online are not free – charging even thousands of dollars for data – while others contain data that is simply not updated regularly (such as this kaggle data source). Today I am going to talk about an easy and reliable way to get cryptocurrency data, allowing you to update your data whenever you want and get access to years of daily historical data for many different cryptocurrencies and exchanges in only a few seconds.

import numpy as np
import datetime
import time
import re
import pandas as pd
import requests
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('-e', '--exchange')
parser.add_argument('-p','--pair')
args = parser.parse_args()

pair = args.pair
exchange = args.exchange

resp = requests.get("https://api.cryptowat.ch/markets/{}/{}/ohlc?after=1000000000&periods=86400".format(exchange, pair))
resp = resp.json()
resp = resp["result"]["86400"]

all_candles = []
for c in resp:
    all_candles.append({"Date": time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(c[0])), "Open": float(c[1]), "High": float(c[2]), "Low": float(c[3]), "Close": float(c[4]), "Volume": float(c[5])})

df = pd.DataFrame(all_candles)
df = df.set_index(["Date"])
df = df[["Open", "High", "Low", "Close", "Volume"]]
print df
df.to_csv("{}_{}_1440.csv".format(exchange, pair))

Although different exchanges offer interfaces that allow us to download data from each one of them it is quite cumbersome to implement a separate solution to download data from each different cryptocurrency exchange. Thankfully there is a website called cryptowatch which has pooled cryptocurrency data from several different exchanges into an easy to use resource. Even better, cryptowatch offers a REST API that allows us to easily access their service in order to obtain historical data and general cryptocurrency information (even order-book data). Today we are going to be taking advantage of their OHLC request function in order to download historical data from their servers from any particular broker that interests us.

The script I have posted above allows you to easily obtain cryptocurrency OHLC daily data for any exchange/pair you want that is carried by cryptowatch. The python script needs to be executed with the parameters “-p” (pair) and “-e” (exchange) which tell it exactly what exchange/pair combination you desire. For example running the above scripts by using the command “python script.py -p btcusd -e kraken” will download all available historical data for bitcoin (for kraken this is from 2013 to 2017), the data is then printed on the terminal and also saved into a csv file for future loading/use without having to call the servers again. The OHLC prices seem to be Bid prices and the time zone of the candles is GMT 0:00. 

Use of this script has some limits as well, when using the cryptowatch servers you’re limited to a given amount of server time per hour, so it is a good idea to avoid calling scripts that request a lot of data frequently. If you want to update your data more frequently then you should modify the API calls and dataframe processing to append and modify your already downloaded full history file instead of having to redownload the entire historical data set each time. It is also worth noting that you can also request candles for other timeframes if you wish, but you will have to modify the calls to do this (you can refer to the cryptowatch documentation for more information about the parameter calls for this API functions). For lower timeframes the total amount of data available will be less and – if the dataframe is too small – you will probably need several hours and calls structured with the before/after parameters to properly get all the data in chunks.

It is also worth mentioning that although there are several python libraries to interface with the cryptowatch API – such as this one and this one – I couldn’t get any of these libraries to work correctly as their data processing seemed to always give me incorrect times when asking for OHLC data, probably due to some issues with how these libraries parse the return OHLC values. The easiest way I found to get the data was to actually write the REST API call myself and to then parse the returned json values manually to put them into a pandas dataframe. Besides showing you how to download the data the script also shows you how to get the data into a pandas dataframe, a very useful thing if you want to perform some trading or data analysis in python using these data sets.

The above should give anyone interested in the design of cryptocurrency trading systems – or even just the analysis of cryptocurrency data – a good start. The first problem with system development is always finding some historical data, once this is solved we can start working with it. Whether something useful can be done with it is an entirely different matter altogether. If you would like to learn more about how we design systems and how you too can learn how to trade using portfolios of algorithmic trading strategies please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

In my previous post we discussed the use of return thresholds in the creation of a classifier in order to improve the out-of-sample (OS) performance of trading strategies. In essence instead of simply predicting whether a system’s future return was above or below zero we tried to predict whether the return was above or below a given threshold (Th). This showed to significantly increase relative performance within testing sets for a sample random forest (RF) machine learning algorithm for OS result classification.  However – after trying on a few other algorithms – it became clear that the idea of using a non-zero return threshold for classification does not work effectively across all OS classifiers. In today’s post I want to discuss these results and when it might be a good or bad idea to apply this technique.

The entire idea of predicting a non-zero return instead of a zero return comes from a desire to eliminate marginal decision making. When you use the results of an OS classifier for trading a strategy you might want to avoid trading not only when the algorithm predicts you will have a negative return, but you might even want to avoid getting into the market unless the predicted reward is high enough. In order to achieve this effect the simplest thing to do is to change the classification threshold to our “minimum desired average return” and then train our OS classifier to predict whether or not we will be able to obtain a result that is above this point. If our classifier then tells us that we have a higher than 50% probability to obtain a result above this return then we should enter the market to take advantage of this opportunity.

This sounds really good and showed very good improvements in results in my previous post. However – when looking at other algorithms – it became clear that this is not always the case. The picture above shows you the relative improvement in results for 4 different random forest classifiers that use different types of inputs and windows for predictions, you can see the results when using zero as a threshold (RFX) or when using a Th value of 0.05% for classification (RFX TH) at different probability thresholds. The results show that for the RF1 and RF2 algorithms the introduction of the non-zero Th for classification actually makes results worse, almost regardless of the actual probability threshold used to make the trading decision. This is in great contrast to algorithms RF3 and RF4 where the introduction of this probability threshold increases the improvement in results dramatically, even taking us above the 1000% mark for the RF4 TH when making the decision to trade only above a 0.6 probability.

But why is this the case? The 4 different algorithms aren’t extremely different in the quality or number of inputs they use. It is true that RF3 and RF4 do use more inputs, but the actual improvements in results obtained at the zero Th level is actually similar for RF1, RF2 and RF3, although for RF3 there is a huge increase in profitability when using the Th of 0.05% that doesn’t really exist for the RF1 or RF2 algorithms. Looking at the second plot in this post, which shows how the sensitivity changes for the 4 different algorithms, we start to see some hints about the real reasons that might make the use of a non-zero Th better in some cases than in others.

I want you to take a look at the difference in sensitivity between the zero and non-zero Th results and especially the difference in sensitivity between both cases. The sensitivity drops as a function of the forecasting probability in all cases – as is expected – but in the case of the RF1 and RF2 algorithms the drop in sensitivity is such that when we go from zero to non-zero Th the sensitivity does not have a lot of way to drop beyond where it has already gone. This tells us that the classifiers in RF1 and RF2 are already having trouble getting true positives at a zero threshold and increasing the Th to a non-zero value does not actually help the classifier since the classifier does not even understand that well how to make a zero-based classification effectively. In contrast the sensitivity for the zero case is much higher at high forecasting probability thresholds for the zero Th value for RF3 and RF4, showing us that the classifier has a much better idea of how to discern strategies using the information at hand and therefore it can be pushed further. It can benefit from a non-zero Th.

Ultimately this boils down to a matter of how much useful information the classifier is actually being able to gather, which is reflected in the sensitivity. A classifier that is able to have a slower drop in sensitivity as a function of its forecasting probability has a higher amount of “domain knowledge” and is therefore able to apply it to more stringent analysis. In all my tests algorithms that have benefited from a higher classification Th have always had sensitivities of at least 20% at a forecasting probability of 0.6 when using a zero Th value. This shows that the speed of the decline of sensitivity as a function of forecasting probability is a good tool to evaluate whether or not a non-zero Th for classification makes sense. If you would like to learn more about our machine learning algorithms for system selection and how you too can trade our system repository using this type of ML derived insights please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

On last week’s post (here) we discussed a new random forest based model that I created for the prediction of out-of-sample (OS) returns in our price-action based trading system repository at Asirikuy. This is the last from a series of currently 7 different models used for OS predictions in our community. Thinking about how I could improve these models even further I decided to implement and idea – borrowed from my work in reinforcement learning – to change the way in which the models perform classification. Today I am going to talk about what this modification is all about and how it can drastically change the results of machine learning models for OS predictions towards more positive territory. We’ll see this exemplified as I apply this modification to one of our existing random forest algorithms.

When I started working in reinforcement learning for the construction of trading strategies it soon became clear that it was difficult for systems to decide when not to trade. If you always reward a system for being long or short and getting it right then the system will see that it is almost always better to either enter the market long or enter the market short. However in reality you wouldn’t want to always trade – because this level of prediction accuracy is practically impossible – but you want to only trade if you’re very certain that you will be able to make money under those market conditions. This is why it is common to introduce reward modifiers in reinforcement learning that make you only reward the trading system when it gets a long or a short right above a certain gain. That way the system won’t just always trade, but will trade only if there is a reward that’s worth it.

When building OS classifiers to determine whether or not a system should trade we can take a similar approach. Instead of trading a classifier to predict whether the system’s results will be positive or not along a given future frontier we can instead train it to predict whether the system’s average return will be above a certain threshold. The first image in this post shows you the results of doing exactly that. If we train one of our classifiers to predict the probability of a system having an average return above Th we can get a substantial improvement in the relative change of the average return within our testing set. In this case we go from a relative improvement of 2x in the average return – trading systems that are predicted to be profitable with a 60%+ probability – to a value of almost 6x when using a Th value of 0.05% instead at the same probability threshold. This means that by training our machine learning algorithm to predict the probability of a substantial profit instead of any profit, we get a significant additional gain in how we can improve our performance.

However this gain does not come without consequences. As you can see in the plot above, the sensitivity of the different algorithms at the same probability threshold decreases significantly as the Th value increases. This is somewhat expected since you will be able to select less systems if you have trained the classifier for a more demanding scenario. The number of systems that will be predicted to have an average return above 0% with a 50% probability will be higher than those predicted to have an average return above 0.05% with the same probability. This is why this type of additional demand on the classification cannot be applied blindly to any algorithm, a primordial requirement is for there to be a significant sensitivity at a 50% classification threshold such that when the Th value for the algorithm is increased the sensitivity drop will not affect the selection algorithm very unfavorably.

It is also important to mention that an increase in Th is not entirely analogous to an increase in the probability threshold demand. Although a similar relative change in profitability exists for the Th|probability threshold cases 0.05%|50% and 0.01|60%, the sensitivity for the 0.05% Th algorithm at 50% is actually higher than that of the 0.01% Th algorithm at 60%. This means that the number of systems predicted to be profitable is greater for the Th 0.05% case, which means that a larger number of strategies would have traded, which is often a desirable case since higher system numbers usually imply a lower volatility in the trading result, given that the relative increase in the mean return is the same. This difference happens because the relative returns of the algorithms are not the same and some algorithms have average returns that are much smaller than other, so a higher threshold in probability is not a straight forward equivalent of a higher Th.

The above opens up the road to an important diversification of our algorithms since it opens up another degree of freedom that can be explored in their use and trading. Higher thresholds imply that we can become pickier traders, allowing us to ignore a trading algorithm unless the return expected from it in the future is especially substantial. Certainly some deeper study of this is granted and we will explore it within some of my following posts. If you would like to learn more about our ML based selection algorithms and how you too can trade systems from our more than 11K system repository please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

I have written several posts in the past about the building of RF models for the prediction of OS returns in our price-action based trading system repository at Asirikuy (you can read more here, here and here). So far we have built 7 different models to attempt to solve this problem, 2 of these models are meant to predict only the first sixth months of OS performance for systems that have just been mined (new systems without an OS) while the other 5 models aim to provide continuous predictions for the following weeks/months for our trading strategies, training with data that comes solely from the real out-of-sample of the strategies (data that didn’t exist when the systems were mined). Some of these continuous prediction models are really single models that use data from all the strategies in the repository while others are actually a single model per system that uses market conditions as inputs to make predictions. Today I want to share with you some data on the latest model I have created to tackle the problem of OS return predictions, what inputs it uses and how it innovates relative to the other models we have.

From the models we currently have, the most successful have been those created using data for all systems to make predictions with a 180 day look back window and a 90 day forward looking window. This means that the examples used to build the model are created using past sets of 180 days to predict the following 90 days. In essence these models output the probability that a system will be able to give a positive return in the following 90 days, given the systems’ characteristics, which never change with time, the systems’ performance and some properties of the general market during this time. To enhance the results of previous models I decided to test a wider array of trading system statistics and to include some volatility measurements of the overall market to see if these could be used to enhance predictions.

The graph above shows you the final variable importance plot for the random forest (RF) algorithm I ended up with. Compared to other models the ulcer index and Pearson correlation are now present as some of the most important inputs. I also included the standard deviation and returns for 9 general market ETFs. However because I included so many new variables it was necessary to perform some sort of pruning to avoid curve-fitting bias, so I removed the 10 least important variables from the model before arriving at the plot above. This importance was figured out looking only at the training data – which is 80% of the available examples – which means that there isn’t any forward looking bias in this step.

As usual, the Bayes statistics evaluated over the testing set – which is the last 20% of the data – are the most interesting part when building these models. As you can see the sensitivity, specificity and posterior probability values are better for previous models since in this case we are able to achieve a higher sensitivity at a lower probability threshold with a higher accuracy and posterior probability. This means that we end up being more certain about positive predictions and generating a higher positive change in the mean return compared with the average return in the testing set without any predictions being made. In the end we select a larger number of systems more accurately which leads to an improvement of +250% relative to the mean return without predictions in the testing set. In line with previous models this value increases even further at a 0.55 probability threshold, then declining when going to 0.6 as the sensitivity of the model becomes virtually zero (almost no positive predictions are made so the probability of being hit by randomly selecting something bad becomes larger).

This new model is a relatively significant advance towards better predictions at lower variable complexities using better predictors. The previous model of this type used a significant amount of additional variables – since no pruning was involved – meaning that it was subject to more significant bias than this model. With the elimination of predictors that are not very relevant and the use of market volatility descriptors – which do seem to confer an additional edge – we now have a model that has been able to make the best improvements up until now within testing sets. As always it is worth pointing out that the testing set contains some hindsight – as it is obviously data that exists right now – so we will need to see how this model improves things across real out-of-sample testing to say for sure whether it’s better or worse than our current RF system selection models.

There are however certain improvements to the above model – and potentially our other models – that can take them to even better performance. Next week we will explore the effect of using thresholds for predictions in this model to see what happens when we don’t try to only predict whether the period was just profitable but above a certain profitability threshold. If you would like to learn more about machine learning and how you too use machine learning to select which systems to trade from a repository of nearly 11 thousand please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Last week we took a look into the Asirikuy PA system repository and how the correlation between in-sample (IS) and real out-of-sample (OS) Profit Factor (PF) values changed as a function of the group of systems we looked at. This week we are going to be doing the same exercise looking at the IS/OS Sharpe ratio (SR) values of the repository in order to evaluate how the Pearson correlation of these values change as a function of different group selections among systems above a given trade threshold. I would also like to point out that the Pearson correlation is probably not an ideal metric for this analysis due to the variations in group size when changing among different percentile groups – except for groups where the number of points is similar – so although this analysis is indicative a more robust approach would probably use something like the Spearman’s rank coefficient (thanks to Fd for the suggestion). I will explore using this coefficient within a later blog post.

In the case of the Sharpe ratio for systems with more than 100 trades – first graph above – we find that the correlation between IS/OS values is higher (0.39) for systems below the 50 percentile line (in green) and lower for systems above it (-0.006). The red line shows you a linear regression of the systems to the left of the green line while the blue line shows you a linear regression for systems to the right of the green line (the number of systems in both groups is fairly similar, 870 vs 810 points). As it was the case with the PF, in the SR we see that as we go to higher IS values a proportional increase in the OS SR becomes less and less guaranteed. However if we analyze only systems with more than 200 trades – second image above – we do notice an increase in both linear regression lines with the R values changing to 0.63 (red) and 0.14 (blue). This is a significant increase from the values seen in the 100 trade threshold, showing that with an increase in the number of trades we continue to see an increase in the IS/OS correlation, even if the correlation is indeed much weaker for larger IS values.

Repeating the analysis we did before using different percentile values – image below – we see the same pattern as in the case of the PF. In this case we see that when looking below the lowest percentiles the correlation is initially higher and drops continuously as we look below increasingly high percentile values. This again points out to a deterioration of the correlation between IS/OS values as we include higher IS SR values. When doing the opposite exercise and looking at values above these percentiles we see that the correlation remains small most of the time and actually starts to increase as we move towards the highest SR values. This is interesting as it points out that increases in the IS SR are more likely to lead to increases in the OS SR at the highest SR values, although much smaller than in the case of the smallest IS SR values. However since the highest percentiles are made up of much smaller groups it is very probable that this small increase in correlation arises merely due to random chance when looking at increasingly small groups.

This analysis again points out to a limit in the increases in OS SR performance that can be achieved by pushing up the IS SR. It seems clear that pushing up the 30 year IS SR indefinitely does not seem to lead to ever increasing values in the OS SR. This may be related to the fact that pushing the IS SR higher requires higher complexity in the algorithms used which leads to difficulties in the proper evaluation of bias sources and can lead to increases in curve-fitting bias. This becomes necessary when pushing Sharpe ratios above certain thresholds since we might be demanding a level of performance that might be achieved much more probably from bias sources than from true market inefficiencies. Contrary to the profit factor we have used the Sharpe ratio as an explicit filter when mining, reason why we have are now accumulating systems only with a SR above 0.5 (the reason why there are so few samples below an IS SR of 0.5).

What these PF and SR analysis show us is that increases in trade number tend to show increases in IS/OS variable correlations but most of these gains are related to lower PF and SR values. This again points to the fact that there is an optimum upper threshold of SR and PF values to demand when searching for systems and searching for higher values is not something that will be rewarded in the OS by the market. This helps explain why trading is not merely a computational strength problem – the ability to search for the better IS inefficiencies – but a problem that is both related with the complexity of what is searched for and the performance that is demanded. Try to get things that are at the top percentiles of IS potential performance and your chances of being disappointed will increase exponentially. This might lead us to think that if you limit system complexity and degrees of freedom you might have a better chance at achieving positive results, although that is also a topic for a future blog post — repeating the above analysis as a function of complexity.

With the above it seems clear that the search for IS/OS correlations is neither linear nor simple. Finding ways to increase OS performance depends on much more than having control over IS statistical variables and demanding the very best systems – this is actually counterproductive – it instead depends more on being able to attempt to predict what the future holds with whatever IS/OS relationships can be observed in past periods. Machine learning offers us a very powerful tool to attempt to do this, although it can also be plagued with sources of statistical bias and similar issues that have to be properly addressed. If you would like to learn more about our PA portfolio and how you too can trade systems, even using ML algorithms that attempt to predict OS performance please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

On last week’s post we talked about how the in-sample profit factor (PF) and sharpe ratio (SR) statistics in our PA system repository correlate with their out-of-sample values. It was clear from those findings that the correlation between in-sample and out-of-sample variables is increasing as a function of trade number, corroborating earlier evidence using pseudo out-of-sample periods that showed this same behavior. However it is also interesting to consider whether this correlation is fundamentally located across a certain portion of the in-sample variable values or whether it is evenly distributed across the aboard. On today’s post we are going to look into this matter to find out whether the correlation is weaker or stronger for higher or lower in-sample PF values. This should help us understand how evenly distributed the data is and whether there is any difference in how in-sample and out-of-sample correlations happen depending on the range of the in-sample statistic we study.

It is clear that if the in-sample has anything to do with the out-of-sample then the correlation between PF/SR statistics across both periods must increase as a function of trade number as the statistical significance of the out-of-sample measurement increases to better match that of the in-sample period. A higher PF in the in-sample will not correspond to a higher PF in the out-of-sample unless the out-of-sample is long enough to reflect at least a part of the edge that was obtained by the system in the IS. We now know that this indeed happens in our PA repository. However it is also a common observation that systems that are particularly well performing in the IS tend to show the inverse relationship in the OS, mainly because the large performance values might tend to mean revert more significantly than values for lower performing IS strategies. With this in mind we would expect the largest PF and SR values to show some tendency towards a negative correlation in the OS.

The image above shows the OS and IS PF values with a green line located at the 50% percentile line of the IS PF (1.27) for systems with more than 100 trades in the OS. There are also two linear regression lines showing the linear regression for the bottom 50% of PF values (red) and the top 50% of PF values (blue). The Pearson correlation coefficient (R) for the entire data is 0.148 while the R for the other lines are 0.27 (red) and 0.04 (blue). This shows us that the correlation is in fact greater for the lower half of IS PF values and the correlation drops significantly when we go to the top half. This implies that there is in fact a decaying tendency for a higher PF in the IS to show a higher PF in the OS as the IS PF gets larger. We look into this in more detail by examining how the R changes for groups above/below different percentile values of the IS PF from 10 to 95%.

In this case what we see is fairly telling. When we separate the PF by looking only at values above a certain percentile we can see that the R is always low, this means that always including the top IS PF values and cutting the lower parts heavily penalizes the correlation while going the other way and removing top performers – by only looking at groups below a certain percentile – actually increases the correlation. The highest benefit in the correlation actually comes from the bottom IS PF performers showing that the highest tendency of increases in the IS PF to become increases in the OS PF is when the PF in the IS is low. The benefit is lost rather quickly as the IS PF becomes larger and it’s evident that demanding ever large increases in the IS PF won’t lead to larger increases in the OS PF. It is most likely true that increasing the IS PF above 1.3 brings little benefit in terms of the expected increase in the OS PF at least with systems that have at least 100 trades in the OS.

As the number of trades grows larger the overall correlation of the entire data set becomes larger – as shown in last week’s post – and the above relationship between the different percentiles remains similar. For example when looking at systems with more than 200 trades in the OS the IS/OS global PF correlation is 0.258 while for systems below the 50% percentile it’s 0.568 and for systems above the 50% percentile it’s 0.046. This means that most of the gains in correlation as system numbers increase occur for the lower performing ranks of the IS PF while for higher rankings the correlation remains close to 0 the entire time. This further supports the notion that demanding an IS PF of 1.3 is logical – since increases in the IS PF are bound to lead to increases in the OS PF up to this point – but forcing systems to perform better in the IS might not lead to better gains in the OS, at least in terms of the PF for our PA system repository.

Stay tuned next week as I uncover a similar analysis for the SR which will show us how the SR statistic correlations behave across our PA repository using different percentile cuts. If you would like to learn more about our work and how you too can trade a repository of more than 10,000 price action Forex strategies without having to use a VPS  please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

The key to profitable trading under real out-of-sample (OS) conditions – meaning under data that didn’t exist when you created the strategy – is to be able to draw some type of predictions about which strategies will be better performing in the future. This means that you must have some idea about how the past characteristics of the system will relate with its future characteristics. I have discussed this several times in the past – read here and here – where it seems that in-sample (IS) return related statistical characteristics bear little or no relationship with future system characteristics, that is, a system’s in-sample back-testing sharpe, profit factor, etc have little to nothing to do with its statistics under real new market conditions. However I had also written a post related with pseudo out-of-sample tests – see here – where it was somewhat clear that although correlations would be low when systems started trading, there would be an increase in these correlations as the systems traded for longer periods of time. This week I wanted to put this hypothesis to the test using our PA repository’s IS and OS Sharpe and Profit Factor statistics.

It is certainly discouraging to look at a system’s Profit Factor or Sharpe ratio in back-tests and see that it really has nothing to do with what you got in live trading. When you have many systems and you try to see if there is any correlation and you see that there is none, then it becomes even more discouraging as it’s easy to think that you’re simply wasting your time by trying to generate systems that conformed to some performance benchmark in the past. What is the point of generating a system with a higher PF if higher PF in the IS does not mean a higher PF in the OS? Of course an important fact here is that we’re usually comparing a much longer IS with a much shorter OS, which means that things should be expected to be very different. It is certainly naive to think that the PF of a 2 month period will be proportional to the PF of a 30 year old period, a 2 month period is much more random and – just by looking at the distribution of PF values in the IS – it becomes clear that drawing any type of expectation that can be related in terms of a correlation is difficult.

However once systems trade for longer they should show stronger and stronger IS/OS correlations if there is any influence of the IS period into the OS. That is, if there is some merit to selecting better systems in the IS then this should show as some type of correlation that increases in value as the OS grows longer. If no such correlation exists then the systems are hopeless victims of some type of statistical bias source (curve-fitting or data-mining bias to name the most difficult to eradicate) and the effort to filter strategies in the IS is meaningless. To put this to the test I used our PA repository – presently with more than 10K strategies – with some of these strategies now trading for more than 3 years under real OS conditions (meaning these market conditions didn’t exist when the systems were created).

The results in the first image of this post show that what we have is indeed a linearly increasing correlation, in line with my post on pseudo out-of-sample periods, of both the Sharpe and Profit Factor IS to OS values. As we go from all systems with more than 10 trades with systems with more than 220 trades the pearson correlations increase dramatically from near zero to more than 0.5 for systems with more than 220 trades. This relationship is very encouraging as the increase is faster than what was expected from pseudo out-of-sample testing. This might be because these p-OS tests did not include data-mining bias assessments which are key to eliminate an important source of bias that might make this evolution slower.

Of course it is also worth mentioning that correlation does not imply that the PF values are equivalent. The fact that IS/OS PF/Sharpe values become more correlated does not mean that a PF of 1.5 in the IS becomes a PF close to 1.5 in the OS, it just means that a higher overall PF in the IS is related with a higher PF in the OS but the correspondence – the slope of this relationship – is not one due to alpha decay. The second graph in this post shows you the IS and OS PF values for systems with more than 250 trades – where the correlation goes up to 0.50 for these variables (as it’s evident we don’t have many systems with so many trades in the OS). As you can see a PF of 1.20 does not immediately imply a PF of 1.20 in the OS but in this case having an IS PF of 1.25 actually implies that the created system’s OS performance is expected to be just close to BE.

The above results are very important because they point to the fact that IS mining of higher PF/Sharpe values makes sense because higher IS values do tend to lead to higher OS values for our systems. While the slope of this relationship is clearly less than 1 – due to alpha decay caused by changes in market conditions – the clear conclusion is that in the long term using the top IS performers of the Sharpe and Profit Factor variables should lead to better results than doing otherwise, a hypothesis that we’ll put to the test in live trading soon enough. As time passes and we obtain more data from our repository it will become possible for us to go deeper into this analysis and see how high the correlations really get as we move into longer and longer OS periods, so far the trend remains linear with no plateau in sight. If you would like to learn more about how you too can mine PA systems and trade our system repository with more than 10K systems please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.

Read Full Article
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Reinforcement learning (RL) has been an important focus for me since I finished my machine learning nanodegree at Udacity. This is because reinforcement learning is substantially different from our other machine learning strategies – which use moving window supervised learning approaches – and therefore a potentially important source of diversification for our trading. However it is often very difficult to understand how reinforcement learning systems work and perhaps more importantly, to be able to predict what sorts of actions they might take given some market conditions. Today I want to offer you a peak into the brain of a Q-table reinforcement learning algorithm so that we can take a look into what it ends up doing and how it ends up making trading decisions.

When I started studying RL approaches in trading it soon became clear that you cannot just throw powerful RL approaches at the market and expect useful algorithms. An RL algorithm can very easily curve-fit to noise and can generate results that are not general and basically the result of data-mining bias (you can read more about that here). For this reason the successful algorithms we have found have turned out to be Q-learning approaches that are simple enough as to be represented and understood in a graphical manner.

To put it simply our algorithms study a set of pre-established market conditions – market states – that are defined by the state of a set of predefined variables. In a very simple case if you had just one variable, say yesterday’s price, you could define three market states, yesterday was bullish, yesterday was bearish or yesterday was neutral. This would create a q-table with three entries and the algorithm will train through the market data assigning rewards depending on the profits/losses generated by going long, short or staying out of the market. Generally we use 60% of our data to train the algorithm – where it’s executed 10 times to refine the Q-table – and we then execute it one time across the entire data. This means that improvements in the pseudo out-of-sample period can only be made with training without hindsight (the algorithm just learns from what it’s doing not from the future in any way). The first image in this post shows the balance curve resulting from carrying out this process with one of our RL algorithms.

Since this Q-learning approach generates a Q-table we can explore what the algorithm learned by looking at the values of this table and color coding them according to their sign. The image above shows you the result of doing this. In the case of this algorithm we have 25 different market states that are each separated as well by the day of the week. This means that in total we have 125 possible market states. As you can see in most cases the algorithm is deciding not to trade as it has learned that – per its market state definitions – trading most of the time does not reliably lead to profits. This is why most BUY/SELL rows are negative – the algorithm expects a negative reward from trading under these conditions – while the STAY OUT row is always green. Some rows are yellow because these market states never presented themselves across the entire testing process.

However there are clearly some exceptions, otherwise the algorithm would never trade. The exceptions are highlighted in black border boxes so that you can see them more easily. We have mainly five instances where the algorithm has decided that trading is better than staying out of the market, four that trigger long positions and another that triggers a short position. For state 15 we can see that this generates a long signal on both Friday and Wednesday while for Thursday and Tuesday the algorithm determined that the edge it received is not large enough. With this info we can see that the algorithm will never trade on Mondays and will take only long trades on Tuesday, Wednesday and Friday and only shorts on Thursday. We can also see that the q-table value is highest for Friday state 15, implying that this behavior will be the hardest to change (and should be the most profitable if what has been learned is not curve-fitted). Wednesday is also the only day with two states that can generate positions.

Although the behavior of reinforcement learning algorithms can be a bit puzzling we can perform some basic and graphical analysis on simple Q-learning algorithms to learn more about how they trade and what the result of the trading process is. With this we can get some expectations about how the algorithms trade and how well ingrained the trading behavior actually is (by comparing the magnitude of the values), something that can be harder to do with other types of machine learning algorithms. The above shows that complexity is also unnecessary to get profitable historical results. Well chosen market state descriptors can lead to successful strategies with much less risk of over-fitting or high data-mining bias.

If you would like to learn more about reinforcement learning and how you too can create strategies that constantly learn from new market conditions please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.strategies.

Read Full Article

Read for later

Articles marked as Favorite are saved for later viewing.
close
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Separate tags by commas
To access this feature, please upgrade your account.
Start your free month
Free Preview