Package:  Diamond Backtesting with Walk-Forward Manager (BTWFMgr)

Through basic backtesting one can arrive at evidence-based statements about what trading rules would have been profitable had they been operating during the past as it occurred.  However significant issues with basic backtesting include:

In contrast, the process of Walk-Forward Analysis (WFA) is a way to model the periodically repeated re-generation of trading rule parameters based on existing historical data, with subsequent application of those rules to later data that was "unseen and unused" by the system at the point the parameters were generated.

In other words, WFA tests whether putting a particular trading model through the optimization process over some period of historical data (a defined past period) results in tradable predictability over some period of subsequent data (a defined future period).  The past period is commonly called the "in-sample" (IS) period, and the future period is commonly called the "out-of-sample" (OOS) period.

A single Walk-Forward Optimization (WFO) looks like this:

The WFO answers the question:  "If I had reoptimized this often, re-choosing my strategy parameters (per a rule/fitness function), then traded this far forward each time using those settings, how would my account balance have changed over the entire period."

Famously written about by Robert Pardo in his book Design, Testing and Optimization of Trading Systems, the Walk-Forward Optimization (WFO) has for many years now been considered to be the "gold standard of testing" for giving a trader confidence to trade a system going forward.

CodeForTraders is pleased to offer a truly outstanding WFA testing package for TradeStation users:  Diamond Backtesting with Walk Forward Manager (BTWFMgr) by Burkhard Eichberger.

What It Does

BTWFMgr anticipates the technician's need to define and run large sets of WFA's using differing IS and OOS periods in order to help find a robust re-optimization interval for any given strategy.

Unlike some other products, BTWFMgr doesn't merely take a single IS/OOS period definition and then run a single WFA.

Instead, BTWFMgr flexibly and easily lets the user define many candidate periods (storing the definitions in a period library), and then lets the user build a higher-level library of named multi-WFA sequences.

To run a test, the user chooses a WFA sequence, a Filter and a Sort.   One run of BTWFMgr is a combination of these 3 concepts executed together as a whole.

This is actually only one of quite a few complex testing, analysis and visualization operations that BTWFMgr can do.

What It Looks Like (WFA Setup)

Here's the 3-way combination choice dialog:

What are those 9 periods (in this case)?  Let's have a look:

On the right-hand side we see the 9 entries in the 'List of selected Period Setups'.

On the left-hand side we see the library of such definitions.  You can add all the definitions you want to the library, and you can use any/all of the definitions from the library in a particular WFA sequence.

Period definitions are allowable in days, weeks, or Runs/Pct, making them very flexible.

Filters and Sorts also have their own interfaces for creating named definitions and saving them in libraries:

The Filters and Sorts relate to fitness, and choosing the most desirable WFA results for further study and possible final choice to trade.  Filters let you rule out (eliminate) results that don't meet with your desired criteria (e.g. too few trades, PF too low, too many max losers in a row, etc.). 

Sorts perform ranking on the results that are left after the filter.  Another phrase for Sorts is "fitness ranking".  There are many ways to Sort (a virtually infinite set of definitions), and testing different ways to sort is a key advantage in discovering great WF trading setups.  IOW, the ubiquitous NetProfit statistic may not be the best way to rank optimized results for robustness out-of-sample.  BTWFMgr anticipates and meets the need for flexibility in fitness!

What It Looks Like (Results Display)

So far, we've only discussed (some of) the setup side of the program  BTWFMgr also shines on the output side, with extensive graphing, reporting, and Excel integration capabilities.

To begin with, BTWFMgr offers a highly organized, superior graphical interface to normal optimization results as initially generated during optimization in TradeStation.

Many viewpoints are provided into the platform's "across all data" optimization performance, which is run prior to handing off the walk-forward chores to BTWFMgr itself.  You might be interested in the iterations with the Max Equity (of course!), or the lowest Avg Drawdown, or the highest Probability, etc.  The top-performers in each category are readily accessible (with the lesser performers grouped a level down).  All it takes is a click on an iteration to show its equity curve.  You can click your way thru the categories of interest very quickly.

If you want to really drill-in, you can view the course of every single trade that made up the equity curve:

Trades and their stats are directly exportable to Excel for those who want to do further "number crunching" or spreadsheet archiving on their own.

 Back up at the top level, there is even more.

Graphs are available to show you the effect on performance (across all the iterations) of every separate parameter of your strategy.

This kind of information can be extremely valuable in helping you to save time on future optimizations.  Take the example above.  The graph shows us definitively that there is little point in doing future optimizations on ResetLvlChoice.  It should simply be fixed at 2 (for this market and timeframe).  By doing this, you make all your future optimizations only 25% of the size they would have been, which allows you to process them 4 times faster.

Further, complete seasonality information for your strategy (performance by Day of Week, Time of Day, Month of Year) is automatically generated for each run:

No other package that we know of even comes close to this level of output analysis and presentation.

Then, once you have chosen a Filter/Sort/Period combination and allowed BTWFMgr to run the WFOs, you will have detailed information available for the the stitched-together OOS equity curves which result from the WFA.

At this point, this is not merely the info on runs of your strategy, this is info on walk-forward runs of your strategy.

Here we see walk-forward equity curves chosen from the high-sorting equity and probability iterations.

With the provided graphs, we can see at a glance which walk-forward Filters, Sorts and Periods performed for the market/strategy/timeframe combination:

BTWFMgr doesn't just leave you to find walk-forward setup (Filter/Sort/Period) in a haystack by hunting-and-pecking.  BTWFMgr tests for you all the possible setups you are interested in (per your own definitions/requirements) and gives you brilliantly organized "very best" and "near best" information that you can decide and act upon in short order.

By now you should be starting to get the picture.  BTWFMgr testing is so far beyond simple backtesting in TradeStation that traders with this information and traders without it are not even on the same playing field.

Other Elements of Note

BTWFMgr is very fast.  After a single pass of optimizing your strategy inside of TradeStation, all the necessary raw information is exported and  BTWFMgr's subsequent WFA is done completely outside of TradeStation at compiled language speed.

BTWFMgr has a special "Strategy Potential Analysis" mode which can be used to evaluate your strategy's entries before any exit rules are built in.  Why waste time and energy optimizing exits if your entries are terrible to begin with?!

Indeed, BTWFMgr introduces an innovative (but completely optional) strategy development lifecycle testing model that is logical, effective, and unlike anything you've seen elsewhere. 

Here's a list of specific features (too many to describe every one in detail here!):

What You Get

For each permanent license purchased, a permanent ability to operate BTWFMgr on a single designated computer. 

For each term license purchased, a temporary ability to operate BTWFMgr on a single designated computer. 

What It Costs

Single computer, permanent licenses begin at $1200, with discounts for additional computers.

Single computer, temporary licenses begin at $135 for a 3 month block ($45/mo), with discounts for additional computers or longer subscription periods.

So, under the most expensive plan (which requires very little commitment), $45 per month buys you access to a tool that provides you serious walk-forward strategy testing which is leaps and bounds beyond your competition (unless they are using this tool also).

This is less than the profit/loss of a single typical trade!  At these offering prices, there is simply no rational reason to forego the advantages that WFA provides.    Remember, these are the financial markets we analyzing here - what you don't know will hurt you! 

How You Get It

Click on one of the buttons below to purchase 1, 2, or 3 permanent licenses, or a single-license term subscription.  If a multi-license term subscription is desired, please inquire to questions@CodeForTraders.com.

Further instructions will be emailed to you after your PayPal payment is processed.  Please allow up to 3 business days for the complete transaction.

Please be aware of the caveats before you order.

Permanent Licenses: