Wednesday, November 6, 2024

Mission Not possible: MetaTrader 5 doesn’t assist testing and optimization of buying and selling robots based mostly on renko indicators – Different – 26 February 2024

It is a continuation of a collection of blogposts about buying and selling by indicators of renko charts. Earlier than this level we have now mentioned many points of utilizing customized symbols for renko implementation. The newest findings are described within the article Use of customized tick modeling to forestall illusory grail-like backtests of Renko-driven buying and selling robots.

One of the vital apparent questions on this collection is why will we use customized symbols for Renko implementation fairly than one thing else. In actual fact, the “one thing else” may be very restricted. Drawing Renko on a canvas or utilizing graphical objects are impractical, as a result of it doesn’t permit for backtesting. It appears way more pure to make use of Renko indicators as an alternative.

Sadly, indicators in MetaTrader 5 are designed in such a manner that it is unimaginable to work with Renko indicators within the tester. And right here is why.

Renko indicator

To start out our analysis we’d like an indicator that calculates graphical collection which seem like Renko packing containers. Let’s don’t invent the wheel and take considered one of present indicators, for instance, Blue Renko Bars.

Because it turned out, this program required to make some bug fixes and enhancements, most vital of which we’ll clarify one after the other. The ultimate modification is connected to the publish.

Because the drawing kind of the plot is DRAW_CANDLES, the unique directive indicator_buffers is inaccurate, as a result of this kind doesn’t assist extra buffer for colours (versus DRAW_COLOR_CANDLES, which makes use of 4+1 buffers).

#property indicator_separate_window
#property indicator_buffers 4         
#property indicator_plots   1

DRAW_CANDLES requires 4 buffers. The buffer array brickColors is ineffective and has been eliminated in every single place.

double brickColors[];

The opposite buffer arrays are initialized at start-up:

int OnCalculate(const int rates_total,      
                const int prev_calculated,  
                ...)
{
   ...
   if(prev_calculated == 0)
   {
      ArrayInitialize(OpenBuffer, 0);
      ArrayInitialize(HighBuffer, 0);
      ArrayInitialize(LowBuffer, 0);
      ArrayInitialize(CloseBuffer, 0);
   }
   ...
}

We now have launched new variable lastBar to detect formation of latest bar on the host chart. At these moments we have to initialize simply added buffer components (below sure circumstances).

datetime lastBar;
   
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],...)
{
   ...
   if(lastBar != time[0])
   {
      OpenBuffer[0] = 0;
      HighBuffer[0] = 0;
      LowBuffer[0] = 0;
      CloseBuffer[0] = 0;
   }
   
   lastBar = time[0];
   ...
}

The counter of obtainable renko packing containers in renkoBuffer array of MqlRates was not dealt with accurately for all conditions and will produce “out of sure” exception (the indicator could be stopped and unloaded).

int OnCalculate(const int rates_total,      
                const int prev_calculated,  
                ...)
{
   
   int measurement = ArraySize(renkoBuffer);
   
   ... 
   
   int first;
   if(prev_calculated == 0) 
   {
      ...
      first = (rates_total > measurement) ? measurement : rates_total; 
   }
   else
   {
      
      
      
      first = measurement;
   }
   
   for(int i = first - 2; i >= 0; i--)
   {
      ... 
      HighBuffer[shift + i + 1] = renkoBuffer[i].excessive;
      LowBuffer[shift + i + 1] = renkoBuffer[i].low;
      ...
   }
}

Within the perform RenkoAdd which provides new field to the renkoBuffer we modified the precept of the operation: as an alternative of heavy ArrayCopy, we wrap the decision to ArrayResize with two calls to ArraySetAsSeries.

void RenkoAdd()
{
   int measurement = ArraySize(renkoBuffer);
   ArraySetAsSeries(renkoBuffer, false);       
   ArrayResize(renkoBuffer, measurement + 1, 10000);
   ArraySetAsSeries(renkoBuffer, true);        
                                               
   ...
}

Additionally the brand new ingredient is initilized by empty struct.

void RenkoAdd()
{
   ...
   const static MqlRates zero = {};
   renkoBuffer[0] = zero;
}

Try N1 (ChartSetSymbolPeriod)

Now allow us to recall a bit how indicators work and what this implies for the renko packing containers.

When a brand new bar is added to the chart, the charges and regular indicators (if utilized) are shifted to the left by 1 bar, however the renko packing containers ought to keep nonetheless (as a result of new packing containers seem by their very own “schedule”). Then again, when a brand new field is generated, we have to shift all earlier packing containers to the left, however charges and different indicators stay on the identical place.

It is vital to notice that any renko indicator should tackle these issues.

To resolve this desynchronization this indicator reserves a variable RedrawChart (which isn’t even an enter parameter) holding various bars to redraw. By default it is 1 and is substituted by CHART_VISIBLE_BARS. In consequence, renko is appropriate solely on the final CHART_VISIBLE_BARS bars. Furthermore, ChartGetInteger(0, CHART_VISIBLE_BARS, 0) returns all the time 0 bars whereas testing/optimizing with out visible mode! This answer is partial and never common, doubtlessly resulting in miscalculations, if utilized in automated buying and selling.

Particularly, it is flawed within the following side. Many buying and selling methods use a mixture of indicators to generate buying and selling indicators. For instance, during the collection of the blogposts we have now been utilizing a easy take a look at technique on 2 MAs crossing on high of renko. To implement it with renko indicator we have to apply MAs to the renko indicator.

And right here is the issue: indicators in MetaTrader, calculated as soon as on all bars, are then re-calculated in “quick circuit” method – solely on the most recent bar. Even in case you replace CHART_VISIBLE_BARS bars within the renko indicator, the MAs (or different indicators) utilized on high of the renko, will replace solely on the most recent bar. In consequence, it is unimaginable to get appropriate crossing of MAs.

To beat the issue we have added a brand new function to the renko indicator. After new bar creation or after new renko field formation we request the chart to replace fully, together with all moreover utilized indicators. For that puspose the calls to ChartSetSymbolPeriod are added into 2 locations: OnCalculate and RenkoAdd.

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],...)
{
   ...
   if(lastBar != time[0])
   {
      ...
      ChartSetSymbolPeriod(0, _Symbol, _Period);
   }
   
   lastBar = time[0];
   ...
}
   
void RenkoAdd()
{
   ...
   if(lastBar)
   {
     ChartSetSymbolPeriod(0, _Symbol, _Period);
   }
}

Now the MAs are correctly up to date in sync with renko re-positioning.

Renko indicator with applied MA with period 1 (to make sure it works by close prices)

Renko indicator with utilized MA with interval 1 (to verify it really works by shut costs, CHLO see beneath)

OHLC -> CHLO

But there’s one other small downside. The renko is represented as 4 buffers with Open, Excessive, Low, and Shut costs of the candles. When an extra indicator is utilized to a different indicator, it makes use of the very first buffer. Therefore our 2 MAs are utilized on Open costs, which isn’t usually a desired impact within the case of renko-based technique. As a substitute, the MAs ought to be utilized to the Shut costs of the renko packing containers. To take action we have to alternate Open and Shut buffers within the renko indicator.

The brand new mode is switched on or off by new parameter SwapOpenClose.

enter bool SwapOpenClose = false; 
   
int OnInit()
{
   ...
   if(SwapOpenClose) PlotIndexSetString(0, PLOT_LABEL, "Shut;Excessive;Low;Open");
   ...
}
   
int OnCalculate(...)
{
   ...
   for(int i = first - 2; i >= 0; i--)
   {
      OpenBuffer[i + 1] = SwapOpenClose ? renkoBuffer[i].shut : renkoBuffer[i].open;
      HighBuffer[i + 1] = renkoBuffer[i].excessive;
      LowBuffer[i + 1] = renkoBuffer[i].low;
      CloseBuffer[i + 1] = SwapOpenClose ? renkoBuffer[i].open : renkoBuffer[i].shut;
      ...
   }
}

This seems like a completed renko indicator appropriate for buying and selling automation. It is a deception, however we’ll uncover this a bit later, and can attempt to add different options to get it to work as anticipated.

Knowledgeable adviser based mostly on 2 MAs crossing on the renko indicator

Presently let’s attempt to adapt our take a look at EA – MA2Cross, initially utilizing renko customized symbols – for working with the renko indicator. The modified model has the title MA2CrossInd.mq5.

Enter paramaters are added for underlying indicator:

enter int  BrickSize    = 100;    
enter bool ShowWicks    = true;   
enter bool TotalBars    = false;  
enter bool SwapOpenClose = true;  

The indicator with the given parameters is created in OnInit, and its deal with is handed to the sign filter as an alternative of former Signal_2MACross_MAPrice parameter (really it was Shut worth on a regular basis).

int OnInit()
{
  ...
  const int deal with = iCustom(_Symbol, _Period, "Blue Renko Bars", BrickSize, ShowWicks, TotalBars, SwapOpenClose);
  if(deal with == INVALID_HANDLE)
  {
    Print("Cannot create indicator, ", _LastError);
    return(INIT_FAILED);
  }
  ChartIndicatorAdd(0, 1, deal with);
  ...
  filter0.MAPrice((ENUM_APPLIED_PRICE)deal with); 
  ...
}

This program might really commerce on an internet chart! But it surely can’t be backtested and optimized!

The rationale for it’s because the perform ChartSetSymbolPeriod isn’t working within the tester. In consequence, 2 MAs are usually not recalculated correctly and provides incoherent indicators.

What can we do?

Try N2 (PLOT_SHIFT)

One in all concepts was to implement the renko indicator with one other precept of re-positioning renko packing containers in opposition to common bars. It is based mostly on the property PLOT_SHIFT.

As we are able to shift visible illustration of indicator’s buffer to the suitable or to the left, relying from the property: optimistic values transfer curves to the suitable for the desired variety of bars, whereas unfavorable values transfer them to the left. So, when a brand new common bar is created, we are able to apply the shift +1 to our renko packing containers to maintain them visually on the identical place. And when a brand new renko field is added, we are able to apply the shift -1 to maneuver previous packing containers to the left, conserving alignment with the final common bar.

As a result of we do not know beforehand the route through which future worth will shift out graph extra, we have to make a reserve of empty invisible packing containers on the proper facet. The reserve is supplied in corresponding enter parameter and used to initialize a variable with present shift.

enter int Reserve = 0;
   
int shift;
   
int OnInit()
{
   ...
   shift = Reserve;
   PlotIndexSetInteger(0, PLOT_SHIFT, shift);
   ...
}

Then in OnCalculate improve the shift on new bars. Nonzero Reserve can be used as a flag to allow the brand new mode.

int OnCalculate(...)
{
   ...
   if(lastBar != time[0]) 
   {
     if(!Reserve)
     {
       ChartSetSymbolPeriod(0, _Symbol, _Period);
     }
     else
     {
       PlotIndexSetInteger(0, PLOT_SHIFT, ++shift);
       Remark("++", shift);
       OpenBuffer[0] = 0;
       HighBuffer[0] = 0;
       LowBuffer[0] = 0;
       CloseBuffer[0] = 0;
     }
   }
   ...
}

Additionally lower the shift on new packing containers in RenkoAdd.

void RenkoAdd()
{
   ...
   if(!Reserve)
   {
      ChartSetSymbolPeriod(0, _Symbol, _Period);
   }
   else
   {
      PlotIndexSetInteger(0, PLOT_SHIFT, --shift);
      Remark("--", shift);
   }
}

After all, the shift have to be used to regulate indices throughout writing knowledge into the buffers.

int OnCalculate(...)
{
   ...
   for(int i = first - 2; i >= 0; i--)
   {
      OpenBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].shut : renkoBuffer[i].open;
      HighBuffer[shift + i + 1] = renkoBuffer[i].excessive;
      LowBuffer[shift + i + 1] = renkoBuffer[i].low;
      CloseBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].open : renkoBuffer[i].shut;
      
      if(i == 0) 
      {
         OpenBuffer[shift + i] = SwapOpenClose ? shut[i] : renkoBuffer[i].shut;
         HighBuffer[shift + i] = upWick ? upWick : MathMax(renkoBuffer[i].shut, renkoBuffer[i].open);
         LowBuffer[shift + i] = downWick ? downWick : MathMin(renkoBuffer[i].shut, renkoBuffer[i].open);
         CloseBuffer[shift + i] = SwapOpenClose ? renkoBuffer[i].shut : shut[i];
      }
   }
}

Sadly, even though the brand new method is working excellent visually, the info shift isn’t detected correctly by indicators utilized on high of the renko.

It is a identified common limitation of MetaTrader platform. The property PLOT_SHIFT can’t be detected outdoors an indicator, and when OnCalculate is named in dependent indicators, they obtain and course of knowledge unshifted. Let’s remind you the way the quick type of OnCalculate seems like:

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const int start,
                const double &knowledge[]);

You possibly can see right here that indicator receives comparable and considerably associated property PLOT_DRAW_BEGIN, which is handed by way of start parameter. However there’s nothing telling that knowledge array makes use of shifted indexing. The one method to overcome that is to enter the shift to all indicators as enter. However in our case we alter the shift dynamically in renko, and therefore it is unimaginable to regulate the shifts in MAs on the fly (until you re-implement all required indicators your self and ship the shift by way of chart occasions, or world variables, or one thing else – that is too costly).

Try N3 (say “no” to rates_total)

Another method to strive which involves thoughts supposes to not replace renko packing containers in any respect. Simply pile them in the beginning of the tester time and return precise quantity from OnCalculate, as an alternative of rates_total.

We are able to use unfavorable Reserve worth as a flag to allow this mode. When it is on, buffer indices ought to be mapped into the vary [0..size], which is finished by adjusting shift variable.

int OnCalculate(...)
{
   ...
   if(Reserve < 0)
   {
     shift = rates_total - measurement;
     if(shift < 0) Print("Renko buffer overflow, will terminate...");
   }
   
   for(int i = first - 2; i >= 0; i--)
   {
      OpenBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].shut : renkoBuffer[i].open;
      HighBuffer[shift + i + 1] = renkoBuffer[i].excessive;
      LowBuffer[shift + i + 1] = renkoBuffer[i].low;
      CloseBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].open : renkoBuffer[i].shut;
      ...
   }
   ...
   return(Reserve < 0 ? measurement + 1 : rates_total);
}

As well as, all calls to vary PLOT_SHIFT property ought to be wrapped into acceptable guard circumstances:

int OnInit()
{
   ...
   if(Reserve > 0)
   {
     shift = Reserve;
     PlotIndexSetInteger(0, PLOT_SHIFT, shift);
   }
   ...
}
   
int OnCalculate(...)
{
   ...
   if(Reserve > 0)
   {
      PlotIndexSetInteger(0, PLOT_SHIFT, ++shift);
      Remark("++", shift);
   }
   ...
}
   
void RenkoAdd()
{
   ...
   else if(Reserve > 0)
   {
      PlotIndexSetInteger(0, PLOT_SHIFT, --shift);
      Remark("--", shift);
   }
}

The indicator is calculated and displayed correctly on this mode whereas operating within the visible tester (keep in mind, it’s essential scroll to the start of the chart to see the packing containers). However that is one other deception.

If it could work as anticipated, we would want to vary the sign technology module Signal2MACross.mqh barely by including the next methodology (see connected Signal2MACrossDEMA.mqh):

class Signal2MACross : public CExpertSignal
{
   ...
   int StartIndex(void) override
   {
      const int base = iBars(_Symbol, PERIOD_CURRENT) - ::BarsCalculated(m_type);
      return((m_every_tick ? base : base + 1));    
   }
}

Right here m_type is a sort of worth on which MA indicators are utilized, and we are able to assign a deal with of the renko indicator to it, because it was talked about earlier than:

  ...
  const int deal with = iCustom(_Symbol, _Period, "Blue Renko Bars", BrickSize, ShowWicks, TotalBars, SwapOpenClose);
  ...
  filter0.MAPrice((ENUM_APPLIED_PRICE)deal with);
  ...

This fashion the renko shut worth from our EA will likely be used for MA calculations.

Sadly, all of this isn’t working as a result of the worth returned from BarsCalculated is all the time rates_total, not precise worth returned from OnCalculate.

We’re attaching a barely modified instance of DEMA indicator (DEMAtest.mq5, to be positioned in MQL5/Indicators/Examples/), which lets you hint and output precise OnCalculate’s parameters obtained by the indicator, when it is utilized on the renko indicator with diminished variety of calculated bars (packing containers). This indicator can be used within the Signal2MACrossDEMA.

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const int start,
                const double &worth[])
{
   const int restrict = (_AppliedTo >= 10) ? BarsCalculated(_AppliedTo) : rates_total; 
   ...
   #ifdef _DEBUG
   Print(rates_total, " ", prev_calculated, " ", restrict, " ", start);
   #endif
   ...
}

You possibly can make it possible for the variety of out there bars is all the time reported as rates_total, and in consequence – the sign module above will learn knowledge at incorrect indices.

I think about this a bug of the platform, as a result of it is passing appropriate values by way of start parameter, however not by way of rates_total.

Backside line

On the time of writing, it is unimaginable in MetaTrader 5 to run a backtest or optimization of EA which is predicated on indicators of a renko indicator. There are some workarounds out there.

You should utilize customized symbols with renko, because it’s described in my earlier blogposts.

You possibly can calculate Renko nearly inside EA. This will likely change into a tough routine process if it’s essential apply completely different technical indicators to Renko, as a result of it’s essential re-implement them from scratch on your digital buildings.

Or you should use solely a restricted subset of indicators relying to renko packing containers (with out extra indicators), for instance, checking Shut[i] in opposition to one another. Blue Renko Bars indicator is prepared for this state of affairs.

Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles