Many merchants like renko and different “timeless” charts for his or her readability and worth noise discount. The charts are used not just for technical evaluation however for constructing buying and selling methods on high of them. They usually require a correct backtesting. Nevertheless, not many merchants notice that Renko backtesting could also be (and often is) extremely misleading attributable to specificities of the chart and platforms that present it.
On this publish we’ll tackle the pitfalls of renko realization in MetaTrader 5 and how you can overcome a few of them. However earlier than we start and for the sake of brevity allow us to remind you of earlier publications on this matter.
The story begins with the introduction of the Renko chart generator – RenkoCharts – it is technical preview (out of date) first appeared on this weblog as effectively. It builds renko from quotes/charges and don’t take into account precise worth actions inside each bar. If the field dimension is smaller than a variety of particular bar, this may occasionally result in a simplified, unrealistic formation of renko bins on historical past. In consequence, backtests can run smoother than in actuality. Please notice that, when on-line, the generator works by incoming ticks, so the standard of renko is unaffected, however not everybody can afford to gather ticks for months on-line to construct a devoted renko (although it will be unfeasible attributable to possible interruptions attributable to bugs, connection losses).
The plain resolution of the issue is to construct renko on the historical past from ticks as an alternative of quotes. After all, that is potential provided that the tick historical past is out there.
Let’s be aware of the issue and its resolution:
A. Keep away from constructing renko from common timeframe bars, even M1, and use ticks if potential
This method is carried out in one other product – RenkoFromRealTicks.
It supplies higher high quality than the previous generator, however does so on the expense of upper sources utilization (disk area, bandwidth, CPU load) and execution occasions.
And guess what? It doesn’t clear up all the issues with renko accuracy.
One in every of them arises from the truth that implementation of renko charts in MetaTrader 5 relies on {custom} symbols. So far as they’re generated regionally in your PC and have distinctive names completely different to the names of unique symbols supplied by dealer, it is not possible to commerce renko symbols on-line. Paradoxically, we will commerce {custom} symbols within the tester simply. However this is able to be theoretical backtests, which can’t be mapped to actual buying and selling with out further coding and don’t assure precisely the identical (good) outcomes.
To satisfy renko charts and unique symbols your robotic ought to difficulty orders for unique symbols by indicators detected on corresponding renkos. There’s a devoted blogpost describing some strategies of Adapting EA robots to buying and selling on {custom} renko charts in MetaTrader 5 and high quality metrics of their backtests.
In reality, such robots are multi-symbol ones and will maintain synchronization between not less than 2 tickers: {custom} and unique. Principally, you need to detect formation of renko bins, however this isn’t so trivial as chances are you’ll assume. Nuances exist relying from buying and selling library you utilize (for instance, normal library) and concerned instruments (resembling indicators or charges evaluation straight, any of which depends on bars completion). We’ll focus on this matter a bit later, in part D.
Now simply be aware of the duty:
B. Guarantee cross-symbol synchronization
Some consideration on constructing extra dependable renko-driven assessments have been made in one other blogpost – Buying and selling on renko charts constructed from actual ticks.
At these stage there have been nonetheless 3 fundamental issues – their frequent impact might be described as peeking into the long run, which produces unrealistically good buying and selling outcomes:
- a niche in entrance of reversal field;
- each field is M1 bar aligned by 1 minute boundary, whereas its precise formation (both of open time and finish time) occur at some moments inside close by minutes;
- quick and outstanding worth actions could produce bins forward of actual time, as a result of they’re M1 bars (bars cannot be positioned extra tightly to one another than as soon as each 1 minute);
Allow us to take into account the clauses extra intently, ranging from the 1-st.
Do not forget that certainly one of potential approaches to adapt EA to work on a {custom} image is to position EA on the chart (renko in our case, which is used for synchronization and technical evaluation) however redirect buying and selling orders to unique image. Usually, all renko indicators are taken from closed bins (as a result of the right-most forming field can find yourself in any course, and even its “open” worth). And each reversal field is began with the hole, as a result of the tester is able to simulating common bars solely. In actuality, there isn’t a such hole between the shut worth of the earlier field and the open worth of the subsequent field. It is only a conference of drawing reversal renko field with the hole.
Execution of trades on 100pt renko chart of EURUSD with synthetic ticks or no ticks (left) vs actual ticks (proper). Be aware how faux gaps on the left chart reduce losses compared to the suitable chart the place there are not any gaps.
As you may even see, if a buying and selling sign is occured in the mean time when reversal field is began, the deal worth is shifted from the true worth for that second by the field dimension (in common). Often this unfair correction goes in favour of the dealer – a shedding commerce turns into much less unprofitable, or worthwhile commerce will increase.
In case your EA is positioned on a chart with unique image and trades it straight, acquiring indicators from bins and/or indicators on renko image, then you definitely get rid of the 1-st drawback. But when it is extra appropriate so that you can place EA on renko (for instance, EA is constructed so by a developer and cannot be tailored), we’ll present an alternate resolution under.
After all, the gaps are there in typical renko solely. As chances are you’ll know, there are extra sorts of renko, which can not require the hole. For instance, certainly one of particular renkos is the gapless renko (rail-to-rail), which can be supported by RenkoFromRealTicks. As typical renkos are extra frequent, most of buying and selling indicators are designed with typical shapes in thoughts. That is why a nonconventional renko require to re-think buying and selling indicators and re-implement EA for nonconventional renko. We’ll not take into account these variations right here.
Now allow us to take into account the 2-nd case.
The second when a worth crosses a boundary for subsequent field completion is random. It lies someplace between two consecutive 1 minute marks. When it occurs we create a brand new field with a timestamp rounded as much as nearest entire minute. The actual worth at this actual minute (with out seconds and its fractions) was not the identical as the value in the mean time when the earlier field was closed and the brand new field was began. The imply of this distinction is a half of M1 bar vary in common. Once more, it signifies that trades are opened and closed by extra useful worth than in actuality, upfront, so to talk. It’s possible you’ll give it some thought as if renko chart is at all times just about shifted half-minute forward of time.
Execution of trades on common EURUSD chart pushed by sign from 100pt renko with synthetic ticks or no ticks (left) vs actual ticks (proper). The sign is shaped by upward field accomplished at 1.08100 on 16:05 M1 bar. The left purchase is executed too early (at bar opening, therefore with too low worth and sudden bonus), whereas the suitable purchase is executed at actual time 16:05:46 and actual worth.
The three-rd drawback might be eradicated by selecting sufficiently huge field dimension that point overflows don’t occur (not at all times potential for devices with massive adjustments in volatility), however overwise it’s sadly unsolvable by canonical charts, and requires to invent some tips (mendacity outdoors the scope). It would not essentially imply that each one of “run-through” bins results in a displaced sign, however a few of them might be mishandled.
Execution of the identical indicators on overflown (trying forward) bins when EA is positioned on common M1 chart (left) vs renko (proper). Huge draw back transfer round 13:30 M1 was mapped into a number of renko bins with “future” time, the place 2 MAs crossing sign produced at synthetic 13:35. When that sign is handed to EA working on common M1 chart, 13:35 bar stands at earlier second, consequently the promote is executed upfront with increased (higher) worth.
One option to probe right here is to run EA solely on renko chart (when within the tester). So far as EA takes indicators from newest accomplished field, it is unimportant that the time marks of the bins could go forward of actual time.
Fortunately the 1-st and 2-nd issues talked about above might be fastened by the next method:
C. Generate actual ticks for renko bins
Do not forget that bars (bins) are simply containers for ticks, so an M1 bar would not have to start out with a tick at 0-th second or find yourself with a tick at 59-th second. As we create our {custom} image, we will generate ticks for renko field with actual timestamp when the field was open. The primary tick will comprise actual precise worth as effectively. All different ticks can even comprise actual costs, however we have to regulate their time to the identical timestamp as a way to hold them contained in the field. If we protect actual time to the renko ticks, the tester and the terminal would type extreme bars (based on they “understanding”), which break correct renko formations.
With such “actual” ticks inside renko bins we will run assessments based mostly on actual ticks for renko symbols and execute trades with actual costs. No synthetic gaps on reversals, no parasitic half-M1 deviations.
So okay, we’d like the true ticks. The truth that RenkoFromRealTicks builds bins from actual ticks doesn’t imply mechanically that there are ticks contained in the bins. Till model 1.4 RenkoFromRealTicks didn’t generate ticks for the bins on historical past.
Now it may possibly.
New parameter TickHistoryMode permits you to select one of many tick technology modes (so as of accelerating accuracy on the expence of useful resource consumption):
- Single tick per field
- OHLC ticks per field
- Bulk ticks per field
The primary mode is sort of the identical because it was earlier than. It generates a single tick per field in historical past. The distinction is that new variations hold timing with seconds and milliseconds (the outdated ones used M1 time).
The second mode generates a number of ticks per field, often 2 or 3 (relying from whether or not a field has a wick), which mimics OHLC attribute costs of a bar (therefore the identify), with 1 further heat up tick.
The third mode generates all ticks from unique image.
It’s possible you’ll select an optimum mode based on your buying and selling algorithm and sources.
When OHLC or bulk ticks modes are enabled, the ensuing renko image helps testing with the settings “each tick based mostly on actual ticks”. It is advisable to verify tester log to ensure that the true ticks are in impact. For instance, if the primary image, chosen within the tester, has tick historical past, the start of the log comprises some strains about ticks synchronization (downloading them from the terminal, if it is 1-st run for that image, or about synchronized already ticks, in any other case).
A number of strains under there ought to be particular line:
EURUSD_r100,M1 (Your Dealer): producing based mostly on actual ticks
When you use a number of renko symbols, all of them also needs to be talked about as having actual ticks:
... EURUSD_T_r100: ticks synchronized already [57 bytes] EURUSD_T_r200: ticks synchronized already [57 bytes] EURUSD_T_r300: ticks synchronized already [57 bytes] ... EURUSD_T_r100 : actual ticks start from 2023.01.01 00:00:00 EURUSD_T_r200 : actual ticks start from 2023.01.02 00:00:00 EURUSD_T_r300 : actual ticks start from 2023.01.02 00:00:00 ...
Make sure that to generate renko historical past with not less than 6-12 months reserve forward of deliberate beginning date of testing. The dates within the log excerpt above is chosen by the tester based on the settings to check from starting of 2023 12 months. The renko symbols had been generated because the starting of 2022.
The tester logs this data just for the primary tester image, for instance:
EURUSD_r100: historical past knowledge begins from 2022.01.02 00:00 EURUSD_r100: ticks knowledge begins from 2022.01.02 00:00
For all further symbols, loaded out of your EA ad-hoc (through CopyRates and so forth) and added to the tester’s market watch, this data is just not supplied within the log (I do not know why).
When you use different {custom} image mills for comparability or cross-validation, please, regulate additional messages in logs. Probably, even when actual ticks are enabled, there might be issues with tick historical past on some bars. RenkoFromRealTicks doesn’t permit this, however different instruments could do.
Right here is an instance of a {custom} image with 17 problematic bars:
EURUSD_T_r100 : 2023.01.02 23:59 - actual ticks absent for 1 minutes out of 16 complete minute bars inside a day EURUSD_T_r100 : 2023.01.12 23:59 - actual ticks absent for 4 minutes out of 51 complete minute bars inside a day EURUSD_T_r100 : 2023.02.03 23:59 - actual ticks absent for 3 minutes out of 37 complete minute bars inside a day EURUSD_T_r100 : 2023.03.12 23:59 - actual ticks absent for 2 minutes out of 8 complete minute bars inside a day EURUSD_T_r100 : 2023.03.22 23:59 - actual ticks absent for 1 minutes out of 22 complete minute bars inside a day EURUSD_T_r100 : 2023.05.05 23:59 - actual ticks absent for 1 minutes out of 14 complete minute bars inside a day EURUSD_T_r100 : 2023.07.07 23:59 - actual ticks absent for 2 minutes out of 18 complete minute bars inside a day EURUSD_T_r100 : 2023.09.20 23:59 - actual ticks absent for 1 minutes out of 15 complete minute bars inside a day EURUSD_T_r100 : 2023.10.08 23:59 - actual ticks absent for 1 minutes out of 3 complete minute bars inside a day EURUSD_T_r100 : 2023.11.14 23:59 - actual ticks absent for 1 minutes out of 18 complete minute bars inside a day EURUSD_T_r100 : actual ticks start from 2023.01.01 00:00:00 EURUSD_T_r100 : 2023.01.01 00:00 - 2024.02.01 00:00 actual ticks absent for 17 minutes of 3230 complete minute bars, each tick technology used
For these bars the tester will generate synthetic ticks that are incompatible with renko and should have an effect on accuracy.
One vital factor to recollect: the tester retains its personal native cache of all symbols utilized in assessments. When you recreate {custom} image with altered properties (together with full elimination and constructing from scratch) the terminal ought to usually present new historical past to the tester, or in different phrases, the tester ought to detect the adjustments and obtain the historical past anew. Sadly, it would not work so on the time of writing, so it’s essential to clear up the tester cache manually when you plan to alter {custom} image. The cache is positioned within the following 2 folders for charges and ticks, correspondingly:
/Tester/bases/<dealer>/historical past/<custom-symbol>/ /Tester/bases/<dealer>/ticks/<custom-symbol>/
One might imagine that whenever you delete a {custom} image from the terminal, the image is mechanically deleted from the tester cache as effectively, but it surely’s not.
D. Implementation
Because it’s talked about earlier, a multi-symbol robotic wants synchronization between symbols. The indicators of renko is smart just for accomplished bars, therefore we have to verify bar completion (or initiation of a brand new bar, if you want). As , MetaTrader 5 is an asynchronous factor. The incoming ticks are processed by the platform and gathered as bars, however it might occur with a bit of delay. In different phrases, your EA can obtain new tick marked by subsequent minute, however corresponding M1 bar is just not but out there (remains to be being constructed). In consequence, indices utilized in expressions like iClose(…,0), iClose(…,1), CopyBuffer(…,1) nonetheless returns values from the previous (nearly outdated, about to be shifted) bars.
Often this unsychronized state lasts not more than 1 tick, if it occurs in any respect. However it’s enough to get incorrect indicators from outdated bars and run into losses.
Sadly, this facet is just not taken under consideration in the usual library (when you use MQL5/Embrace/Knowledgeable) and in addition could not correctly dealt with in different supply libraries.
For instance, allow us to look into the tactic CExpert::Refresh which is known as upon new ticks.
bool CExpert::Refresh(void) { MqlDateTime time; if(!m_symbol.RefreshRates()) return(false); TimeToStruct(m_symbol.Time(), time); if(m_period_flags != WRONG_VALUE && m_period_flags != 0) if((m_period_flags & TimeframesFlags(time)) == 0) return(false); m_last_tick_time = time; m_indicators.Refresh(); return(true); }
RefreshRates of m_symbol calls SymbolInfoTick internally, and the time from the tick is used as a latch for not processing the bars twice. For that goal the tick is theoretically mapped in all timeframes (calculated in TimeframesFlags). If the tick advances on one of many timeframes, the code shops m_last_tick_time and refresh indicators. In different instances, the refresh is skipped. However as we already know, the brand new tick might be not but propagated to bars, so the refresh (which is definitely equal of required CopyBuffer’s) shops the outdated knowledge, which is about to alter in a fraction of a second. But when the charges are up to date, and indexing is shifted, the latch doesn’t permit to refresh anymore (as a result of subsequent ticks have occasions throughout the theoretical bar, detected on the 1-st tick).
That is why we have to apply some patches to the usual library or do related stuff in our supply codes.
Right here is an instance, the way it may very well be achieved in an skilled adviser, based mostly on the usual library.
class SyncSymbol { protected: const string image; const ENUM_TIMEFRAMES timeframe; const string interval; datetime lastUnsync; lengthy countUnsync; public: SyncSymbol(const string s = NULL, const ENUM_TIMEFRAMES tf = PERIOD_CURRENT): image(!StringLen(s) ? _Symbol : s), timeframe(tf == PERIOD_CURRENT ? _Period : tf), interval(StringSubstr(EnumToString(timeframe), StringLen("PERIOD_"))) { } bool isReady() { MqlTick t; SymbolInfoTick(image, t); const string now = StringFormat("%s'%03d", TimeToString(t.time, TIME_SECONDS), t.time_msc % 1000); if(!SeriesInfoInteger(image, timeframe, SERIES_SYNCHRONIZED)) { PrintFormat("%s wait unsync %s %s", now, image, interval); return false; } if(t.time / 60 * 60 != iTime(image, timeframe, 0)) { PrintFormat("%s wait unsync with %s on %s %s", now, (string)iTime(image, timeframe, 0), image, interval); lastUnsync = t.time / 60 * 60; countUnsync++; return false; } if(lastUnsync == t.time / 60 * 60 && countUnsync) { PrintFormat("Synced after %ld ticks on %s %s", countUnsync, image, interval); countUnsync = 0; } return true; } }; void OnTick() { static SyncSymbol image; if(!image.isReady()) return; ExtExpert.OnTick(); }
This demonstrates check-up of present _Symbol solely. For multi-symbol EAs it is a good suggestion to verify all symbols in the identical method.
In case of unsynchronization the thing will output one thing like this:
2023.01.03 10:25:53 10:25:53'645 wait unsync with 2023.01.03 09:42:00 on EURUSD_r100 M1 2023.01.03 10:25:53 Synced after 1 ticks on EURUSD_r100 M1
It’s possible you’ll discover a new revision of the take a look at EA MA2Cross2.mq5 and its sign module hooked up to the publish. Additionally a few vital patches for ExpertSignal.mqh and ExpertTrade.mqh are included.
And listed below are some outcomes achieved with this EA backtested in numerous modes.
Tester report for buying and selling EURUSD by artificially generated ticks in EURUSD 100pt renko, equal of finest accuracy supplied by former variations of RenkoFromRealTicks (pre-1.4)
Tester report for buying and selling EURUSD by actual ticks in EURUSD 100pt renko, with improved accuracy
Tester report for buying and selling straight on EURUSD 100pt renko by actual ticks, which demonstrates related outcomes to earlier one and is appropriate for validation at smaller useful resource footprint
You’ll be able to take a look at EA in numerous modes and count on on-line buying and selling efficiency to be close to the decrease certain amongst all outcomes.
F. Closing notes
Visualization of the renko within the tester can look unusual – bins could also be smaller or bigger than predefined dimension. It could’t be fastened – that is how the tester works.
Allow us to make clear. Renko bins/bars are correctly formed by the generator in static historical past and correctly re-shaped on-the-fly on-line. When the ticks are “performed” within the visible tester, the generator is uncontrolled, and the bars are shaped by ticks solely. That is why they give the impression of being unusual.
On this regard, it is vital to notice that the attribute worth of the renko field is its shut worth. So, when you use situations for checking indicators, as an alternative of shut(i) vs open(i) think about using shut(i) vs shut(i+1). Open(i) might be someplace in a variety of 0-2 field sizes from shut(i) and even equal to shut(i), if i-th bar is a single tick bar drawn throughout huge leap of worth.
If you are going to use indicators (like MAs) utilized to renko, additionally setup them for shut costs solely. Although highs and lows also needs to be dependable.
The same issues of exact backtesting pertain to different particular charts which get rid of a time in a way that bars should not alined by time scale however shaped upon knowledge pushed situations, for instance, by quantity restrict. Additionally lots of such charts have gaps in entrance of reversal bars, for instance Level And Figures.
