RegularExpressions in MQL5 for working with regular expressions – library MetaTrader 5

Real author:

Microsoft Corporation. Source code taken from .Net Framework 4.6.1. 

Here is a translation of the RegularExpressions from .Net Framework 4.6.1.

To work with the library, include the Regex.mqh file from the “\MQL5\Include\RegularExpressions\” directory in your code.

Several illustrative examples are provided with the library, which at the same time serve as the test cases. All the samples are taken from the official site of Microsoft Corporation, they vividly demonstrate the main differences from the regular expressions in C# and the features of their use in the MQL5.

Below is more detailed information about RegularExpressions MQL5 ported library packages:

Packages	Description
CharUnicodeInfo.mqh	Archived txt file to determine the Unicode categories for all symbols (including non-Latin characters).
RegexCapture.mqh	Represents the results from a single successful subexpression capture.
RegexCaptureCollection.mqh	Represents the set of captures made by a single capturing group.
RegexGroup.mqh	Represents the results from a single capturing group.
RegexGroupCollections.mqh	Represents a collection of Group objects.
RegexMatch.mqh	Represents the results from a single regular expression match.
RegexMatchCollection.mqh	Represents a collection of successful matches found by iteratively applying the regular expression pattern to the input string.
Regex.mqh	Represents an immutable regular expression
RegexOptions.mqh	Provides enumerated values to use to set regular expression options.

The regular expression parameters from the file RegexOptions.mqh:

Parameter	Description
None	Specifies that no options are set.
IgnoreCase	Specifies case-insensitive matching.
Multiline	Specifies multiline mode.
ExplicitCapture	Do not capture unnamed groups. Specifies that the only valid captures are explicitly named or numbered groups of the form (?<name> subexpression).
Singleline	Specifies single-line mode.
IgnorePatternWhitespace	Eliminates unescaped white space from the pattern and enables comments marked with #.
RightToLeft	Specifies that the search will be from right to left instead of from left to right.
Debug	Specifies that the program works in the debug mode.
ECMAScript	Enables ECMAScript-compliant behavior for the expression. This value can be used only in conjunction with the IgnoreCase and Multiline values.

Working with RegularExpressions for MQL5:

1. As withe the version for .Net, this library implements a storage (static cache) of regular expressions. All implicitly created regular expressions (instances of the Regex class) are written to that storage. This approach speeds up the operation of the scripts, as it eliminates the need to rebuild the regular expression if its pattern matches any of the existing ones. The default size of the storage is 15. The Regex::CacheSize() method returns or sets the maximum number of entries in the current static cache of the complied regular expressions.
   
2. The second feature of working with regular expression in MQL5 directly follows from the first one. And it lies in the fact that the above storage must be cleared. To do that, call the Regex::ClearCache() static function. It is recommended to call this function only after the work with the regular expressions has been completed, otherwise there is a risk to delete necessary pointers.
3. Unlike the .Net, the MQL5 does not implement the foreach loop, and hence the enumeration handling will be different. Example:

   //--- Code in C#
   Regex rx = new Regex(@"\b(?<word>\w+)\s+(\k<word>)\b", RegexOptions.IgnoreCase);   
   string text = "The the quick brown fox  fox jumped over the lazy dog dog.";
   MatchCollection matches = rx.Matches(text);
   foreach (Match match in matches) 
     {
      //--- handling
     }
   
   //--- Code in MQL5
   Regex *rx = new Regex("\\b(?<word>\\w+)\\s+(\\k<word>)\\b", RegexOptions::IgnoreCase);        
   string text = "The the quick brown fox  fox jumped over the lazy dog dog.";
   MatchCollection *matches = rx.Matches(text);
   IEnumerator<Match*> *en = matches.GetEnumerator();
   while(en.MoveNext()) 
     {
      Match *match = en.Current();
      //--- handling
     }
   delete rx;
   delete matches;
   delete en;
   Regex::ClearCache();
   

4. As can be seen from the above example, the C# syntax allows to put the ‘@’ symbol in front of the strings to ignore all formatting marks in it. In MQL5, this approach is not provided, so all control characters in a regular expression pattern should be explicitly defined.

Example of working with RegularExpressions for MQL5:

As an example of regular expressions, consider their application to parse the trading history downloaded from the terminal in the form of an html file.

To do that, create an expert with a single input parameter of the ‘string’ type, which will be the name of the sandbox file:

This document contains two main tables: “Orders” and “Deals”. 

Create a regular expression to parse the file:

Regex *rgx=new Regex("(>)([^<>]*)(<)");

Consider this regular expression:

(>)	Find the ‘>’ character
(^[<>]*)	Any symbol except ‘>’ and ‘<‘, repeated zero or more times
(<)	Fin the ‘<‘ symbol

Next, read the file line by line and get all the matches which correspond to this regular expression:

string str=FileReadString(m_handel);
MatchCollection *matches=rgx.Matches(str);

The strings of the html file, that are entries (tuples) for the “Orders” and “Deals” tables, will have the most of such matches, namely 23 and 27 matches respectively. Therefore, the only thing left to do is to extract all the necessary information from these strings.

For the “Orders” table:

if(matches.Count()==23)
  {
   string in[11];
   for(int i=0,j=1; i<11; i++,j+=2)
     {
      in[i]=StringSubstr(matches[j].Value(),1,StringLen(matches[j].Value())-2);
     }
   m_list1.Add(new OrderRecord(in));
  }

It has been confirmed that the number of matches is 23, and therefore, the handled entry is from the “Orders” table. Create a string array as the initial representation of our entry. Create iteration over all the odd matches, get the values of those matches using the matches[j][.Value() method, trim the first and the last characters of each match, which correspond to the characters ‘>’ and ‘<‘. Each formatted match is written to a previously declared in array. After that create an instance of the OrderRecord(in) class, which represents a single entry of the “Orders” table, and add it to the m_list1 list. This list will interpret the “Orders” table.

The “Deals” table will be processed in a similar way:

if(matches.Count()==27)
  {
   string in[13];
   for(int i=0,j=1; i<13; i++,j+=2)
     {
      in[i]=StringSubstr(matches[j].Value(),1,StringLen(matches[j].Value())-2);
     }
   m_list2.Add(new DealRecord(in));
  }

Here m_list2 is a list of pointers to the DealRecord class. It represents the “Deals” table.

Both these lists are members of the TradeHistory class. This class is the representation of the entire original html file. In addition, it allows to apply simple filters to the “Orders” and “Deals” tables using the methods: FindAllOrders(const int columnIndex,const T value) and FindAllDeals(const int columnIndex,const T value).

Create a simple graphical interface in the EA to demonstrate these capabilities:

When working with this form, select the desired table, select a column and its value to filter the table. Press the Find button, and the filtered table will be displayed below, along with some statistics on it. The Save button stores the currently displayed table to a csv file. The saved file will be located in the sandbox as well, and have the name Result.csv.

To learn more about the RegularExpressions for MQL5 and its features, use the provided Tests.mqh expert. It implements numerous examples of regular expression usage, which cover all the main functionality of the library.