THE USE OF AI IN BENCHMARKING PERFORMANCE
Artificial Intelligence (AI) tools have become quite prolific in all types of media of late, but in fact these tools have been with us for quite some time. The tools range from common regression techniques to machine learning tools, and networks that mimic neural logic. While many of the articles and news stories are focused on things like Large Language Models, e.g., ChatGPT, Bard, etc., in truth the application of many of these tools are quite a bit more operational in nature. However, please do not read “operational” as boring, these tools provide no less dramatic insights and performance, just not as eye catching. An area that has received less attention, but that has been found to benefit quite significantly is benchmarking. Frontier Foundry’s work with clients has provided unique insights, when combined with our team’s extensive academic and industry experience, related to the practical application of AI in benchmarking and other areas. In our experience, many of the findings that AI has provided would not be possible, and certainly never in the periods that AI facilitates.
The subject of benchmarks is one that often confuses people, what is commonly referred to as “benchmarks” and which are used for a variety of purposes can be interpreted in any number of ways. Benchmarks are often constructed using historic observations in the form of data points to look backwards, sometime for an indefinite period, sometimes for just microseconds. These results or output of these benchmarks may be used for purposes as specific as compliance purposes or for other models as a forecasting input. In our work, we have found that there are many performance measures in historical data, but these are often unavailable to users of standard benchmarking tools. The problem many firms experience in leveraging this historic data to extract findings and to discover trends is that traditional means of analysis are quite limiting. This results in missed opportunities, performance and at times regulatory issues. While the output of these is used to address anything from operational efficiencies and client service to investment management securities selection, these are all moot without a very robust solution, which some types of artificial intelligence provide.
Why is AI needed to understand events that have occurred in the past? Well, consider that number of variables that may affect a standard benchmark, sometimes these are known, but often they are not. AI not only provides insights into the known and unknown variables, but also how much of a particular benchmark change can be attributed to the individual variables, which is extremely helpful when using these outputs to affect change.
AI has been shown to produce benchmarks that utilize:
- Machine learning regression algorithms, that not only provide a “line of best fit” like simple regressions, but also an improved means to forecast via the learning methodology.
- Support Vector Machines, which can assist in taking data with little dimensionality to understand the effects of specific activity and providing insight into how this activity affects the subject of interest.
- Unsupervised algorithms, that can be used to find associations in data without any type of “learning” data being provided.
The results of these algorithmic models are insights into the variables and their unique weightings, providing the user with actionable insights, in addition to the required benchmarks. The actionability is the direct result of the identification of variables that can only be completed using AI tools. Identifying these variables also provides an opportunity to utilize these features, and others, to forecast the behavior of certain benchmarks under various conditions. These conditions, defined by the user by adjusting the significant variables, and utilizing such tools as machine learning, we can “train” models to provide probable outcomes given a set of specific circumstances.
As AI use becomes more institutionalized, the use in benchmarking will no doubt provide users with quite distinct advantages by way of insights and forecasts that to date have been unavailable to most. Furthermore, in all types of industries there will no doubt be new findings that materialize, not the least of which will be more benchmarks that provide not only views on what has occurred, but the ability to know what will happen with specific probabilities, and the opportunity to alter these.