Benchmark is one of the main experiences at aiXplain, which means it resides in the Navigation Bar located on the top left side of your screen under “BENCHMARK”. 

When you click on Benchmark you will be directed to the benchmarking page. And under that you will find three boxes to add the different parameters for your benchmarking job.

Step 1 - Add a Dataset

A Dataset is a large collection of data that will be used to evaluate the performance of the models you want to benchmark against. To add a dataset, click on "Find Datasets" and that will generate a popup with a list of available datasets to choose from in the form of cards showing details about each dataset. You can use the search box on the top of the popup to search for a dataset or you can use the function, as well as input and output filters based on the language direction that you want (if you choose translation) . Click "Collect" to select it After collecting the dataset, click "Add to Benchmark".

Step 2- Add Models

Benchmarking allows you to compare the performance of different machine translation models. The models will use the dataset that you’ve previously selected to run. Similar to adding a dataset, click on "Find Models" will generate a popup. Use the search bar at the top of the popup to find the models you’re looking for as cards that show brief details (e.g. translation from Spanish to Greek). Note that the models are already pre-filtered based on your selection of dataset. When you hover over a model card you can click on the three dots, then Show Details to view further details of that model or "Collect" to choose this as a model for your benchmarking job. After selecting the different models that you’d like to use in your benchmarking job click on "Add to Benchmark" and you’ll see the models you added in the benchmarking page.

Step 3- Add performance metrics

The scores of the performance metrics are used to benchmark the models that you have selected against each other. When you click on "Find Metrics" it will create a list of our available metrics. Check on the metrics that you would like to be used for benchmarking. Collect your desired metrics, and add them to your Benchmark.

 
You have added the Dataset, Models, and Performance Metrics, an estimate of the cost of usage for your benchmarking job is displayed, as well as an option to turn "AutoMode" option ON. 

If you do not have enough credits, a warning will display to let you know to add credits to your wallet to proceed.

When your credit balance checks out, you are ready to go! Click on Start on your screen to run your benchmarking job.