Motivation

Although the factorial feature in Apsim Next Generation(ApsimNG) can generate multiple simulations files on the fly, it is very CPU demanding to run a large .apsimx file with many simulations in it. Particular in the High Performance Cluster (HPC) environment, small .apsimx files that require less CPU time are desirable so that the HPC can run small jobs parallel within a relative short time. A various solutions have been developed to cater the need of multiplying small .apsimx files in R, Python and C#. However, they are tailored case by case. ApsimNG has a built in feature, /Edit, can perform such task in command line.

Note: This documentation is for models in the prototype stages on a custom build ApsimNG.

ApsimNG built in feature in the command line

Step by step in Window Powershell

  1. A compiled version of APSIM next generation.

  2. An executable file Models.exe can be found in the Bin directory.

  3. Run .\Bin\Models.exe in command line will trigger the help documentation as below

/Edit <PathToConfigFile>
Edits the .apsimx file. Path to a config file must be specified which contains lines of parameters to change in the form 'path = value'

  1. The command requires two inputs:

a. The .apsimx file that needs to multiply. b. A configuration file contains the parameter address with associated values to be modified.

  1. To construct the configuration file, we need to identify the absolute path to the wanted nodes. The easiest way is using the user interface (UI) to copy the path to particular nodes. For example, we can right click the clock node and select the copy path to node or use the short cut key shows in snapshot 1.

Snapshot 1

  1. We can paste the path to a plain text file. To change specific parameter values, we still need to define the name of the parameter. For instance, we need to use .Start to instruct the absolute path address for modifying the clock start date. The updated values can be placed after the equal sign with the same format in the .apsimx file. See an example of formatting the configuration file. Notepad++ is recommended for Windows user to search the exact key - value pair to track the target parameters. Please be aware that no quotation is needed.

Get-Content .\config20200312.txt

.Simulations.New Zealand.Iversen_12.Simulation3.clock.Start = 1970-02-03T00:00:00
.Simulations.New Zealand.Iversen_12.Simulation3.clock.End = 1980-02-03T00:00:00
.Simulations.New Zealand.Iversen_12.Simulation3.paddock.Soil.Physical.LucerneSoil.KL = 1,2,3,4,5,6,7,8,9,0
.Simulations.Replacements.Lucerne.Leaf.Photosynthesis.FW.XYPairs.X = 2,3,4,5
.Simulations.New Zealand.Iversen_12.Simulation3.Weather.FileName = C:\Data\01raw-data\ClimateAndObserved\lincoln.met
  1. The command below can trigger the editing on a test .aspimx file. Please be aware of the working directory for deciding the absolute and relative paths.
 C:\ApsimX\Bin\Models.exe .\test.apsimx /Edit .\config20200312.txt
```

8. Run the test `.apsimx` file to make sure the expected modification has happened.

C:\ApsimX\Bin\Models.exe .\test.apsimx


More discussion about this feature can be find in this [github issue](https://github.com/APSIMInitiative/ApsimX/issues/4905)

9. To optimise the power of `/Edit`. We believe that multiple configuration files can be automatically generated by additional tools such as R, Python and C#. A simple for loop can be written to iterate over all configuration files and output corresponding modified `.apsimx` files. An automated workflow will be discussed. 

## Linux 

The process is exactly the same as it in Powershell. The only difference might be that `mono` is required to invoke the `Models.exe` in the Linux environment. The command, therefore, will be as below:

mono C:\ApsimX\Bin\Models.exe .\test.apsimx /Edit .\config20200312.txt ```

This document provides details about how to compile ApsimX into Linux enviroment.

Other solutions for multiplying .apsimx files

Except for C#, the R and Python solutions require specific tweaks to suit for different purposes. Will briefly describe how these solutions work in this section. In general, the logic in all solutions is the same as the build-in /Edit feature, which is searching and updating. Specifically, customised functions are developed to perform searching the path of wanted nodes and replacing the value in those nodes by new values.

R solution

Coming soon

Python solution

Coming soon

C# solution

Coming soon