Chapter 8 Using Scripts in jEPlus

Version 2.1, © 2019, 2020 Energy Simulation Solutions Ltd

jEPlus v2.1 allows user to use any scripting language that can execute a script file using command-line. An example project has been included in the distribution package to show how Python, R, Ruby and PHP scripts can be used for both pre-processing (@script() syntax for parameters) and results collection (scripts object in RVX). This chapter makes comprehensive coverage of how scripts can be used in jEPlus projects.

Before you can use scripts in jEPlus, you need to first configure the language's platform. This is done using the Configure External Programs dialogue window (through Tools/Configure External Programs… menu). The screenshot below shows the important fields to fill when setting up a language.

Configue script languages

Here are example configurations for Python, R, Ruby and PHP languages. You can copy and paste these blocks into your own tools.json in the jEPlus folder, and then edit the location of the executables to make them work on your computer.

{
  ...,
  "scripConfigs" : {
    "PHP" : {
      "exec" : "C:\\bin\\php-7.4.6\\php.exe",
      "args" : "",
      "scriptExt" : "php",
      "verCmd" : "-v",
      "screenFile" : "console.log"
    },
    "R" : {
      "exec" : "C:\\bin\\R-4.0.0\\bin\\Rscript.exe",
      "args" : "--vanilla",
      "scriptExt" : "r",
      "verCmd" : "--version",
      "screenFile" : "console.log"
    },
    "python3" : {
      "exec" : "C:\\bin\\Python36\\python.exe",
      "args" : "",
      "scriptExt" : "py",
      "verCmd" : "-V",
      "screenFile" : "console.log"
    },
    "ruby" : {
      "exec" : "C:\\bin\\Ruby26-x64\\bin\\ruby.exe",
      "args" : "",
      "scriptExt" : "rb",
      "verCmd" : "-v",
      "screenFile" : "console.log"
    }
  },
  ...
}

Scripts can be used during the process of preparing simulation cases in jEPlus, through the @script() syntax for parameter values. Utilising the power of the libraries that support EnergyPlus, e.g. Eppy, it offers great flexibility in manipulating the IDF models.

The diagram below is the pre-processing steps for preparing the EnergyPlus model for each case. The step of using an arbitrary Python script to manipulate the model is inserted right before the last step, calling ExpandObject. At this stage jEPlus has done all its processes, and the in.idf is ready in the case's folder. So the idea is that a users script can apply further operations on the in.idf file, using the arguments defined in the parameter and paths names passed in by jEPlus.

jEPlus calls the named script file in the project folder and passes in four arguments:

1. Sys.argv[1] - project's base folder where the project files (e.g. project.jep) are located
2. Sys.argv[2] - folder of the current case where in.idf (or in.dck and so forth) is located
3. Sys.argv[3] - Other arguments specified in the parameter definition, as a ',' delimited string
4. Sys.argv[4] - The location of the binary files of the simulation program, e.g. the location of Energy+.idd. This argument is only relevant with EnergyPlus simulations

It is expected that the script will take the existing in.idf, make changes, and produce a valid new in.idf that is ready for E+ simulation.

Here is an example of the parameter definition:

@script(python3, pre_test_args_py3.py, P1, 222, abc)

It is led by the keyword @script. The first field in the brackets is the script language, which must have been configured as described in section 9.1. The second field is the paths of the script file. Both relative and absolution paths names can be used here. If a relative path is used, it is relative to where the project file is located.

The following fields in the brackets are arguments to be passed to the script. Please note if jEPlus parameter names present in the project are referenced, the values of the parameters for each case will be used. This allows the scripts to work with existing parameters.

In the given example, arguments 222 and abc will be passed to the script as they are, whereas P2 will be replaced by e.g. 180 before being passed to the script.

If the syntax is valid, you shall see it being displayed in the Preview box, as:

{call(python3, pre_test_args_py3.py, @@orientation@@, 222, abc)}

The included folder example_2-scripts_E+v8.9/ contains some script skeleton files for pre-processing, namely in Python 3, R, Ruby and PHP languages. The scripts do not really do anything except echo what arguments they have received from the system calls. If the example project runs successfully, the arguments that have been passed to the scripts will be written to the logs.

Example project and the console log if successful

The contents of the skeleton files are shown below.

# python script for pre-processing
# Arguments:
#   sys.argv[1]  -  project's base folder where the project files are located
#   sys.argv[2]  -  folder of the current case where in.idf is located
#   sys.argv[3]  -  Other arguments specified in the parameter definition. They are passed in as a ',' delimited string
#   sys.argv[4]  -  folder of the binary files of the simulation program, e.g. the location of Energy+.idd

import os
import sys
import math

print ('Pre-process script:')
print ('sys.argv[0]: ' + sys.argv[0])
print ('sys.argv[1]: ' + sys.argv[1])
print ('sys.argv[2]: ' + sys.argv[2])
print ('sys.argv[3]: ' + sys.argv[3])
print ('sys.argv[4]: ' + sys.argv[4])

# path to E+ idd file
# os.path.join(sys.argv[4], 'Energy+.idd')

# path to energyplus input file within each simulated folder
# os.path.join(sys.argv[2], 'in.idf')

# parameters passed through args
# sys.argv[3]

# path to E+ idd file
# os.path.join(sys.argv[4], 'Energy+.idd')

# Done

#!/usr/bin/env Rscript

# R script for pre-processing
# Arguments:
#   args[0]  -  project's base folder where the project files are located
#   args[1]  -  folder of the current case where in.idf is located
#   args[2]  -  Other arguments specified in the parameter definition. They are passed in as a ',' delimited string
#   args[3]  -  folder of the binary files of the simulation program, e.g. the location of Energy+.idd

args = commandArgs(trailingOnly=TRUE)

# test the number of expected arguments: if fail, return an error
# if (length(args)<4) {
#   stop("Exactly 4 arguments must be supplied", call.=FALSE)
# }

i <- 0
for (arg in args) {
	cat (sprintf ("Argument %g: %s\n", i, arg))
	i <- i + 1
}

# Done

# Ruby script for pre-processing
# Arguments:
#   ARGV[0]  -  project's base folder where the project files are located
#   ARGV[1]  -  folder of the current case where in.idf is located
#   ARGV[2]  -  Other arguments specified in the parameter definition. They are passed in as a ',' delimited string
#   ARGV[3]  -  folder of the binary files of the simulation program, e.g. the location of Energy+.idd

ARGV.each_with_index do|arg, i|
  puts "Argument #{i}: #{arg}"
end

# Done

#!/usr/bin/php
<?php
	# PHP script for pre-processing
	# Arguments:
	#   $argv[1]  -  project's base folder where the project files are located
	#   $argv[2]  -  folder of the current case where in.idf is located
	#   $argv[3]  -  Other arguments specified in the parameter definition. They are passed in as a ',' delimited string
	#   $argv[4]  -  folder of the binary files of the simulation program, e.g. the location of Energy+.idd

    var_dump($argv);

	# Done
?>

Running Python scripts for result collection and post-processing was first introduced in jEPlus v1.6. V2.1 now supports any script languages that can run script files using command-line. This gives the user virtually infinite possibilities for post-processing simulation results. Here is a snippet of the Scripts section of the included example project.

{
  ...
  "rvx" : {
    ...
    "scripts" : [ {
      "fileName" : "post_test_each_py3.py",
      "onEachJob" : true,
      "arguments" : "some;args",
      "tableName" : "script_table1",
      "language" : "python3"
    }, {
      "fileName" : "post_test_args_py3.py",
      "onEachJob" : false,
      "arguments" : "some;args",
      "tableName" : "script_table2",
      "language" : "python3"
    }, 
    ...  
  ],
  ...
}

One field to note is the onEachJob option, which determines where and when the script is called, and the number of arguments it will receive from jEPlus.

The fields in each script item require a bit of explaination:

• fileName – The file name of the Python script. If full path is not provided, it is relative to the location of this RVX file.
• pythonVersion – Unfortunately the Python language differs between version 2 and 3. There are also restrictions depending on the interpreter used. In this field you can select jython, python2 or python3.
• onEachJob – true or false. If true, the script will be executed in each job folder, otherwise in the project's output folder where the individual job folders are located. If not onEachJob, a list of jobs in the project will be passed to the script as the second argument.
• arguments – You can provide additional arguments to be passed to the script. All additional arguments should be specified in one text string, separated by ,.
• tableName – The file name for the output table. Value of this field will be passed to the Python script as the third argument. The script is then responsible for producing a csv table similar to the RVI result.

The number of arguments jEPlus passes to the script varies depending on the onEachJob option. If the onEachJob field is set to false, jEPlus will pass five arguments in total to the script. The arguments can be read within the script using sys.argv[]. Note that sys.argv[0] always returns the name of the script.

1. sys.argv[1] – the full paths of the location of the project itself
2. sys.argv[2] – the full path of the output folder of the project where the individual job folders are located
3. sys.argv[3] – the list of the job IDs of the simulations that have been executed
4. sys.argv[4] – the expected output table name, as defined by tableName. jEPlus will add .csv to the table name before calling the script
5. sys.argv[5] – Other arguments as specified in the scripts object in the RVX file

Otherwise, if the onEachJob option is true, the arguments passed are as below:

1. sys.argv[1] – the full paths of the location of the project itself
2. sys.argv[2] – the full path of the output folder of the simulation case
3. sys.argv[3] – the expected output table name, as defined by tableName. jEPlus will add .csv to the table name before calling the script
4. sys.argv[4] – Other arguments as specified in the scripts object in the RVX file

The Python scripts are responsible to produce suitable output tables that jEPlus can read and include into its result collection process. Depending on where the script is run, the output table formats are different.

If a script is run in the individual jobs folders, the output table should mimic the format of a typical eplusout.csv file generated by ReadVarsESO. Basically, the first column is date and time; the rest are data. Here is an example:

Date/Time,InteriorLights:Electricity [J](Hourly),InteriorEquipment:Electricity [J](Hourly),Heating:DistrictHeating [J](Hourly)
 01/01  01:00:00,0.0,2276640.,13278931.1044949
 01/01  02:00:00,0.0,2276640.,32229908.1477126
 01/01  03:00:00,0.0,2276640.,17895859.9832406
 01/01  04:00:00,0.0,2276640.,38784519.4821989
...

A script running in the project's output folder should produce a table similar to SimResults.csv. The table should have three columns before the start of data. These three columns are the serial IDs, the job IDs, and a reserved column that can be anything or left empty. Below is an example. Please note the header row must start with #

#, Job_ID, Date/Time, Electricity:Facility [J](RunPeriod)
0, LHS-000000, simdays=62, 233879205202.003 
1, LHS-000001, simdays=62, 236359323510.063 
2, LHS-000002, simdays=62, 248514348464.105 
3, LHS-000003, simdays=62, 232542002313.733 
4, LHS-000004, simdays=62, 248299129214.135 
5, LHS-000005, simdays=62, 250977825693.01 
6, LHS-000006, simdays=62, 239737768305.779 
...

Example project showing how scripts are used in RVX

The following R scripts are included in the example_2-scripts_E+v8.9/ folder. These are skeletons just for demonstrating how arguments are passed from jEPlus to the script, depending on whether it is onEachJob or not.

# This script is for testing the args passed by jEPlus. It is expected to run in the individual case folders, 
# not in the project's working folder.
#
# Arguments:
#   args[0]  -  project's base folder where the project files are located
#   args[1]  -  output folder of the project where the RunTimes.csv is located
#   args[2]  -  user-defined output table name + .csv
#   args[3..] - Optional - other arguments specified in the RVX ScriptItem object

# Show args

args = commandArgs(trailingOnly=TRUE)

# test the number of expected arguments: if fail, return an error
# if (length(args)<4) {
#   stop("Exactly 4 arguments must be supplied", call.=FALSE)
# }

i <- 0
for (arg in args) {
	cat (sprintf ("Argument %g: %s\n", i, arg))
	i <- i + 1
}

# Done

# This Ruby script is for testing the args passed by jEPlus. It is expected to run in the working folder, 
# not in individual cases folder
#
# Arguments:
#   args[0]  -  project's base folder where the project files are located
#   args[1]  -  output folder of the project where the RunTimes.csv is located
#   args[2]  -  the list of jobs have been executed in the project. If this length
#               of the list is over 8,000 characters, this argument will contain
#               a file name in the output folder (argv[2]) where the actual ';' delimited  
#               list is stored; or 'NA' if jEPlus fails to write the list to file.
#   args[3]  -  user-defined output table name + .csv
#   args[4..] - Optional - other arguments specified in the RVX ScriptItem object

# Show args

args = commandArgs(trailingOnly=TRUE)

# test the number of expected arguments: if fail, return an error
# if (length(args)<4) {
#   stop("Exactly 4 arguments must be supplied", call.=FALSE)
# }

i <- 0
for (arg in args) {
	cat (sprintf ("Argument %g: %s\n", i, arg))
	i <- i + 1
}

# Done

Another example shown here is a Python 2 script for reading RunTimes.csv, do some calculations on the reported computing times, and then write to a CSV table, to demonstrate the whole process. This is a “global” script, i.e. the onEachJob option is set to false.

# Example python script: This script reads from RunTimes.csv, calculates CPU time used in seconds,
# and then write to the different table specified by the user.
# Arguments:
#   sys.argv[1]  -  project's base folder where the project files are located
#   sys.argv[2]  -  output folder of the project where the RunTimes.csv is located
#   sys.argv[3]  -  the list of jobs have been executed in the project
#   sys.argv[4]  -  user-defined output table name + .csv
#   sys.argv[5..] - Other arguments specified in the RVX file

import sys
import csv
import math

ifile  = open(sys.argv[2] + "RunTimes.csv", "rt")
reader = _csv.reader(ifile)
ofile = open(sys.argv[2] + sys.argv[4], "wb")
writer = _csv.writer(ofile)

rownum = 0
timelist = []
for row in reader:
    # Save header row.
    if rownum == 0:
        header = row[0:3]
        header.append("CpuTime")
        writer.writerow(header)
    else:
        time = [float(t) for t in row[5:]]
        seconds = time[0]*3600+time[1]*60+time[2]
        timelist.append(seconds)
        temprow = row[0:3]
        temprow.append(seconds)
        writer.writerow(temprow)
    rownum += 1
ifile.close()
ofile.close()

n = len(timelist)
mean = sum(timelist) / n
sd = math.sqrt(sum((x-mean)**2 for x in timelist) / n)

# Console output will be recorded in PyConsole.log
print '%(n)d jobs done, mean simulation time = %(mean).2fs, stdev = %(sd).2fs' % {'n':n, 'mean':mean, 'sd':sd}