Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion example/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# %PDI examples {#PDI_example}
\page PDI_example PDI example
# %PDI examples

The example is made of a single source file (one version in each supported
programming language), this file loads its configuration and %PDI specification
Expand Down
8 changes: 5 additions & 3 deletions pdi/docs/Doxyfile
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ LAYOUT_FILE = ./_template/layout.xml
CITE_BIB_FILES =
QUIET = NO
WARNINGS = YES
WARN_IF_UNDOCUMENTED = YES
WARN_IF_UNDOCUMENTED = NO
WARN_IF_DOC_ERROR = YES
WARN_IF_INCOMPLETE_DOC = YES
WARN_NO_PARAMDOC = YES
Expand All @@ -130,11 +130,13 @@ WARN_AS_ERROR = NO
WARN_FORMAT = "$file:$line: $text"
WARN_LINE_FORMAT = at line $line of file $file
WARN_LOGFILE =
INPUT = @DOXYGEN_INPUT_SP@
INPUT = @DOXYGEN_INPUT_SP@ \
../../plugins \
../../tutorial
INPUT_ENCODING = UTF-8
INPUT_FILE_ENCODING =
FILE_PATTERNS = *.c *.cc *.cxx *.cxxm *.cpp *.cppm *.c++ *.c++m *.java *.ii *.ixx *.ipp *.i++ *.inl *.idl *.ddl *.odl *.h *.hh *.hxx *.hpp *.h++ *.ixx *.l *.cs *.d *.php *.php4 *.php5 *.phtml *.inc *.m *.markdown *.md *.mm *.dox *.py *.pyw *.f90 *.f95 *.f03 *.f08 *.f18 *.f *.for *.vhd *.vhdl *.ucf *.qsf *.ice
RECURSIVE = NO
RECURSIVE = YES
EXCLUDE =
EXCLUDE_SYMLINKS = NO
EXCLUDE_PATTERNS = */.git/* */.svn/* */.hg/* */CMakeFiles/* */_CPack_Packages/* DartConfiguration.tcl CMakeLists.txt CMakeCache.txt
Expand Down
2 changes: 1 addition & 1 deletion pdi/docs/Plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
|:----------------------------------------------------|:------------------------------------------------------------------|
|\subpage Decl_HDF5_plugin "Decl'HDF5 plugin" |Read and write data from HDF5 files in a declarative way. |
|\subpage Decl_NetCDF_plugin "Decl'NetCDF plugin" |Read and write data from NetCDF files in a declarative way. |
|Deisa plugin |Expose data to a Dask analysis script. |
|\subpage deisa_plugin "Deisa plugin" |Expose data to a Dask analysis script. |
|\subpage mpi_plugin "MPI plugin" |Enables MPI support in %PDI and plugins. |
|\subpage pycall_plugin "Pycall plugin" |Call python scripts from C application |
|\subpage JSON_plugin "JSON plugin" |Export data in JSON format. |
Expand Down
4 changes: 2 additions & 2 deletions pdi/include/pdi/datatype.h
Original file line number Diff line number Diff line change
Expand Up @@ -110,15 +110,15 @@ class PDI_EXPORT Datatype: public Datatype_template
*
* \param[in] to the pointer to the allocated memory to fill (dense data)
* \param[in] from the pointer to the copied data (size of buffersize)
* \return updated `to' pointer
* \return updated 'to' pointer
*/
virtual void* data_to_dense_copy(void* to, const void* from) const = 0;

/** Creates a sparse deep copy of dense data
*
* \param[in] to the pointer to the allocated memory to fill (size of buffersize)
* \param[in] from the pointer to the copied data (dense data)
* \return updated `to' pointer
* \return updated 'to' pointer
*/
virtual void* data_from_dense_copy(void* to, const void* from) const = 0;

Expand Down
2 changes: 1 addition & 1 deletion pdi/include/pdi/expression.h
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ class PDI_EXPORT Expression

/** Moves an expression
*
* \param[in] expr the expression to move`
* \param[in] expr the expression to move
* \return *this
*/
Expression& operator= (Expression&& expr);
Expand Down
3 changes: 2 additions & 1 deletion plugins/decl_hdf5/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# The Decl'HDF5 plugin {#Decl_HDF5_plugin}
\page Decl_HDF5_plugin Decl'HDF5 plugin
# The Decl'HDF5 plugin

**WARNING** This documentation is a work in progress and does not reflect the
full Decl'HDF5 potential.
Expand Down
2 changes: 2 additions & 0 deletions plugins/decl_hdf5/dataset_op.h
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ class Dataset_op
* \param dir the operation direction
* \param name the value name
* \param default_when the default "when" clause as read from the file level (optional)
* \param file_collision_policy the collision policy string
*/
Dataset_op(Direction dir, std::string name, PDI::Expression default_when, Collision_policy file_collision_policy = Collision_policy::WRITE_INTO);
/** Builds a Dataset_op from its yaml config
Expand All @@ -127,6 +128,7 @@ class Dataset_op
* \param name the value name
* \param default_when the default "when" clause as read from the file level (optional)
* \param tree the configuration tree
* \param collision_policy the collision policy string
*/
Dataset_op(
Direction dir,
Expand Down
2 changes: 1 addition & 1 deletion plugins/decl_hdf5/hdf5_wrapper.h
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ Raii_hid make_raii_hid(hid_t value, Destroyer&& dst, const char* message = NULL)
/** builds a HDF5 dataspace that represents a PDI Datatype
*
* \param type the datatype to represent in HDF5
* \param select whether to create a dense type instead of a type with a selection
* \param dense whether to create a dense type instead of a type with a selection
* \return a tuple containing the Raii_hid for (dataspace, datatype)
*/
std::tuple<Raii_hid, Raii_hid> space(PDI::Datatype_sptr type, bool dense = false);
Expand Down
3 changes: 2 additions & 1 deletion plugins/decl_netcdf/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# The Decl'NetCDF plugin {#Decl_NetCDF_plugin}
\page Decl_NetCDF_plugin Decl'NetCDF plugin
# The Decl'NetCDF plugin

The Decl'NetCDF plugin was created to read from and write to NetCDF files in a declarative way.

Expand Down
1 change: 1 addition & 0 deletions plugins/decl_netcdf/dnc_group.h
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ class Dnc_group
/** Creates a NetCDF group information from yaml
*
* \param ctx Context of this group
* \param path File name of the netCDF dataset
* \param config (optional) config with group attributes
*/
Dnc_group(PDI::Context& ctx, const std::string& path, PC_tree_t config = {});
Expand Down
1 change: 0 additions & 1 deletion plugins/decl_netcdf/dnc_io.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,6 @@ class Dnc_io
/** Creates I/O operation (read or write) on NetCDF file
*
* \param ctx Context of this I/O
* \param file File associated with this I/O operation
* \param config Configuration node of this I/O
*/
Dnc_io(PDI::Context& ctx, PC_tree_t config);
Expand Down
6 changes: 3 additions & 3 deletions plugins/decl_netcdf/dnc_netcdf_file.h
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ class Dnc_netcdf_file
* \param right_flag right flag which be used to open file
* \param mpi_comm_expr if not empty, opens NetCDF file in parallel using this MPI_Comm
*/
Dnc_netcdf_file(PDI::Context& ctx, const std::string& filename, int rights_flag, PDI::Expression mpi_comm_expr);
Dnc_netcdf_file(PDI::Context& ctx, const std::string& filename, int right_flag, PDI::Expression mpi_comm_expr);

/// Deleted copy constructor
Dnc_netcdf_file(const Dnc_netcdf_file& other) = delete;
Expand Down Expand Up @@ -140,7 +140,7 @@ class Dnc_netcdf_file
/** Gets variable from the file
*
* \param variable variable to get
* \param write Dnc_io that deteremines the read operation
* \param read Dnc_io that deteremines the read operation
* \param ref_w reference where the data will be written
*/
void get_variable(const Dnc_variable& variable, const Dnc_io& read, PDI::Ref_w ref_w);
Expand All @@ -151,7 +151,7 @@ class Dnc_netcdf_file
* \param sizeof_variable variable to read its size
* \param ref reference where the data will be written
*/
void get_sizeof_variable(const std::string& variable, const std::string& sizeof_var, PDI::Ref ref);
void get_sizeof_variable(const std::string& variable, const std::string& sizeof_variable, PDI::Ref ref);

/// Destructor
~Dnc_netcdf_file();
Expand Down
1 change: 1 addition & 0 deletions plugins/decl_netcdf/dnc_variable.h
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ class Dnc_variable
/** Creates NetCDF variable information from yaml
*
* \param ctx Context of this variable
* \param path File name of the netCDF dataset
* \param config Configuration node of this variable
*/
Dnc_variable(PDI::Context& ctx, const std::string& path, PC_tree_t config);
Expand Down
31 changes: 16 additions & 15 deletions plugins/deisa/README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,31 @@
# The DEISA plugin {#deisa_plugin}
\page deisa_plugin Deisa plugin
# The DEISA plugin

**WARNING** This documentation is a work in progress and does not reflect the full DEISA plugin potential.

The DEISA plugin is used to expose data to [Dask](https://www.dask.org/).
This mechanism decouples analytics code from simulation code by providing an asynchronous pipeline for analytics.

The DEISA plugin uses the following configuration:
- `scheduler_info`: File name defined by the dask scheduler. This file contains information on how to connect to the dask cluster.
- `init_on`: Name of the PDI event called after sharing all metadata.
- `time_step`: Timestep variable.
- `deisa_arrays`: List of Deisa virtual arrays.
- `type`: Type of the Deisa array. Usually: `array`.
- `subtype`: Type stored in the `Deisa array.
- `size`: Size of the Deisa array.
- `subsize`: How the Deisa array is chunked. In other words, the size of each chunk.
- `start`: How to find the start of each chunk.
- `+timedim`: Define where the time dimension is. Currently, only dimension 0 is supported.
- `map_in`: Define how a local data is mapped to a Deisa array.
- 'scheduler_info': File name defined by the dask scheduler. This file contains information on how to connect to the dask cluster.
- 'init_on': Name of the PDI event called after sharing all metadata.
- 'time_step': Timestep variable.
- 'deisa_arrays': List of Deisa virtual arrays.
- 'type': Type of the Deisa array. Usually: 'array'.
- 'subtype': Type stored in the 'Deisa array.
- 'size': Size of the Deisa array.
- 'subsize': How the Deisa array is chunked. In other words, the size of each chunk.
- 'start': How to find the start of each chunk.
- '+timedim': Define where the time dimension is. Currently, only dimension 0 is supported.
- 'map_in': Define how a local data is mapped to a Deisa array.


---


# Example: A 2D MPI space with ghost cells.

```yaml
@code{.yaml}
metadata:
nx: int # Domain size per proc
ny: int # Domain size per proc
Expand All @@ -51,5 +52,5 @@ plugins:
start: [$nstep, '$mpi_coords_x*($nx+2)', '$mpi_coords_y*($ny+2)']
+timedim: 0
map_in:
main_field: global_t # `main_field` is mapped to `global_t`
```
main_field: global_t # 'main_field' is mapped to 'global_t'
@endcode
3 changes: 2 additions & 1 deletion plugins/json/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# JSON plugin {#JSON_plugin}
\page JSON_plugin JSON plugin
# JSON plugin

**WARNING** This documentation is a work in progress and does not
reflect the full potential of the JSON plugin.
Expand Down
3 changes: 2 additions & 1 deletion plugins/mpi/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# The MPI plugin {#mpi_plugin}
\page mpi_plugin MPI plugin
# The MPI plugin

**WARNING** This documentation is a work in progress and does not reflect the
full MPI plugin potential.
Expand Down
3 changes: 2 additions & 1 deletion plugins/pycall/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Pycall plugin {#pycall_plugin}
\page pycall_plugin Pycall plugin
# Pycall plugin

The Pycall plugin allows calling python scripts from yaml file, that can work either if wanted data was shared or program calls specific event.

Expand Down
3 changes: 2 additions & 1 deletion plugins/serialize/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Serialize plugin {#serialize_plugin}
\page serialize_plugin Serialize plugin
# Serialize plugin

Serialize plugin allows to serialize shared data. The plugin will convert all
arrays from sparse to dense and dereference all pointers.
Expand Down
3 changes: 2 additions & 1 deletion plugins/set_value/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Set_value plugin {#set_value_plugin}
\page set_value_plugin Set Value plugin
# Set_value plugin

The Set_value plugin allows setting values to data and metadata descriptors from
the yaml file.
Expand Down
1 change: 0 additions & 1 deletion plugins/set_value/operation.h
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ class Operation
public:
/** Creates operation
* \param[in] ctx context of the operation
* \param[in] release_value_node yaml config tree of operation
*/
Operation(PDI::Context& ctx);

Expand Down
3 changes: 2 additions & 1 deletion plugins/trace/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Trace plugin {#trace_plugin}
\page trace_plugin Trace plugin
# Trace plugin

The trace plugin is intended to generate a trace of what happens in
\ref Data_store "PDI data store".
Expand Down
3 changes: 2 additions & 1 deletion plugins/user_code/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# The user-code plugin {#user_code_plugin}
\page user_code_plugin user-code plugin
# The user-code plugin

The `user-code` plugin enables one to call a user-defined function
when a specified event occur or certain data becomes available.
Expand Down
Loading