File Formats

MEDYAN.jl uses Zarr v2 zip store for saving snapshots.

Other Languages

To open the snapshots in python use zarr

trajectory output directory

Trajectory outputs and logs are stored in a directory.

The output directory contains a traj sub directory with a header.json file, many snap$(NUMBER).zarr.zip files where NUMBER is the frame number of the snapshot, and finally a footer.json.

The initial state returned by setup is stored in snap0.zarr.zip.

Inside the snap$(NUMBER).zarr.zip there is a snap/medyan group containing the snapshot of the MEDYAN.Context at the step. Other state may be stored in child groups of snap.

The header.json file also has a top level "medyan" key with a value describing the system being simulated.

Other header information may be under other top level keys.

See https://github.com/medyan-dev/MEDYANSimRunner.jl for more details on the output directory structure.

"medyan" Header JSON object.

header.json["medyan"] contains static metadata about the simulation that is constant over time.

Example header.json "medyan" value

using MEDYAN
import JSON3
cinit, s = MEDYAN.example_all_sites_context()
JSON3.pretty(MEDYAN.header(cinit); allow_inf = true)
{
    "version": "0.1.0",
    "medyanInfo": {
        "title": "MEDYAN.jl",
        "version": "0.1.0",
        "sourceCodeUrl": "https://github.com/medyan-dev/MEDYAN.jl"
    },
    "size": {
        "x(nm)": 2000,
        "y(nm)": 500,
        "z(nm)": 500
    },
    "chem_grid_size": {
        "nx": 4,
        "ny": 1,
        "nz": 1,
        "voxel_x(nm)": 500,
        "voxel_y(nm)": 500,
        "voxel_z(nm)": 500
    },
    "diffusing_species": [
        {
            "name": "b"
        },
        {
            "name": "c"
        }
    ],
    "membrane_diffusing_species": [
        {
            "name": "ma"
        },
        {
            "name": "mb"
        },
        {
            "name": "mc"
        }
    ],
    "fixed_species": [
        {
            "name": "d"
        },
        {
            "name": "a"
        }
    ],
    "filaments": [
        {
            "name": "a",
            "radius(nm)": 3,
            "monomerstates": [
                "me",
                "a",
                "b",
                "c",
                "pe"
            ]
        },
        {
            "name": "b",
            "radius(nm)": 3,
            "monomerstates": [
                "me",
                "a",
                "b",
                "c",
                "pe"
            ]
        }
    ],
    "link_2mons": [
        {
            "name": "a"
        },
        {
            "name": "b"
        },
        {
            "name": "c"
        },
        {
            "name": "d"
        }
    ]
}

Snapshot group

A snapshot describes the state of a Context at a single point in time.

A snapshot can be used to set the state of a Context if it was constructed with the same system and parameters as the Context used to create the snapshot. Trying to load a snapshot into a Context constructed with a different system or parameters may lead to unexpected results.

By default units are in nm, pN, and s. Coordinates are relative to the center of the grid.

The snapshot doesn’t hold the exact full state of a Context, simulations restarted from a snapshot may not have the exact same results because:

  1. The snapshot doesn’t store the state of the random number generator.
  2. Coordinates may be rounded to save disk space.
  3. Dictionaries and other internal data structures may get reordered and or resorted when the snapshot is loaded.
  4. Multithreading may be non deterministic.

But if rounding isn’t too extreme, the reloaded Context should have the same statistics.

Snapshot Versioning

The snapshot format is versioned by the “version” attribute. The current snapshot version is:

using MEDYAN
MEDYAN.SNAPSHOT_VERSION
v"0.8.0"

Currently before snapshot version 1.0.0 anything goes.

After snapshot version 1.0.0 is released.

Snapshots written with a previous snapshot version above v"1" should be readable. Snapshots written with a newer snapshot version are generally not readable.

If new agent types are added to MEDYAN.jl, usually only the minor version needs to be updated, as nothing special needs to be done to read older snapshot versions without that added agent type.

If new data is added to an existing agent type, also usually only the minor version needs to be increment, though the case of the new data not existing must be handled with some default.

If the way an existing agent type is stored significantly changes, such that external code analyzing the snapshot would need to be modified, the major version must be incremented. Also if possible there should be a function to update a snapshot from the old version to the new version.

“#experimental” and “#comment” prefixes

Any group, dataset, or attribute name prefixed with with “#” can change format or be removed without changing the snapshot version.

“#experimental” is used for saving new types of agents or other data that doesn’t have a stable format yet.

“#comment” is used for saving human readable comments that could change in format or wording.

Example snapshot

using MEDYAN
using SmallZarrGroups
cinit, s = MEDYAN.example_all_sites_context()
group = MEDYAN.snapshot(cinit)
Snapshot 📂
🏷️
time (s) 
attrs(group)["time (s)"]
0.0
Default: No change
See MEDYAN.set_time!
version 
attrs(group)["version"]
"0.8.0"
Default: No change
See Snapshot Versioning
uuid 
attrs(group)["uuid"]
"37eee81f-88ae-4d11-b6b3-d38e1ccf0a08"
Must be set to exactly “37eee81f-88ae-4d11-b6b3-d38e1ccf0a08”
to be considered a valid MEDYAN snapshot.
🔢
diffusingcounts 
collect(group["diffusingcounts"])
2×4 Matrix{Int32}:
 0   100   40  0
 0  1000  400  0

Default: Empty

Indexed by [species id, compartment id] to give count.

See MEDYAN.chem_adddiffusingcount!
fixedcounts 
collect(group["fixedcounts"])
2×4 Matrix{Float64}:
 10.25  0.0  0.0   0.0
  0.0   0.0  0.0  20.25

Default: Empty

Indexed by [species id, compartment id] to give count.

See MEDYAN.chem_addfixedcount!
📂
chemboundary

There is a dataset for each type of boundary, the total boundary is an intersection of the following.

See MEDYAN.Boundary

🔢 capsules 
collect(group["chemboundary/capsules"])
8×1 Matrix{Float64}:
     0.0
     0.0
     0.0
     0.0
     0.0
     0.0
 10000.0
     1.0

Default: Empty

Each column of the dataset is a capsule.
🔢 planes 
collect(group["chemboundary/planes"])
4×6 Matrix{Float64}:
   -10.0     10.0     0.0     0.0     0.0     0.0
     0.0      0.0   -10.0    10.0     0.0     0.0
     0.0      0.0     0.0     0.0   -10.0    10.0
 10000.0  10000.0  2500.0  2500.0  2500.0  2500.0

Default: Empty

Each column of the dataset is a plane.
filaments
🏷️
position_scale 
attrs(group["filaments"])["position_scale"]
3

Default: No effect

Filament positions were rounded to the nearest 2^-position_scale nm when saved.

There is a subgroup for each filament type with at least one filament. The subgroups are named by their filament type id. For example:

📂 1
🔢 endloadforces 
collect(group["filaments/1/endloadforces"])
2×2 Matrix{Float64}:
 0.0  0.0
 0.0  0.0

Filament end load forces (pN).

Each row is the load force on the minus and plus ends of a filament.
🔢 fids 
collect(group["filaments/1/fids"])
2-element Vector{Int32}:
 1
 2
Filament ids.
🔢 minusend_num_notminimized 
collect(group["filaments/1/minusend_num_notminimized"])
2-element Vector{Int32}:
 0
 0
Number of newly added monomers to the filament minus ends since last minimization.
🔢 plusend_num_notminimized 
collect(group["filaments/1/plusend_num_notminimized"])
2-element Vector{Int32}:
 0
 0
Number of newly added monomers to the filament plus ends since last minimization.
🔢 num_cylinders 
collect(group["filaments/1/num_cylinders"])
2-element Vector{Int32}:
 1
 1
Number of cylinders per filament.
🔢 num_monomers 
collect(group["filaments/1/num_monomers"])
2-element Vector{Int32}:
 9
 9
Number of monomers per filament.
🔢 node_mids 
collect(group["filaments/1/node_mids"])
2-element Vector{Int32}:
 1
 1

The monomer ids at the minus ends of the cylinders.

                                   |
                        -----+-----|-----+-----
    minus end <----       M  |  M  | (M) |  M        ----> plus end
                        -----+-----|-----+-----
                                   |
                                   ^ A nodeposition is indicated by the line.
The monomer id with parenthesis (M) will in node_mids
🔢 nodepositions 
collect(group["filaments/1/nodepositions"])
4×3 Matrix{Float64}:
 -530.0  -50.0  -50.0
 -440.0  -50.0  -50.0
 -520.0  -49.0  -50.0
 -430.0  -49.0  -50.0
Each row is a node position in nm.
🔢 monomerstates 
collect(group["filaments/1/monomerstates"])
18-element Vector{UInt8}:
 0x01
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x05
 0x01
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x05
Monomer states.
link_2mons

There is a subgroup for each link_2mon type with at least one link_2mon. The subgroups are named by their link_2mon type id. For example:

📂 1
🏷️
num_link_2mons 
attrs(group["link_2mons/1"])["num_link_2mons"]
2
Number of link_2mons of this type.
next_lid 
attrs(group["link_2mons/1"])["next_lid"]
3

The default next link id for new links.

This should be greater than all the existing lids.
🔢 endnames 
collect(group["link_2mons/1/endnames"])
2×6 Matrix{Int64}:
 1  1  2  1  1  2
 1  1  2  1  2  2

The pairs of monomers that the link_2mons are linked between. The monomer pairs are directed and unique.

Each row of this matrix represents two monomers. The 6 integers per row are in order:

  1. Minus end filament type id.
  2. Minus end filament id.
  3. Minus end monomer id.
  4. Plus end filament type id.
  5. Plus end filament id.
  6. Plus end monomer id.
🔢 lids 
collect(group["link_2mons/1/lids"])
2-element Vector{Int64}:
 1
 2

The link_2mon ids.

Links retain their ids while they remain bound.
📂 state

Default: default link_2mon_state

See MEDYAN.chem_setlink_2mon_state! and MEDYAN.Link2MonState

The state of a link_2mon, organized in a nested struct of vector like form. Any static arrays will be unwrapped into a tuple of vectors, in column major order. The “name” attribute of each subgroup and dataset is the corresponding property name in the StructArray Any property in the default state that isn’t in the snapshot will stay at its default value.

group["link_2mons/1/state"]
📂
├─ 📂 1 🏷️ name => "chemstate",
|  ├─ 🔢 1: 2 Int64  🏷️ name => "a",
|  └─ 🔢 2: 2 Float64  🏷️ name => "b",
├─ 📂 2 🏷️ name => "mechstate",
|  └─ 📂 1 🏷️ name => "â",
|     ├─ 🔢 1: 2 Float32  🏷️ name => "1",
|     ├─ 🔢 2: 2 Float32  🏷️ name => "2",
|     └─ 🔢 3: 2 Float32  🏷️ name => "3",
└─ 🔢 3: 2 Bool  🏷️ name => "is_minimized",
mechboundary

There is a dataset for each type of boundary, the total boundary is an intersection of the following.

See MEDYAN.Boundary

🔢 capsules 
collect(group["mechboundary/capsules"])
8×1 Matrix{Float64}:
     0.0
     0.0
     0.0
     0.0
     0.0
     0.0
 10000.0
     1.0

Default: Empty

Each column of the dataset is a capsule.
🔢 planes 
collect(group["mechboundary/planes"])
4×6 Matrix{Float64}:
   -10.0     10.0     0.0     0.0     0.0     0.0
     0.0      0.0   -10.0    10.0     0.0     0.0
     0.0      0.0     0.0     0.0   -10.0    10.0
 10000.0  10000.0  2500.0  2500.0  2500.0  2500.0

Default: Empty

Each column of the dataset is a plane.
membranes
🏷️
num_membranes 
attrs(group["membranes"])["num_membranes"]
1
Total number of membranes.
position_scale 
attrs(group["membranes"])["position_scale"]
3

Default: No effect

Positions were rounded to the nearest 2^-position_scale nm when saved.

There is a subgroup for each membrane. The subgroups are named by the membrane index. For example:

📂 1
🏷️
type_id 
attrs(group["membranes/1"])["type_id"]
1
This membrane’s type id.
🔢 trilist 
collect(group["membranes/1/trilist"])
3×1914 Matrix{Int64}:
   2  871  836  517  520  462  409  416  …  792  688  226  201  450  326  164
 848  807  732  412  362  635  656  845     913  354  652  920  923  956  781
 238  630  203  190  623  853  733  608     875  944  373  958  958  959  959
Each column is the 3 vertex indexes of a triangle. Indexes are one based, and follow the right hand rule. Looking at the triangle from the outside in, they have counterclockwise winding.
🔢 vertlist 
collect(group["membranes/1/vertlist"])
3×959 Matrix{Float64}:
  115.0   -590.25   -552.0    226.75  …  182.375  -316.125   157.75
 -222.75  -232.875  -193.125  137.5      238.125  -429.875   -34.5
 -252.25   275.25   -343.25   -15.5      -49.125  -237.875  -279.875
Each column is a vertex coordinate in nm.
🔢 id 
collect(group["membranes/1/id"])
959-element Vector{Int64}:
  7814
 10608
  3334
 14756
 12541
  9498
 10903
  2092
  9775
    10
 12855
 12136
  1949
     ⋮
  3653
  5295
  3951
  8003
 14307
 12258
  7204
  4182
  7834
  9058
  6485
 10738
Array of vertex IDs. Each vertex has an id which is assigned upon initialization and persists throughout its lifetime.
optional 🔢 copynumbers 
collect(group["membranes/1/copynumbers"])
3×959 Matrix{Int64}:
 0  0  0  0  0  0  0  0  0  0  0  0  0  …  0  0  0  0  0  0  0  0  0  0  0  0
 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
Array of vertex membrane species copynumbers. If the membrane has no defined species this dataset will not exist.