File Formats

MEDYAN.jl uses Zarr v2 zip store for saving snapshots.

Other Languages

To open the snapshots in python use zarr

trajectory output directory

Trajectory outputs and logs are stored in a directory.

The output directory contains a traj sub directory with a header.json file, many snap$(NUMBER).zarr.zip files where NUMBER is the frame number of the snapshot, and finally a footer.json.

The initial state returned by setup is stored in snap0.zarr.zip.

Inside the snap$(NUMBER).zarr.zip there is a snap/medyan group containing the snapshot of the MEDYAN.Context at the step. Other state may be stored in child groups of snap.

The header.json file also has a top level "medyan" key with a value describing the system being simulated.

Other header information may be under other top level keys.

See https://github.com/medyan-dev/MEDYANSimRunner.jl for more details on the output directory structure.

"medyan" Header JSON object.

header.json["medyan"] contains static metadata about the simulation.

Example header.json "medyan" value

using MEDYAN
import JSON3
cinit, s = MEDYAN.example_all_sites_context()
JSON3.pretty(MEDYAN.header(cinit); allow_inf = true)

{
    "version": "0.11.0",
    "medyanInfo": {
        "title": "MEDYAN.jl",
        "version": "0.2.1",
        "sourceCodeUrl": "https://github.com/medyan-dev/MEDYAN.jl"
    },
    "size": {
        "x(nm)": 2000,
        "y(nm)": 500,
        "z(nm)": 500
    },
    "chem_grid_size": {
        "nx": 4,
        "ny": 1,
        "nz": 1,
        "voxel_x(nm)": 500,
        "voxel_y(nm)": 500,
        "voxel_z(nm)": 500
    },
    "diffusing_species": [
        {
            "name": "b"
        },
        {
            "name": "c"
        }
    ],
    "membrane_diffusing_species": [
        {
            "name": "ma"
        },
        {
            "name": "mb"
        },
        {
            "name": "mc"
        }
    ],
    "fixed_species": [
        {
            "name": "d"
        },
        {
            "name": "a"
        }
    ],
    "filaments": [
        {
            "name": "a",
            "radius(nm)": 3,
            "monomerstates": [
                "me",
                "a",
                "b",
                "c",
                "pe"
            ]
        },
        {
            "name": "b",
            "radius(nm)": 3,
            "monomerstates": [
                "me",
                "a",
                "b",
                "c",
                "pe"
            ]
        }
    ],
    "links": [
        {
            "name": "fila_tip_restraint",
            "places": [
                "fila_tip"
            ]
        },
        {
            "name": "fila_mono_restraint",
            "places": [
                "fila_mono"
            ]
        },
        {
            "name": "fila_mono_dummy",
            "places": [
                "fila_mono"
            ]
        },
        {
            "name": "fila_mono_2bonds",
            "places": [
                "fila_mono"
            ]
        },
        {
            "name": "fila_mono_distance_bond",
            "places": [
                "fila_mono",
                "fila_mono"
            ]
        },
        {
            "name": "a",
            "places": [
                "fila_mono",
                "fila_mono"
            ]
        },
        {
            "name": "b",
            "places": [
                "fila_mono",
                "fila_mono"
            ]
        },
        {
            "name": "c",
            "places": [
                "fila_mono",
                "fila_mono"
            ]
        },
        {
            "name": "memb_vert_restraint",
            "places": [
                "memb_vert"
            ]
        }
    ]
}

Snapshot group

A snapshot describes the state of a Context at a single point in time.

A snapshot can be used to set the state of a Context if it was constructed with the same system and parameters as the Context used to create the snapshot. Trying to load a snapshot into a Context constructed with a different system or parameters may lead to unexpected results.

By default units are in nm, pN, and s. Coordinates are relative to the center of the grid.

The snapshot doesn’t hold the exact full state of a Context, simulations restarted from a snapshot may not have the exact same results because:

1. The snapshot doesn’t store the state of the random number generator.
2. Coordinates may be rounded to save disk space.
3. Dictionaries and other internal data structures may get reordered and or resorted when the snapshot is loaded.
4. Multithreading may be non deterministic.

But if rounding isn’t too extreme, the reloaded Context should have the same statistics.

Snapshot Versioning

The snapshot format is versioned by the “version” attribute. The current snapshot version is:

using MEDYAN
MEDYAN.SNAPSHOT_VERSION

v"0.11.0"

Currently before snapshot version 1.0.0 anything goes.

After snapshot version 1.0.0 is released.

Snapshots written with a previous snapshot version above v"1" should be readable. Snapshots written with a newer snapshot version are generally not readable.

If new agent types are added to MEDYAN.jl, usually only the minor version needs to be updated, as nothing special needs to be done to read older snapshot versions without that added agent type.

If new data is added to an existing agent type, also usually only the minor version needs to be increment, though the case of the new data not existing must be handled with some default.

If the way an existing agent type is stored significantly changes, such that external code analyzing the snapshot would need to be modified, the major version must be incremented. Also if possible there should be a function to update a snapshot from the old version to the new version.

“#experimental” and “#comment” prefixes

Any group, dataset, or attribute name prefixed with with “#” can change format or be removed without changing the snapshot version.

“#experimental” is used for saving new types of agents or other data that doesn’t have a stable format yet.

“#comment” is used for saving human readable comments that could change in format or wording.

Example snapshot

using MEDYAN
using SmallZarrGroups
cinit, s = MEDYAN.example_all_sites_context()
group = MEDYAN.snapshot(cinit)

Snapshot 📂

🏷️

time (s)

attrs(group)["time (s)"]

0.0

Default: No change
See MEDYAN.set_time!

version

attrs(group)["version"]

"0.11.0"

Default: No change
See Snapshot Versioning

uuid

attrs(group)["uuid"]

"37eee81f-88ae-4d11-b6b3-d38e1ccf0a08"

Must be set to exactly “37eee81f-88ae-4d11-b6b3-d38e1ccf0a08”
to be considered a valid MEDYAN snapshot.

🔢

diffusingcounts

collect(group["diffusingcounts"])

2×4 Matrix{Int32}:
 0   100   40  0
 0  1000  400  0

Default: Empty

Indexed by [species id, compartment id] to give count.

See MEDYAN.chem_adddiffusingcount!

fixedcounts

collect(group["fixedcounts"])

2×4 Matrix{Float64}:
 10.25  0.0  0.0   0.0
  0.0   0.0  0.0  20.25

Default: Empty

Indexed by [species id, compartment id] to give count.

See MEDYAN.chem_addfixedcount!

📂

chemboundary

There is a dataset for each type of boundary, the total boundary is an intersection of the following.

See MEDYAN.Boundary

🔢 capsules

collect(group["chemboundary/capsules"])

8×1 Matrix{Float64}:
     0.0
     0.0
     0.0
     0.0
     0.0
     0.0
 10000.0
     1.0

Default: Empty

Each column of the dataset is a capsule.

🔢 planes

collect(group["chemboundary/planes"])

4×6 Matrix{Float64}:
   -10.0     10.0     0.0     0.0     0.0     0.0
     0.0      0.0   -10.0    10.0     0.0     0.0
     0.0      0.0     0.0     0.0   -10.0    10.0
 10000.0  10000.0  2500.0  2500.0  2500.0  2500.0

Default: Empty

Each column of the dataset is a plane.

filaments

🏷️

position_scale

attrs(group["filaments"])["position_scale"]

3

Default: No effect

Filament positions were rounded to the nearest 2^-position_scale nm when saved.

There is a subgroup for each filament type with at least one filament. The subgroups are named by their filament type id. For example:

📂 1

🔢 endloadforces

collect(group["filaments/1/endloadforces"])

2×2 Matrix{Float64}:
 0.0  0.0
 0.0  0.0

Filament end load forces (pN).

Each row is the load force on the minus and plus ends of a filament.

🔢 fids

collect(group["filaments/1/fids"])

2-element Vector{Int32}:
 1
 2

Filament ids.

🔢 minusend_num_notminimized

collect(group["filaments/1/minusend_num_notminimized"])

2-element Vector{Int32}:
 0
 0

Number of newly added monomers to the filament minus ends since last minimization.

🔢 plusend_num_notminimized

collect(group["filaments/1/plusend_num_notminimized"])

2-element Vector{Int32}:
 0
 0

Number of newly added monomers to the filament plus ends since last minimization.

🔢 num_cylinders

collect(group["filaments/1/num_cylinders"])

2-element Vector{Int32}:
 1
 1

Number of cylinders per filament.

🔢 num_monomers

collect(group["filaments/1/num_monomers"])

2-element Vector{Int32}:
 9
 9

Number of monomers per filament.

🔢 node_mids

collect(group["filaments/1/node_mids"])

2-element Vector{Int32}:
 1
 1

The monomer ids at the minus ends of the cylinders.

                                   |
                        -----+-----|-----+-----
    minus end <----       M  |  M  | (M) |  M        ----> plus end
                        -----+-----|-----+-----
                                   |
                                   ^ A nodeposition is indicated by the line.

The monomer id with parenthesis (M) will in node_mids

🔢 nodepositions

collect(group["filaments/1/nodepositions"])

4×3 Matrix{Float64}:
 -530.0  -50.0  -50.0
 -440.0  -50.0  -50.0
 -520.0  -49.0  -50.0
 -430.0  -49.0  -50.0

Each row is a node position in nm.

🔢 monomerstates

collect(group["filaments/1/monomerstates"])

18-element Vector{UInt8}:
 0x01
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x05
 0x01
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x02
 0x05

Monomer states.

links

There is a subgroup for each link type with at least one current or past link. The subgroups are named by their link type id.

🏷️

version

attrs(group["links"])["version"]

1

link data major version, version 1 is described here.

For example:

📂 8

🏷️

num_links

attrs(group["links/8"])["num_links"]

4

Number of links of this type.

next_lid

attrs(group["links/8"])["next_lid"]

5

The default next link id for new links.

This should be greater than all the existing link ids of this type.

🔢 be

collect(group["links/8/be"])

4×1 Matrix{Bool}:
 1
 1
 1
 1

The bonds that are enabled.

Each row of this matrix represents a link, each column represents a bond.

🔢 ids

collect(group["links/8/ids"])

4-element Vector{Int64}:
 1
 2
 3
 4

The link ids.

🔢 im

collect(group["links/8/im"])

4-element Vector{Bool}:
 0
 0
 0
 1

Is the link minimized. A link is marked as not minimized when created, and then marked as minimized after running mechanics.

🔢 re

collect(group["links/8/re"])

4×6 Matrix{Bool}:
 1  1  1  1  1  1
 1  1  1  1  1  1
 1  1  1  1  1  1
 1  1  1  1  1  1

The reactions that are enabled.

Each row of this matrix represents a link, each column represents a reaction.

🔢 tags

collect(group["links/8/tags"])

4×4 Matrix{UInt32}:
 0x00000001  0x00000001  0x00000001  0x00000001
 0x00000005  0x00000001  0x00000006  0x00000001
 0x00000000  0x00000000  0x00000006  0x00000001
 0x00000005  0x00000001  0x00000001  0x00000001

The tags of the places the links are attached to.

Each row of this matrix represents a link.

Each place has two adjacent columns. First a index into the tags, and next a tag generation. The place type can be found in the header file. If the place is not attached, the generation and index will be zero.

📂 bs

Default: default bond states

See MEDYAN.update_link!.

The state of link bonds, organized in a nested struct of vector like form. Any static arrays will be unwrapped into a tuple of vectors, in column major order. The “name” attribute of each subgroup and dataset is the corresponding property name in the StructArray Any property in the default state that isn’t in the snapshot will stay at its default value.

group["links/8/bs"]

📂
└─ 📂 1 🏷️ name => "1",
   ├─ 🔢 1: 4 Float64  🏷️ name => "k",
   └─ 🔢 2: 4 Float64  🏷️ name => "L0",

📂 s

Default: default states

See MEDYAN.update_link!.

The state of link, organized in a nested struct of vector like form. Any static arrays will be unwrapped into a tuple of vectors, in column major order. The “name” attribute of each subgroup and dataset is the corresponding property name in the StructArray Any property in the default state that isn’t in the snapshot will stay at its default value.

group["links/8/s"]

📂
├─ 🔢 1: 4 Int64  🏷️ name => "a",
└─ 🔢 2: 4 Float64  🏷️ name => "b",

mechboundary

There is a dataset for each type of boundary, the total boundary is an intersection of the following.

See MEDYAN.Boundary

🔢 capsules

collect(group["mechboundary/capsules"])

8×1 Matrix{Float64}:
     0.0
     0.0
     0.0
     0.0
     0.0
     0.0
 10000.0
     1.0

Default: Empty

Each column of the dataset is a capsule.

🔢 planes

collect(group["mechboundary/planes"])

4×6 Matrix{Float64}:
   -10.0     10.0     0.0     0.0     0.0     0.0
     0.0      0.0   -10.0    10.0     0.0     0.0
     0.0      0.0     0.0     0.0   -10.0    10.0
 10000.0  10000.0  2500.0  2500.0  2500.0  2500.0

Default: Empty

Each column of the dataset is a plane.

membranes

🏷️

num_membranes

attrs(group["membranes"])["num_membranes"]

1

Total number of membranes.

position_scale

attrs(group["membranes"])["position_scale"]

3

Default: No effect

Positions were rounded to the nearest 2^-position_scale nm when saved.

There is a subgroup for each membrane. The subgroups are named by the membrane index. For example:

📂 1

🏷️

typeid

attrs(group["membranes/1"])["typeid"]

1

This membrane’s type id.

🔢 trilist

collect(group["membranes/1/trilist"])

3×1954 Matrix{Int64}:
 490  680  317  602  572  565  448  286  …  688  972  447  552   53  635  354
  42  418  518  398  288  279   82  624     166  581  460  293  898   49  397
 182  711  808  792  978  958  501  172     371  216  343  355  782  979  979

Each column is the 3 vertex indexes of a triangle. Indexes are one based, and follow the right hand rule. Looking at the triangle from the outside in, they have counterclockwise winding.

🔢 vertlist

collect(group["membranes/1/vertlist"])

3×979 Matrix{Float64}:
 -395.375  228.25   -631.625    32.125  …  -566.0   -344.5    -556.875
 -266.125  124.875   234.0    -184.875      367.25   205.375   341.5
  393.25    38.5     212.0    -364.0        103.25  -441.625  -189.0

Each column is a vertex coordinate in nm.

🔢 id

collect(group["membranes/1/id"])

979-element Vector{Int64}:
  4702
 14762
 10420
 11066
  6022
  2495
  1262
  5013
 11796
    10
     ⋮
 12731
  5861
 10661
   974
  1224
  5415
  3107
  9714
 12351

Array of vertex IDs. Each vertex has an id which is assigned upon initialization and persists throughout its lifetime.

optional 🔢 copynumbers

collect(group["membranes/1/copynumbers"])

3×979 Matrix{Int64}:
 0  0  0  0  0  0  0  0  0  0  0  0  0  …  0  0  0  0  0  0  0  0  0  0  0  0
 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0

Array of vertex membrane species copynumbers. If the membrane has no defined species this dataset will not exist.

Tags

There is a subgroup for each tag type with at least one current or past tag. Each tag type stores tag generations as a vector of UInt32 in g and packed places in p_n where n is the packed format version. If the generation is odd the tag is valid. otherwise the tag is an empty slot à la https://docs.rs/slotmap/latest/slotmap/.

📂 fila_mono

🔢 p_1

collect(group["tags/fila_mono/p_1"])

16-element Vector{UInt64}:
 0x0000000100000002
 0x0000000100000004
 0x0000000200000002
 0x0000000200000002
 0x0000000100000002
 0x0000000100000002
 0x0000000200000004
 0x0000000200000004
 0x0000000000000004
 0x0000000000000004
 0x0000000000000008
 0x000000000000000a
 0x0000000000000002
 0x0000000000000012
 0x0000000000000002
 0x0000000000000012

The filament monomer places for each tag stored as a vector of UInt64. Not all monomers will have tags. For tag i of n total tags, p_1[i]>>>32 is the filament type id, (p_1[i]%UInt32)>>1 is the filament index, and p_1[i+n] is the zigzag encoded monomer index.

📂 fila_tip

🔢 p_1

collect(group["tags/fila_tip/p_1"])

8-element Vector{UInt64}:
 0x0000000100000003
 0x0000000100000002
 0x0000000100000005
 0x0000000100000004
 0x0000000200000003
 0x0000000200000002
 0x0000000200000005
 0x0000000200000004

The filament tip places for each tag stored as a vector of UInt64. All filament tips will have tags. For tag i of n total tags, p_1[i]>>>32 is the filament type id, (p_1[i]%UInt32)>>1 is the filament index, and p_1[i] is odd for the minus end of the filament and even for the plus end.