DC data I/O
When working with DC data, you will inevitably run into the situation where you would like to write some part of dataset (be it an .rtdc file or data on DCOR) to a new file. Depending on the situation, one or more of the following subsection will probably cover what you need.
Exporting data
The RTDCBase
class has the attribute
RTDCBase.export
which allows to export event
data to several data file formats. See Export for more
information.
In [1]: import dclab
In [2]: ds = dclab.new_dataset("data/example.rtdc")
# Restrict the deformation to 0.15
In [3]: ds.config["filtering"]["deform min"] = 0
In [4]: ds.config["filtering"]["deform max"] = .15
In [5]: ds.apply_filter()
In [6]: print("Deformation mean before filtering:", ds["deform"][:].mean())
Deformation mean before filtering: 0.0287258
In [7]: print("Deformation mean after filtering:", ds["deform"][ds.filter.all].mean())
Deformation mean after filtering: 0.026480934
# Export to .tsv
In [8]: ds.export.tsv(path="export_example.tsv",
...: features=["area_um", "deform"],
...: filtered=True,
...: override=True)
...:
# Export to .rtdc
In [9]: ds.export.hdf5(path="export_example.rtdc",
...: features=["area_um", "aspect", "deform"],
...: filtered=True,
...: override=True)
...:
Note that data exported as HDF5 files can be loaded with dclab (reproducing the previously computed statistics - without filters).
In [10]: ds2 = dclab.new_dataset("export_example.rtdc")
In [11]: ds2["deform"][:].mean()
Out[11]: np.float32(0.026480934)
Writing to an .rtdc file
If you would like to create your own .rtdc files, you can
make use of the RTDCWriter
class.
In [12]: with dclab.RTDCWriter("my-data.rtdc", mode="reset") as hw:
....: hw.store_metadata({"experiment": {"sample": "my sample",
....: "run index": 1}})
....: hw.store_feature("deform", np.random.rand(100))
....: hw.store_feature("area_um", np.random.rand(100))
....:
In [13]: ds_custom = dclab.new_dataset("my-data.rtdc")
In [14]: print(ds_custom.features)
['area_um', 'deform', 'index']
In [15]: print(ds_custom.config["experiment"])
{'event count': 100, 'run index': 1, 'sample': 'my sample'}
Copying (parts of) a dataset
In some situations, you would only like to copy an entire feature column
from one dataset to a new file without modification. The copier
submodule enables this on a low-level.
Use the
rtdc_copy()
method to create a compressed version of a DC dataset opened as an HDF5 file (RTDC_HDF5
orRTDC_S3
).Use the
h5ds_copy()
method to copy parts of an HDF5 dataset to another HDF5 file, with the option to enforce compression (if the sourceh5py.Dataset
is not compressed properly already).