Notation

When coding with dclab, you should be aware of the following definitions and design principles.

Events

An event comprises all data recorded for the detection of one object (e.g. cell or bead) in an RT-DC measurement.

Features

A feature is a measurement parameter of an RT-DC measurement. For instance, the feature “index” enumerates all recorded events, the feature “deform” contains the deformation values of all events. There are scalar features, i.e. features that assign a single number to an event, and non-scalar features, such as “image” and “contour”. The following features are supported by dclab:

Scalar features

scalar features

description [units]

area_cvx

Convex area [px]

area_msd

Measured area [px]

area_ratio

Porosity (convex to measured area ratio)

area_um

Area [µm²]

aspect

Aspect ratio of bounding box

bg_med

Median frame background brightness [a.u.]

bright_avg

Brightness average [a.u.]

bright_bc_avg

Brightness average (bgc) [a.u.]

bright_bc_sd

Brightness SD (bgc) [a.u.]

bright_perc_10

10th Percentile of brightness (bgc)

bright_perc_90

90th Percentile of brightness (bgc)

bright_sd

Brightness SD [a.u.]

circ

Circularity

deform

Deformation

emodulus

Young’s modulus [kPa]

fl1_area

FL-1 area of peak [a.u.]

fl1_dist

FL-1 distance between two first peaks [µs]

fl1_max

FL-1 maximum [a.u.]

fl1_max_ctc

FL-1 maximum, crosstalk-corrected [a.u.]

fl1_npeaks

FL-1 number of peaks

fl1_pos

FL-1 position of peak [µs]

fl1_width

FL-1 width [µs]

fl2_area

FL-2 area of peak [a.u.]

fl2_dist

FL-2 distance between two first peaks [µs]

fl2_max

FL-2 maximum [a.u.]

fl2_max_ctc

FL-2 maximum, crosstalk-corrected [a.u.]

fl2_npeaks

FL-2 number of peaks

fl2_pos

FL-2 position of peak [µs]

fl2_width

FL-2 width [µs]

fl3_area

FL-3 area of peak [a.u.]

fl3_dist

FL-3 distance between two first peaks [µs]

fl3_max

FL-3 maximum [a.u.]

fl3_max_ctc

FL-3 maximum, crosstalk-corrected [a.u.]

fl3_npeaks

FL-3 number of peaks

fl3_pos

FL-3 position of peak [µs]

fl3_width

FL-3 width [µs]

flow_rate

Flow rate [µLs⁻¹]

frame

Video frame number

g_force

Gravitational force in multiples of g

index

Index (Dataset)

index_online

Index (Online)

inert_ratio_cvx

Inertia ratio of convex contour

inert_ratio_prnc

Principal inertia ratio of raw contour

inert_ratio_raw

Inertia ratio of raw contour

ml_class

Most probable ML class

nevents

Number of events in the same image

pc1

Principal component 1

pc2

Principal component 2

pos_x

Position along channel axis [µm]

pos_y

Position lateral in channel [µm]

pressure

Pressure [mPa]

size_x

Bounding box size x [µm]

size_y

Bounding box size y [µm]

temp

Chip temperature [°C]

temp_amb

Ambient temperature [°C]

tex_asm_avg

Texture angular second moment (avg)

tex_asm_ptp

Texture angular second moment (ptp)

tex_con_avg

Texture contrast (avg)

tex_con_ptp

Texture contrast (ptp)

tex_cor_avg

Texture correlation (avg)

tex_cor_ptp

Texture correlation (ptp)

tex_den_avg

Texture difference entropy (avg)

tex_den_ptp

Texture difference entropy (ptp)

tex_ent_avg

Texture entropy (avg)

tex_ent_ptp

Texture entropy (ptp)

tex_f12_avg

Texture First measure of correlation (avg)

tex_f12_ptp

Texture First measure of correlation (ptp)

tex_f13_avg

Texture Second measure of correlation (avg)

tex_f13_ptp

Texture Second measure of correlation (ptp)

tex_idm_avg

Texture inverse difference moment (avg)

tex_idm_ptp

Texture inverse difference moment (ptp)

tex_sen_avg

Texture sum entropy (avg)

tex_sen_ptp

Texture sum entropy (ptp)

tex_sva_avg

Texture sum variance (avg)

tex_sva_ptp

Texture sum variance (ptp)

tex_var_avg

Texture variance (avg)

tex_var_ptp

Texture variance (ptp)

tilt

Absolute tilt of raw contour

time

Time [s]

userdef0

User-defined 0

userdef1

User-defined 1

userdef2

User-defined 2

userdef3

User-defined 3

userdef4

User-defined 4

userdef5

User-defined 5

userdef6

User-defined 6

userdef7

User-defined 7

userdef8

User-defined 8

userdef9

User-defined 9

volume

Volume [µm³]

In addition to these scalar features, it is possible to define a large number of features dedicated to machine-learning, the “ml_score_???” features: The “?” can be a digit or a lower-case letter of the alphabet, e.g. “ml_score_rbc” or “ml_score_3a3”. If “ml_score_???” features are defined, then the ancillary “ml_class” feature, which identifies the most-probable feature for each event, becomes available.

Non-scalar features

non-scalar features

description [units]

contour

Event contour

image

Gray scale event image

image_bg

Gray scale event background image

mask

Binary mask labeling the event in the image

trace

Dictionary of fluorescence traces

Examples

deformation vs. area plot

import matplotlib.pylab as plt
import dclab
ds = dclab.new_dataset("data/example.rtdc")
ax = plt.subplot(111)
ax.plot(ds["area_um"], ds["deform"], "o", alpha=.2)
ax.set_xlabel(dclab.dfn.get_feature_label("area_um"))
ax.set_ylabel(dclab.dfn.get_feature_label("deform"))
plt.show()

(Source code, png, hires.png, pdf)

_images/sec_av_notation-1.png

event image plot

import matplotlib.pylab as plt
import dclab
ds = dclab.new_dataset("data/example_video.rtdc")
ax1 = plt.subplot(211, title="image")
ax2 = plt.subplot(212, title="mask")
ax1.imshow(ds["image"][6], cmap="gray")
ax2.imshow(ds["mask"][6])

(Source code, png, hires.png, pdf)

_images/sec_av_notation-2.png

Ancillary features

Not all features available in dclab are recorded online during the acquisition of the experimental dataset. Some of the features are computed offline by dclab, such as “volume”, “emodulus”, or scores from imported machine learning models (“ml_score_xxx”). These ancillary features are computed on-the-fly and are made available seamlessly through the same interface.

Filters

A filter can be used to gate events using features. There are min/max filters and 2D polygon filters. The following table defines the main filtering parameters:

filtering

parsed

description [units]

enable filters

{f}

Enable filtering

hierarchy parent

str

Hierarchy parent of the dataset

limit events

{f}

Upper limit for number of filtered events

polygon filters

{f}

Polygon filter indices

remove invalid events

{f}

Remove events with inf/nan values

Min/max filters are also defined in the filters section:

filtering

explanation

area_um min

Exclude events with area [µm²] below this value

area_um max

Exclude events with area [µm²] above this value

aspect max

Exclude events with an aspect ratio above this value

Examples

excluding events with large deformation

import matplotlib.pylab as plt
import dclab
ds = dclab.new_dataset("data/example.rtdc")

ds.config["filtering"]["deform min"] = 0
ds.config["filtering"]["deform max"] = .1
ds.apply_filter()
dif = ds.filter.all

f, axes = plt.subplots(1, 2, sharex=True, sharey=True)
axes[0].plot(ds["area_um"], ds["bright_avg"], "o", alpha=.2)
axes[0].set_title("unfiltered")
axes[1].plot(ds["area_um"][dif], ds["bright_avg"][dif], "o", alpha=.2)
axes[1].set_title("Deformation <= 0.1")

for ax in axes:
    ax.set_xlabel(dclab.dfn.get_feature_label("area_um"))
    ax.set_ylabel(dclab.dfn.get_feature_label("bright_avg"))

plt.tight_layout()
plt.show()

(Source code, png, hires.png, pdf)

_images/sec_av_notation-3.png

excluding random events

This is useful if you need to have a (sub-)dataset of a specified size. The downsampling is reproducible (the same points are excluded).

import matplotlib.pylab as plt
import dclab
ds = dclab.new_dataset("data/example.rtdc")
ds.config["filtering"]["limit events"] = 4000
ds.apply_filter()
fid = ds.filter.all

ax = plt.subplot(111)
ax.plot(ds["area_um"][fid], ds["deform"][fid], "o", alpha=.2)
ax.set_xlabel(dclab.dfn.get_feature_label("area_um"))
ax.set_ylabel(dclab.dfn.get_feature_label("deform"))
plt.show()

(Source code, png, hires.png, pdf)

_images/sec_av_notation-4.png

Experiment metadata

Every RT-DC measurement has metadata consisting of key-value-pairs. The following are supported:

experiment

parsed

description [units]

date

str

Date of measurement (‘YYYY-MM-DD’)

event count

{f}

Number of recorded events

run index

{f}

Index of measurement run

sample

str

Measured sample or user-defined reference

time

str

Start time of measurement (‘HH:MM:SS[.S]’)

fluorescence

parsed

description [units]

baseline 1 offset

{f}

Baseline offset channel 1

baseline 2 offset

{f}

Baseline offset channel 2

baseline 3 offset

{f}

Baseline offset channel 3

bit depth

{f}

Trace bit depth

channel 1 name

str

FL1 description

channel 2 name

str

FL2 description

channel 3 name

str

FL3 description

channel count

{f}

Number of active channels

channels installed

{f}

Number of available channels

laser 1 lambda

float

Laser 1 wavelength [nm]

laser 1 power

float

Laser 1 output power [%]

laser 2 lambda

float

Laser 2 wavelength [nm]

laser 2 power

float

Laser 2 output power [%]

laser 3 lambda

float

Laser 3 wavelength [nm]

laser 3 power

float

Laser 3 output power [%]

laser count

{f}

Number of active lasers

lasers installed

{f}

Number of available lasers

sample rate

{f}

Trace sample rate [Hz]

samples per event

{f}

Samples per event

signal max

float

Upper voltage detection limit [V]

signal min

float

Lower voltage detection limit [V]

trace median

{f}

Rolling median filter size for traces

fmt_tdms

parsed

description [units]

video frame offset

{f}

Missing events at beginning of video

imaging

parsed

description [units]

flash device

str

Light source device type

flash duration

float

Light source flash duration [µs]

frame rate

float

Imaging frame rate [Hz]

pixel size

float

Pixel size [µm]

roi position x

{f}

Image x coordinate on sensor [px]

roi position y

{f}

Image y coordinate on sensor [px]

roi size x

{f}

Image width [px]

roi size y

{f}

Image height [px]

online_contour

parsed

description [units]

bg empty

{f}

Background correction from empty frames only

bin area min

{f}

Minium pixel area of binary image event

bin kernel

{f}

Disk size for binary closing of mask image

bin threshold

{f}

Threshold for mask from bg-corrected image

image blur

{f}

Odd sigma for Gaussian blur (21x21 kernel)

no absdiff

{f}

Do not use OpenCV ‘absdiff’ for bg-correction

online_filter

parsed

description [units]

target duration

float

Target measurement duration [min]

target event count

{f}

Target event count for online gating

setup

parsed

description [units]

channel width

float

Width of microfluidic channel [µm]

chip identifier

{f}

Unique identifier of the chip used

chip region

{f}

Imaged chip region (channel or reservoir)

flow rate

float

Flow rate in channel [µL/s]

flow rate sample

float

Sample flow rate [µL/s]

flow rate sheath

float

Sheath flow rate [µL/s]

identifier

str

Unique setup identifier

medium

str

Medium used

module composition

str

Comma-separated list of modules used

software version

str

Acquisition software with version

temperature

float

Mean chip temperature [°C]

Example: date and time of a measurement

In [1]: import dclab

In [2]: ds = dclab.new_dataset("data/example.rtdc")

In [3]: ds.config["experiment"]["date"], ds.config["experiment"]["time"]
Out[3]: ('2017-07-16', '19:01:36')

Analysis metadata

In addition to inherent (defined during data acquisition) metadata, dclab also supports additional metadata that are relevant for certain data analysis pipelines, such as Young’s modulus computation or fluorescence crosstalk correction.

calculation

parsed

description [units]

crosstalk fl12

float

Fluorescence crosstalk, channel 1 to 2

crosstalk fl13

float

Fluorescence crosstalk, channel 1 to 3

crosstalk fl21

float

Fluorescence crosstalk, channel 2 to 1

crosstalk fl23

float

Fluorescence crosstalk, channel 2 to 3

crosstalk fl31

float

Fluorescence crosstalk, channel 3 to 1

crosstalk fl32

float

Fluorescence crosstalk, channel 3 to 2

emodulus lut

str

Look-up table identifier

emodulus medium

str

Medium used (e.g. ‘0.49% MC-PBS’)

emodulus temperature

float

Chip temperature [°C]

emodulus viscosity

float

Viscosity [Pa*s] if ‘medium’ unknown

emodulus viscosity model

str

Viscosity model for known media

User-defined metadata

In addition to the registered metadata keys listed above, you may also define custom metadata in the “user” section. This section will be saved alongside the other metadata when a dataset is exported as an .rtdc (HDF5) file.

Note

It is recommended to use the following data types for the value of each key: str, bool, float and int. Other data types may not render nicely in ShapeOut2 or DCOR.

To edit the “user” section in dclab, simply modify the config property of a loaded dataset. The changes made are not written to the underlying file.

Example: Setting custom “user” metadata in dclab

In [4]: import dclab

In [5]: ds = dclab.new_dataset("data/example.rtdc")

In [6]: my_metadata = {"inlet": True, "n_channels": 4}

In [7]: ds.config["user"] = my_metadata

In [8]: other_metadata = {"outlet": False, "RBC": True}

# we can also add metadata with the `update` method
In [9]: ds.config["user"].update(other_metadata)

# or
In [10]: ds.config.update({"user": other_metadata})

In [11]: print(ds.config["user"])
{'inlet': True, 'n_channels': 4, 'outlet': False, 'RBC': True}

# we can clear the "user" section like so:
In [12]: ds.config["user"].clear()

If you are implementing a custom data acquisition pipeline, you may alternatively add user-defined meta data (permanently) to an .rtdc file in a post-measurement step like so.

Example: Setting custom “user” metadata permanently

import h5py
with h5py.File("/path/to/your/dataset.rtdc") as h5:
    h5.attrs["user:inlet"] = True
    h5.attrs["user:n_channels"] = 4
    h5.attrs["user:outlet"] = False
    h5.attrs["user:RBC"] = True
    h5.attrs["user:project"] = "strangelove"

User-defined metadata can also be used with user-defined plugin features. This allows you to design plugin features which utilize your pipeline-specific metadata.