Adaptive optical correction for in vivo two-photon fluorescence microscopy with neural fields

像素 计算机科学 计算机视觉 光学 人工智能 堆栈(抽象数据类型) 位图 算法 灰度 样品(材料) 显微镜 图像处理 图像质量 亚像素渲染 泽尼克多项式 显微镜 光圈(计算机存储器) 二进制数 数字图像相关 图像分辨率 人工神经网络 二值图像 薄层荧光显微镜 自适应光学 物理 光流 运动估计 迭代重建 运动场 职位(财务) 图像复原 光学显微镜 数学
作者
Iksung Kang,Hyeonggeon Kim,Ryan G. Natan,Qinrong Zhang,Stella X. Yu,Na Ji
出处
期刊:
标识
DOI:10.24433/co.2804276.v1
摘要

This repository contains the code for Neural fields for Adaptive Optical Two-photon Fluorescence Microscopy (NeAT). NeAT operates in three stages: 1. **Aberration and structural estimation** From a single 3D image stack, NeAT estimates system or sample aberrations and recovers the underlying structural information, without any external training data. 2. **Conjugation error correction** NeAT estimates and corrects conjugation errors in the imaging system, typically caused by incomplete conjugation or alignment errors. 3. **Sample motion correction** NeAT maintains performance even with sample motion during the acquisition of the 3D input stack, by adaptively registering and correcting slice-to-slice motion artifacts. --- ### Code overview This capsule includes three main scripts: #### `neat_learning.py` Core routine that takes a 3D image stack as input and outputs aberration and structural estimations. Key physical parameters can be set via command-line arguments (a full list of tunable options is in the `args` parser): - `psf_dx, psf_dy, psf_dz`: pixel sizes (µm) along $x$, $y$, $z$ axes - `cnts`: center coordinates of the input stack - `dims`: dimensions of the input stack - `na_exc`: excitation numerical aperture (NA) - `sample_motion`: Boolean parameter for sample motion correction. True if sample motion correction is performed. **Expected outputs:** - **`rec.h5`** — HDF5 file containing: - `out_x_m`: estimated structure - `out_k_m`: estimated PSF - `out_y`: computed image stack - `wf`: Zernike coefficients - `loss_list`: losses per epoch - `y`: normalized input stack - `y_min`, `y_max`: normalization bounds - **`est_aber_map.bmp`** — bitmap of the estimated aberration map from `wf` (unit: waves) - **`slm_pattern.bmp`** — 8-bit grayscale image (with a pixel value of 255 corresponding to 1 wave) that is the wrapped corrective phase pattern to be applied to an SLM for aberration correction, if needed. Feel free to explore the code and adjust parameters to suit your imaging setup. #### `neat_conj_est.py` This script estimates conjugation errors that may be present in a two-photon imaging system. It should be run after `neat_learning.py` on six image stacks. For testing, example stacks acquired using a microscope at UC Berkeley are available under `/commercial_beads_zeros/`, `/commercial_beads_mode4/`, `/commercial_beads_mode6/`, … and `/commercial_beads_mode13/`. Users should acquire these image stacks using their own system to estimate conjugation errors specific to their microscope. To acquire these stacks, users need to display on the SLM images corresponding to a flat phase pattern and five different Zernike modes for calibration (modes 4, 6, ..., 13), respectively. These patterns are available in `/slm_patterns_for_calibration/` as `k_vis_zeros.bmp`, `k_vis_mode4.bmp`, `k_vis_mode6.bmp`, ..., `k_vis_mode13.bmp.` (To apply these patterns directly on a SLM, the SLM needs to be calibrated so that a pixel value of 255 corresponds to 1 wave. The patterns can also be scaled and formatted for a deformable mirror.) Conjugation errors are computed as an affine transformation $\hat{H}$, from the reconstructions of these stacks and saved to `H.h5`. #### `neat_conj_corr.py` This script compensates for the conjugation errors estimated by `neat_conj_est.py` by applying the inverse affine transformation $\hat{H}^{-1}$ to an SLM pattern (generated as `slm_pattern.bmp` by `neat_learning.py`, which does not include conjugation error correction). The corrected pattern is saved as `k_vis_(args.dataset)_with_H.bmp`. > **Note:** If you are using a microscope with perfect conjugation and alignment, you can skip `neat_conj_est.py` and `neat_conj_corr.py`. --- ### Running the Capsule This capsule demonstrates NeAT’s application on both custom-built and commercial microscopes: 1. **Microscope with perfect conjugation and alignment** Run NeAT on a fixed mouse-brain-slice image stack (data in `/custom_built_brain_slice/`), where conjugation-error correction is not required. 2. **Calibration for a microscope with possible conjugation and alignment issues (e.g., most commercial systems)** Process five calibration stacks, estimate conjugation errors, and apply those corrections to generate an error-corrected SLM pattern. 3. **In vivo imaging with motion** Run NeAT on an in vivo mouse-brain image stack with sample motion. If the samples do not require motion correction (e.g., zebrafish larvaes or plants) but still need conjugation error correction, simply set `sample_motion = False` and proceed with this step.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
合法的天空完成签到,获得积分10
1秒前
1秒前
Zz发布了新的文献求助200
1秒前
大胆的马里奥完成签到,获得积分10
2秒前
水煮电吹风的应助被萧笙采纳,获得10
3秒前
橙巧巧和乌霉霉完成签到,获得积分10
5秒前
6秒前
6秒前
7秒前
甘甚么完成签到,获得积分10
8秒前
9秒前
9秒前
可爱的函函的应助被cuipi采纳,获得30
9秒前
YiRan完成签到,获得积分10
10秒前
英姑的应助被开心德地采纳,获得10
11秒前
踏实凡桃发布了新的文献求助10
11秒前
Wynne的应助被業業采纳,获得20
11秒前
苦茶完成签到,获得积分10
11秒前
甘甚么发布了新的文献求助10
12秒前
爱笑圆圆完成签到 ,获得积分10
12秒前
12秒前
小马甲的应助被君澔采纳,获得10
12秒前
恩涵贺完成签到,获得积分10
13秒前
13秒前
挚友发布了新的文献求助30
15秒前
之之完成签到,获得积分10
15秒前
元元发布了新的文献求助10
15秒前
小肥天鹅完成签到,获得积分10
16秒前
ding的应助被guoyan采纳,获得10
16秒前
18秒前
机灵紫夏完成签到 ,获得积分10
18秒前
qqq发布了新的文献求助10
18秒前
19秒前
19秒前
沐沐完成签到,获得积分10
20秒前
21秒前
22秒前
核桃发布了新的文献求助10
23秒前
Desirable发布了新的文献求助20
23秒前
fshell完成签到,获得积分10
24秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
Sex, Aging, and Death in a Medieval Medical Compendium: Trinity College Cambridge MS R.14.52, Its Texts, Language, and Scribe 500
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
基于结构调控的功能化水诱导发电材料与器件研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8188654
求助须知:如何正确求助?哪些是违规求助? 9576372
关于积分的说明 21847161
捐赠科研通 7786741
什么是DOI,文献DOI怎么找? 3435402
关于科研通互助平台的介绍 2559632
邀请新用户注册赠送积分活动 2482360