Deep-learning-augmented computational miniature mesoscope

Abstract

Fluorescence microscopy is essential to study biological structures and dynamics. However, existing systems suffer from a trade-off between field of view (FOV), resolution, and system complexity, and thus cannot fulfill the emerging need for miniaturized platforms providing micron-scale resolution across centimeter-scale FOVs. To overcome this challenge, we developed a computational miniature mesoscope ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msup> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">C</mml:mi> <mml:mi mathvariant="normal">M</mml:mi> </mml:mrow> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> </mml:math> ) that exploits a computational imaging strategy to enable single-shot, 3D high-resolution imaging across a wide FOV in a miniaturized platform. Here, we present <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msup> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">C</mml:mi> <mml:mi mathvariant="normal">M</mml:mi> </mml:mrow> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> </mml:math> V2, which significantly advances both the hardware and computation. We complement the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mn>3</mml:mn> </mml:mrow> <mml:mo>×</mml:mo> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mn>3</mml:mn> </mml:mrow> </mml:math> microlens array with a hybrid emission filter that improves the imaging contrast by <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mn>5</mml:mn> </mml:mrow> <mml:mo>×</mml:mo> </mml:math> , and design a 3D-printed free-form collimator for the LED illuminator that improves the excitation efficiency by 3×. To enable high-resolution reconstruction across a large volume, we develop an accurate and efficient 3D linear shift-variant (LSV) model to characterize spatially varying aberrations. We then train a multimodule deep learning model called <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msup> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">C</mml:mi> <mml:mi mathvariant="normal">M</mml:mi> </mml:mrow> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">N</mml:mi> <mml:mi mathvariant="normal">e</mml:mi> <mml:mi mathvariant="normal">t</mml:mi> </mml:mrow> </mml:math> , using only the 3D-LSV simulator. We quantify the detection performance and localization accuracy of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msup> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">C</mml:mi> <mml:mi mathvariant="normal">M</mml:mi> </mml:mrow> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">N</mml:mi> <mml:mi mathvariant="normal">e</mml:mi> <mml:mi mathvariant="normal">t</mml:mi> </mml:mrow> </mml:math> to reconstruct fluorescent emitters under different conditions in simulation. We then show that <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:msup> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">C</mml:mi> <mml:mi mathvariant="normal">M</mml:mi> </mml:mrow> <mml:mn>2</mml:mn> </mml:msup> </mml:mrow> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">N</mml:mi> <mml:mi mathvariant="normal">e</mml:mi> <mml:mi mathvariant="normal">t</mml:mi> </mml:mrow> </mml:math> generalizes well to experiments and achieves accurate 3D reconstruction across a <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mo>∼</mml:mo> </mml:mrow> <mml:mn>7</mml:mn> <mml:mtext>-</mml:mtext> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">m</mml:mi> <mml:mi mathvariant="normal">m</mml:mi> </mml:mrow> </mml:math> FOV and 800-µm depth, and provides <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mo>∼</mml:mo> </mml:mrow> <mml:mn>6</mml:mn> <mml:mtext>-</mml:mtext> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mtext>µ</mml:mtext> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">m</mml:mi> </mml:mrow> </mml:mrow> </mml:math> lateral and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mo>∼</mml:mo> </mml:mrow> <mml:mn>25</mml:mn> <mml:mtext>-</mml:mtext> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mtext>µ</mml:mtext> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mi mathvariant="normal">m</mml:mi> </mml:mrow> </mml:mrow> </mml:math> axial resolution. This provides an <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mo>∼</mml:mo> </mml:mrow> <mml:mn>8</mml:mn> <mml:mo>×</mml:mo> </mml:math> better axial resolution and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"> <mml:mrow class="MJX-TeXAtom-ORD"> <mml:mo>∼</mml:mo> </mml:mrow> <mml:mn>1400</mml:mn> <mml:mo>×</mml:mo> </mml:math> faster speed compared to the previous model-based algorithm. We anticipate this simple, low-cost computational miniature imaging system will be useful for many large-scale 3D fluorescence imaging applications.

Topics & Concepts