THE HARDIN LABClick on the links below to go to various sections of this page...
• 4d macros and plugins for Fiji/ImageJ
• FRAP scripts for Fiji/ImageJ
• Legacy plugins
4d macros and plugins for Fiji/ImageJ
Introduction: Reading and writing data from movies is challenging
Reading: BioFormats Importer can handle some compressed QuickTime movies in Fiji/ImageJ and other programs that supports the BioFormats libraries. Unfortunately, BioFormats doesn't handle Motion JPEG A compressed movies, which our lab made quite a few of back in the day.
QuickTime for Java (QTJ) has been deprecated for many years by Apple. Due to its 32-bit architecture, it will not work under 64-bit only operating systems like OS X Catalina. It is possible to use virtualization software to run OS X Mojave or earlier, to retain support for QTJ. I (Jeff) have successfully tested Mojave with VMWare Fusion.
Instead, it may be easier to use plugins under ImageJ to import QuickTime movies as virtual stacks using Ffmpeg, which will work on 64-bit operating systems and under newer, 64-bit versions of ImageJ (see below). Some really old flattened (i.e., non-resource fork-based .MOV files, if you even know what resource forks are!) work this way if the JPEG or Motion JPEG image codecs were used. More importantly, files saved in widely used AVI (with compression) and H264/MP4 formats can be imported this way. This means that files can be resaved into a modern format from legacy datasets, salvaging old, non-raw 4d data.
Writing: ImageJ has major limitations writing to modern video formats such as H264 using the H264 codec and convenient wrappers (.MOV, .MP4, .AVI), One approach is to write to uncompressed AVIs, within ImageJ which is supported natively in ImageJ/Fiji, then use your favorite app of choice (Adobe Media Converter, VirtualDub on Windows, or Ffmpeg or wrapper apps that make calls to Ffmpeg on Linux/Mac/Windows) to convert to MP4. [In the last category, HandBrake is free and easy. An elegant commercial example for Mac I like is FFworks.] A downside is that uncompressed AVIs can be huge (often an order of magnitude larger than movies saved using some codecs). For Nomarski data this means a major storage hit. For fluorescence data one normally wouldn't compress primary data using a lossy compressor.
Stanislav Chizhik's FFmpeg-based plugins: Fortunately, Stanislav Chizhik (Institute of Solid Chemistry and Mechanochemistry SB RAS, Novosibirsk, Russia) has provided both Ffmpeg-based read and write functionality using JavaCV and Ffmpeg. The easiest way to access these plugins is by adding his "FFmpegVideoImportExport" update site to Fiji. A Wiki page describing these plugins can be found here.
4d Tools Update Site (Fiji)
I (Jeff) have developed plugins that run within the public domain program Fiji/ImageJ, to process, and play back 4d microscopy data as an outgrowth of our own imaging needs. The advantage of this software suite is that it is free, and that it allows 4d datasets to be viewed without special, proprietary software. Please recognize that I am a professor, and program in my "spare" time, so support for these plugins is limited and bound to be sporadic!
The easiest way to get current versions of the 4d macros/plugins is by enabling the "4d-Tools" update site using the Fiji Updater. Please see below or consult the 4d-Tools update site Wiki page for details.
- Sample 4d movie with 8000 frames (20 focal planes, 400 time points) in xyctz format. Download sample dataset and info file as a ZIP archive (large: 985 Mb)
- Sample 4d movie with 1920 frames (30 focal planes, 64 time points) in xyctz format. Download sample dataset and info file as a ZIP archive (smaller: 52 Mb)
Overview: 4d Tools is a set of macros/plugin for (1) import of legacy 4d datasets as movies (.MOV, .MP4, .AVI); (2) simplified import of Imaris .IMS and Micro-manager 1.4/2.0 datasets into Fiji; and (3) experimental playback of maximum-intensity projections on the fly as a crude attempt to recapitulate legacy Perkin-Elmer UltraView software.
Rationale
- Import of legacy datasets is useful for many long-established C. elegans laboratories, for example, which have large numbers of legacy 4d DIC datasets as movies in xytc format. For this vintage approach, see, for example, Thomas, C., DeVries, P., Hardin, J., and White, J. (1996). Four-dimensional imaging: computer visualization of 3D movements in living specimens. Science 273, 603-607.
- Micro-Manager has an integrated playback mechanism built on top of ImageJ, and a free multi-platform Imaris player is available, but in some cases direct import into Fiji is useful, and these macros simplify this process for routine work.
- Oddly, there doesn’t seem to be a straightforward way to display a “range” of Z slices as a maximum intensity projection in ImageJ/Fiji. Old Perkin-Elmer Ultraview software had this functionality.
Add the 4d-Tools update site using the Fiji update sites manager. After updating, five new menu items will be added to Fiji. A “4d” submenu will be added to the “Import” menu in Fiji. Four items will be available in this submenu:
- Classic 4d AVI to Hyperstack - Imports a 4d AVI video as a Virtual Hyperstack. It looks for a plain text file with the same root name as video. If none is found the macro will ask for the number of focal planes.
- Classic 4d Movie to Hyperstack - Imports a 4d Quicktime .MOV or an MPEG4 .MP4 video as a 4d Virtual Hyperstack using FFMpeg. It looks for a plain text file with the same root name as the video. If none is found the macro will ask for the number of focal planes.
- Imaris to Hyperstack - Imports the highest- resolution dataset in an Imaris .IMS file (a variant of the HDF5 format) as a Virtual Hyperstack using the BioFormats Importer built into Fiji.
- MM Images to Hyperstack - Imports a Micro-Manager 1.4 or 2.0 dataset as a Virtual Hyperstack, using the metadata in the dataset.
- Hyperstack Projector - A new “Hyperstack Projector” item will be added to the “Image -> Hyperstacks” submenu. Running this plugin will attempt to recapitulate the functionality of the old Perkin-Elmer Ultraview software, which performed a maximum-intensity projection on a specified range of images on the fly. The plugin assumes a 4d Hyperstack is open, and temporarily replaces the stack window with a custom window that allows specification of the number of focal planes to be projected. Closing the window restores the original stack window. NOTE: This plugin is extremely experimental. The code in the installed .JAR file can likely be massively improved.
Classic 4d AVI to Hyperstack
- Select menu “Import->4d-> Classic 4d AVI to Hyperstack”.
- Specify the video file.
- If no text file with focal plane information is found, specify the number of focal planes.
- The dataset will be loaded as a Virtual Hyperstack.
- Select menu “Import->4d-> Classic 4d Movie to Hyperstack”.
- Specify the video file.
- If no text file with focal plane information is found, specify the number of focal planes.
- The dataset will be loaded as a Virtual Hyperstack.
- A window with the FFMpeg import data from the FFmpegVideoImportExport plugin should be visible.
Imaris to Hyperstack
- Select menu “Import->4d-> Imaris to Hyperstack”.
- Specify the data file.
- The dataset will be loaded as a Virtual Hyperstack.
MM Images to Hyperstack
- Select menu “Import->4d-> MM Images to Hyperstack”.
- Specify the data directory.
- The dataset will be loaded as a Virtual Hyperstack.
Hyperstack Projector
- With a 4d Hyperstack in the frontmost window, select menu “Import->Hyperstacks-> Hyperstack Projector”.
- The Hyperstack window will be replaced with a custom window.
- Enter the number of focal planes to be projected centered on the current focal plane in the “d” text field.
- A running projection will be displayed centered on the current z plane and time point.
- Navigate up and down in space using the “z” slider and forward and backward in time using the “>” slider. Clicking the “v” button will play the movie, which will loop if the “loop” checkbox is selected.
- Space bar toggles playback on/off, and arrow keys allow navigation in t and z (right/left and up/down, respectively). Holding down the option key and using the right/left arrow keys switches channels.
top
FRAP Scripts for Fiji/ImageJ
Overview: FRAP-Tools is a set of scripts for (1) generating and saving FRAP data from an image stack and (2) display of saved data using either built-in ImageJ plot functions or the JFreeChart library.
Rationale
1. There are many options for FRAP analysis, but most are not integrated into Fiji/ImageJ.
2. There are relatively few options for semi-automated pooling of individual FRAP experiments and curve-fitting of pooled data.
3. Plot functions in ImageJ are powerful, but the resulting plots are bitmaps. Adding JFreeChart plot options allows saving as .SVG files.
FRAP Analysis
FRAP Analysis is a .py script inspired by hackathon code available here.
This script automates locating the bleach timepoint and trimming fluorescence recovery after photobleaching (FRAP) experimental data, and then allows analysis of the recovery kinetics.
FRAP Analysis uses the double normalization method outlined in:
Phair RD, Gorski SA, Misteli T. (2004). Measurement of dynamic protein binding to chromatin in vivo, using photobleaching microscopy. Methods Enzymol. 375:393-414.
doi: 10.1016/s0076-6879(03)75025-3. PMID: 14870680.
This normalization method relies on a non-FRAPed region containing signal, a second region containing non-bleached actual signal, and a third containing background signal. It assumes an image stack is open containing a FRAP experiment and three ROIs saved in the ROI Manager. Options are provided for automatically saving the data for a given dataset in a folder, saving a snapshot image with ROIs, and there are various options provided for how curve fitting is done. The resulting data can be plotted using the provided plotting scripts.
FRAP Plotting Scripts
Scripts are provided for plotting (a) single FRAP experiments (which may or may not contain a curve fit) using built-in ImageJ plot functions or JFreeChart; (b) similar generalized scripts for plotting data in .CSV files that may be useful for other purposes; and (c) scripts for combining multiple FRAP experiments so that they can be plotted on the same plot, mean +/- SEM/SD recovery data can be calculated/plotted, and a recovery curve fit of the pooled data can be generated and plotted.
Resulting output from the "FRAP Analysis" and "Combine FRAP data..." scripts can be easily imported into other programs.
Installation in Fiji
Add the FRAP-Tools update site using the Fiji update sites manager. After updating, seven new menu items will be added to Fiji bin the submenu "FRAP" in the "Analyze" menu:
Usage
FRAP Analysis
1. With the desired ImageStack open, create a ROI containing the FRAP region. Add it to the ROI Manager by typing "t", which will open the ROI Manager.
Alternatively, use the menu command "Analyze->Tools->"ROI Manager..." to open it and click the "Add" button. Repeat for a region containing signal but which was not photobleached, and for a region containing no signal (a background region). The ROI Manager should now contain three and only three ROIs saved in the specified order.
2. Invoke the "Analyze->FRAP->"FRAP Analysis" script.
3. A dialog box with options will appear, allowing you to set options:
a. Curve fitting method: Single or double exponential recovery can be selected.
b. Force curve fit through origin for single exponential: in theory, the curve should pass through x = 0 (first time point after bleach) and the minimum signal (y). This may or may not give the best overall fit to the recovery curve.
c. Stretch intensities to full range (0-1): normalizes the range to 0 (immediately post-bleach) to 1 (average signal pre-bleach).
d. Time interval: different platforms with different metadata (or no such data) are not parsed; instead, the time interval is manually entered here.
e. Create image with ROls: creates a snapshot image with the three ROIs for future reference. Note: ROIs can also be re-loaded from a saved ROI .ZIP file later if this was saved.
f. Autosave settings: the results can be saved to a folder and the windows closed after each run to reduce clutter and tedium.
4. Click "OK" to run the analysis. If "Autosave" options were selected, plot and analysis windows will be closed and the data saved to a folder with the name (Image name) + "-FRAP". If this folder already exists, you will be asked if you want to overwrite it.
Import FRAP data and plot Jython and Import FRAP data and plot JFree
1. Invoke the "Analyze->FRAP->"Import FRAP data and plot Jython" (or "...JFree") script. The script will open a recovery curve dataset or a recovery curve with a second set of XY columns that contains the fitted curve.
2. Multiple FRAP data files can be selected in the File opener dialog; the script will open all files and plot them individually.
3. Select the legend position (JFree version only): The default position of the legend in JFreeChart is outside the chart; a custom routine can place it inside the chart.
4. Click "OK" to run the script.
Read CSV and plot Jython and Read CSV and plot JFree
1. Invoke the "Analyze->FRAP->"Read CSV file and plot Jython" (or "...JFree") script. The script will open a file containing one or more columns of XY data and plot the data.
2. Multiple FRAP data files can be selected in the File opener dialog; the script will open all files and plot them individually.
3. Select the data structure: the data can have a single X column with multiple Y columns or separate pairs of XY data.
4. Select the legend position (JFree version only): The default position of the legend in JFreeChart is outside the chart; a custom routine can place it inside the chart.
5. Click "OK" to run the script.
Combine FRAP data Jython and Combine FRAP data JFree
1. Invoke the "Analyze->FRAP->"Combine FRAP data Jython" (or "...JFree") script. The script will open two or more single XY recovery curve datasets, plot them on the same graph, calculate and plot mean +/- SEM (or SD) FRAP recovery data, and calculate and plot a fitted curve to the mean recovery data. The script assumes that the datasets (a) have the same X axis (i.e., the same time interval of sampling; and (b) have the same Y range (typically normalized to a 0-to-1 range), and that the datasets all begin with the first post-bleach time point (i.e., there are no pre-bleach time points).
2. A dialog box with options will appear, allowing you to set options:
a. Combine plots: Plots all datasets on a single graph.
b. Plot Y mean values: Calculates mean Y value at each time point and displays on a separate plot.
c. Show error bars: Calculates standard error of the mean (SEM) or standard deviation (SD) for Y values at each time point and displays on the mean plot.
d. Curve fit means: performs a curve fit on the mean recovery values.
e. Curve fit method: Single or double exponential recovery can be selected.
f. Force curve fit through origin for single exponential: in theory, the curve should pass through x = 0 (first time point after bleach) and the minimum signal (y). This may or may not give the best overall fit to the recovery curve.
NOTE: for the Jfree version there is an additional option for legend placement. The default position of the legend in JFreeChart is outside the chart; a custom routine can place it inside the chart.
3. Click "OK" to run the script.
NOTE: The results, curve fit data, and plot(s) must be saved manually.
Sample TIFF stack (ZIP archive; 53 Mb): sample data
top
Legacy ImageJ plugins (and even older software)
Some of my really, really old plugins and software can be found on the Legacy software page.
Support for development of code provided here was provided by the National Institute for General Medical Sciences of the National Institutes of Health (current award #R35 GM145312).
