2. Installation & Analysis Setup

Brieflow and Brieflow Analysis are designed to work together to analyze optical pooled screens. Follow these steps to get set up for a screen analysis!

Standard Workflow (Recommended)

1. Screen Analysis Repository Setup

Brieflow-analysis is a template for each screen analysis. Create a new repository for a screen to get started.

1. Create a new screen repository with the “Use this template” button for each new screen analysis.

   Click the “Use this template” button at brieflow-analysis and give it a name like your-screen-name.

   

   See the GitHub documentation for using a template for more information.

2. Clone the newly created repository to your local machine:

   git clone https://github.com/YOUR-USERNAME/YOUR-SCREEN-REPO.git
   cd YOUR-SCREEN-REPO
   

2. Brieflow Setup

We use brieflow to process data on a very large scale from each screen. We use brieflow as a git submodule in this repository. Please see the Git Submodules basic explanation for information on how to best install, use, and update this submodule.

1. Initialize the Brieflow submodule

   The brieflow submodule is already configured in this repository.

   (Optional) If you want to use your own fork of brieflow instead of the default cheeseman-lab/brieflow:

   git submodule set-url brieflow https://github.com/YOUR-USERNAME/brieflow.git
   

   Then all users should initialize the submodule:

   git submodule update --init --recursive
   

2. Create and activate a screen-specific conda environment (~10 min)

   # Create and activate brieflow_SCREEN_NAME conda environment
   # NOTE: Replace brieflow_SCREEN_NAME with the name of your screen to ensure a screen-specific installation
   # Using this screen-specific installation will refer to library code in ./brieflow/workflow/lib
   conda create -n brieflow_SCREEN_NAME -c conda-forge python=3.11 uv pip -y
   conda activate brieflow_SCREEN_NAME
   
   # Navigate to brieflow directory
   cd brieflow
   
   # Install external packages
   uv pip install -r pyproject.toml
   
   # Install editable version of brieflow
   uv pip install -e .
   
   # install optional packages (as needed):
   uv pip install cellpose==4.0.4 torch==2.7.0 torchvision==0.22.0  # Cellpose 4.x with CPSAM model
   uv pip install "numpy<2.0" stardist==0.9.1 csbdeep==0.8.1  # StarDist (requires numpy<2.0)
   

   Notes:

   • We recommend a screen-specific installation because changes to this particular ./brieflow/workflow/lib code will live within this specific installation of brieflow, and an explicit name helps keep track of different brieflow installations. One could also install one version of brieflow that is used across brieflow-analysis repositories.

   • For a rule-specific package consider creating a separate conda environment file and using it for the particular rule as described in the Snakemake integrated package management notes.

3. Configure Slurm resources (if using HPC)

   We use the HPC integration for Slurm as detailed in the setup instructions. To use the Slurm integration for Brieflow, configure the Slurm resources in analysis/slurm/config.yaml.

3. Test Analysis

Due to the large nature of OPS data, it is impractical to provide a realistic dataset to test Brieflow on. However, we provide a minimal set of data useful for testing Brieflow. This data is uploaded to Zenodo at https://zenodo.org/records/15276612. We also provide default configuration files for processing this data at brieflow/tests/small_test_analysis/config/

This minimal test analysis data can be used to test your installation of Brieflow as well as any update Brieflow code.

Note: the test analysis folder is NOT the location for actual analysis data.

You can run the following commands to download/unpack the test analysis data from Zenodo and perform a Brieflow run. On our local machine, this takes about 15 minutes.

# activate brieflow env
conda activate brieflow_SCREEN_NAME
cd brieflow/tests/small_test_analysis
# download and unpack test analysis data
python small_test_analysis_setup.py
# run brieflow on test analysis data
bash run_brieflow.sh
cd ../../..
pytest   # from inside the brieflow directory

Running Modules

You are now ready to start configuring and running your screen. See 3. Running Modules to determine how to proceed!

Video Walk-Through

Note: This video is a little outdated but still shows the general installation and setup processes.

Developer Workflow

If you are contributing new features or fixes back to the core Brieflow codebase, follow the standard workflow above with these key differences:

1. Fork Brieflow first: Create a fork of brieflow as described here.

2. Use the optional fork setup in step 2.1 of Brieflow Setup to point the submodule to your fork before initializing.

3. Configure the remote repository for your fork (more info here):

   # Navigate to brieflow directory
   cd brieflow
   
   # Set remote upstream repo
   git remote add upstream https://github.com/cheeseman-lab/brieflow.git
   
   # Check origin and upstream repos
   git remote -v
   # Confirm you see the below output
   > origin    https://github.com/YOUR-USERNAME/brieflow.git (fetch)
   > origin    https://github.com/YOUR-USERNAME/brieflow.git (push)
   > upstream  https://github.com/cheeseman-lab/brieflow.git (fetch)
   > upstream  https://github.com/cheeseman-lab/brieflow.git (push)
   

   Follow the GitHub documentation to sync changes between your fork and cheeseman-lab/brieflow (e.g., to pull a new branch).

4. Create feature branches inside the brieflow submodule (not in the top-level analysis repo) for your changes.

5. Contribute changes back: Track changes to computational processing in a new branch on your fork. Contribute these changes to cheeseman-lab/brieflow with a pull request. See GitHub’s documentation for contributing to a project and Brieflow’s development guide for more info.

Note: We highly recommend reading the GitHub documentation for explanation of forks to understand how your fork of brieflow syncs with the cheeseman-lab brieflow. From the documentation:

A fork is a new repository that shares code and visibility settings with the original “upstream” repository. Forks are often used to iterate on ideas or changes before they are proposed back to the upstream repository, such as in open source projects or when a user does not have write access to the upstream repository. For more information, see Working with forks.

Full contribution details are in 4. Development & Contributing.