5. Development

Overview

Brieflow is meant to be very adaptable for new screening contexts. As OPS develops, we expect users to make changes for their specific context. Pieces of these changes that are useful for the broad OPS community may be included in the main branch of brieflow.

The general process of screen adaptation is as follows:

1. Make a new fork/branch for a new screening context. Adapt Brieflow for this screening context.

2. If there is code that would be useful to the main branch, make a PR to contribute this code. See the Contributing section below for more info.

Screen Adaptation

Users should do the following as they adapt Brieflow to their own screen:

1. Make a user-specific fork of brieflow.

2. Make a screening context-specific branch with the screening context code changes. Make sure to self-review the PR to ensure the additional code is only what is useful for the main branch.

Development Conventions

When developing new computational methodologies for Brieflow, follow these conventions:

Code Structure

The brieflow submodule is organized into four key directories:

brieflow/workflow/
├── lib/              # Library functions (edit first)
│   ├── preprocess/
│   ├── sbs/
│   ├── phenotype/
│   ├── merge/
│   ├── aggregate/
│   └── cluster/
├── scripts/          # Scripts that call lib functions
│   └── {module}/
├── rules/            # Snakemake rules (params, resources)
│   └── {module}.smk
└── targets/          # Output definitions (wildcards, paths)
    └── {module}.smk

Code Organization

Type	Scope	What Changes
Function	Small change to existing function	lib/ only
Process	New/modified script	lib/, scripts/, possibly rules/
Module	Large changes or new functionality	lib/, scripts/, rules/, targets/

Edit Order

When modifying brieflow code, always follow this order:

1. lib/ - Modify library functions first

2. scripts/ - Update scripts that call those functions

3. rules/ - Change rule parameters if needed

4. targets/ - Update output definitions if needed

For a new module, also update the main Snakefile to include it.

Contributing

Brieflow is still actively under development and we welcome community use and development.

How to Contribute

We aim to strike a balance between simplicity and functionality in the main branch. For this reason, we will only incorporate code into main that is helpful for a general/standardized OPS screening context. If you have code that would help other brieflow users, please make a pull request to main. Matteo and Roshan will review this pull request and guide it to be merged!

To contribute:

• File a GitHub issue to share comments and issues

• File a GitHub PR to contribute to Brieflow as detailed in the pull request template (.github/pull_request_template.md)

• Read about how to contribute to a project to understand forks, branches, and PRs

Development Tools

We use ruff for linting and formatting code. Ruff is automatically installed within each Brieflow environment.

cd brieflow/
ruff check workflow/        # Check for linting errors
ruff format workflow/       # Format code
ruff check --fix workflow/  # Auto-fix linting errors

Coding Conventions

When contributing code, please follow these conventions:

Documentation and Formatting

• Use one sentence per line convention for markdown files

• Use Google format for function docstrings

File Formats

• Use tsv file format for saving small dataframes that require easy readability

• Use parquet file format for saving large dataframes

Naming Conventions

• Data files: Data location information (well, tile, cycle, etc) + __ + type of information (cell features, phenotype info, etc) + . + file type

• Data is stored in its respective analysis directories

• Example: brieflow_output/preprocess/metadata/phenotype/P-1_W-A2_T-571__metadata.tsv

Versioning

• We use semantic versioning for brieflow and brieflow-analysis versions

• These two repositories should always have the same version

• Small version changes are only tracked in the pyproject.toml file of brieflow (not in brieflow-analysis)

Integration Tests

To ensure new features don’t break Brieflow, we have integration tests set up to automatically run Brieflow on test data when a PR to main is created.

The integration test runs the Test Installation code from 2. Installation & Analysis Setup (step 3) in a GitHub action.

Running tests locally before creating a PR:

cd brieflow/tests/small_test_analysis/analysis/
sh run_brieflow.sh

Depending on the contribution, it may be necessary to update the test analysis config files at brieflow/tests/small_test_analysis/config.