# 5. Development

## Overview

Brieflow is meant to be very adaptable for new screening contexts.
As OPS develops, we expect users to make changes for their specific context.
Pieces of these changes that are useful for the broad OPS community may be included in the `main` branch of brieflow.

The general process of screen adaptation is as follows:

1) Make a new fork/branch for a new screening context.
Adapt Brieflow for this screening context.
2) If there is code that would be useful to the `main` branch, make a PR to contribute this code.
See the *Contributing* section below for more info.

## Screen Adaptation

Users should do the following as they adapt Brieflow to their own screen:

1) Make a user-specific [fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo) of brieflow.
2) Make a screening context-specific [branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-and-deleting-branches-within-your-repository) with the screening context code changes.
Make sure to self-review the PR to ensure the additional code is only what is useful for the main branch.

### Development Conventions

When developing new computational methodologies for Brieflow, follow these conventions:

#### Code Structure

The brieflow submodule is organized into four key directories:

```
brieflow/workflow/
├── lib/              # Library functions (edit first)
│   ├── preprocess/
│   ├── sbs/
│   ├── phenotype/
│   ├── merge/
│   ├── aggregate/
│   └── cluster/
├── scripts/          # Scripts that call lib functions
│   └── {module}/
├── rules/            # Snakemake rules (params, resources)
│   └── {module}.smk
└── targets/          # Output definitions (wildcards, paths)
    └── {module}.smk
```

#### Code Organization

| Type | Scope | What Changes |
|------|-------|--------------|
| **Function** | Small change to existing function | `lib/` only |
| **Process** | New/modified script | `lib/`, `scripts/`, possibly `rules/` |
| **Module** | Large changes or new functionality | `lib/`, `scripts/`, `rules/`, `targets/` |

#### Edit Order

When modifying brieflow code, always follow this order:

1. **lib/** - Modify library functions first
2. **scripts/** - Update scripts that call those functions
3. **rules/** - Change rule parameters if needed
4. **targets/** - Update output definitions if needed

For a new module, also update the main `Snakefile` to include it.

## Contributing

Brieflow is still actively under development and we welcome community use and development.

### How to Contribute

We aim to strike a balance between simplicity and functionality in the `main` branch.
For this reason, we will only incorporate code into main that is helpful for a general/standardized OPS screening context.
If you have code that would help other brieflow users, please make a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) to `main`.
Matteo and Roshan will review this pull request and guide it to be merged!

To contribute:
- File a [GitHub issue](https://github.com/cheeseman-lab/brieflow/issues) to share comments and issues
- File a GitHub PR to contribute to Brieflow as detailed in the pull request template (`.github/pull_request_template.md`)
- Read about how to [contribute to a project](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project) to understand forks, branches, and PRs

### Development Tools

We use [ruff](https://github.com/astral-sh/ruff) for linting and formatting code.
Ruff is automatically installed within each Brieflow environment.

```bash
cd brieflow/
ruff check workflow/        # Check for linting errors
ruff format workflow/       # Format code
ruff check --fix workflow/  # Auto-fix linting errors
```

### Coding Conventions

When contributing code, please follow these conventions:

#### Documentation and Formatting
- Use [one sentence per line](https://nick.groenen.me/notes/one-sentence-per-line/) convention for markdown files
- Use [Google format](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) for function docstrings

#### File Formats
- Use [tsv](https://en.wikipedia.org/wiki/Tab-separated_values#:~:text=Tab%2Dseparated%20values%20(TSV),similar%20to%20comma%2Dseparated%20values.) file format for saving small dataframes that require easy readability
- Use [parquet](https://www.databricks.com/glossary/what-is-parquet) file format for saving large dataframes

#### Naming Conventions
- Data files: Data location information (well, tile, cycle, etc) + `__` + type of information (cell features, phenotype info, etc) + `.` + file type
- Data is stored in its respective analysis directories
- Example: `brieflow_output/preprocess/metadata/phenotype/P-1_W-A2_T-571__metadata.tsv`

#### Versioning
- We use [semantic versioning](https://semver.org/) for brieflow and brieflow-analysis versions
- These two repositories should always have the same version
- Small version changes are only tracked in the `pyproject.toml` file of brieflow (not in brieflow-analysis)

### Integration Tests

To ensure new features don't break Brieflow, we have integration tests set up to automatically run Brieflow on test data when a PR to main is created.

The integration test runs the `Test Installation` code from [2. Installation & Analysis Setup](https://brieflow.readthedocs.io/en/latest/2.installation_analysis_setup.html) (step 3) in a GitHub action.

**Running tests locally before creating a PR:**

```bash
cd brieflow/tests/small_test_analysis/analysis/
sh run_brieflow.sh
```

Depending on the contribution, it may be necessary to update the test analysis config files at `brieflow/tests/small_test_analysis/config`.