BEP-038: Atlases #1714

effigies · 2024-02-28T16:23:10Z

This PR proposes the inclusion of BEP-038 into the BIDS specification.

The BEP is hosted on its own branch in the main repository, and BEP leads have been given write permissions to the branch. If you are a BEP lead and having issues writing to the branch, please let a maintainer know and we can make sure you have access.

At present this is a starting point, and I anticipate significant rewriting and reorganization before final merge. There is extensive discussion in #1281 that needs to be turned into concrete proposals. I would suggest keeping the discussion on this PR minimal so that newcomers can follow the thread. When opening PRs against this branch, please include a reference to this PR (#1714) so that it shows up in the thread.

Rendered draft

https://bids-specification--1714.org.readthedocs.build/en/1714/derivatives/atlas.html

Related discussions

Associated PRs

I made a first draft of the atlas BEP based on the Google doc

Added atlas content

All the content of the BEP038 Google document has been ported to the atlas.md file. Additionally atlas and a definition of what of means was added as an entity. There are still smaller things to fix as some links, references and the tables still need to be inserted.

Fixed details such as linking to entities and suffixes as well as clarified some of the example text. The tables need still to be fixed.

[BEP038] Style and similar cleanups

CPernet · 2024-03-31T07:52:57Z

@erdalkaraca @tsalo this has passed the review period now, can this be checked and merged? thx

oesteban · 2024-04-01T19:16:55Z

I'm working on a review. Please hold on.

effigies · 2024-04-01T19:33:15Z

I also have not finished my review/proposed changes. I am unlikely to get to it before the meeting next week as I am traveling for another meeting this week.

effigies · 2024-04-04T19:52:58Z

This BEP currently needs:

Schema implementation of required checks and filename rules
Examples implementing all significant (ideally all) features proposed to https://github.com/bids-standard/bids-examples
Implementation of all rules in https://github.com/bids-standard/bids-validator and tested on the examples.
A two week public review period announced on all communication channels.

I'm converting this to draft to reflect this status. I'm going to temporarily restrict comments, pending synchronous discussion.

This reverts commit 0511805.

Co-authored-by: Mark Mikkelsen <[email protected]>

yarikoptic

Overall I think it is largely great work, needs some clarifications, but IMHO a larger problem is that introduces a new principled file type (atlas-<label>_description.json) without considering/adopting otherwise existing already pattern in BIDS (atlases.tsv).
At large IMHO that violated existing BEP Guidelines recommendation on atomic changes: https://bids.neuroimaging.io//extensions/guidelines.html#facilitate-atomic-changes and not mentioning there also tpl and cohort for others to become aware in https://bids.neuroimaging.io//extensions/guidelines.html#proposed-entities .

I think this inconsistently does not necessarily should stop this BEP from being merged, but I think it would be great to talk through on either we would like to either

introduce that as some kind of a generic principle to have such _description.json files
or refactor/migrate later into some more common atlases.tsv or atlases.jsonl (see #2283)

src/derivatives/atlas.md

src/common-principles.md

tools/schemacode/uv.lock

src/derivatives/atlas.md

yarikoptic · 2025-12-17T18:59:43Z

src/derivatives/atlas.md

+
+### Atlas identification and metadata
+
+The `atlas-<label>_description.json` file provides metadata to uniquely identify,


Was it considered at all to just make "one atlas per dataset" and thus introduce DatasetType "atlas" and thus use dataset_description.json to provide all the gory details (with huge overlap which what current dataset_description.json has)???

then, when necessary, those could be composed into some BIDS super-dataset across atlases if so desired similar to

what was contemplated in https://bids.neuroimaging.io/bep035 : https://bids.neuroimaging.io/bep035

inline with potential assembly of BIDS dataset from smaller subj/session ones Allow composition of a BIDS dataset (dataset level) from smaller (subj or subj/ses) level bids-2-devel#59

but the question is even - how common is to bring up multiple atlases here into a single dataset?

Was it considered at all to just make "one atlas per dataset" and thus introduce DatasetType "atlas" and thus use dataset_description.json to provide all the gory details (with huge overlap which what current dataset_description.json has)???

This has taken probably 80% of the time for this BEP (meaning, this is not proposed this way lightly).

how common is to bring up multiple atlases here into a single dataset?

A lot (big lot).

replying to @oesteban 's comment in the main thread (can't be collapsed/cleaned there):

At large IMHO that violated existing BEP Guidelines recommendation on atomic changes: https://bids.neuroimaging.io//extensions/guidelines.html#facilitate-atomic-changes

Creating a new DatasetType seems way less atomic than creating new entities to me,

I proposed above defining a new DatasetType (if needed to tell it apart from other derivative datasets, but may be not even needed) as an alternative not to a new entity (or entities, here - atlas, tpl, cohort which I did not voice against) but to a new principled file type {entity}-<label>_description.json, which is not following any prior known convention AFAIK to be defined on top level. I would not discuss "precedence cases" of other BEPs here ATM.

What I mean is that, for much I like the atomicity principle it is not applicable as it stands at the moment. Every community member has a slightly different understanding of BIDS and what is atomic and what's not.

Are you arguing against {entity}-<label>_description.json or indicating that, for the atomicity principle it should be introduced in another BEP as a more general rule?

Are you arguing against {entity}-_description.json or indicating that, for the atomicity principle it should be introduced in another BEP as a more general rule?

yes and yes. As if not considered more generally, it should not be introduced and rather already existing approaches should be used even if they are neither formalized fully yet (#2283).

DatasetType "atlas"

This was proposed in 2023 and abandoned in 2024.

Are you arguing against {entity}-<label>_description.json or indicating that, for the atomicity principle it should be introduced in another BEP as a more general rule?

yes and yes. As if not considered more generally, it should not be introduced and rather already existing approaches should be used even if they are neither formalized fully yet (#2283).

Not sure what else to say, but we've had several years of back and forth on this, and finally a review period that produced no dissenting votes. I'm not sure what we should have done, procedurally, but it is out of scope for a PR review to propose such changes.

Not sure what else to say, but we've had several years of back and forth on this

ah, having atlases.tsv was discussed and rejected? (I wonder if "lists" formatting for authors etc was the reason)

I'm not sure what we should have done, procedurally, but it is out of scope for a PR review to propose such changes.

as we already have for entities, IMHO there should have been a proposal/discussion to introduce a principally new file type {entity}-<label>_description.json file with atlas-<label>_description.json as an example (pointing to this BEP), in favor of already present in BIDS {entity_plural}.tsv files.

NB IMHO it is actually a relatively small "serialization" aspect here as all those could be easily expressed as a more conventional atlases.tsv (and missing ATM templates.tsv and spaces.tsv) and we all could remain friends -- stay consistent at a minor question of formatting of those few list entries in the .tsv, while discussing potentially a better fit format to replace .tsv in such cases.

oesteban · 2025-12-17T20:35:46Z

At large IMHO that violated existing BEP Guidelines recommendation on atomic changes: https://bids.neuroimaging.io//extensions/guidelines.html#facilitate-atomic-changes

Creating a new DatasetType seems way less atomic than creating new entities to me, and also potentially by the principle you linked to (it would fall under BIDS principles). Creating new datatype folders seems way less atomic than creating new entities (to me again, from what I understand from the guidelines), and yet most BEPs happily add new datatypes with no objection.

With its current implementation (a paragraph that points to a PR), the atomicity principle is largely in the eyes of the beholder (unfortunately, because I think it is a fundamental perspective we should keep in mind and try to enforce in some way).

This is to say, this BEP currently tries to make changes as atomic as possible (saving the cohort discussion for which I have no more time, if it drops, I will not say a thing).

Co-authored-by: Yaroslav Halchenko <[email protected]>

Concrete suggestions addressed. Other conversations out of scope for post-BEP review.

effigies

I had a full read through, and AFAIC this is ready for merge. One last comment for consistency.

src/schema/objects/metadata.yaml

This change makes BEP038's atlas metadata approach consistent with existing BIDS conventions for entity-specific TSV files (like participants.tsv, sessions.tsv, descriptions.tsv). Changes: - Replace atlas-<label>_description.json with atlases.tsv - Add templates.tsv for template metadata - Add tpl-<label>_cohorts.tsv for cohort metadata within templates - Support tpl-<label>_atlases.tsv for template-level atlas metadata The atlases.tsv includes columns derived from the original JSON fields: - atlas_id, name, license (REQUIRED) - description, authors (RECOMMENDED) - curators, funding, references_and_links, species, sample_size, derived_from, level_type, rrid (OPTIONAL) Array fields (authors, curators, funding, references_and_links) use JSON-encoded strings in TSV cells. Schema changes: - Added column definitions for all atlas/template/cohort TSV columns - Added suffix definitions for atlases, templates, cohorts - Added tabular_data rules for column validation - Added file rules supporting both root-level and template-level naming - Removed old atlas_description JSON-based rules and associations This addresses the inconsistency noted in: - Issue #2285: atlas-<label>_description.json pattern introduced by BEP038 - Issue #2283: Proposal to formalize {entities_plural}.tsv pattern This is a follow-up to PR #1714 (BEP038: Templates and atlases). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

Replace atlas-<label>_description.json with atlases.tsv for atlas metadata, consistent with existing entity-specific TSV files (participants.tsv, sessions.tsv). Key changes: - atlases.tsv: placed at derivative dataset root or tpl-<label>/ directory - Metadata fields converted to snake_case column names - Array fields (authors, curators, funding, references_and_links) use JSON encoding Schema changes: - Add atlas_id and all atlas-specific columns to columns.yaml - Add atlases suffix to suffixes.yaml - Add Atlases rule to common_derivatives.yaml - Add atlases file rule to tables.yaml - Remove old atlas_description association, context, and metadata rules - Delete atlas.yaml check and file rules Documentation: - Update atlas.md with atlases.tsv section and updated examples - Add JSON-encoded array note - Replace all atlas-<label>_description.json references with atlases.tsv Closes: #2285 Related to: #2283 Related to: #1714 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

yarikoptic · 2025-12-19T00:52:27Z

note: sorry, I made some wrong push, reverted right away. those changes are now in dedicated PR(s)

Replace atlas-<label>_description.json with atlases.tsv for atlas metadata, consistent with existing entity-specific TSV files (participants.tsv, sessions.tsv). Key changes: - atlases.tsv: placed at derivative dataset root or tpl-<label>/ directory - Metadata fields converted to snake_case column names - Array fields (authors, curators, funding, references_and_links) use JSON encoding Schema changes: - Add atlas_id and all atlas-specific columns to columns.yaml - Add atlases suffix to suffixes.yaml - Add Atlases rule to common_derivatives.yaml - Add atlases file rule to tables.yaml - Remove old atlas_description association, context, and metadata rules - Delete atlas.yaml check and file rules Documentation: - Update atlas.md with atlases.tsv section and updated examples - Add JSON-encoded array note - Replace all atlas-<label>_description.json references with atlases.tsv Closes: #2285 Related to: #2283 Related to: #1714 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>

melanieganz and others added 17 commits January 23, 2024 09:23

First draft of the atlas spec

9878702

I made a first draft of the atlas BEP based on the Google doc

Added proper content

8304769

Added atlas content

Fixed details

81d5ccc

Fixed details such as linking to entities and suffixes as well as clarified some of the example text. The tables need still to be fixed.

case 3 files renamed

7626992

update wording and fix typos in atlas.md

11ed800

add draft for outlining desc- usage

1129d8d

add ref to template and lnter

6881c9c

Remove trailing whitespace, normalize indentation to two spaces

e2b3ddb

Run codespell and aspell

f7a268d

Replace i.e. and e.g.

a96ab79

Line break at sentences, format nested list as markdown

18e2bcc

Replace unicode quotes with ASCII

803f0d4

Include atlas.md in table of contents

dda4790

Semantic line breaks, mostly at commas

ba30c0e

FIX: Whitespace in schema.objects.entities.atlas.description

9c803d7

Merge pull request #1 from effigies/bep038

33e55b5

[BEP038] Style and similar cleanups

effigies added the BEP label Feb 28, 2024

effigies requested review from erdalkaraca and tsalo as code owners February 28, 2024 16:23

Merge branch 'master' into bep038

9baaeec

melanieganz mentioned this pull request Mar 5, 2024

BEP Proposal: Atlas specification #1281

Open

effigies marked this pull request as draft April 4, 2024 19:53

bids-standard locked as too heated and limited conversation to collaborators Apr 4, 2024

christinerogers marked this pull request as ready for review April 5, 2024 17:02

sappelhoff marked this pull request as draft April 5, 2024 17:12

effigies and others added 5 commits December 4, 2025 17:10

Revert "Update src/schema/objects/metadata.yaml"

d9b2404

This reverts commit 0511805.

Update src/schema/objects/metadata.yaml

5ffae33

Co-authored-by: Mark Mikkelsen <[email protected]>

Update src/schema/objects/metadata.yaml

2b9cd15

Co-authored-by: Mark Mikkelsen <[email protected]>

Update src/schema/objects/metadata.yaml

c5f6770

Co-authored-by: Mark Mikkelsen <[email protected]>

Update src/schema/objects/metadata.yaml

5d30520

Co-authored-by: Mark Mikkelsen <[email protected]>

yarikoptic mentioned this pull request Dec 17, 2025

Formalize the concept of [{leading entities}_]{entity_plural}.{tsv,json} file(s) #2283

Open

2 tasks

yarikoptic previously requested changes Dec 17, 2025

View reviewed changes

effigies and others added 3 commits December 18, 2025 13:03

Update src/derivatives/atlas.md

6f24a19

Co-authored-by: Yaroslav Halchenko <[email protected]>

Apply suggestions from code review

b26b3fd

Co-authored-by: Yaroslav Halchenko <[email protected]>

chore: Remove accidentally added lockfile

bdb34e0

effigies mentioned this pull request Dec 18, 2025

test(bst): Narrow glossary validation test #2284

Merged

effigies commented Dec 18, 2025

View reviewed changes

src/schema/objects/metadata.yaml Show resolved Hide resolved

Merge branch 'master' into bep038

8f49cd2

yarikoptic mentioned this pull request Dec 18, 2025

Formalize concept of '{entity}-<label>_description.json' #2285

Open

yarikoptic force-pushed the bep038 branch from c1098d7 to 8f49cd2 Compare December 18, 2025 23:07

yarikoptic mentioned this pull request Dec 18, 2025

Replace atlas-<label>_description.json with atlases.tsv pattern and also add templates.tsv and cohorts.tsv #2286

Open

effigies added 3 commits December 19, 2025 02:27

Merge branch 'master' into bep038

5d6e720

chore: Prettify YAML

c85adba

Merge branch 'master' into bep038

e7ab432

yarikoptic mentioned this pull request Dec 19, 2025

BEP on controlled vocabularies support/linkage #2289

Open

8 tasks

Update src/schema/objects/metadata.yaml

7b71063

effigies merged commit 074ae6f into master Jan 7, 2026
27 checks passed


		### Atlas identification and metadata

		The `atlas-<label>_description.json` file provides metadata to uniquely identify,

BEP-038: Atlases #1714

BEP-038: Atlases #1714

Conversation

effigies commented Feb 28, 2024 • edited by yarikoptic Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rendered draft

Related discussions

Associated PRs

Uh oh!

CPernet commented Mar 31, 2024

Uh oh!

oesteban commented Apr 1, 2024

Uh oh!

effigies commented Apr 1, 2024

Uh oh!

effigies commented Apr 4, 2024

Uh oh!

yarikoptic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yarikoptic Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

oesteban Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

yarikoptic Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

oesteban Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

yarikoptic Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

effigies Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

yarikoptic Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

oesteban commented Dec 17, 2025

Uh oh!

effigies left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yarikoptic commented Dec 19, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

effigies commented Feb 28, 2024 •

edited by yarikoptic

Loading