Revise QA-04.01 to apply only to projects with subprojects, move to level 2

[evankanderson]

As discussed in the 2025-11-25 meeting.

In my opinion, if projects with only a single repository don't need to do anything (e.g. no security-insights or SBOM) to indicate that status, then the assessment is not harmful but maybe not useful. In particular, it becomes difficult to assess whether an absence of information means that there is are no sub-projects, or that the control has not been followed. (In general, sub-projects and the scope of "what is a project" is tricky, and I believe we aren't clear enough in our own documentation yet. Our current definition is "A group of people and resources that coordinate to produce a release.") I would be _happy_ to support more work on allowing projects to define "this is our set of assets" in an affirmatory way, and to require it at a certain maturity level. Ideally, this would include not just software repositories, but also distribution channels and other resources.

I believe the concern behind QA-04 as a whole is that a level-3 project ("do-important-thing") might choose to put all their less-savory practices in a sub-project (e.g. "dirty-laundry") and then consume that project through a dependency mechanism. It's not clear to me how relevant that concern is, given that a project might also depend on a third-party or generic library with the same security properties.

A more important property (to me) is being able to affirmatively know that consuming things from the @babel NPM prefix are owned by https://github.com/babel/babel (for example). I don't know if we have a mechanism or control for that today, but (if widely adopted) that seems helpful for distinguishing typoquatting and other supply chain attacks.

[funnelfiasco]

Strictly speaking, this is (as I understand it) less about dependencies and more about different repos for different parts of the same end "product" (e.g. a CLI tool and a web app). That's probably not important for this change, but I just wanted to write it down in case it sparks further conversation.

I initially supported keeping this as a Level 1, but on further discussion, I was convinced that the intersection of "immature" projects and projects with this complexity is small enough to make a 2+ reasonable.

[eddie-knight]

I like adding the condition to omit this requirement for projects with only one repo. A manual assessment of the baseline should be able to skip over this without any fuss. 👍

But I strongly disagree on it being a maturity 2 item.

Any project with multiple repos should already be doing this. It is unlikely that they won't document their subproject repos.

And adding it is a very simple change. For projects that want manual review this could go in the README.md, for projects that want automated reviews this could live in the SECURITY-INSIGHTS.yml.

If they fail to provide this information, we can't confidently assess whether the project is actually following the baseline. And for end users of projects with build or delivery processes hosted in separate repos, this will NOT be tracked in dependency management. Those will become shadow repos, unseen and unknown by anyone outside of core contributors.

[funnelfiasco]

Adding a fake nack to allow more time for discussion

[evankanderson]

If they fail to provide this information, we can't confidently assess whether the project is actually following the baseline. And for end users of projects with build or delivery processes hosted in separate repos, this will NOT be tracked in dependency management. Those will become shadow repos, unseen and unknown by anyone outside of core contributors.

Ah -- you're imagining a situation where (for example) the CI for a project is in a different repo from the main software development, and the linkage isn't clear? I'll buy that, though I'm not sure for a 1-person project that the payoff is substantial.

If we are going to require this, it should be an inventory of all the project components, including additional services beyond source code repos that are used by the project. For example, a Jenkins or TravisCI instance which is configured outside the repo itself, or services like Codecov.

[eddie-knight]

I agree that if it was purely CI we're discussing, that complexity probably wouldn't merit level 1.

But it's not always just CI. And the simplicity of just naming your subproject repos renders that moot anyhow. There is virtually no complexity, it affects a low number of projects with the added condition statement, and it creates a LOT of value for folks who are doing security reviews.

[evankanderson]

Okay, looking at all the text in the control and assessment, we get:

Any additional subproject code repositories produced by the project and compiled into a release MUST enforce security requirements as applicable to the status and intent of the respective codebase.

and then:

While active, the project documentation MUST contain a list of any codebases that are considered subprojects.

With the follow-on .02 assessment adding:

When the project has made a release comprising multiple source code repositories, all subprojects MUST enforce security requirements that are as strict or stricter than the primary codebase.

So this pretty clearly does not cover CI-only repos (or CI at all). Thinking further about CI, supporting repos containing CI configuration might span multiple projects. In fact, one common case would be to have workflows in the .github repo which apply to all the projects under a larger organization which release separate projects. The .github files aren't owned by any one project, but rather are consumed like a service from the central organization.

My $0.02 would be to change this to:

1. Have a list of projects assets (e.g. repos, CI accounts, package manager accounts, etc)
2. All project assets enforce security requirements as strict or stricter than the baseline standard

The current language and requirements is sort of weird -- we already (QA-02) require a list of dependencies which should capture additional first and third party repos "compiled into a release". I'm not arguing against having a consistent policy across a project's repos, but I'm trying to figure out why that's something that baseline cares about, and why "subprojects" are an important distinction from other types of assets.

Concretely, I seem to recall at one point that the GUAC visualizer was a different project from the GUAC backend. Is there a subproject relationship there, or are they two projects that work well (or even have a dependency), but QA-04 doesn't apply at all.

[funnelfiasco]

Concretely, I seem to recall at one point that the GUAC visualizer was a different project from the GUAC backend. Is there a subproject relationship there, or are they two projects that work well (or even have a dependency), but QA-04 doesn't apply at all.

We consider them as distinct projects from a "things we ship to users" perspective. GUAC Visualizer is "experimental" and not released with GUAC proper, even though the Visualizer is useless without a running GUAC instance.

The current language and requirements is sort of weird -- we already (QA-02) require a list of dependencies which should capture additional first and third party repos "compiled into a release".

There's compiled and then there's compiled. QA-02 is about things that are compiled in the "we created a binary" sense. QA-04 is about things that are compiled in the "we put a bunch of things into a nice box for your convenience" sense. We should probably not use "compiled" in -04 since that term means something specific to computer people, even though the usage is technically correct from a general English perspective.