<andi->
ekleog: good point. I am trying to avoid flat files for now. It does very often not feel right. I see the point of having a file for that though.. In the initial mock I did I put a $something.toml/yaml file in each CVEs directory which would have that job. Loading all CVEs in memory (parsing those files) will be doable for the foreseeable future I think..
<andi->
I also started thinking about different platforms. E.g. ams64, i686, aarch64,... how much differentiation must we do there?
ckauhaus has joined #nixos-security
<ckauhaus>
andi-: I saw your issue db example yesterday - what is it supposed to do?
<ckauhaus>
tell me more about it
ckauhaus has quit [Remote host closed the connection]
<andi->
and gone he is m(
* andi-
writes writing a README on the TODO list
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Ping timeout: 276 seconds]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-security
<ekleog>
andi-: Theoretically CVEs can apply to only some platforms. IMO, this is significant only for things like the GNU/Linux kernel (sorry couldn't resist) or Xen
__Sander__ has joined #nixos-security
<ekleog>
And even when it's significant… well, I think all useful metadata we could add are CVE-level only (like, is it fixed or not, is it nonsa/wontfix or not…), and if an end-user wants to know whether they are affected, they likely will want to check the full CVE details anyway, so they may as well just read the architecture at the same time
<ekleog>
ckauhaus: as people often (try to) answer when you're gone, I'd suggest you find a bouncer somewhere -- if you can't self-host one, an account on matrix.org does the job, with the bridge ;) (and there are likely other free bouncers, but I don't know them)
<ckauhaus>
yeah, that would be a good idea
<ckauhaus>
I'm sometimes on a quite flaky Internet connection
<andi->
ekleog: I agree. Sounds like a reasonable approach to that. I'll try to finally fix the symlinking tonight.. turned out to be more complex to do from within rust then with bash... :) (at least the "place a relative symlihk somewhere else" thing)
ckauhaus has quit [Remote host closed the connection]
<ekleog>
Great, thanks!
<andi->
ekleog: what would be a (meta) data format for us to keep information on such issues? A simple text file? A YAML file where we can address each affected pkg of a CVE itself? What would we even store there? (URLs to upstream bugs, fixes, annoucements, ...)
ckauhaus has joined #nixos-security
<ekleog>
Hmm… URLs to upstream bugs, fixes and announcements are (theoretically) in the CVE database by NIST, though sometimes it's… hard to figure out which link is which
<ekleog>
Currently I can only think of “fixed” and “wontfix”, and as “fixed” is auto-computed I can only think of “wontfix” that would have to be manually set
<ekleog>
Having it as a YAML file in eg. issues/[CVE]/metadata.yaml sounds good to me (and would allow removing the `.keep`)
<ekleog>
(containing something like, `nonsa: true` for wontfix'd tasks, and that leaves space for other metadata)
<ekleog>
Oh, we could add in metadata like “tracked by nixpkgs issue #X”
<ekleog>
Which would likely be great for hard-but-important-to-fix issues
<ekleog>
That means the tool must provide a way to list only the not-yet-wontfix'd CVEs for a given / the last eval, though
<ekleog>
(as that can change between filesystem layout generation and read)
<ekleog>
Oh, another piece of metadata potentially important: “public”. Currently we only handle public CVEs, but if we are to ever set foot in distros@, then it'll come in handy to have prepared for these
<andi->
Yeah, I'd see those as a version of the repository that is "hidden" (access restricted, not on GitHub)
<andi->
would then basically be a branch that we merge after the disclosure
<gchristensen>
nice, elegant
<andi->
One could even REQUIRE a `public: true` field in metadata.xml for things to be published
<andi->
and then automate the publishing..
<andi->
but that is probably still very far away
<gchristensen>
maybe should enforce a pre-push hook though that prevents publish pushing of private things
<andi->
yep that was also my tought. I would not put too much effort into that right now. We are able to just grep `public:\s*[tT]` that later on
<ekleog>
Elegant indeed :)
ckauhaus has quit []
andi- has quit [Ping timeout: 276 seconds]
andi- has joined #nixos-security
<pie_>
why yaml if you're already using json
<gchristensen>
he's been using yaml this whole time
<pie_>
oh
<gchristensen>
I say slightly trollingly, since json is valid yaml
<pie_>
the loop | outside | me
<pie_>
wait i thought yaml was some indented "name: value" without {} stuff
<joepie91>
pie_: that's just *one* of the valid syntaxes!
<gchristensen>
"The primary objective of this revision is to bring YAML into compliance with JSON as an official subset." -- http://yaml.org/spec/1.2/spec.html :)
<andi->
pie_: I am biased towards YAML if I am supposed to edit manually.. I don't deal with {,}, ", [ & ] if I don't have to.. :)
<gchristensen>
just all the other weird stuff that yaml brings :)
<joepie91>
YAML annoys me because it has a great premise and fucks it up by being unreasonably ambiguous and extensive
<joepie91>
but it's Good Enough that nobody seriously tries to make something better with the same premise
__Sander__ has quit [Quit: Konversation terminated!]
<andi->
joepie91: I am open for everything that has reasonable well language support in many languages.. After all we would want to stick with it for a while and/or be able to migrate in $years
<gchristensen>
+1
<andi->
I would probably like markdown for the freedom it gives but parsing data from that wasn't enjoyable the last time I tried that
<gchristensen>
+1
<joepie91>
andi-: right, the problem is *getting* that language support, because when you try to write a somewhat-better alternative (or a much-better alternative where the benefits are not immediately obvious) for something that broken-ish but also works and is widely deployed, the first question is usually "why don't you just use $existingThing?" and that seriously hampers adoption
<joepie91>
and language support comes from many people being interested in supporting it
<joepie91>
s/that/that's/
<gchristensen>
sure
<gchristensen>
but we don't need to fight that fight right here
<gchristensen>
I hear what you're saying
<joepie91>
I've seen this issue a lot; things that are subtly broken rarely get fixed, whereas things that are really really broken get fixed quickly
<joepie91>
oh yeah, sure
<joepie91>
just as a general note of annoyance :)
<joepie91>
not towards people here specifically
<gchristensen>
I think it is reasonable to assume this tooling could exist a decade from now
<andi->
joepie91: what are the features what we need to get right in a language for our use cases? I don't see much where the typical YAML things get in my way. I am just curious what should be improved/changed in YAML from your point of view.
<joepie91>
andi-: the two major issues in YAML are 1) the extensibility to the point of security vulnerabilities when used naively, and 2) the rather ridiculous amount of permutations of valid syntax and the resulting syntax ambiguities, resulting in difficult-to-spot misparses
<joepie91>
1 is fixable on a per-project basis (only ever use safeload) but is still an ecosystem issue
<joepie91>
2 is just... painful
<andi->
the extensibility is an issue. I agree... It is also not practical if you want to share documents since the other party might not support it.
<andi->
I've seen the YAML extensions being used in python to execute code.. not very nice
<flokli>
andi-: what about the initial idea, keeping the structure of the yaml inside a file-system-like hierarchy?
<andi->
there is no YAML /o\
<flokli>
Fine, json :-P
<andi->
there are JSON fields that are scan results of EVERYTHING. I am not going to store every single unrelated package just now. I'll keep an archive of those. THey can be regenerated easily (in a few hours of CPU time) anyway.
<andi->
I do not want to also invent a tool to work with the repository... I can barely work on this...
* andi-
goes back to debugging freeradius
<flokli>
Oh yay
<gchristensen>
andi-: screw the feedback, write it with whatever tool you want
<gchristensen>
let's get it done we can kibitz over formatting when we have a cool new tool helping us kick ass with security patching
<andi->
I like feedback :) Even negative one... It keeps one grounded and open minded.
<joepie91>
andi-: gchristensen: btw, to be clear, I'm not against using YAML in things; just important to be aware of the issues :)
<gchristensen>
yeah, of course, I agree
<gchristensen>
but "how do we store the human readable stuff on disk?" isn't a happy place to get stugck
<flokli>
Right :-)
<andi->
worst case: Write it in any format you like. THere will just be not tooling that reads/aggegrates it :D
<gchristensen>
+1
<gchristensen>
(how about we store everything as GPG keys on public SKS servers)