samueldr changed the topic of #nixos-dev to: #nixos-dev NixOS Development (#nixos for questions) | NixOS 20.03 Feature Freeze Feb 10 https://discourse.nixos.org/t/nixos-20-03-feature-freeze/5655 | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | https://r13y.com | 19.09 RMs: disasm, sphalerite; 20.03: worldofpeace, disasm | https://logs.nix.samueldr.com/nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
callahad8707 has joined #nixos-dev
Willi_Butz has joined #nixos-dev
cole-h_ has joined #nixos-dev
callahad870 has quit [Quit: Ping timeout (120 seconds)]
cole-h has quit [Ping timeout: 272 seconds]
WilliButz has quit [Ping timeout: 272 seconds]
marek has quit [Ping timeout: 272 seconds]
MoreTea has quit [Ping timeout: 272 seconds]
marek has joined #nixos-dev
MoreTea has joined #nixos-dev
betawaffle has quit [Quit: Oh noes, my ZNC!]
betawaffle has joined #nixos-dev
FRidh has quit [Ping timeout: 272 seconds]
FRidh has joined #nixos-dev
cole-h_ has quit [Quit: WeeChat 2.7]
cole-h has joined #nixos-dev
ris has quit [Ping timeout: 265 seconds]
lovesegfault has quit [Ping timeout: 260 seconds]
Emantor has quit [Quit: ZNC - http://znc.in]
Emantor has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
orivej has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.6]
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
<{^_^}> resolved: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
justanotheruser has quit [Ping timeout: 260 seconds]
bhipple has quit [Ping timeout: 260 seconds]
bhipple has joined #nixos-dev
zarel_ has joined #nixos-dev
zarel has quit [Ping timeout: 265 seconds]
orivej has quit [Ping timeout: 265 seconds]
justanotheruser has joined #nixos-dev
cole-h has quit [Ping timeout: 240 seconds]
bhipple has quit [Ping timeout: 240 seconds]
eyJhb has quit [Quit: Clever message]
eyJhb has joined #nixos-dev
eyJhb has joined #nixos-dev
eyJhb has quit [Changing host]
eyJhb has quit [Quit: Clever message]
eyJhb has joined #nixos-dev
eyJhb has joined #nixos-dev
__monty__ has joined #nixos-dev
janneke_ has joined #nixos-dev
janneke has quit [Ping timeout: 260 seconds]
orivej has joined #nixos-dev
ris has joined #nixos-dev
ris has quit [Ping timeout: 272 seconds]
<yorick> gchristensen: I use unsafeDiscardStringContext to get /nix/store/... paths that aren't in the store yet
<gchristensen> I feel that if you do something with a file and get a `/nix/store` path, it is reasonable to expect it *might* be in the store with that name, something that should never be true with files containing secrets
<Profpatsch> why do you people want to handle secrets with nix anyway. It’s a bad fit.
<gchristensen> +100
<gchristensen> we're not equipped to do it with our current tools
<simpson> Profpatsch: Often, doing some mind-reading, it seems like folks' *current* secret-handling practices aren't safe. The unsafety only becomes obvious when they are forced by Nix to assemble everything up-front.
<Profpatsch> i’ve researched into secrets handling for a deployment recently, and it’s heck of a mindbending topic for sure
<gchristensen> it is super important to handle secret lifecycle properly, and ideally support rotation out of the box
<Profpatsch> My current tl;dr is 1) use fifos 2) do it at runtime 3) never ever use tempfiles
<Profpatsch> And that’s just the scripting part, not even the operational things like rotation.
<gchristensen> my tl;dr would be "use an existing tool designed to securely handle secrets" :P
<Profpatsch> Ideally you never have any secrets in transport that are valid for longer than a few minutes
<Profpatsch> however, then you need an authority like a Vault server, which has to be actively running somewhere and is a single point of failure.
<simpson> Yep. We need tools and languages that make it easy to manipulate a secret as to manipulate a non-secret; we need systems where values can be secret by default.
<Profpatsch> And then there’s the split between authorization and identification (did I get that right?) that would ideally be handled by different tools
ixxie has quit [Ping timeout: 240 seconds]
<gchristensen> secrets should be treated as monadic by default (who am I and who taught me that word) ~~ usually it is worded as authentication (who are you) and authorization (are you allowed to)
<Profpatsch> yeah, and if you use “identification” instead of “authentication” it’s even possible to tell the words apart :)
<simpson> Profpatsch: Only authorization matters; identification is not just a red herring, but actively harmful. http://habitatchronicles.com/2017/05/what-are-capabilities/ for an introduction to this set of ideas.
<gchristensen> simpson++ :D
<{^_^}> simpson's karma got increased to 17
<Profpatsch> see, we don’t even have a common ground for these things yet.
<Profpatsch> Which reminds me that Information Theory isn’t even a hundred years old yet.
<simpson> gchristensen: Sure, we can fuse these things together. In ocap literature, this is known as the Horton pattern; these would be authorizations which have additional tamperproof metadata indicating issuer identity.
<gchristensen> I think you're describing traditionally how orgs think about access control, and simpson is thinking about capability theory which (IMO, and Isimpson'sO) is a much safer, more powerful mechanism
<simpson> Like a cashiers' check or a TLS certificate, while you can't know who is currently trying to present it, you can know who originally issued it.
<Profpatsch> simpson: But very thanks, great link, will read.
<Profpatsch> simpson++
<{^_^}> simpson's karma got increased to 18
<gchristensen> Profpatsch: definitely read :) simpson's perspective has given me a lot to think about... and helped me build better things
<simpson> I'm just trying to be less obscurantist than the typical ocap author. I know that they mean well, but MarkM and kpreid can be almost completely impossible to decipher.
<simpson> Just like with other design patterns, ocap patterns are descriptive, so when I say things like "Nix tries to talk about packages as capabilities", I'm not trying to make confusing metaphors, but to explain how I'm interpreting the design features.
<gchristensen> +1 you've made it really accessible to me -- somebody who is nearly completely unequipped to read, and inexperienced at reading, academic writing
<simpson> Thanks! I try. I think Chip (habitatchronicles.com) does a much better job, which is why I link him.
<gchristensen> nice, thank you
johnny101m2 has joined #nixos-dev
johnny101m2 has quit [Client Quit]
johnny101m has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
zarel has joined #nixos-dev
zarel_ has quit [Ping timeout: 268 seconds]
<gchristensen> does anybody recognize this problem on 20.03, and know what fixed it on unstable?in job ‘tested’:cannot coerce null to a string, at /nix/store/m2bnkxvdwsyxkrsx3l0dfh4va7k6c0fd-source/nixos/modules/installer/cd-dvd/channel.nix:17:5
justanotheruser has quit [Ping timeout: 272 seconds]
cole-h has joined #nixos-dev
<gchristensen> hydra's evaluator is emitting unhappy messages like this: Feb 15 17:58:19 ceres hydra-evaluator[26990]: aggregate job ‘nixpkgs.buildRustCrateTests.test.x86_64-linux’ has a constituent ‘/nix/store/1jk8l8njqmmnhcnwcw9qsaiy5a74y6il-run-buildRustCrate-rustLibTestsDefault-test.drv’ that doesn't correspond to a Hydra build
<gchristensen> cc Profpatsch who I think knows thinsg about these .test things
ris has joined #nixos-dev
<gchristensen> there is a problem with the new test framework
<gchristensen> flokli: you around?
<gchristensen> I don't want to go reverting a bunch of stuff
<Profpatsch> gchristensen: what problem?
<Profpatsch> gchristensen: andi- added this attribute according to git
<gchristensen> what problem -> with the test framework?
* andi- never touched the python tests
<Profpatsch> I don’t know about hydra, but that attrset has been in nixpkgs since late 2018
<Profpatsch> andi-: It’s about buildRustCrateTests as far as I can see
<andi-> Profpatsch: yeah, I can see that. It has been in master for a while
<Profpatsch> yeah, that’s what I’m saying
<gchristensen> note hydra isn't super-super mad about it, but hydra is only a little unhappy about it
<Profpatsch> I don’t even know what the hydra message means :)
<andi-> same
<gchristensen> the problems with the test framework are 100% blockers
<andi-> I can work on a solution if I would understand why it is mad :/
<Profpatsch> gchristensen: one above?
<andi-> It probably requires all of the constituents to be individual goals and not just the combined attribute?
<gchristensen> I don't know about the `.test` message, I've just found it while debugging a different issue
<Profpatsch> (I just grepped for "tested"
* gchristensen is writing a bit
<Profpatsch> worldofpeace added that last November ^
justanotheruser has joined #nixos-dev
<flokli> gchristensen: pong
<gchristensen> that is unrelated
<Profpatsch> Then I don’t know, I can’t find another "tested".
<gchristensen> tested is a job, in the nixos jobset (writing)
<Profpatsch> Ah, `tested.=` gives some results
<gchristensen> the issue I'm chasing with the test framework is this: in job ‘tested’:cannot coerce null to a string, at /nix/store/m2bnkxvdwsyxkrsx3l0dfh4va7k6c0fd-source/nixos/modules/installer/cd-dvd/channel.nix:17:5 which is happening for these NixOS tests: nixos.tests.installer.lvm.x86_64-linux nixos.tests.containers-imperative.x86_64-linux nixos.tests.boot.biosCdrom.x86_64-linux
<gchristensen> nixos.tests.installer.separateBoot.x86_64-linux nixos.tests.containers-ip.x86_64-linux nixos.tests.installer.simple.x86_64-linux
<flokli> gchristensen: when did this start to appear?
<Profpatsch> who in the world uses null anyway
<flokli> Thanks Profpatsch :-/
<flokli> I'm pretty sure I ran some of these tests before they were merged.
<gchristensen> the last evaluation to have the `tested` jobset (ie: the last evaluation which did not have this problem) is 8130f3c1c2b
<gchristensen> that hasn't changed in months, but it is possible related
<flokli> gchristensen: I'm not on my computer right now, so I can't dive into it in detail. But I'm very sure ofborg eval passed.
<gchristensen> I know it did too
<flokli> Could it be some of the "instantiate nixpkgs earlier" fixes, which caused this specifically to break?
<gchristensen> I'm not looking for blame or to find problems in our process right now, right now I just want to find the solution
<flokli> Yeah sure
<gchristensen> on on the release-20.03 branch, this w ill replicate: nix-build --arg nixpkgs '{ outPath = ./scratch/nixpkgs/source; rev = "50edd0f565aa26a557b8848316bdc4631782b004"; shortRev = "50edd0f"; revCount = 212937; }' -I nixpkgs=./scratch/nixpkgs/source --arg stableBranch false --arg supportedSystems '[ "x86_64-linux" ]' --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux
<gchristensen> ./scratch/nixpkgs/source/nixos/release-combined.nix -A tested --show-trace
<gchristensen> oops, delete ./scratch/scratch and just point to a checkout
<gchristensen> flokli: I don't know of those patches ...
<flokli> gchristensen: I'm wondering if https://github.com/NixOS/nixpkgs/issues/79907#issuecomment-586296798 is included in 20.03 now, and it could have caused that.
<gchristensen> interesting
<flokli> I could run a bisect in like two hours or so, but currently only on my mobile phone, sorry.
<gchristensen> flokli: I'll start a bisect, good thinking
<Profpatsch> wait, that’s haskell packages stuff
<Profpatsch> gchristensen: What do you call to reproduce the error?
<gchristensen> on on the release-20.03 branch, this w ill replicate: nix-build --arg nixpkgs '{ outPath = ./scratch/nixpkgs/source; rev = "50edd0f565aa26a557b8848316bdc4631782b004"; shortRev = "50edd0f"; revCount = 212937; }' -I nixpkgs=./scratch/nixpkgs/source --arg stableBranch false --arg supportedSystems '[ "x86_64-linux" ]' --option extra-binary-caches https://hydra.nixos.org/ --option system x86_64-linux
<gchristensen> ./scratch/nixpkgs/source/nixos/release-combined.nix -A tested --show-trace
<Profpatsch> Ah, missed your message
<Profpatsch> wtf
<Profpatsch> How do you even find out stuff like this
<gchristensen> go to any failing build: https://hydra.nixos.org/build/112827931 click Actions, click Reproduce
<gchristensen> download, edit the "-A" to "tested"
<Profpatsch> when I run that I just get the pseudo-package error error: Please be informed that this pseudo-package is not the only part of
<Profpatsch> Arghl, the message is split
<Profpatsch> Is there a more specific argument that prompts the error?
<gchristensen> dunno
<gchristensen> nixos.tests.containers-ip.x86_64-linux probably
<Profpatsch> doesn’t look like it
<gchristensen> then I don't know :P
<gchristensen> time for breakfast
<Profpatsch> I’m afraid I don’t have the amount of RAM necessary for evaluating tested
<gchristensen> oack
<gchristensen> thank you for trying :)
<gchristensen> ah Profpatsch I have more info for you
<gchristensen> duplicate the nixpkgs repo and then `rm -rf .git` and then run -A nixos.tests.installer.simple.x86_64-linux
<gchristensen> that should replicate. back in a couple hours or less
<Profpatsch> gchristensen: Ah, you can restrict if you comment out ever constituent except the failing test
<Profpatsch> my guess: a builtin returns `null` for a missing directory
<Profpatsch> that’s 5 days ago
<Profpatsch> would you look at that! https://a.uguu.se/bxvpXqztgUKn_screenshot.png
<Profpatsch> ^ gchristensen
<Profpatsch> found it
<Profpatsch> cc niksnut ^
<Profpatsch> can’t just embiggen the type of an option in an untyped language without swaths of pain for all downstream users.
pie_[bnc] has quit [Quit: pie_[bnc]]
pie_[bnc] has joined #nixos-dev
justanotheruser has quit [Ping timeout: 272 seconds]
<arianvp> Maybe not silently merge large experimental changes just before a branch-off? :/ Also shouldn't this have been caught by running ofborg tests before merging?
ixxie has joined #nixos-dev
<arianvp> Sorry that came out more salty than intended. Is meant to be constructive.
zarel has quit [Ping timeout: 260 seconds]
zarel has joined #nixos-dev
<flokli> It's good we found it - now we should see how to fix it ;-)
<flokli> I wonder if ofborg eval would have caught it as well
<niksnut> arianvp: what large experimental features?
<arianvp> I was referring to flake support PR. But large is maybe a bit hyperbole
<arianvp> But something like "NixOS does not eval" is definitely something I wish would be caught before merging to master. Preferably in an automated fashion :)
bhipple has joined #nixos-dev
justanotheruser has joined #nixos-dev
janneke_ is now known as janneke
<rycee> Anybody know the status of `nix ping-store`? Specifically, would I be future safe if I use it in Home Manager?
<rycee> My specific use case is to create the per-user gcroots and profiles directory, if it doesn't exist already.
drakonis has joined #nixos-dev
<niksnut> rycee: the 'nix' command is experimental
<rycee> niksnut: Yeah, that's why I'm a bit worried about using this. But I didn't find a similar command among the old tools.
<niksnut> gchristensen: I wouldn't use the "reproduce" script since you can also just do a "git clone"
<rycee> Ideally I'd like something that just creates the appropriate per-user gcroot and profile paths for the current user.
<gchristensen> niksnut: the "reproduce" script was the only way to really reproduce it, since the value of the `nixpkgs` argument, and the lack of a `.git` directory were significant. the reproduce script was important because it let me get a known-bad and compare
<rycee> (without creating a `~/.nix-profile` link)
<clever> gchristensen: something ive done, since i have ssh to some hydra's, is just nix-copy-closure the .drv right out of the hydra itself
<clever> so its all pre-evaled, and should be identical (baring nix.conf on the build machine)
<gchristensen> in this case, the problem was the evaluation was failing
<clever> ah
<clever> if you go to the inputs tab of an eval that did work, you can nix-store -r those, to download them
<clever> and then re-run nix-instantiate on those dirs
<gchristensen> no, that wan't sufficient either
<clever> hydra-eval-jobs?
<Profpatsch> gchristensen: was the commit I linked the source of the problem?
<gchristensen> clever: I also chased down a problem based on this: --arg nixpkgs '{ outPath = ./scratch/nixpkgs/source; rev = "50edd0f565aa26a557b8848316bdc4631782b004"; shortRev = "50edd0f"; revCount = 212937; }'
<gchristensen> Profpatsch: I think so!
<gchristensen> thank you!
<clever> gchristensen: yeah, getting that --arg right is sometimes important
<Profpatsch> great!
ixxie has quit [Ping timeout: 260 seconds]
__monty__ has quit [Quit: leaving]