lassulus has quit [(Ping timeout: 268 seconds)]
lassulus has joined joined #nixos-dev
dtzWill has quit [(Ping timeout: 252 seconds)]
ris has quit [(Ping timeout: 248 seconds)]
dtzWill has joined joined #nixos-dev
orivej has quit [(Ping timeout: 276 seconds)]
Sonarpulse has quit [(Ping timeout: 240 seconds)]
{^_^} has joined joined #nixos-dev
{^_^} has quit [(Changing host)]
{^_^} has joined joined #nixos-dev
mbrgm has quit [(Ping timeout: 260 seconds)]
ma27 has quit [(Ping timeout: 255 seconds)]
mbrgm has joined joined #nixos-dev
ma27 has joined joined #nixos-dev
ma27 has quit [(Ping timeout: 255 seconds)]
orivej has joined joined #nixos-dev
goibhniu has joined joined #nixos-dev
__Sander__ has joined joined #nixos-dev
MichaelRaskin has quit [(Quit: MichaelRaskin)]
FRidh has quit [(Remote host closed the connection)]
FRidh has joined joined #nixos-dev
phreedom has quit [(Ping timeout: 255 seconds)]
phreedom has joined joined #nixos-dev
<__Sander__> sigh
<__Sander__> implementing the new NPM support remains painful
ma27 has joined joined #nixos-dev
orivej has quit [(Ping timeout: 255 seconds)]
orivej has joined joined #nixos-dev
ma27 has quit [(Ping timeout: 240 seconds)]
orivej has quit [(Ping timeout: 260 seconds)]
vcunat has joined joined #nixos-dev
<Mic92> LnL: Does the build bot also build for darwin?
<LnL> the carnix stuff doesn't work yet
<LnL> I've just been running it on my hydra machine
<Mic92> not a general solution, but for some applications this is quite useful.
<LnL> how can that work properly
<LnL> any references to node or system libraries would break the hash after a mass rebuild
<Mic92> ignore-scripts maybe?
<Mic92> It does not seem to use system libraries. I am not sure about node upgrades
<LnL> hmm, right system libraries just won't work and it won't run the fixupPhase so scripts probably won't have absolute references
<Mic92> Is there an aquivalent to https://github.com/alexcrichton/cargo-vendor in node land?
{^_^} has quit [(Remote host closed the connection)]
{^_^} has joined joined #nixos-dev
<gchristensen> if anyone has any ideas, I'd _love_ to be able to use "checkMeta" on only the attrs changed by a PR: https://github.com/NixOS/nixpkgs/pull/32342
<__Sander__> Mic92: yes I have seen it
<__Sander__> it's a nice stop-gap solution, but
<__Sander__> there are scenarios it does not support
<__Sander__> such as supporting native dependencies
<__Sander__> or dependencies that need build scripts to run
<Mic92> __Sander__: do you see any way, how the fetching part of npm can be seperated from the build/installation part?
<Mic92> except having to explicitly list every npm source
<__Sander__> Mic92: you mean some derivation that only obtains the dependencies?
<Mic92> __Sander__: Yes, so we have one checksum over the dependencies and nixpkgs build the final package out of that.
<vcunat> gchristensen: if we clear up master, you can use it in each PR unconditionally
<vcunat> (but still keep it off by default, so "production" evaluations don't get slowed down)
<gchristensen> right
<gchristensen> I wonder what it'd take to clean up master
<gchristensen> I could set it up to run only on a specific PR
<gchristensen> vcunat: https://gist.github.com/grahamc/6f77471d220b25e5bafd8f1da5eab5a6 d'you think this is all we'd need to do to effectively checkMeta?
<vcunat> gchristensen: yes, I think that should work.
<__Sander__> Mic92: I have to think about that
<gchristensen> ok then I think after oxij's PR the only one left to fix is pkgs/desktops/gnome-3/core/gucharmap/unicode-data.nix -- is that possible? :P
<vcunat> yes. He wrote he has the checking in his config. Gnome3 might be a new change.
<vcunat> gchristensen: or not. Apparently it stops on the first error.
<gchristensen> after I fixed gucharmap I saw no further errors
<vcunat> ah, I tested a different commit
<gchristensen> vcunat: if I deploy this change now, can you backport the meta fixes?
<gchristensen> should make sure they get to 17.09 and staging to prevent erroneous errors
<vcunat> hmm, it doesn't work for a different reason
<gchristensen> eh?
<vcunat> well, how do you use the amount.nix file?
<gchristensen> HOME=/homeless-shelter NIX_REMOTE=daemon NIX_PATH=nixpkgs=$(pwd) nix-env -f ./amount.nix -qaP --no-name --out-path --show-trace --option restrict-eval true --option build-timeout something --argstr system x86_64-linux
<vcunat> right, that's what I did, and it won't work
<gchristensen> what does it do instead?
<vcunat> nix-env just skips derivations that throw an error
<vcunat> so the faulty ones get omitted
<vcunat> but the error isn't even shown
<gchristensen> it definitely throws errors for me
orivej has joined joined #nixos-dev
<vcunat> It depends on what it fails.
<vcunat> But this is the same throw as for unfree or broken packages.
<gchristensen> https://gist.github.com/grahamc/76db3a5991a4b6fff53f2969cd43cf95 this is what I get when running it on 17.09
<vcunat> yes, that's an error when composing the thrown string
<gchristensen> vcunat: I'm also interested in enabling broken and unfree in this list: https://gist.github.com/grahamc/76db3a5991a4b6fff53f2969cd43cf95#file-outpaths-nix-L7 what do you think?
<gchristensen> oh I see...!
<vcunat> I get more errors if I do
<vcunat> (if reason == "unknown-meta" then abort else throw)
<gchristensen> would nix-instantiate do a better job here?
<vcunat> I agree we should check unfree packages as well (and perhaps broken)
<vcunat> well, nix-instantiate will fail with assertion in systemd on darwin
<vcunat> got the same one :-)
<gchristensen> :)
<vcunat> What about doing just trace instead of throw?
<vcunat> That way we get all errors.
<gchristensen> that would be hard to make sure we fail only if meta is bad
<vcunat> empty stderr or not
<vcunat> but we have many errors
<gchristensen> I'm not surprised
<vcunat> I think tracing might make more sense than throwing in this case.
<vcunat> 493 errors
<gchristensen> nice!
<vcunat> almost all seem simple typos
ma27 has joined joined #nixos-dev
<gchristensen> the issue about stderr is concerning, because the check isn't actually directly checking what it says it is
<vcunat> gchristensen: what do you mean
<gchristensen> well the check is if there is stderr output, not if the meta is valid, and there are potentially other reasons stderr isn't empty
<vcunat> hmm, yes, there might be other evaluation errors
<gchristensen> what if we could specify a checkMetaErrorCallback set that to a fn that throws, then do something with scopedImport where we remap all other `throws` to a noop
<gchristensen> :$
<vcunat> but you would have to parse the throwed strings
<vcunat> and determine the reason of the throw
<vcunat> we do have function to catch throws
<gchristensen> ouch
<gchristensen> ok
<gchristensen> where is this bit: (if reason == "unknown-meta" then abort else throw) ?
<gchristensen> that might be the best route
<vcunat> I'll push to somewhere what I'm using now
<gchristensen> once we fix the 493 errors (hopefully mostly a sed expression?) it won't be too painful to only get one error at a time, since every PR will be checked presumably each PR will only introduce 1 or 2 errors
<vcunat> this commits prints all
<vcunat> I used: nix-env -f checkMeta.nix -qaP --no-name --out-path --show-trace >/dev/null
<vcunat> but I think it behaves reasonably also in standard nix-build and other usage
<srhb> How do I allocate a uid/gid pair? Do I just pick an unused combination in ids.nix ?
<srhb> Following the guidelines to not stray above 399.
<gchristensen> for in nixpkgs or for your own?
<srhb> For the logstash service that I'm repackaging, hopefully. :-)
<srhb> As in, the module.
<gchristensen> yeah just pick the next available number for the user / group
<srhb> Alright, thanks!
<vcunat> `abort` might be too harsh for nixpkgs (impossible to catch), though probably possible if checkMeta remains false by default
<gchristensen> if we keep checkMeta passing it won't be a problem :P
<vcunat> the errors can change with some other config, override, overlay
<gchristensen> ah
<gchristensen> well if we can provide a checkMetaErrorCallback then we can insert an abort ourselves
<vcunat> even overloading checkMeta meaning is possible
<gchristensen> :(
<gchristensen> well sure
<vcunat> this reminds me our nixpkgs-lint tool - it does a slightly different set of checks :-D
<gchristensen> yeah
<gchristensen> another thing I wouldn't mind running on everything all the time, or just changed stuff
<gchristensen> the crappy thing about doing it just when stuff is changed is we subject patchers who don't have anything to do with the problem to it
<gchristensen> so they may not even know how to fix it or diagnose the problem
<vcunat> If we really add such checks into PR, we need to keep master green.
<vcunat> I can't see a way around that.
<gchristensen> I agree
<gchristensen> and not a progressive application of the checks, but a check across all of nixpkgs
<vcunat> For that, we would better add a simple command-line script so direct pushers can do the checks easily.
<vcunat> They tend to do less mistakes, but still...
<gchristensen> yeah
<gchristensen> that is tricky because ofborg doesn't actually use anything stored in nixpkgs for its checks (Just In Case)
mbrgm_ has joined joined #nixos-dev
mbrgm has quit [(Ping timeout: 248 seconds)]
mbrgm_ is now known as mbrgm
Mic92 has quit [(Ping timeout: 260 seconds)]
orivej has quit [(Ping timeout: 246 seconds)]
<globin> gchristensen: you could use <nixpkgs> to run the check script?
<gchristensen> like the host's <nixpkgs>?
<globin> gchristensen: yep
<globin> vcunat: also add an easy way to create git hook for pre-push/pre-commit
<globin> vcunat: where you'd run those
<gchristensen> these checks take a Long Time
<gchristensen> it'd be good to have a way to run them locally
<gchristensen> maybe even `ofborg` the cli tool, available from the cache
<gchristensen> * not a thing that currently exists
<vcunat> if you push to the official repo directly, the meta checks might be cheap enough
<vcunat> 65.95user 0.92system 0:38.14elapsed 175%CPU (0avgtext+0avgdata 3911104maxresident)k
<vcunat> well, 4G memory is not too little
<gchristensen> yeah it is a bit aggressive there
<vcunat> with x86_64-linux only it seems bearable
<vcunat> 36.37user 0.64system 0:20.89elapsed 177%CPU (0avgtext+0avgdata 2633452maxresident)k
<globin> I think that should be fine for a pre-push hook
<vcunat> meta shouldn't normally depend on platform anyway
JosW has joined joined #nixos-dev
FRidh has quit [(Quit: Konversation terminated!)]
FRidh has joined joined #nixos-dev
FRidh has quit [(Remote host closed the connection)]
FRidh has joined joined #nixos-dev
FRidh has quit [(Remote host closed the connection)]
<mbrock> is there any precedence for a setup to protect users from a binary cache being compromised? for example, an installer that verifies identical results from several cache servers
<gchristensen> where is the compromise: the cache server, or the thing populating the cache?
<gchristensen> cache server: yes, all your binary caches should use signatures, then there is no need to check multiple cache servers, just check the signature.
<gchristensen> thing populating the cache: it'll equally taint all cache servers, making that check not useful
<gchristensen> nix places no trust in the cache _itself_
orivej has joined joined #nixos-dev
<mbrock> like if my Hydra server is compromised then the attacker can sign whatever they want, so I would tell someone else to operate an independent Hydra... ok, I don't have a fully thought out threat model, just gott a little flash of paranoia
<simpson> mbrock: Feel free to not use binary caches, but you'll get the full Gentoo experience.
<gchristensen> unfortunately that requires perfectly binary reproducibility and we don't have thaht
<gchristensen> using no binary cache gets you a Gentoo+ experience: we recompile more than gentoo does
<mbrock> this would be for a subset of my own curated packages, but I don't know how practically unreproducible things tend to be in Nix
<gchristensen> do you have a threat model?
<simpson> It depends on the language being compiled. Some compilers are good at reproducibility, and some compilers...some compilers need to be helped.
<gchristensen> (If you don't have a threat model, I'd suggest you're just being paranoid. if you have a defined threat model, it can help explore how to defend yourself)
<clever> systemd has decided that my fstab is invalid, so it must kill all the processes!!!
<mbrock> I was imagining multiple independent signatures on Git hashes of our nixpkgs, so a user can know they're using an audited package set -- but then I'm the only operator of the Hydra/cache, and I could be compromised (quite plausibly)
<simpson> Well, audited by who? I don't think that this improves the TCB as far as the number of people or machines being trusted.
<mbrock> it would be audited by say minimum 3 members of my team, so that one compromised member can't push out a backdoor
<gchristensen> so ... do you have a threat model?
<simpson> Sounds like the threat model is moles and backdoors.
<gchristensen> given a stable base of packages you're building against and knowing your tools build reproducibly, you can expect your results to have perfect reproducibility
<gchristensen> since the stable base is consistent and being built by a consistent upstream
<mbrock> I have a realistic concern and I'm drafting a threat model document, hence my question. Thanks for your answers, I'll get back to writing
<simpson> It's a good question, but I've not known anybody to ever have an answer for any distro other than "Uh...."
__Sander__ has quit [(Quit: Konversation terminated!)]
<gchristensen> IMO nix provides far better guarantees there than any other distro
<gchristensen> if you can, please do share your threat model when you have it written out so we can properly address them
<simpson> It does. Nix doesn't have an answer to generalized Trusting Trust, but everything short of that is covered.
goibhniu has quit [(Ping timeout: 250 seconds)]
jtojnar has joined joined #nixos-dev
orivej has quit [(Quit: No Ping reply in 180 seconds.)]
orivej has joined joined #nixos-dev
Sonarpulse has joined joined #nixos-dev
ma27 has quit [(Ping timeout: 250 seconds)]
ma27 has joined joined #nixos-dev
FRidh has joined joined #nixos-dev
FRidh has quit [(Client Quit)]
vcunat has quit [(Ping timeout: 240 seconds)]
orivej has quit [(Ping timeout: 258 seconds)]
orivej has joined joined #nixos-dev
JosW has quit [(Quit: Konversation terminated!)]
vcunat has joined joined #nixos-dev
Mic92 has joined joined #nixos-dev
orivej has quit [(Ping timeout: 240 seconds)]
<Sonarpulse> I'm now getting weird failures on 32-bit arm
<Sonarpulse> (64-bit works!)
<Sonarpulse> I cannot reproduce in nix-shell, though that does bypass builder script
<Sonarpulse> basically the issue is float abis
<Sonarpulse> evidentally gcc 5 isn't defining __ARM_PCS_VFP
<Sonarpulse> as it should
<Sonarpulse> so glibc tries to include stubs-hard instead of stubs-soft
<Sonarpulse> which doesn't exist
<gchristensen> vcunat: would you like to take an editorial look at //github.com/grahamc/ofborg/pull/15 ?
<gchristensen> it'd be nice to run aspell over these when the PR comes in
<vcunat> let me fix those meta problems...
<gchristensen> I'm going to be somewhat absent this evening due to some personal news, but leave me messages or PRs :P
phreedom has quit [(Remote host closed the connection)]
<jtojnar> do we still use this https://github.com/NixOS/nixpkgs/projects/7?
orivej has joined joined #nixos-dev
<vcunat> jtojnar: I don't use it. The labels feel good enough for me.
<vcunat> (there's a WIP label for the "not-ready" status, and people often put it into title as well)
ris has joined joined #nixos-dev
phreedom has joined joined #nixos-dev
ris has quit [(Read error: Connection reset by peer)]
ris has joined joined #nixos-dev
<grahamc> Let’s delete the project
jtojnar has quit [(Remote host closed the connection)]
jtojnar has joined joined #nixos-dev
<gchristensen> deleted
<gchristensen> vcunat: what are you going to do w.r.t. meta?
<vcunat> fixed all the problems on x86_64-linux
<vcunat> doing other platforms now
<gchristensen> _all_ of the 500 problems?
<vcunat> yes, well, some were shown multiple times
<gchristensen> :o :D
<vcunat> evaluated multiple times for some reason
<gchristensen> (1) we'll need to backport to 17.09, (2) how will we then integrate the check in to borg?
<vcunat> +1
<gchristensen> :)
<gchristensen> https://github.com/grahamc/ofborg/pull/16 is the partial / broken implementation (broken b/c it isn't a thorough check)
zraexy has quit [(Ping timeout: 276 seconds)]
ma27 has quit [(Ping timeout: 260 seconds)]
<Sonarpulse> Dezgeg: see also my message to bgamari above about the https://hydra.nixos.org/jobset/nixpkgs/ericson2314-cross-trunk failures
<Sonarpulse> you also might have some insight into why it's trying to use the software float headers
<Sonarpulse> also both of you: https://github.com/NixOS/nixpkgs/pull/32361 is my progress so far fixing the regressions
<Sonarpulse> doesn't do anything alone
<Sonarpulse> but narrows the gap from a known-good state to the first one where the build != host == target ones fail
<Sonarpulse> and generally makes things nicer too boot
<Sonarpulse> *to boot
zraexy has joined joined #nixos-dev
<Sonarpulse> vcunat: left a github comment, but should I just drop outputsToInstall?
<Sonarpulse> you tell me what to do, and I'll do it; you are the multiple outputs expert
<vcunat> Sonarpulse: just reading those.
<vcunat> I would drop it.
<Sonarpulse> vcunat: cool, thanks
<Sonarpulse> I got an unrelated build in flight now, so figure I might as well fix it
<vcunat> For the "man" "info" and similar it seems better.
<Sonarpulse> waiting for the general solution
<vcunat> though with nix-env we still don't have a nice way for users to choose
<Sonarpulse> so we don't litter a mess everywhere
<Sonarpulse> sounds fine to me
<vcunat> But for systemPackages at least we do.
<Sonarpulse> vcunat: relatedly, if we every move the propagated build input logic into Nix
<Sonarpulse> we can do nix-shells with man pages
<Sonarpulse> which is probably what people *really* want, anyways
<vcunat> hmm, I _think_ this we will get for free with the nix-env -enabling change
<Sonarpulse> oh, the thing you linked me was an issue
<Sonarpulse> is there a PR linked there?
<Sonarpulse> I've seen it before, and didn't remoember or notice again at a quick glance
<vcunat> I'm not sure if there's a PR
<vcunat> I had some WIP code, but needed to check breaking some buildEnv and other usages
<vcunat> (and there's always more things to do coming in)
<Sonarpulse> indeed there always is
<Sonarpulse> I only recently got around to looking at some build != host == target GCC regressions from the summer