pie_ has quit [Ping timeout: 248 seconds]
ma271 has joined #nixos-dev
ma271 has quit [Ping timeout: 256 seconds]
jtojnar has joined #nixos-dev
pie_ has joined #nixos-dev
jtojnar has quit [Remote host closed the connection]
Nadrieril has quit [Remote host closed the connection]
Nadrieril has joined #nixos-dev
<LnL> nice!
<LnL> is that the stats pr?
contrapumpkin is now known as Whale
Whale is now known as contrapumpkin
contrapumpkin is now known as Whale
Whale is now known as contrapumpkin
<gchristensen> yeah
pie_ has quit [Ping timeout: 248 seconds]
<gchristensen> that one is a bit weird of a graph... doesn't really make sense in that format, I think, but I'm not sure how else to show it
<gchristensen> LnL: if you have ideas on making that dashboard better... please do it :)
orivej has joined #nixos-dev
mbrgm has quit [Ping timeout: 240 seconds]
mbrgm has joined #nixos-dev
Lisanna has quit [Quit: Lisanna]
Lisanna_ has joined #nixos-dev
pie_ has joined #nixos-dev
pie_ has quit [Ping timeout: 240 seconds]
Lisanna_ has quit [Quit: Lisanna_]
Lisanna_ has joined #nixos-dev
Lisanna_ has quit [Ping timeout: 256 seconds]
<Profpatsch> What’s John Ericson’s IRC handle again?
<Profpatsch> Is it okay from a cross-compilation standpoint to patch ${stdenv.cc.libc}/bin/getconf into the source code?
pie_ has joined #nixos-dev
<Profpatsch> line 23
<Profpatsch> Sonarpulse: ^
taktoa has joined #nixos-dev
<taktoa> so I understand that Nix is written in C++ because it makes bootstrapping really easy, but I'm curious how people would feel about rewriting Nix in Rust _if_ there were an easy way to get a portable rust compiler (I think I have come up with such a technique)
<taktoa> like imagining that, hypothetically, there were a rust compiler written in portable C
<taktoa> (I'm thinking that I can compile rustc to wasm and then write a simple portable wasm JIT in C)
orivej has quit [Ping timeout: 240 seconds]
<makefu> taktoa: i think the more obvious choice would be go, no?
<taktoa> makefu: Nix performance is already not so great, so the extra cost of GC would be painful, and personally I think the intersection between would-be Nix developers and Rust people is much larger than the intersection between would-be Nix developers and Go people
<taktoa> also personally I'm not a fan of really any of Go's design decisions
<adisbladis> And a huge advantage of rust over something like go is that a rust migration could be incremental. With go this is much, much harder.
<makefu> taktoa: i am not so sure about rust ppl vs go ppl in the nix community. also go compiles down to a single executable which is what would be one of the primary requirements. however i am not in favor to actually rewrite the whole thing anyway
JosW has joined #nixos-dev
<Profpatsch> taktoa: So you are the local RESF ambassador?
<taktoa> Profpatsch: lmao
<Profpatsch> taktoa: Give me one reason why we should rewrite nix.
<taktoa> Profpatsch: Nix needs to be rewritten as a compiler rather than an interpreter _anyway_, and Rust is much better suited to AST manipulation than C++
<Profpatsch> where does this _anyway_ come from?
<Profpatsch> > Rust is much better suited to AST manipulation than C++
<taktoa> Nix evaluator is slow
<Profpatsch> what makes you say that?
<taktoa> Profpatsch: I've written compilers in languages with and without sum types?
<simpson> taktoa: How fast should Nix be? Do you have a benchmark?
<Profpatsch> taktoa: Do you have a link to plans of rewriting nix as a compiler?
<taktoa> Profpatsch: no, it just strikes me as a pretty obvious next step given the performance issues we've been experiencing
<taktoa> to clarify, I'm not arguing that Nix needs to be AOT compiled. JIT would probably be fine
<taktoa> simpson: that's a good question, and I don't.
<simpson> taktoa: But you *have* been experiencing performance issues?
<Profpatsch> Do you know how much work JIT compiling is (I don’t know, it doesn’t sound easy)?
<Profpatsch> Do you know how JIT would be applicable for a lazy language?
<simpson> Profpatsch: It's easy using a JIT toolkit like RPython, but the generated executables aren't single-static-binary.
<Profpatsch> Do you think parsing the AST is the biggest problem when you have to implement JIT?
<taktoa> Profpatsch: a bytecode JIT is pretty easy, and would likely be faster than what we're doing now.
<adisbladis> simpson: Well neither is nix currently.
<Profpatsch> simpson: Do these JIT toolkits work for non-imperative languages?
<simpson> Profpatsch: Yes, RPython versions of Prolog and Haskell exist.
<taktoa> Profpatsch: lol no, that's a gross misinterpretation. I'm talking about manipulating the IR (which is a form of AST) in compiler passes etc.
<Profpatsch> simpson: I have never heard of such a thing for Haskell.
<simpson> Profpatsch: It never got to the stage where cfbolz could write a paper for it~
<Profpatsch> taktoa: Are you familiar with the internals of nix?
<taktoa> Profpatsch: yes, fairly familiar, though it's been a while since I looked at the details of how laziness is implemented
<taktoa> I've written PRs
<Profpatsch> taktoa: And do you know what causes the current performance issues
<taktoa> Profpatsch: I suspect that a decent portion of the current performance issues could be fixed by adding a `memoize` primitive that saves evaluations to an on-disk database
<Profpatsch> memoization doesn’t sound like a very functional concept to me.
<taktoa> ???
<taktoa> in a pure functional language you can't tell the difference?
<Profpatsch> Adding disk IO to the evaluation sounds like it might have performance implications of its own.
<taktoa> of course, but there's a lot of repeated evaluation work, and I suspect that the DB would be small enough that the whole thing could just be prefetched
<Profpatsch> I haven’t looked into evaluation performance very much, but from what I hear lots of overhead comes from evaluating the module system multiple times.
<taktoa> obviously to preserve correctness you'd have to check if the subexpression uses an impure builtin and error if that is the case
<Profpatsch> Is there a way to generate e.g. flamegraphs of evaluation times
<Profpatsch> ?
<taktoa> not that I know of
<Profpatsch> Then before attempting a rewrite maybe improving the inspection mechanisms might be a good first step.
<taktoa> I also think rewriting Nix in Rust would give us a chance to separate parsing and evaluation (which are currently a bit coupled), which would be useful for a lot of things (editor plugins, adding a type system, etc.)
<Profpatsch> > adding a type system
<Profpatsch> x: if x < 10 then 42 else x: x + 23
<Profpatsch> x: if x < 10 then 42 else y: y + 23
<taktoa> Profpatsch: static analysis is conservative. I know that those things are valid Nix, but they don't have to be. think of it as a linter.
<Profpatsch> static analysis /= type system
<taktoa> type systems are a specific kind of static analysis, you're being pedantic
<simpson> taktoa: Well, you're being optimistic. It's quite easy to *say* that all of these changes will happen.
<taktoa> simpson: okay, that's a different argument though. saying "we should not rewrite it" is not the same as "we do not have the manpower to rewrite it".
<Profpatsch> Well, I’d say we should not rewrite it
<Profpatsch> Because one goes with the other.
<Profpatsch> It took one person a good part of a year to rewrite the perl parts in C++
<taktoa> condition all of the things I have said above on us having enough manpower to do these things
<simpson> taktoa: How about "we do not have the manpower to rewrite it in Rust" but "we probably do have the manpower to rewrite it in a theoretical non-existent high-level language".
<Profpatsch> And you shouldn’t forget that nix contains 20+ man-years of work.
<Profpatsch> And that Rust is a pretty new language with a very fast moving ecosystem.
<Profpatsch> So not really a good platform to base your long-living software on.
<Profpatsch> Stuff like this makes the whole re-write proposal kind of theoretical.
<Profpatsch> Btw. there is a library called hnix that implements the nix AST with sum types (even recursion schemes!)
<taktoa> I find it likely that the tech debt in the current Nix codebase is more of a maintenance burden than the hypothetical problems associated with depending on "fast-moving" Rust packages. Anyway, I have to go for a bit.
<simpson> Rust is definitely not stable enough yet to write "long-living software". But that's okay.
<Profpatsch> Give it 10 more years. :)
<Profpatsch> I say rewrite it in urbit
<Profpatsch> Or Common Lisp
orivej has joined #nixos-dev
<Profpatsch> Fun fact: the hnix library is by the current Emacs maintainer.
<Profpatsch> so yeah
<simpson> I see Nix as a transitional technology. Hopefully, in a decade, we won't need it anymore.
<Profpatsch> simpson: What’s your current bet?
<Profpatsch> If we can extrapolate from other tech, it’s here to stay, though. And we should really try to keep evaluation times small right now. :P
<Profpatsch> Otherwise we won’t have much fun in 10 years.
orivej has quit [Read error: Connection reset by peer]
goibhniu has joined #nixos-dev
goibhniu has quit [Ping timeout: 255 seconds]
<sphalerite_> is there any reason why FdSink::write (src/libutil/serialise.cc) warns about large paths? Surely if they're getting written out to a file descriptor it doesn't matter how much data there is?
<sphalerite_> Also, that _good = true; and discarding the exception looks wrong. Might be what causes #1762
ma271 has joined #nixos-dev
goibhniu has joined #nixos-dev
romildo has joined #nixos-dev
gleber_ has quit []
gleber_ has joined #nixos-dev
cbarrett has quit []
cbarrett has joined #nixos-dev
<gchristensen> does this make sense? $ nix-shell --builders "https://foobar.com" (cc shlevy)
<shlevy> I don't know the "--builders" flag...
<gchristensen> heh
<gchristensen> cool :)
<gchristensen> when using it like that, it starts uploading NARs to foobar.com as if it is a remote store
<shlevy> Ah, I see
<shlevy> Seems to be a shortcut for --option builders foo
<gchristensen> does it make sense for the builders option to accept binary caches?
<shlevy> Yes, binary caches are just store implementations
<shlevy> It will fail of course because they can't build
<shlevy> So perhaps it should fail earlier
<gchristensen> will it fail? here I'm just seeing it fail because it can't upload to itt
<shlevy> Sure, but if it could upload to it (it tries with PUT and your netrc options) it would fail at the actual build stage :)
<gchristensen> amazing
<gchristensen> so I guess I'm thinking that according to the implementation it makes sense this is possible, but in terms of what the option means it should probably reject binary caches as a builder
<LnL> don’t you want —option store
<shlevy> Yeah, should be checked that it supports building
<gchristensen> shall I open a bug report?
<shlevy> Please
<gchristensen> on it
<shlevy> niksnut: where is the "builders" flag parsed??? I see it in globals.hh but nothing that seems to check for it in parseCmdLine or anything
<niksnut> settings.convertToArgs(*this, cat);
<shlevy> Ah, cool
<shlevy> gchristensen: It was niksnut's anwer, not yours :)
<gchristensen> oh :D
romildo has quit [Quit: Leaving]
<dtz> --builder isn't documented, is it? should it be or is it really an internals-only thing?
<shlevy> Ideally there'd be automatic documentation generated from any Setting that is an alias
<shlevy> That isn't an alias, I mean
<gchristensen> I think UX bugs are bugs, a user doesn't care that a binary cache is an implementation of the abstract store concept. a binary cache isn't capable of building, how isn't it a bug that a binary cache can be provided as a builder?
__Sander__ has joined #nixos-dev
orivej has joined #nixos-dev
<niksnut> gchristensen: why would a user try to use a binary cache as a builder?
<niksnut> I mean, what would the expected result be?
orivej_ has joined #nixos-dev
orivej has quit [Ping timeout: 264 seconds]
<domenkozar> I think this stopped working in Nix 2.0
<domenkozar> or it changed behavior?
<domenkozar> ah
<domenkozar> let's try that!
<domenkozar> we use builtins.storePath hack for pinning nixpkgs, hopefully it will work
<gchristensen> niksnut: they didn't intend on it being a binary cache, and my impression is that they got to using http in --builders through trial and error, and the success of a binary cache being acceptable at all made them continue down the path of thinking it was the right thing
infinisil has quit [Quit: Configuring ZNC, sorry for the join/quits!]
infinisil has joined #nixos-dev
adisbladis has quit [Ping timeout: 264 seconds]
alunduil has quit [Ping timeout: 255 seconds]
<shlevy> I agree with gchristensen
pie__ has joined #nixos-dev
pie_ has quit [Remote host closed the connection]
pie__ has quit [Ping timeout: 260 seconds]
JosW has quit [Quit: Konversation terminated!]
<contrapumpkin> the outputHash feeds into a .drv hash, right?
JosW has joined #nixos-dev
<Sonarpulse> Profpatsch: hi
<Sonarpulse> dtz: no more WIP?!
<Sonarpulse> taktoa: rust would be my ideal choice I think
<Sonarpulse> mrustc is a thing
<Sonarpulse> Profpatsch: that's fine for now
<Sonarpulse> Profpatsch: i mean, that binary won't work for cross
<Sonarpulse> Profpatsch: does it matter which libc getconf comes from?
<Sonarpulse> getBin buildPackages.stdenv.cc.libc
<Sonarpulse> will be the same on native
<Sonarpulse> and get a binary you can still run on cross
<Sonarpulse> but will come from a different libc than the one being linked by the program
<dtz> I mean I can work more, always more to do, but not obviously unready I don't think, although normal PR review and care are warranted :). LMK how you want to proceed, or if there are parts that are questionable and whatnot
<Sonarpulse> dtz: so right now there is isMusl but not isGlibc, right?
<Sonarpulse> and you squashed PRs to your liking?
<dtz> Sonarpulse: yes re:isMusl/isGlibc
<dtz> Sonarpulse: I am slightly burned on squashing further, but I'll happily squash or address anything that folks think could be better, even if it takes some work
<Sonarpulse> dtz: that's fine
<dtz> note that the PR does include the possibly significant XZ fixes
<Sonarpulse> I'm just going to look at the whole thing at once
<dtz> okay great
<Sonarpulse> and assume the commit-by-commit is good :)
<Sonarpulse> I don't wanna burn you further
<Sonarpulse> so xz we should maybe get something else to review
<Sonarpulse> I don't have any package-specific knowledge, basically
<dtz> well I figure if the commit-by-commit is a smidge out of place it's not the worst, esp if it's because it tells the story a bit more honestly :)
<Sonarpulse> more `libc == "musl"` :):):):)
<dtz> yeah that's is another PR already
<dtz> just drawing your attention to it so we don't merge this as-is without seeing how that's resolved
<Sonarpulse> dtz: woah libbfd
<Sonarpulse> target shouldn't matter
<dtz> yeah probably could do a search-replace for "isMusl" and replace with 'libc == "musl"', modulo parens
<Sonarpulse> whatever on the isMusl
<Sonarpulse> can do that later or not at all :)
<Sonarpulse> but libbfd is just a library
<Sonarpulse> --target should have no effect
<dtz> well kinda, it's really binutils right. But okay I'll investigate!
<contrapumpkin> niksnut: is there a reference somewhere with all the different ways hashes are constructed in Nix?
<contrapumpkin> (that isn't the source code :P)
<Sonarpulse> dtz: depsBuildBuilds should also be depsBuildBuild
<Sonarpulse> probably not needed then, as its having no effect
<shlevy> contrapumpkin: I think the thesis has the basic ideas already
<shlevy> But would begood to have in the manual
<Sonarpulse> I was worried the incremental intensional store RFC made *more* hashes
<Sonarpulse> when we should have the same or fewer types
<contrapumpkin> yeah it just would be cool to have a reference of the different hash shapes
* contrapumpkin digs around
<dtz> ha! d'oh! lemme revisit that, I think I tried to clean but punted to what bgamari had (which he probably flagged as WIP or whatnot, no criticism there). I wonder if typo'ing depsBuildBuild is what made some of that necessary
<dtz> to the debug-mobile!
<Sonarpulse> dtz: oh if that was a bgamari thing, it might predate the finalization of my changes too
<Sonarpulse> libbfd is a fairly new creation of mine
<Sonarpulse> dtz: I'd try just dropping everything but the new patch
FRidh has joined #nixos-dev
FRidh has quit [Remote host closed the connection]
<fpletz> dtz: thanks so much for your work on musl!
<fpletz> have you tested if nixos works with musl? systemd will probably need glibc :)
<dtz> :D:D you're very welcome!
<dtz> I have not, and yes it will. Although until recently NixOS's systemd could be patched to build/maybe-work, but I never tried it
<dtz> probably some minor details to sort if you tried to use NixOS this way, I never actually tried doing so just building pkgs O:)
<dtz> Sonarpulse: ty for making comments on the PR, easier to track that way :innocent: . But feel free to chat about it here too and whatnot
<Sonarpulse> dtz: I changed them to be all at once gbtw
<Sonarpulse> happy to chat here too
<Sonarpulse> but i started doing that for visibility
<Sonarpulse> etc etc
<Sonarpulse> dtz can we make getent a top-level thing?
<Sonarpulse> see iconv
<Sonarpulse> and glibcIconv
<Sonarpulse> for an analog
<dtz> okay!
<domenkozar> can't wait to build musl backed haskell static executable
<shlevy> niksnut: How hard would it be to have a way to append to an option list with --option in addition to replacing it?
<domenkozar> it's just a monoid how hard can it be!
__Sander__ has quit [Quit: Konversation terminated!]
ma271 has quit [Ping timeout: 256 seconds]
_rvl has quit [Quit: ZNC 1.6.5 - http://znc.in]
_rvl has joined #nixos-dev
<clever> `nixpkgs ? fetchTarball channel:nixos-17.09`
<clever> what, what???
<clever> when has that ever been valid??
<gchristensen> nix 2.0!
<clever> lol
<clever> well, now i cant build nix with nix, lol
<Dezgeg> yeah that's a bit of pain, you need nix-shell -p nixUnstable
<clever> { nix ? builtins.fetchGit ./.
<clever> and even that!!
<clever> Dezgeg: i want to modify nix, to add more debugging, to figure out the cause of error: imported archive of ‘/nix/store/jcmcikcs1f69aj6d8415blcjpdlz6f4d-5c09cdc187b014143302551e62d4973b5bf52814.tar.gz’ lacks a signature
<Dezgeg> yes, you just need to build it with nixUnstable
<clever> Dezgeg: that also fails in other ways, one min
<Dezgeg> well :P
<clever> Dezgeg: the perl-bindings are just //'d on
<clever> and .overrideAttrs isnt aware of them, and strips them out
<clever> also, thats a let block, so its imposible to re-add them
<clever> why wasnt passthru used??
<clever> also, i just noticed, i was working with a 1.12pre nixUnstable, but the nixUnstable i just linked is 2.0!
<clever> i wonder, will that break things more, or fix things, lol
<dtz> aww I forgot everything depends on systemd, lol utillinux ffs D:
<clever> dtz: one sec
<clever> dtz: with this, and runit.nix, i was able to boot a linux based image, with no systemd in the closure
<dtz> ooo neat!
<clever> Dezgeg: dang, nix 2.0 also fails with error: imported archive of ‘/nix/store/izbja1sg9g7bsswsa8k4r0dwii9rqdwd-source’ lacks a signature
JosW has quit [Quit: Konversation terminated!]
orivej_ has quit [Ping timeout: 256 seconds]
<clever> Dezgeg: strange, `nix-build '<nixpkgs>' -A hydra` is now consuming 100% cpu, the nix-build itself
romildo has joined #nixos-dev
<niksnut> shlevy: I've thought about that, since it would be nice to get rid of those extra-* options
<niksnut> probably something like 'option += value' in nix.conf, and --option option +value on the command line
<clever> error: imported archive of ‘/nix/store/jcmcikcs1f69aj6d8415blcjpdlz6f4d-5c09cdc187b014143302551e62d4973b5bf52814.tar.gz’ lacks a signature
<clever> niksnut: any idea why both nix 1.12 and 2.0 are failing when i use builtins.fetchTarball with a sha256 provided?, with a 1.11 nix-daemon
<niksnut> is the daemon also 2.0?
<LnL> oh is this 1.11 -> 1.12+?
<clever> niksnut: 1.11 daemon
<Sonarpulse> domenkozar: me too!
<LnL> err the other way around
<niksnut> yeah that doesn't work
<Sonarpulse> dtz: you do any haskell btw?
<Sonarpulse> i suppose i never asked
<Sonarpulse> just tagged you in that cabal thread :)
<clever> niksnut: so if i used a 1.12 daemon, builtins.fetchTarball would work fine?
<niksnut> yes
<dtz> not... "yet". I'm certainly interested in compilers and tooling for haskell, but I don't use it for anything nor do I know it very well.
<LnL> could the 1.12 client sign fixed output paths then talking to another daemon?
<LnL> an older*
<clever> LnL: i was under the understanding that it was virtually identical to just src = ./.; behind the scenes, "here is a nar stream, save it to its own hash"
<clever> so the signatures shouldnt have come into play at all
<LnL> I think the 1.11 client signs all paths
<LnL> only 1.12 uses the hash for fixed outputs IIRC
<clever> LnL: i dont see how that works when importing a local path, because the client is under the control of a potential attacker, and they would also have access to the keys
<clever> LnL: and src = ./.; works without setting up keypairs
<LnL> dunno how it works exactly, just remember running into that
Lisanna has joined #nixos-dev
ma271 has joined #nixos-dev
<dtz> oh yay ty for review Sonarpulse --will go through right now ^_^
<Sonarpulse> dtz mostly things I already said :)
<Sonarpulse> went to lunch and came back
<shlevy> niksnut: Any thoughts on how a plugin could add a setting? I was thinking there could just be a SettingsRegistration or whatever but then we need to defer the "unknown setting" errors until initPlugins. Would that be OK?
<Sonarpulse> and figured i better just paste what i got
<clever> niksnut: yeah, switching to a 1.12 nix-daemon does solve it, now i'm on to a new issue, i cant nix-build the result of builtins.fetchTarball, because its just a string with context
<shlevy> clever: nix-instantiate it
<shlevy> with --read-write-mode
<shlevy> and --eval
<clever> shlevy: except, the nix file has an if statement, and will either return a derivation, or a string with context
<clever> shlevy: depending on if you have nix 1.11 or 1.12
<clever> maybe "${import ./foo.nix}" might work
<clever> then the --eval would build the derivation
<shlevy> yep
<shlevy> well
<shlevy> what are you trying to do?
<clever> i have a nix expression that fetches nixpkgs and returns it
<clever> with 1.11, it returns a derivation that unpacks it
<shlevy> OK. Why do you need to nix-build it directly?
<clever> with 1.12, it uses builtins.fetchTarball
<clever> export NIX_PATH="nixpkgs=$(${NIX_BUILD} fetch-nixpkgs.nix -o nixpkgs)"
<clever> ${NIX_SHELL} -E 'with import (import ./fetch-nixpkgs.nix) { config = {}; }; stdenv.mkDerivation { name = "shell"; buildInputs = [ nix bash coreutils zlib gmp ncurses purescript ]; }' "$@"
<clever> shlevy: but this should also work, and just unset NIX_PATH entirely
<shlevy> Yep
<clever> "fun"
<clever> the 1.11 derivation to unpack, has a different hash from the 1.12 fetchTarball
<clever> oh, i see why
<clever> the 1.11 expression is based around the hash of the tar, while the 1.12 is based around the recursive hash of its contents
<clever> which means the expression on https://nixos.wiki/wiki/How_to_fetch_Nixpkgs_with_an_empty_NIX_PATH is just broken, the hash only works on a single version of nix
<clever> and anybody using nixUnstable hasnt noticed, because nix 1.12pre is "older" then 1.12, and they have been using the fallback path
<LnL> well that explains the problems I was having with it before
<LnL> heh
<Sonarpulse> dtz: hmm i am scared to merge everything
<Sonarpulse> because my attention span i don't think is good enough to review all at once :D
<Sonarpulse> but i don't want to make you do more work?
<Sonarpulse> is there ways to do subseuences of commits such that we can add tests in release-cross and its good?
<Sonarpulse> like a) mass rebuild stdenv b) mass rebuild some other milestone c) non-mass rebuild random crap d) more like c...
<Sonarpulse> dtz: i'm sort of addicted to rebase
<Sonarpulse> if you tell me how to chop up
<Sonarpulse> and move tests around
<Sonarpulse> I'm happy to help to
<dtz> hmm, maybe. I think reviewing the big stack helps see "where" we want to go / what is possible.... now that that's somewhat solidified it might be easier to break back into pieces
<dtz> creating small PR's in the first place was difficult, too many things xD. But I like what you're saying. Hmm.
<dtz> (and already i've broken off the LLVM-related bits for example, as not being immediately related)
pie__ has joined #nixos-dev
<Sonarpulse> bfd i might be overly attached to because I made it with some difficulty :)
<Sonarpulse> others, whatever
<Sonarpulse> like most if it is obviously low impact / nobody should care / merge the damn thing!
<Sonarpulse> (care about neg impact, obviously we do care about benenfit :))
<clever> + ./result/bin/hydra-eval-jobs -I . -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/5c09cdc187b014143302551e62d4973b5bf52814.tar.gz release.nix
<clever> error: access to URI 'https://github.com/NixOS/nixpkgs/archive/5c09cdc187b014143302551e62d4973b5bf52814.tar.gz' is forbidden in restricted mode
<clever> niksnut: and after getting all of the hashes to align correctly, it still fails, any way to allow it to fetch a fixed-output url?
jtojnar has joined #nixos-dev
<Profpatsch> Sonarpulse: So is there a “right” way to get binaries from the correct libc?
<clever> ahh, allowedUris isnt part of the same magic as -I and local files!
<dtz> what do you want to use them for? if it's to patch into scripts then stdenv.cc.libc will probably do, although if you're looking for getconf/getent there'll be special "getconf" and "getent" attributes soon-ish
<dtz> if you want to use getconf/getent during the build, things get a bit trickier, not sure I've seen this much though
<dtz> giving you the answer I can until Sonarpulse gets back to ya ;)
<Sonarpulse> Profpatsch: "right" binaries and "right libc is hard to reconcile
<Sonarpulse> Profpatsch: it was getconf, right?
<Sonarpulse> and during build?
<Sonarpulse> (if its not during build, then your thing was already correct, but dtz's makes it better!)
<Profpatsch> Sonarpulse: Yes, getconf, but during runtime.
<Sonarpulse> Profpatsch: oh cool
<Sonarpulse> your patch is legit
<Profpatsch> getconf is for finding out kernel system limits, right?
<Sonarpulse> fuck if I know :D
<Profpatsch> :D
<Sonarpulse> the parametric approach to cross compilation: it's almost better if I don't know what any of these packages do :D
<Profpatsch> So true I tell ya
<Profpatsch> Just treat them as opaque and mostly fucked up blobs
<Sonarpulse> heheh yeah
<Sonarpulse> nixpkgs is the bandaid that makes unix tolerable
<Sonarpulse> and also the nice centralized todo list for all the software i'd like to not use
<Profpatsch> lol
<gchristensen> haha
orivej has joined #nixos-dev
<rycee> Hmm, libreoffice times out in the 17.09 builds on hydra?
<rycee> I just bumped my 17.09 channel on my laptop and it started building libreoffice…
<rycee> Which I noped out of pretty quickly :-D
<fpletz> rycee: I'll take a look
<rycee> fpletz: Thank you!
<Sonarpulse> dtz: I'm tempted to just merge the damn thing hah
<Sonarpulse> dtz: ok to merge?
adisbladis has joined #nixos-dev
<dtz> I mean, into staging right? Hmm, trust me I'm as eager as can be to see this merged ASAP haha but mass-breakage is a good way to anger everyone. It'd be nice if we could merge to staging-staging O:)
<fpletz> just had another look at the musl pr, it's a pity that my systemd upgrade broke things but we can fix that later
<dtz> I don't have the build resources to test on various cross-configs, native-musl configs, and current native platforms, haha, :(
<fpletz> dtz: do you anticipate breakage in other platforms than musl?
<Sonarpulse> i don't!
<dtz> well the cross fixes will trigger rebuilds in places and until we see that working it's "possible"
<dtz> but don't particularly see why things'd break l ol
<fpletz> well, in that case… let's merge we merge that wonderful PR asap :>
<fpletz> Sonarpulse: dtz: <3
<fpletz> wah, I shouldn't speak and type %)
<dtz> EEEEE
<dtz> itshappening.gif
<Sonarpulse> :D
<catern> this is a fairly significant vulnerability in multi-user Nix setups I think so maybe we will need a point release? or if 2.0 is released soon enough, maybe we don't need a point release? https://github.com/NixOS/nix/pull/1869
<Sonarpulse> merged!
<Sonarpulse> dtz: let's submit to the nixos weekly thing
<Sonarpulse> at least once it hits master
<Sonarpulse> dtz: also, its still dynamic linking right now, right?
<fpletz> I'll trigger another staging eval
<dtz> yes!
<fpletz> rycee: btw restarted those libreoffice builds
<dtz> and yes
<dtz> ty sir
<dtz> EEEEEEEP O:)
<Sonarpulse> np!
<dtz> well worst case everything breaks and we revert and do better next time, haha
<Sonarpulse> I know what it's like when a pull request hangs around being unmerged
<Sonarpulse> fpletz and globin were the first to save me from that :)
<gchristensen> hey catern, in the future please consider sending such issues to the security team privately first -- http://nixos.org/nixos/security.html
<Sonarpulse> dtz: well that's why I didn't squash and merge!
<Sonarpulse> its not all or nothing if we need a revert
<Sonarpulse> (but I doubt we will, not hard to fix things)
<dtz> ah, indeed
<dtz> that'd just be a monster commit anyway lol
<dtz> and yeah any breakage really should be easy to fix
<Sonarpulse> dtz: is there any stuff of bgamari's that still needs to make it?
<bgamari> I do have quite a number of patches outstanding I believe
<bgamari> I've not rebased recently
<dtz> I.. don't remember! O:). Probably. Last few weeks I've been ignoring it until something doesn't build and I'm not sure why ... and if it's not clear I check if he had any commits touching it
<dtz> mostly because I was trying to keep this "small" lol xD
<dtz> (and it was already giant :))
<Sonarpulse> bgamari: well dtz took some of yours
<Sonarpulse> I guess we'll have to rebase and see what's left!
<dtz> yeah, I tried to preserve commits (sometimes with cherry-pick -x) or mention you by name when "borrowing" your changes, bgamari . Credit where credit's due, apologies if missed anything O:).
<rycee> fpletz: Cool!
<bgamari> dtz, quite alright, thanks again for sorting through my chaos
<fpletz> btw. on the topic of picking up work from others, did you know co-authored-by? https://github.com/blog/2496-commit-together-with-co-authors
<dtz> sokay. I had/have a lot of chaos of my own to sort.... so I was wearing that hat anyway :). And y'alls work has made this work much better than I would've thought was possible ~1 year ago ^_^
<dtz> I did not! Maybe saw it mentioned but .. neat!
* fpletz used that for the systemd update because it was based on work by flokli and andi- :)
<dtz> OMG IT WAS MERGED hahah still don't believe it
<dtz> crossing all the fingers
<dtz> fingers = buildPlatform != hostPlatform
<dtz> oh, awesome
<fpletz> staging eval is running, you will have somthing to watch for the evening/night :>
<Sonarpulse> fingers = buildPlatform != hostPlatform
<Sonarpulse> hahahahhaha
<Sonarpulse> i don't know why I don't find this so funny
<dtz> :D
<dtz> err hopefully second "don't" was unintentional there, otherwise BURN lol
<Sonarpulse> oh oops
<Sonarpulse> dtz: second "don't" indeed in error
<dtz> fpletz: what are the job(s) to watch for the staging eval? the nixpkgs staging job? ( https://hydra.nixos.org/jobset/nixpkgs/staging )
<Sonarpulse> dtz: yeah
<fpletz> exactly
<dtz> it's unfortunate that our platform testing infra assumes "system" is unique per platform, so "x86_64-linux" means glibc and no convenient way to specify testing both glibc and musl variants without reworking the release.nix-esque files
<Sonarpulse> dtz: i want to rework it
<Sonarpulse> tada
<Sonarpulse> dtz I would just get rid of "system"-style doubles
<Sonarpulse> llvm names are better
<Sonarpulse> but that's a nix change
goibhniu has quit [Ping timeout: 248 seconds]
<catern> gchristensen: agh, sorry, my mistake
<gchristensen> :)
<Sonarpulse> dtz: do we still need the useMusl flag for busybox?
<dtz> yes. might be hard to get rid of it since nix repo itself references it
<dtz> although it'll now look for a nixpkgs top-level attribute to use as the sandbox shell first
<dtz> err if you think we can instead construct an appropriate cross and pull busybox from that then that'd be swell
romildo has quit [Quit: Leaving]
<Sonarpulse> dtz: yeah with your staging patch I assume we can!
<dtz> is there a good way to do that, anyway? not sure where else that's done other than busybox lol
<Sonarpulse> do which?
<Sonarpulse> construct an appropriate cross
<Sonarpulse> i meant sort of out of band
<Sonarpulse> not calling nixpkgs in nixpkgs
<Sonarpulse> i'll look at nix repo a i guess
<dtz> oh it just does same thing we do in-tree in make-bootstrap-tools
<Sonarpulse> dtz: hmm ok
<clever> also, there is the concept of what the cpu can run (x86-64, with a linux kernel)
<clever> and that is seperate from your libc (nix-copy-closure can just ship the right one)
<clever> so the system attribute in a derivation shouldnt include the libc your using
<dtz> hmm indeed
<clever> and then when i throw in qemu-user-arm and binfmt-misc, i can run armv7l-linux binaries on an x86-64-linux system!
<dtz> ! yeah I've been using that to run/test aarch64 in a limited sense
<dtz> not nix-integrated, which is what I think you mean
<dtz> just binfmt-misc
<Sonarpulse> clever: fair
<clever> dtz: ive patched my nix-daemon, so it supports a `build-extra-platforms = armv7l-linux` in nix.conf
<Sonarpulse> mmmmm
<clever> dtz: and now nix will just blindly execute arm builders/derivations
<Sonarpulse> kewl
<Sonarpulse> clever: so you are really ready for swl
<Sonarpulse> *wsl
<Sonarpulse> doing windows builds
<clever> Sonarpulse: potentially, i could jam wine into there, and handle builder = "c:\bash.exe"; lol
<Sonarpulse> hahah
<clever> but $WINEPREFIX would have to be configured and present
<Sonarpulse> but IIRC wsl already runs windows binaries fine
<clever> i was thinking of how to handle it from a linux kernel
<clever> but yeah, i could have a "linux" build of nix-daemon inside WSL, that does windows builds
<clever> let me see....
<clever> this builds the patched nix
<dtz> new staging job failure: https://hydra.nixos.org/build/69111344 looks unrelated/spurious but not sure. Anyone know more about this?
<dtz> looks like timeout is probably too low on a loaded builder....? :)
<clever> FAILcmd/go189.719s
<clever> doesnt look like a timeout to me?
<clever> enless there is a timeout within that command
<clever> dtz: hmmm, x86_64-pc-mingw32 or x86_64-cygwin?
<clever> or maybe a new one, where its just bare windows, with no special env
<dtz> "panic: test timed out after 3m0s"
<dtz> so yeah looks to be timeout. Will try to build locally ...
<clever> ah, thats half way into the log, after a giant wall of details
<dtz> haha idk :3
<dtz> haha yeah for some reason my scrolling stopped right there and I got lucky :D
<dtz> didn't even see the nice FAIL you posted xD
<clever> i always hit the end key, then work my way up
<dtz> err bajillion jobs aborted in the new evaluation? :(
<Sonarpulse> dtz: people do that
<Sonarpulse> hey people, don't!
<Sonarpulse> g2g though
<Sonarpulse> there will presumably be new eval
<Sonarpulse> after
<Sonarpulse> that subsumes it
<dtz> oh, haha okay
<dtz> and hooray that go (1.6.4) builds fine locally AFAICT, hopefully np there
<dtz> shlevy: nix plugins, eh?! :D. Whatcha doing with those? exec/importNative?
<gchristensen> shlevy: fetchPijul, fetchHDFS, fetchShlevyCorpDataWarehous
<gchristensen> oops, dtz ^
<shlevy> Well, first we want to iterate on fetchGit without having to update everyone's nix :) But also, at least fetchPypi and fetchHackage (the latter kinda exists but requires us to periodically update an all-hackage tarball hash, which is annoygin)
Sonarpulse has quit [Ping timeout: 260 seconds]
<gchristensen> I'd love to see fetchGit move back out of nix core and in to a plugin
<dtz> ooo interesting
<shlevy> gchristensen: Well keep an eye on nix-plugins then :)
<niksnut> not gonna happen
<gchristensen> aww
<gchristensen> but closure size, niksnut!
<niksnut> fetchMercurial, maybe
<niksnut> but fetchGit is too important
<contrapumpkin> can we make its closure include git but only get retrieved lazily? :) :) :)
<shlevy> gchristensen: I think fetchGit in nix just uses git in $PATH, no?
<contrapumpkin> yeah, it does
<shlevy> I don't think git is in nix's closure
<contrapumpkin> that's what I was just asking to fix :P
<gchristensen> ahh
<shlevy> It's one of my annoyances with it, though I understand why if it's upstream. so I'm going to have it not be :D
<contrapumpkin> there's also this thing: https://github.com/NixOS/nix/pull/1345
<contrapumpkin> I intentionally left git out of it
<contrapumpkin> but there are other things, too
<contrapumpkin> man, that PR is way out of date now
<clever> contrapumpkin: from that pr, `And if someone builds it on a non-Nix system we'll just embed non-store paths and nothing is lost.
<clever> `
<clever> contrapumpkin: ive seen that fail, if you import <nix/config.nix> and then do builtins.storePath on a tool like tar, so it becomes a proper input with context
<clever> contrapumpkin: it fails hard, if the tar isnt actually in the store
* contrapumpkin well, the question is whether that's desirable to allow in the first place :P
<shlevy> clever: So... don't do that? :D
<contrapumpkin> erm, no /me
<contrapumpkin> in general, using random shit from $PATH feels very un-nixy to me
<contrapumpkin> in almost all contexts except those where we're forced to
<clever> shlevy: restricted eval in hydra also blocks that now, due to improvements in nix's restricted mode
<shlevy> That stuff is in <nix/config.nix> is only "exported" because it's needed by other stuff in nix corepkgs
<clever> shlevy: i was abusing it to implement builtins.fetchTarball on nix 1.11
<shlevy> clever: Ah, so it's fixed in 2.0 with proper plugins :)
<clever> shlevy: its also fixed on 1.12, which supports fetchTarball with a sha256
<gchristensen> ~ there is no 1.12 ~
<shlevy> clever: I mean the meta-motivation is fixed
<fpletz> but there is a talk about it! ;)
<dtz> yeah, having nix break hard and badly because it's executed without magic PATH setup seems.. unfortunate
<dtz> at least for things like.. fetchTarball, and lesser fetchGit but IMO definitely that too
<contrapumpkin> how bad is the error right now if you fetchgit and don't have git on the path?
<clever> dtz: in this case, its not about whats in $PATH, its about what paths are in <nix/config.nix>
<fpletz> just like php6, there were even books published but it never got finished and released \o/
<dtz> whoa, so guess what: 1.12 will NEVER BE RELEASED
<dtz> :P
<contrapumpkin> clever: the PR is all about what's in $PATH though
<clever> dtz: i was adding those strings to $PATH, inside the nix sandbox, to make sure it didnt care about the original $PATH
<contrapumpkin> and not having to care
<clever> contrapumpkin: ah, havent fully read it yet
<contrapumpkin> the point is that Nix randomly uses several tools from $PATH
<dtz> oh
<contrapumpkin> during its normal operation
<contrapumpkin> and that PR was trying to rein those in
<shlevy> $ PATH= $(which nix) eval '(fetchGit ./.)'
<shlevy> error: executing 'git': No such file or directory
<shlevy> error: executing 'git': No such file or directory
<shlevy> error: program 'git' failed with exit code 1
<contrapumpkin> shlevy: that's pretty awful :)
<clever> contrapumpkin: i havent confirmed, but i suspect `builtins.fetchTarball` expects `tar` in PATH
<contrapumpkin> yeah, which implicitly expects gzip, and xz, and bzip2
<contrapumpkin> that's what the PR is fixing
<clever> shlevy: ive seen that kind of thing implemented with bare nix already
<shlevy> clever: Nope, it hard-codes the configure-time path
<contrapumpkin> shlevy: it does?
<dtz> fetchGit is too important to be a plugin but not important enough to make including a working git acceptable :P
<contrapumpkin> maybe it's changed significantly from when I made that PR
<shlevy> $ grep -w tar /nix/store/jy01ji8xwrdjp6dq2kh02jp0ncbz3ck0-nix-2.0pre5943_52c777a7/share/nix/corepkgs/config.nix
<shlevy> tar = "/nix/store/1jn750iav5rfbm05345ysi8dvxcz7j26-gnutar-1.30/bin/tar";
<shlevy> Ah, what the hell??
<contrapumpkin> there are a few places where runProgram("tar"
<shlevy> :D
<contrapumpkin> that PR is trying to fix those
<shlevy> We already ahve it in the closure...
<shlevy> Amazing
<contrapumpkin> so that should be uncontroversial
<dtz> oh maybe it's builtins.fetchcTarball that uses raw "tar" then
<contrapumpkin> and then I jump through the hoops needed to give tar gzip and bzip2 and xz
<clever> dtz: fetchTarball uses the line i linked above in download.cc to do the unpack
<dtz> re:runProgram("tar",...)
<dtz> ah great, haha I see
<clever> shlevy: this is where the tar in config.nix gets used
<clever> to unpack channels, so it has to work before nixpkgs exists
<shlevy> Right
<clever> and the newer option everybody recomends, NIX_PATH=nixpkgs=https://example.com/foo.tar.gz probably leads to download.cc, which expects tar in $PATH
<dtz> new staging eval: https://hydra.nixos.org/eval/1433254?compare=1433166 (comparison skipping aborted eval prior)
<clever> i also dont like the idea of that one, things will upgrade without warning
<shlevy> clever: You can pointto a specific revision
<shlevy> But FWIW I just don't use NIX_PATH anywhere and hard-code nixpkgs revisions (ultimately with fetchGit)
<clever> shlevy: but then i might as well just be using a local git checkout or nix-channel
<shlevy> Not if you want to share it
<clever> but for my local use, i use NIX_PATH
<clever> shlevy: also, i just identified 2 bugs with the expression in that wiki today
<clever> first, "1.12pre" is older then "1.12" according to builtins.compareVersions, so it always uses the fallback method
<clever> second, the 2 methods hash different things, so the sha256 will only be right for one
infinisil has quit [Quit: Configuring ZNC, sorry for the join/quits!]
infinisil has joined #nixos-dev
<dtz> :(
<clever> to get the hashes, you have to nix-prefetch-url the .tar.gz, both with and without --unpack
<clever> then edit the expression so it can accept both hashes
Lisanna has quit [Quit: Lisanna]
Lisanna has joined #nixos-dev
Lisanna has quit [Client Quit]
orivej has quit [Ping timeout: 248 seconds]
goibhniu has joined #nixos-dev
ma271 has quit [Ping timeout: 276 seconds]