<taktoa>
so I understand that Nix is written in C++ because it makes bootstrapping really easy, but I'm curious how people would feel about rewriting Nix in Rust _if_ there were an easy way to get a portable rust compiler (I think I have come up with such a technique)
<taktoa>
like imagining that, hypothetically, there were a rust compiler written in portable C
<taktoa>
(I'm thinking that I can compile rustc to wasm and then write a simple portable wasm JIT in C)
orivej has quit [Ping timeout: 240 seconds]
<makefu>
taktoa: i think the more obvious choice would be go, no?
<taktoa>
makefu: Nix performance is already not so great, so the extra cost of GC would be painful, and personally I think the intersection between would-be Nix developers and Rust people is much larger than the intersection between would-be Nix developers and Go people
<taktoa>
also personally I'm not a fan of really any of Go's design decisions
<adisbladis>
And a huge advantage of rust over something like go is that a rust migration could be incremental. With go this is much, much harder.
<makefu>
taktoa: i am not so sure about rust ppl vs go ppl in the nix community. also go compiles down to a single executable which is what would be one of the primary requirements. however i am not in favor to actually rewrite the whole thing anyway
JosW has joined #nixos-dev
<Profpatsch>
taktoa: So you are the local RESF ambassador?
<taktoa>
Profpatsch: lmao
<Profpatsch>
taktoa: Give me one reason why we should rewrite nix.
<taktoa>
Profpatsch: Nix needs to be rewritten as a compiler rather than an interpreter _anyway_, and Rust is much better suited to AST manipulation than C++
<Profpatsch>
where does this _anyway_ come from?
<Profpatsch>
> Rust is much better suited to AST manipulation than C++
<taktoa>
Nix evaluator is slow
<Profpatsch>
what makes you say that?
<taktoa>
Profpatsch: I've written compilers in languages with and without sum types?
<simpson>
taktoa: How fast should Nix be? Do you have a benchmark?
<Profpatsch>
taktoa: Do you have a link to plans of rewriting nix as a compiler?
<taktoa>
Profpatsch: no, it just strikes me as a pretty obvious next step given the performance issues we've been experiencing
<taktoa>
to clarify, I'm not arguing that Nix needs to be AOT compiled. JIT would probably be fine
<taktoa>
simpson: that's a good question, and I don't.
<simpson>
taktoa: But you *have* been experiencing performance issues?
<Profpatsch>
Do you know how much work JIT compiling is (I don’t know, it doesn’t sound easy)?
<Profpatsch>
Do you know how JIT would be applicable for a lazy language?
<simpson>
Profpatsch: It's easy using a JIT toolkit like RPython, but the generated executables aren't single-static-binary.
<Profpatsch>
Do you think parsing the AST is the biggest problem when you have to implement JIT?
<taktoa>
Profpatsch: a bytecode JIT is pretty easy, and would likely be faster than what we're doing now.
<adisbladis>
simpson: Well neither is nix currently.
<Profpatsch>
simpson: Do these JIT toolkits work for non-imperative languages?
<simpson>
Profpatsch: Yes, RPython versions of Prolog and Haskell exist.
<taktoa>
Profpatsch: lol no, that's a gross misinterpretation. I'm talking about manipulating the IR (which is a form of AST) in compiler passes etc.
<Profpatsch>
simpson: I have never heard of such a thing for Haskell.
<simpson>
Profpatsch: It never got to the stage where cfbolz could write a paper for it~
<Profpatsch>
taktoa: Are you familiar with the internals of nix?
<taktoa>
Profpatsch: yes, fairly familiar, though it's been a while since I looked at the details of how laziness is implemented
<taktoa>
I've written PRs
<Profpatsch>
taktoa: And do you know what causes the current performance issues
<taktoa>
Profpatsch: I suspect that a decent portion of the current performance issues could be fixed by adding a `memoize` primitive that saves evaluations to an on-disk database
<Profpatsch>
memoization doesn’t sound like a very functional concept to me.
<taktoa>
???
<taktoa>
in a pure functional language you can't tell the difference?
<Profpatsch>
Adding disk IO to the evaluation sounds like it might have performance implications of its own.
<taktoa>
of course, but there's a lot of repeated evaluation work, and I suspect that the DB would be small enough that the whole thing could just be prefetched
<Profpatsch>
I haven’t looked into evaluation performance very much, but from what I hear lots of overhead comes from evaluating the module system multiple times.
<taktoa>
obviously to preserve correctness you'd have to check if the subexpression uses an impure builtin and error if that is the case
<Profpatsch>
Is there a way to generate e.g. flamegraphs of evaluation times
<Profpatsch>
?
<taktoa>
not that I know of
<Profpatsch>
Then before attempting a rewrite maybe improving the inspection mechanisms might be a good first step.
<taktoa>
I also think rewriting Nix in Rust would give us a chance to separate parsing and evaluation (which are currently a bit coupled), which would be useful for a lot of things (editor plugins, adding a type system, etc.)
<Profpatsch>
> adding a type system
<Profpatsch>
x: if x < 10 then 42 else x: x + 23
<Profpatsch>
x: if x < 10 then 42 else y: y + 23
<taktoa>
Profpatsch: static analysis is conservative. I know that those things are valid Nix, but they don't have to be. think of it as a linter.
<Profpatsch>
static analysis /= type system
<taktoa>
type systems are a specific kind of static analysis, you're being pedantic
<simpson>
taktoa: Well, you're being optimistic. It's quite easy to *say* that all of these changes will happen.
<taktoa>
simpson: okay, that's a different argument though. saying "we should not rewrite it" is not the same as "we do not have the manpower to rewrite it".
<Profpatsch>
Well, I’d say we should not rewrite it
<Profpatsch>
Because one goes with the other.
<Profpatsch>
It took one person a good part of a year to rewrite the perl parts in C++
<taktoa>
condition all of the things I have said above on us having enough manpower to do these things
<simpson>
taktoa: How about "we do not have the manpower to rewrite it in Rust" but "we probably do have the manpower to rewrite it in a theoretical non-existent high-level language".
<Profpatsch>
And you shouldn’t forget that nix contains 20+ man-years of work.
<Profpatsch>
And that Rust is a pretty new language with a very fast moving ecosystem.
<Profpatsch>
So not really a good platform to base your long-living software on.
<Profpatsch>
Stuff like this makes the whole re-write proposal kind of theoretical.
<Profpatsch>
Btw. there is a library called hnix that implements the nix AST with sum types (even recursion schemes!)
<taktoa>
I find it likely that the tech debt in the current Nix codebase is more of a maintenance burden than the hypothetical problems associated with depending on "fast-moving" Rust packages. Anyway, I have to go for a bit.
<simpson>
Rust is definitely not stable enough yet to write "long-living software". But that's okay.
<Profpatsch>
Give it 10 more years. :)
<Profpatsch>
I say rewrite it in urbit
<Profpatsch>
Or Common Lisp
orivej has joined #nixos-dev
<Profpatsch>
Fun fact: the hnix library is by the current Emacs maintainer.
<Profpatsch>
so yeah
<simpson>
I see Nix as a transitional technology. Hopefully, in a decade, we won't need it anymore.
<Profpatsch>
simpson: What’s your current bet?
<Profpatsch>
If we can extrapolate from other tech, it’s here to stay, though. And we should really try to keep evaluation times small right now. :P
<Profpatsch>
Otherwise we won’t have much fun in 10 years.
orivej has quit [Read error: Connection reset by peer]
goibhniu has joined #nixos-dev
goibhniu has quit [Ping timeout: 255 seconds]
<sphalerite_>
is there any reason why FdSink::write (src/libutil/serialise.cc) warns about large paths? Surely if they're getting written out to a file descriptor it doesn't matter how much data there is?
<sphalerite_>
Also, that _good = true; and discarding the exception looks wrong. Might be what causes #1762
ma271 has joined #nixos-dev
goibhniu has joined #nixos-dev
romildo has joined #nixos-dev
gleber_ has quit []
gleber_ has joined #nixos-dev
cbarrett has quit []
cbarrett has joined #nixos-dev
<gchristensen>
does this make sense? $ nix-shell --builders "https://foobar.com" (cc shlevy)
<shlevy>
I don't know the "--builders" flag...
<gchristensen>
heh
<gchristensen>
cool :)
<gchristensen>
when using it like that, it starts uploading NARs to foobar.com as if it is a remote store
<shlevy>
Ah, I see
<shlevy>
Seems to be a shortcut for --option builders foo
<gchristensen>
does it make sense for the builders option to accept binary caches?
<shlevy>
Yes, binary caches are just store implementations
<shlevy>
It will fail of course because they can't build
<shlevy>
So perhaps it should fail earlier
<gchristensen>
will it fail? here I'm just seeing it fail because it can't upload to itt
<shlevy>
Sure, but if it could upload to it (it tries with PUT and your netrc options) it would fail at the actual build stage :)
<gchristensen>
amazing
<gchristensen>
so I guess I'm thinking that according to the implementation it makes sense this is possible, but in terms of what the option means it should probably reject binary caches as a builder
<LnL>
don’t you want —option store
<shlevy>
Yeah, should be checked that it supports building
<gchristensen>
shall I open a bug report?
<shlevy>
Please
<gchristensen>
on it
<shlevy>
niksnut: where is the "builders" flag parsed??? I see it in globals.hh but nothing that seems to check for it in parseCmdLine or anything
<dtz>
--builder isn't documented, is it? should it be or is it really an internals-only thing?
<shlevy>
Ideally there'd be automatic documentation generated from any Setting that is an alias
<shlevy>
That isn't an alias, I mean
<gchristensen>
I think UX bugs are bugs, a user doesn't care that a binary cache is an implementation of the abstract store concept. a binary cache isn't capable of building, how isn't it a bug that a binary cache can be provided as a builder?
__Sander__ has joined #nixos-dev
orivej has joined #nixos-dev
<niksnut>
gchristensen: why would a user try to use a binary cache as a builder?
<niksnut>
I mean, what would the expected result be?
<domenkozar>
we use builtins.storePath hack for pinning nixpkgs, hopefully it will work
<gchristensen>
niksnut: they didn't intend on it being a binary cache, and my impression is that they got to using http in --builders through trial and error, and the success of a binary cache being acceptable at all made them continue down the path of thinking it was the right thing
infinisil has quit [Quit: Configuring ZNC, sorry for the join/quits!]
infinisil has joined #nixos-dev
adisbladis has quit [Ping timeout: 264 seconds]
alunduil has quit [Ping timeout: 255 seconds]
<shlevy>
I agree with gchristensen
pie__ has joined #nixos-dev
pie_ has quit [Remote host closed the connection]
pie__ has quit [Ping timeout: 260 seconds]
JosW has quit [Quit: Konversation terminated!]
<contrapumpkin>
the outputHash feeds into a .drv hash, right?
JosW has joined #nixos-dev
<Sonarpulse>
Profpatsch: hi
<Sonarpulse>
dtz: no more WIP?!
<Sonarpulse>
taktoa: rust would be my ideal choice I think
<Sonarpulse>
Profpatsch: i mean, that binary won't work for cross
<Sonarpulse>
Profpatsch: does it matter which libc getconf comes from?
<Sonarpulse>
getBin buildPackages.stdenv.cc.libc
<Sonarpulse>
will be the same on native
<Sonarpulse>
and get a binary you can still run on cross
<Sonarpulse>
but will come from a different libc than the one being linked by the program
<dtz>
I mean I can work more, always more to do, but not obviously unready I don't think, although normal PR review and care are warranted :). LMK how you want to proceed, or if there are parts that are questionable and whatnot
<Sonarpulse>
dtz: so right now there is isMusl but not isGlibc, right?
<Sonarpulse>
and you squashed PRs to your liking?
<dtz>
Sonarpulse: yes re:isMusl/isGlibc
<dtz>
Sonarpulse: I am slightly burned on squashing further, but I'll happily squash or address anything that folks think could be better, even if it takes some work
<Sonarpulse>
dtz: that's fine
<dtz>
note that the PR does include the possibly significant XZ fixes
<Sonarpulse>
I'm just going to look at the whole thing at once
<dtz>
okay great
<Sonarpulse>
and assume the commit-by-commit is good :)
<Sonarpulse>
I don't wanna burn you further
<Sonarpulse>
so xz we should maybe get something else to review
<Sonarpulse>
I don't have any package-specific knowledge, basically
<dtz>
well I figure if the commit-by-commit is a smidge out of place it's not the worst, esp if it's because it tells the story a bit more honestly :)
<Sonarpulse>
more `libc == "musl"` :):):):)
<dtz>
yeah that's is another PR already
<dtz>
just drawing your attention to it so we don't merge this as-is without seeing how that's resolved
<Sonarpulse>
dtz: woah libbfd
<Sonarpulse>
target shouldn't matter
<dtz>
yeah probably could do a search-replace for "isMusl" and replace with 'libc == "musl"', modulo parens
<Sonarpulse>
whatever on the isMusl
<Sonarpulse>
can do that later or not at all :)
<Sonarpulse>
but libbfd is just a library
<Sonarpulse>
--target should have no effect
<dtz>
well kinda, it's really binutils right. But okay I'll investigate!
<contrapumpkin>
niksnut: is there a reference somewhere with all the different ways hashes are constructed in Nix?
<contrapumpkin>
(that isn't the source code :P)
<Sonarpulse>
dtz: depsBuildBuilds should also be depsBuildBuild
<Sonarpulse>
probably not needed then, as its having no effect
<shlevy>
contrapumpkin: I think the thesis has the basic ideas already
<shlevy>
But would begood to have in the manual
<Sonarpulse>
I was worried the incremental intensional store RFC made *more* hashes
<Sonarpulse>
when we should have the same or fewer types
<contrapumpkin>
yeah it just would be cool to have a reference of the different hash shapes
* contrapumpkin
digs around
<dtz>
ha! d'oh! lemme revisit that, I think I tried to clean but punted to what bgamari had (which he probably flagged as WIP or whatnot, no criticism there). I wonder if typo'ing depsBuildBuild is what made some of that necessary
<dtz>
to the debug-mobile!
<Sonarpulse>
dtz: oh if that was a bgamari thing, it might predate the finalization of my changes too
<Sonarpulse>
libbfd is a fairly new creation of mine
<Sonarpulse>
dtz: I'd try just dropping everything but the new patch
FRidh has joined #nixos-dev
FRidh has quit [Remote host closed the connection]
<fpletz>
dtz: thanks so much for your work on musl!
<fpletz>
have you tested if nixos works with musl? systemd will probably need glibc :)
<dtz>
:D:D you're very welcome!
<dtz>
I have not, and yes it will. Although until recently NixOS's systemd could be patched to build/maybe-work, but I never tried it
<dtz>
probably some minor details to sort if you tried to use NixOS this way, I never actually tried doing so just building pkgs O:)
<dtz>
Sonarpulse: ty for making comments on the PR, easier to track that way :innocent: . But feel free to chat about it here too and whatnot
<Sonarpulse>
dtz: I changed them to be all at once gbtw
<Sonarpulse>
happy to chat here too
<Sonarpulse>
but i started doing that for visibility
<Sonarpulse>
etc etc
<Sonarpulse>
dtz can we make getent a top-level thing?
<Sonarpulse>
see iconv
<Sonarpulse>
and glibcIconv
<Sonarpulse>
for an analog
<dtz>
okay!
<domenkozar>
can't wait to build musl backed haskell static executable
<shlevy>
niksnut: How hard would it be to have a way to append to an option list with --option in addition to replacing it?
<domenkozar>
it's just a monoid how hard can it be!
__Sander__ has quit [Quit: Konversation terminated!]
<Dezgeg>
yeah that's a bit of pain, you need nix-shell -p nixUnstable
<clever>
{ nix ? builtins.fetchGit ./.
<clever>
and even that!!
<clever>
Dezgeg: i want to modify nix, to add more debugging, to figure out the cause of error: imported archive of ‘/nix/store/jcmcikcs1f69aj6d8415blcjpdlz6f4d-5c09cdc187b014143302551e62d4973b5bf52814.tar.gz’ lacks a signature
<Dezgeg>
yes, you just need to build it with nixUnstable
<clever>
Dezgeg: that also fails in other ways, one min
<clever>
dtz: with this, and runit.nix, i was able to boot a linux based image, with no systemd in the closure
<dtz>
ooo neat!
<clever>
Dezgeg: dang, nix 2.0 also fails with error: imported archive of ‘/nix/store/izbja1sg9g7bsswsa8k4r0dwii9rqdwd-source’ lacks a signature
JosW has quit [Quit: Konversation terminated!]
orivej_ has quit [Ping timeout: 256 seconds]
<clever>
Dezgeg: strange, `nix-build '<nixpkgs>' -A hydra` is now consuming 100% cpu, the nix-build itself
romildo has joined #nixos-dev
<niksnut>
shlevy: I've thought about that, since it would be nice to get rid of those extra-* options
<niksnut>
probably something like 'option += value' in nix.conf, and --option option +value on the command line
<clever>
error: imported archive of ‘/nix/store/jcmcikcs1f69aj6d8415blcjpdlz6f4d-5c09cdc187b014143302551e62d4973b5bf52814.tar.gz’ lacks a signature
<clever>
niksnut: any idea why both nix 1.12 and 2.0 are failing when i use builtins.fetchTarball with a sha256 provided?, with a 1.11 nix-daemon
<niksnut>
is the daemon also 2.0?
<LnL>
oh is this 1.11 -> 1.12+?
<clever>
niksnut: 1.11 daemon
<Sonarpulse>
domenkozar: me too!
<LnL>
err the other way around
<niksnut>
yeah that doesn't work
<Sonarpulse>
dtz: you do any haskell btw?
<Sonarpulse>
i suppose i never asked
<Sonarpulse>
just tagged you in that cabal thread :)
<clever>
niksnut: so if i used a 1.12 daemon, builtins.fetchTarball would work fine?
<niksnut>
yes
<dtz>
not... "yet". I'm certainly interested in compilers and tooling for haskell, but I don't use it for anything nor do I know it very well.
<LnL>
could the 1.12 client sign fixed output paths then talking to another daemon?
<LnL>
an older*
<clever>
LnL: i was under the understanding that it was virtually identical to just src = ./.; behind the scenes, "here is a nar stream, save it to its own hash"
<clever>
so the signatures shouldnt have come into play at all
<LnL>
I think the 1.11 client signs all paths
<LnL>
only 1.12 uses the hash for fixed outputs IIRC
<clever>
LnL: i dont see how that works when importing a local path, because the client is under the control of a potential attacker, and they would also have access to the keys
<clever>
LnL: and src = ./.; works without setting up keypairs
<LnL>
dunno how it works exactly, just remember running into that
Lisanna has joined #nixos-dev
ma271 has joined #nixos-dev
<dtz>
oh yay ty for review Sonarpulse --will go through right now ^_^
<Sonarpulse>
dtz mostly things I already said :)
<Sonarpulse>
went to lunch and came back
<shlevy>
niksnut: Any thoughts on how a plugin could add a setting? I was thinking there could just be a SettingsRegistration or whatever but then we need to defer the "unknown setting" errors until initPlugins. Would that be OK?
<Sonarpulse>
and figured i better just paste what i got
<clever>
niksnut: yeah, switching to a 1.12 nix-daemon does solve it, now i'm on to a new issue, i cant nix-build the result of builtins.fetchTarball, because its just a string with context
<shlevy>
clever: nix-instantiate it
<shlevy>
with --read-write-mode
<shlevy>
and --eval
<clever>
shlevy: except, the nix file has an if statement, and will either return a derivation, or a string with context
<clever>
shlevy: depending on if you have nix 1.11 or 1.12
<clever>
maybe "${import ./foo.nix}" might work
<clever>
then the --eval would build the derivation
<shlevy>
yep
<shlevy>
well
<shlevy>
what are you trying to do?
<clever>
i have a nix expression that fetches nixpkgs and returns it
<clever>
with 1.11, it returns a derivation that unpacks it
<shlevy>
OK. Why do you need to nix-build it directly?
<clever>
and anybody using nixUnstable hasnt noticed, because nix 1.12pre is "older" then 1.12, and they have been using the fallback path
<LnL>
well that explains the problems I was having with it before
<LnL>
heh
<Sonarpulse>
dtz: hmm i am scared to merge everything
<Sonarpulse>
because my attention span i don't think is good enough to review all at once :D
<Sonarpulse>
but i don't want to make you do more work?
<Sonarpulse>
is there ways to do subseuences of commits such that we can add tests in release-cross and its good?
<Sonarpulse>
like a) mass rebuild stdenv b) mass rebuild some other milestone c) non-mass rebuild random crap d) more like c...
<Sonarpulse>
dtz: i'm sort of addicted to rebase
<Sonarpulse>
if you tell me how to chop up
<Sonarpulse>
and move tests around
<Sonarpulse>
I'm happy to help to
<dtz>
hmm, maybe. I think reviewing the big stack helps see "where" we want to go / what is possible.... now that that's somewhat solidified it might be easier to break back into pieces
<dtz>
creating small PR's in the first place was difficult, too many things xD. But I like what you're saying. Hmm.
<dtz>
(and already i've broken off the LLVM-related bits for example, as not being immediately related)
pie__ has joined #nixos-dev
<Sonarpulse>
bfd i might be overly attached to because I made it with some difficulty :)
<Sonarpulse>
others, whatever
<Sonarpulse>
like most if it is obviously low impact / nobody should care / merge the damn thing!
<Sonarpulse>
(care about neg impact, obviously we do care about benenfit :))
<clever>
niksnut: and after getting all of the hashes to align correctly, it still fails, any way to allow it to fetch a fixed-output url?
jtojnar has joined #nixos-dev
<Profpatsch>
Sonarpulse: So is there a “right” way to get binaries from the correct libc?
<clever>
ahh, allowedUris isnt part of the same magic as -I and local files!
<dtz>
what do you want to use them for? if it's to patch into scripts then stdenv.cc.libc will probably do, although if you're looking for getconf/getent there'll be special "getconf" and "getent" attributes soon-ish
<dtz>
if you want to use getconf/getent during the build, things get a bit trickier, not sure I've seen this much though
<dtz>
giving you the answer I can until Sonarpulse gets back to ya ;)
<Sonarpulse>
Profpatsch: "right" binaries and "right libc is hard to reconcile
<Sonarpulse>
Profpatsch: it was getconf, right?
<Sonarpulse>
and during build?
<Sonarpulse>
(if its not during build, then your thing was already correct, but dtz's makes it better!)
<Profpatsch>
Sonarpulse: Yes, getconf, but during runtime.
<Sonarpulse>
Profpatsch: oh cool
<Sonarpulse>
your patch is legit
<Profpatsch>
getconf is for finding out kernel system limits, right?
<Sonarpulse>
fuck if I know :D
<Profpatsch>
:D
<Sonarpulse>
the parametric approach to cross compilation: it's almost better if I don't know what any of these packages do :D
<Profpatsch>
So true I tell ya
<Profpatsch>
Just treat them as opaque and mostly fucked up blobs
<Sonarpulse>
heheh yeah
<Sonarpulse>
nixpkgs is the bandaid that makes unix tolerable
<Sonarpulse>
and also the nice centralized todo list for all the software i'd like to not use
<Profpatsch>
lol
<gchristensen>
haha
orivej has joined #nixos-dev
<rycee>
Hmm, libreoffice times out in the 17.09 builds on hydra?
<rycee>
I just bumped my 17.09 channel on my laptop and it started building libreoffice…
<rycee>
Which I noped out of pretty quickly :-D
<fpletz>
rycee: I'll take a look
<rycee>
fpletz: Thank you!
<Sonarpulse>
dtz: I'm tempted to just merge the damn thing hah
<Sonarpulse>
dtz: ok to merge?
adisbladis has joined #nixos-dev
<dtz>
I mean, into staging right? Hmm, trust me I'm as eager as can be to see this merged ASAP haha but mass-breakage is a good way to anger everyone. It'd be nice if we could merge to staging-staging O:)
<fpletz>
just had another look at the musl pr, it's a pity that my systemd upgrade broke things but we can fix that later
<dtz>
I don't have the build resources to test on various cross-configs, native-musl configs, and current native platforms, haha, :(
<fpletz>
dtz: do you anticipate breakage in other platforms than musl?
<Sonarpulse>
i don't!
<dtz>
well the cross fixes will trigger rebuilds in places and until we see that working it's "possible"
<dtz>
but don't particularly see why things'd break l ol
<fpletz>
well, in that case… let's merge we merge that wonderful PR asap :>
<fpletz>
Sonarpulse: dtz: <3
<fpletz>
wah, I shouldn't speak and type %)
<dtz>
EEEEE
<dtz>
itshappening.gif
<Sonarpulse>
:D
<catern>
this is a fairly significant vulnerability in multi-user Nix setups I think so maybe we will need a point release? or if 2.0 is released soon enough, maybe we don't need a point release? https://github.com/NixOS/nix/pull/1869
<Sonarpulse>
merged!
<Sonarpulse>
dtz: let's submit to the nixos weekly thing
<Sonarpulse>
at least once it hits master
<Sonarpulse>
dtz: also, its still dynamic linking right now, right?
<fpletz>
I'll trigger another staging eval
<dtz>
yes!
<fpletz>
rycee: btw restarted those libreoffice builds
<dtz>
and yes
<dtz>
ty sir
<dtz>
EEEEEEEP O:)
<Sonarpulse>
np!
<dtz>
well worst case everything breaks and we revert and do better next time, haha
<Sonarpulse>
I know what it's like when a pull request hangs around being unmerged
<Sonarpulse>
fpletz and globin were the first to save me from that :)
<gchristensen>
hey catern, in the future please consider sending such issues to the security team privately first -- http://nixos.org/nixos/security.html
<Sonarpulse>
dtz: well that's why I didn't squash and merge!
<Sonarpulse>
its not all or nothing if we need a revert
<Sonarpulse>
(but I doubt we will, not hard to fix things)
<dtz>
ah, indeed
<dtz>
that'd just be a monster commit anyway lol
<dtz>
and yeah any breakage really should be easy to fix
<Sonarpulse>
dtz: is there any stuff of bgamari's that still needs to make it?
<bgamari>
I do have quite a number of patches outstanding I believe
<bgamari>
I've not rebased recently
<dtz>
I.. don't remember! O:). Probably. Last few weeks I've been ignoring it until something doesn't build and I'm not sure why ... and if it's not clear I check if he had any commits touching it
<dtz>
mostly because I was trying to keep this "small" lol xD
<dtz>
(and it was already giant :))
<Sonarpulse>
bgamari: well dtz took some of yours
<Sonarpulse>
I guess we'll have to rebase and see what's left!
<dtz>
yeah, I tried to preserve commits (sometimes with cherry-pick -x) or mention you by name when "borrowing" your changes, bgamari . Credit where credit's due, apologies if missed anything O:).
<rycee>
fpletz: Cool!
<bgamari>
dtz, quite alright, thanks again for sorting through my chaos
<dtz>
sokay. I had/have a lot of chaos of my own to sort.... so I was wearing that hat anyway :). And y'alls work has made this work much better than I would've thought was possible ~1 year ago ^_^
<dtz>
I did not! Maybe saw it mentioned but .. neat!
* fpletz
used that for the systemd update because it was based on work by flokli and andi- :)
<dtz>
OMG IT WAS MERGED hahah still don't believe it
<dtz>
crossing all the fingers
<dtz>
fingers = buildPlatform != hostPlatform
<dtz>
oh, awesome
<fpletz>
staging eval is running, you will have somthing to watch for the evening/night :>
<dtz>
it's unfortunate that our platform testing infra assumes "system" is unique per platform, so "x86_64-linux" means glibc and no convenient way to specify testing both glibc and musl variants without reworking the release.nix-esque files
<Sonarpulse>
dtz: i want to rework it
<Sonarpulse>
tada
<Sonarpulse>
dtz I would just get rid of "system"-style doubles
<Sonarpulse>
llvm names are better
<Sonarpulse>
but that's a nix change
goibhniu has quit [Ping timeout: 248 seconds]
<catern>
gchristensen: agh, sorry, my mistake
<gchristensen>
:)
<Sonarpulse>
dtz: do we still need the useMusl flag for busybox?
<dtz>
yes. might be hard to get rid of it since nix repo itself references it
<dtz>
although it'll now look for a nixpkgs top-level attribute to use as the sandbox shell first
<dtz>
err if you think we can instead construct an appropriate cross and pull busybox from that then that'd be swell
romildo has quit [Quit: Leaving]
<Sonarpulse>
dtz: yeah with your staging patch I assume we can!
<dtz>
is there a good way to do that, anyway? not sure where else that's done other than busybox lol
<Sonarpulse>
do which?
<Sonarpulse>
construct an appropriate cross
<Sonarpulse>
i meant sort of out of band
<Sonarpulse>
not calling nixpkgs in nixpkgs
<Sonarpulse>
i'll look at nix repo a i guess
<dtz>
oh it just does same thing we do in-tree in make-bootstrap-tools
<Sonarpulse>
dtz: hmm ok
<clever>
also, there is the concept of what the cpu can run (x86-64, with a linux kernel)
<clever>
and that is seperate from your libc (nix-copy-closure can just ship the right one)
<clever>
so the system attribute in a derivation shouldnt include the libc your using
<dtz>
hmm indeed
<clever>
and then when i throw in qemu-user-arm and binfmt-misc, i can run armv7l-linux binaries on an x86-64-linux system!
<dtz>
! yeah I've been using that to run/test aarch64 in a limited sense
<dtz>
not nix-integrated, which is what I think you mean
<dtz>
just binfmt-misc
<Sonarpulse>
clever: fair
<clever>
dtz: ive patched my nix-daemon, so it supports a `build-extra-platforms = armv7l-linux` in nix.conf
<Sonarpulse>
mmmmm
<clever>
dtz: and now nix will just blindly execute arm builders/derivations
<Sonarpulse>
kewl
<Sonarpulse>
clever: so you are really ready for swl
<Sonarpulse>
*wsl
<Sonarpulse>
doing windows builds
<clever>
Sonarpulse: potentially, i could jam wine into there, and handle builder = "c:\bash.exe"; lol
<Sonarpulse>
hahah
<clever>
but $WINEPREFIX would have to be configured and present
<Sonarpulse>
but IIRC wsl already runs windows binaries fine
<clever>
i was thinking of how to handle it from a linux kernel
<clever>
but yeah, i could have a "linux" build of nix-daemon inside WSL, that does windows builds
<shlevy>
Well, first we want to iterate on fetchGit without having to update everyone's nix :) But also, at least fetchPypi and fetchHackage (the latter kinda exists but requires us to periodically update an all-hackage tarball hash, which is annoygin)
Sonarpulse has quit [Ping timeout: 260 seconds]
<gchristensen>
I'd love to see fetchGit move back out of nix core and in to a plugin
<dtz>
ooo interesting
<shlevy>
gchristensen: Well keep an eye on nix-plugins then :)
<niksnut>
not gonna happen
<gchristensen>
aww
<gchristensen>
but closure size, niksnut!
<niksnut>
fetchMercurial, maybe
<niksnut>
but fetchGit is too important
<contrapumpkin>
can we make its closure include git but only get retrieved lazily? :) :) :)
<shlevy>
gchristensen: I think fetchGit in nix just uses git in $PATH, no?
<contrapumpkin>
yeah, it does
<shlevy>
I don't think git is in nix's closure
<contrapumpkin>
that's what I was just asking to fix :P
<gchristensen>
ahh
<shlevy>
It's one of my annoyances with it, though I understand why if it's upstream. so I'm going to have it not be :D
<contrapumpkin>
I intentionally left git out of it
<contrapumpkin>
but there are other things, too
<contrapumpkin>
man, that PR is way out of date now
<clever>
contrapumpkin: from that pr, `And if someone builds it on a non-Nix system we'll just embed non-store paths and nothing is lost.
<clever>
`
<clever>
contrapumpkin: ive seen that fail, if you import <nix/config.nix> and then do builtins.storePath on a tool like tar, so it becomes a proper input with context
<clever>
contrapumpkin: it fails hard, if the tar isnt actually in the store
* contrapumpkin
well, the question is whether that's desirable to allow in the first place :P
<shlevy>
clever: So... don't do that? :D
<contrapumpkin>
erm, no /me
<contrapumpkin>
in general, using random shit from $PATH feels very un-nixy to me
<contrapumpkin>
in almost all contexts except those where we're forced to
<clever>
shlevy: restricted eval in hydra also blocks that now, due to improvements in nix's restricted mode
<shlevy>
That stuff is in <nix/config.nix> is only "exported" because it's needed by other stuff in nix corepkgs
<clever>
shlevy: i was abusing it to implement builtins.fetchTarball on nix 1.11
<shlevy>
clever: Ah, so it's fixed in 2.0 with proper plugins :)
<clever>
shlevy: its also fixed on 1.12, which supports fetchTarball with a sha256
<gchristensen>
~ there is no 1.12 ~
<shlevy>
clever: I mean the meta-motivation is fixed
<fpletz>
but there is a talk about it! ;)
<dtz>
yeah, having nix break hard and badly because it's executed without magic PATH setup seems.. unfortunate
<dtz>
at least for things like.. fetchTarball, and lesser fetchGit but IMO definitely that too
<contrapumpkin>
how bad is the error right now if you fetchgit and don't have git on the path?
<clever>
dtz: in this case, its not about whats in $PATH, its about what paths are in <nix/config.nix>
<fpletz>
just like php6, there were even books published but it never got finished and released \o/
<dtz>
whoa, so guess what: 1.12 will NEVER BE RELEASED
<dtz>
:P
<contrapumpkin>
clever: the PR is all about what's in $PATH though
<clever>
dtz: i was adding those strings to $PATH, inside the nix sandbox, to make sure it didnt care about the original $PATH
<contrapumpkin>
and not having to care
<clever>
contrapumpkin: ah, havent fully read it yet
<contrapumpkin>
the point is that Nix randomly uses several tools from $PATH
<dtz>
oh
<contrapumpkin>
during its normal operation
<contrapumpkin>
and that PR was trying to rein those in
<clever>
shlevy: this is where the tar in config.nix gets used
<clever>
to unpack channels, so it has to work before nixpkgs exists
<shlevy>
Right
<clever>
and the newer option everybody recomends, NIX_PATH=nixpkgs=https://example.com/foo.tar.gz probably leads to download.cc, which expects tar in $PATH