Lisanna has quit [Remote host closed the connection]
Lisanna has joined #nixos-dev
<angerman>
what tooling do we have to compare derivations? Say two derivations for the same package, but they end up being slightly different. And then go from there to find the relevant part of the expreression that evaluated differently?
<gchristensen>
nix-diff
<gchristensen>
its a haskell package
<angerman>
gchristensen: thanks. I'll look into it.
<disasm>
nix-diff == awesome :) -- clever showed me it about a month ago.
mbrgm has quit [Ping timeout: 240 seconds]
mbrgm has joined #nixos-dev
pie_ has quit [Ping timeout: 260 seconds]
taktoa has joined #nixos-dev
pie_ has joined #nixos-dev
orivej has joined #nixos-dev
Lisanna has quit [Remote host closed the connection]
thefloweringash has quit [Ping timeout: 246 seconds]
<aminechikhaoui>
I just use `diff -u --color <(nix show-derivation <drv1>) <(nix show-derivation <drv2>)` :p
thefloweringash has joined #nixos-dev
Lisanna has joined #nixos-dev
pie_ has quit [Read error: Connection reset by peer]
pie_ has joined #nixos-dev
<clever>
aminechikhaoui: nix-diff is recursive
<aminechikhaoui>
oh nice, I didn't have a use case yet to use it
ma27 has joined #nixos-dev
ma27 has quit [Ping timeout: 240 seconds]
pie_ has quit [Ping timeout: 240 seconds]
goibhniu has joined #nixos-dev
MichaelRaskin has quit [Quit: MichaelRaskin]
<Mic92>
Is there a way to get derivations that are direct dependencies of an updated derivation? That would be sometimes a good alternative to a complete rebuild.
<Mic92>
assuming it is a c library it is unlikely that packages beyond direct ancestors breaks.
<Mic92>
I don't mean direct dependencies, but derivations that directly depend on a certain derivation.
<Dezgeg>
not directly, but you can query the entire nixpkgs drv graph as XML
<Dezgeg>
and process that
<Mic92>
Does the graph fit into my memory?
<Dezgeg>
yes
<Mic92>
Dezgeg: do you know what the command I have to use for that?
<Dezgeg>
I wish I had documented the exact things I did, but I presume it was something like nix-store -q --xml $(nix-instantiate -A stdenv) $(nix-instantiate -A stdenvNoCC) etc. for all the packages
<Dezgeg>
Mic92: ok I figured it out, see latest commit in the repo
<Dezgeg>
hope it helps
<Mic92>
Dezgeg: thanks
<Mic92>
Dezgeg: Maybe I don't even need the whole graph at once. I could also check for each derivation if it directly depends on a package that was updated. That could be then even done in parallel
<Dezgeg>
I expect it to be slow unless you figure out how to do that without launching a nix process each time
<Dezgeg>
the XML file is 400 kB
<Dezgeg>
er
<Dezgeg>
56 M
<Mic92>
56M sounds reasonable
orivej_ has joined #nixos-dev
orivej has quit [Ping timeout: 260 seconds]
__Sander__ has joined #nixos-dev
zybell_ has quit [Ping timeout: 240 seconds]
pie_ has joined #nixos-dev
<sphalerite>
Dezgeg: how is it not directly possible?
<sphalerite>
Mic92: does nix-store -q --referrers /nix/store/*-foo.drv not do what you want?
ma27 has joined #nixos-dev
<Mic92>
sphalerite: do I have to instantiate all packages for that?
<Mic92>
could be a solution though
<Mic92>
because the other variant is doing the same
<sphalerite>
yeah I think so
<Mic92>
Then I can diff against all derivations in the nixpkgs evaluation and it is done.
<sphalerite>
Lisanna: ooooh, maybe I can build haskell on my chromebook now??? :D
<Mic92>
If I build multiple sets it can still be parallel
<sphalerite>
Mic92: right, as long as you don't have evaluation errors in any of the packages in the set
<sphalerite>
so it doesn't work if you're going for all of nixpkgs
<Mic92>
this can be catched with tryEval
<sphalerite>
ooh right that's what list-pkgs does
ma27 has quit [Ping timeout: 256 seconds]
ma27 has joined #nixos-dev
<jtojnar>
peti: revert should be safe but it will be a massive rebuild
<peti>
jtojnar: Well, it can't be helped IMHO. The current state of Nixpkgs is not particularly useful, so all those re-builds feels like a good investment if they yield working binaries.
<peti>
I pushed cb6f88ce5ea98998701e1d49690e294ab2613daa.
<LnL>
Mic92: did you get anywhere with the nix-docker issue?
<jtojnar>
thanks
<jtojnar>
I will try to bisect it during the weekend
<peti>
jtojnar: Thank you very much for the quick response! When I opened the issue, I was convinced that I would never find out what was causing these crashes, so I was pleasantly surprised that the issue turned out to be so easy to solve. :-)
<gchristensen>
!pm peti, jtojnar
<gchristensen>
s/pm/m/
<jtojnar>
we noticed the issue when running some tests but did not think it would affect so many things
<jtojnar>
what is !m?
<sphalerite>
back in the days when botbot was alive, it would say "You're doing good work, <name>"
<LnL>
it should work in #nixos, botbot is not in this channel
<gchristensen>
I mean it when I do the !m, but I think it is sort of a joke-thing, because I know Eelco loves it
<LnL>
heh, no worries I was just wondering if you found something
<LnL>
for reference, I have a version with 1.12pre5619_346aeee1 that works fine
<Mic92>
LnL: there was one executable, where it returned: Permission denied. And I was setting a breakpoint, before execute it to find out, why it returns permission denied.
<Mic92>
when I encountered a bug in my debugger...
<LnL>
have you tried to debug the debugger? :p
<Mic92>
I did and fixed it
<Mic92>
but then it was already late...
<Mic92>
But I can give you the exact file it tries to execute. The problem is not file itself, but the context it is executed in.
<LnL>
yeah, it's something with the build environment
<Mic92>
gdb is also pretty bad when it comes to debugging multiple processes at once
<Mic92>
It can only follow one process at the time. Either parent or child
<LnL>
tracing multiple processes with gdb is definitively out of my comfort zone :)
<Mic92>
radare2 does a better job there, but it requires some time to get used to it.
Synthetica has joined #nixos-dev
<gchristensen>
I have seen twice now this very cool button on documentation pages
<Sonarpulse>
Mic92: syslog-ng != syslog; part of its raison d'etre is sending the logs other places
<Mic92>
and it does so for logs?
<Mic92>
via email?
<Sonarpulse>
if it is so configured
<Sonarpulse>
we could put the dep behind an option, but its a very tiny library it links (judging from build time)
<Sonarpulse>
So i figured unconditional dep + definitely *not* enabling emails by default haha, was fine
<Mic92>
LnL: I have now attached to docker-build with cntr and build my nix-build from git and installed into the container. The error is slightly different it fails to create /nix/store. So the digest is that /nix is not readable/writeable at the time of execution
<Mic92>
but let me first add a breakpoint before that and confirm that.
<Mic92>
it is not possible to deploy 18.03 with the current stable and unstable version
goibhniu has quit [Ping timeout: 246 seconds]
ma27 has joined #nixos-dev
<Mic92>
Wie kompliziert es ist Nicht-NixOS-VMs automatisch zu bauen und zu betreiben.
<Mic92>
oh, wrong channel
<Mic92>
getting late
__Sander__ has quit [Quit: Konversation terminated!]
ma27 has quit [Ping timeout: 276 seconds]
ma271 has joined #nixos-dev
7GHAAFMOQ has joined #nixos-dev
7GHAAFMOQ is now known as zybell_
Synthetica has quit [Quit: Connection closed for inactivity]
MichaelRaskin has joined #nixos-dev
elvishjerricco has joined #nixos-dev
<copumpkin>
anyone seen this before? "error: store path mismatch in file downloaded from" from a builtins.fetchurl call
<Sonarpulse>
copumpkin: sounds like a stupid different name same hash fixed-output drv thing?
<shlevy>
Sonarpulse: So, to reiterate outside the noise: In the ideal world, all code that emits code would have a base package that's emit-system agnostic and emit-system specific plugins/libraries/whatever.
<dtz>
copumpkin: yes
coconnor has joined #nixos-dev
<dtz>
I just yesterday submitted PR to ensure that never confuses someone again
<dtz>
well once a release is made and propagates
<copumpkin>
dtz: nice! how do I fix it? :)
<dtz>
copumpkin: fix your hash
<Sonarpulse>
shlevy: yeah
<Sonarpulse>
plugins so no rebuilding some shim binary
<Sonarpulse>
though I suppose tiny shim binary is fine
<copumpkin>
dtz: but a broken hash is how I figure out the correct hash :P
<shlevy>
But without that, IMO multi-emit is only worth it if it's relatively low cost. *especially* since so much of the base toolchain is single-emit anyway
<copumpkin>
you telling me I have to actually nix-prefetch-url this shit
<copumpkin>
unacceptable!!
<shlevy>
copumpkin: I usually change the hash slightly
<Sonarpulse>
shlevy: relatively little of the chain is single-emit
<copumpkin>
I do too!
<dtz>
shlevy: with 2 and fetchurl that doesn't work
<dtz>
and it just prints a store mismatch
<Sonarpulse>
binutils is multi-emit except for gnu asm
<copumpkin>
but apparently until dtz's PR, that doesn't work on builtins.fetchurl
<Sonarpulse>
llvm is multi-emit
<dtz>
which is... unhelpful
<copumpkin>
:)
<dtz>
:D
<copumpkin>
thanks dtz
<shlevy>
Sonarpulse: "except asm" means "not actually multi-emit" :P
<Sonarpulse>
my escalation would be a) use ninja in llvm and cmake
<shlevy>
We still need to build it anew for each emit system
<Sonarpulse>
shlevy: but it is for llvm
<dtz>
copumpkin: <3
<Sonarpulse>
because internal assembler
<Sonarpulse>
is what llvm can use
<Sonarpulse>
all you need is multi-target ld
<Sonarpulse>
I'd maybe switch to llvm-based cross compilation by default
<Sonarpulse>
and reuse (just rewrap) normal binutils
<shlevy>
Sonarpulse: I guess my question is how common the use case you're optimizing for is
<Sonarpulse>
it wold work!
<copumpkin>
apparently I can't nix-build a builtins.fetchurl either
<copumpkin>
gah :)
<Sonarpulse>
shlevy: well the faster it is to build the more common my use-case will be :P
<Sonarpulse>
yes I'm biased
<Sonarpulse>
I mean if its really bad
<Sonarpulse>
ok single target
<Sonarpulse>
I'll accept it
<shlevy>
OK :P That's all I wanted to get to
<Sonarpulse>
but first let's through in ninja
<shlevy>
I'm not convinced it's really bad
<Sonarpulse>
cool
<shlevy>
And agree let's do multi-emit if we can
<Sonarpulse>
+1
<shlevy>
But really we should split that shit out
<Sonarpulse>
is llvm made to do backend plugins?
<shlevy>
Sure, it's open source :P
<shlevy>
(I don't know the actual answer to your question)
<Sonarpulse>
I just asked in OFTC #llvm
<Sonarpulse>
maybe I can do something for them that won't piss them off....
<shlevy>
:)
<zybell_>
llvm is thought as plugin for emit 'backend' and plugin for lang 'frontend'
<shlevy>
zybell_: But it also has common logic. And you can't have one version of clang choose different versions of llvm at usage time
<zybell_>
no but have llvm choose different version of clang
<zybell_>
the problem with any compiler is: The most stable Part the running main is the most boring one, the optimizer.
<Sonarpulse>
shlevy: no dlopen support it sounds like
<shlevy>
Hmm, why?
<Sonarpulse>
i dunno
<Sonarpulse>
nobody bothered
<shlevy>
Anyway, separate small frontends doesn't seem like a big deal.
<shlevy>
Ah
<zybell_>
So nobody counts That as the app. the interesting interaction is with the plugins. But THEY depend (like all plugins) on the boring part.
<shlevy>
If you're adding it, I'd recommend having the interface pass in any needed symbols in an explicit table rather than relying on symbol resolution, or dtz will be sad
<Sonarpulse>
shlevy: hmm ok
<Sonarpulse>
dtz: btw is your group the original llvm group?
<dtz>
shlevy: <3<3
<shlevy>
I mean, in a perfect world system-specific stuff would just be data
<shlevy>
dtz: You've revolutionized my thinking about all things binary :P
<shlevy>
Down with dynamic loaders
<shlevy>
Just put the code into memory, fix up any within-module relocations, and get out of the way
<dtz>
:D right?!
<dtz>
it's cool to be able to kinda actually do that
<dtz>
at some point want to write up what assumptions/requirements there are to do so for common code
<Sonarpulse>
dtz: your group has plans for dynamic linking ng?
<zybell_>
does somebody remember option -H of cpio?
<dtz>
Sonarpulse: yes it's original llvm group, although of course all those people graduated ages ago ^_^
<dtz>
but same advisor/group ^_^
<shlevy>
dtz: Do you know off-hand if standard C++ ABIs require vtables to be in global symbol tables somehow?
<Sonarpulse>
dtz: nice
<Sonarpulse>
!
<shlevy>
I guess they do... Since there's not necessarily rtti
<shlevy>
Boo
<shlevy>
So niksnut is going to yell at me when I propose changing the Nix plugin API... I guess we'll have to wait until your paper is out :D
<Sonarpulse>
dtz: btw I found an example of HOST and TARGET in the LLVM source, but then somebody else piped up for taking the discussion back to the mailing list
<dtz>
implementations of C++ often require something like that, in order to satisfy ODR
<Sonarpulse>
so my __is_target saga continues
Lisanna has quit [Quit: Lisanna]
<shlevy>
dtz: Might be nice to have a clean wrapper around dlopen that handles symbol stuff properly at compile time
<dtz>
shlevy: so maybe part of some C++ abi rules?
<dtz>
I know that sometimes c++ libs (libc++/etc) use string comparison of names instead of comparing addresses of these things, to make things "work" when folks do things wrong
<shlevy>
dtz: E.g. something that lets me says "the code in this module may depend on symbols foo and bar" and that desugars (maybe only in allmux context?) to something explicitly passing a symbol table in
<dtz>
but libc++ refuses--it has an option to partially enable that in like.. I forget--one particular scenario xD
<dtz>
and on windows, probably
<dtz>
shlevy: yes!
<dtz>
shlevy: I think making that sort of thing would be a great next step, and generally an improvement on how foiks bind things together (dlsym'ing strings is sad-face)
<dtz>
but not sure how to do that best without inventing some IDL ;)
<shlevy>
dtz: Well in the Nix context it's especially fun, as the plugins only action is in ctors
<dtz>
IIRC libtool (lol) has some sort of functionality for this, don't know that it's used by anyone ... xD
<shlevy>
dtz: So we have to "pass" the symbol table to the dynamic loader, not the actual dlsymed function, for it to be seamless
<dtz>
shlevy: oh
<shlevy>
Maybe "seamless" is too high abar though
<dtz>
shlevy: in that case just adding your plugin to the allexe('s) might do the job xD
<shlevy>
Oh, I'm sure it would
<dtz>
oh not if it uses dlsym
<shlevy>
Actually since they're all RTLD_LOCAL they can't even try to get each other's syms
<shlevy>
I don't think there's any dlsym usage there
<shlevy>
There *is* some for builtins.importNative
<dtz>
if it has a constructor that calls a "registerme(table of pointers)" that would work
<shlevy>
But with modern plugins you don't really need importNative
<shlevy>
But anyway, a good replacement for dlsym would be great
<shlevy>
and an IDL seems like exactly what's needed
<zybell_>
C++ *is* an IDL!
<shlevy>
dtz: So how would the include-in-allmux story work? dlopen would do basename-based dispatch for which ctors and dtors to register? :D
<shlevy>
zybell_: Perhaps a more constrained one would be appropriate :P
<dtz>
but esp when publishing it's important to make public the exact code used in experiments
<MichaelRaskin>
zybell_: I do not buy that an _interface_ definition language forces recompilation on adding a new _private_ field
<dtz>
shlevy: but basically yes
<shlevy>
Hmm OK
<shlevy>
that lets many of the cases I care about work without deep changes
<sorear>
private fields are part of the ABI interface if you ever allocate objects
<shlevy>
Though makes them less pluggable of course
<dtz>
shlevy: I already generate code that basically does "am I <A>? if so, run_ctor_A, atexit(dtor_A), run_ctor_B, atexit(dtor_B), ... exit(main(..))" orso
<shlevy>
I mean, allmux aside moving to well-defined ABIs for plugins is a generally good thing for the ecosystem
<dtz>
but one "rule" for allexe's currently is that they're not "allowed" to care about their own "symbols"
<zybell_>
Why don't have discipline? And Macros...Lots of Macros...Inside-Out Macros...I looooove it.
<dtz>
which is what makes something as simple yet ambitious ;) as allmux... kinda actually work :D
<dtz>
basically I'm reinventing lisp machines from the C and LLVM side
<dtz>
:D
<shlevy>
dtz: They're allowed to only at link-time right?
<dtz>
^ should be my phd thesis subtitle
<dtz>
lol
<shlevy>
dtz: so does dlopen actually *work* that way currently? Or is that just he natural extension?
<shlevy>
Is there actually a well-defined "entry point" for dlopen anyway? I guess some kind of top-level ctor?
<dtz>
shlevy: kinda re:link-time. Super rarely do folks actually care they bind to a symbol named "foo" necessarily, they just use that as a convention to mean something more specific like "particular function foo from library X" (X::foo, etc -- yay namespacing)
<shlevy>
dtz: Right, yeah
<dtz>
but what I mean is that if I renamed "foo" to "asdf" but still bound you to it--would your app care about the name? generally, no, although strictly speaking that's not at all safe to assume for "arbitrary" code
<dtz>
errr
<shlevy>
Right
<dtz>
shlevy: I'd probably just encourage doing the equivalent of LD_PRELOAD=myplugin.so ./nix
<dtz>
and generate a new "nix-with-plugin" binary
<dtz>
okay I think you get all of this, sorry haha not used to folks knowing about linking and loading :D
<dtz>
(much rarer in the academic compiler community haha)
<shlevy>
dtz: :( OK.
<zybell_>
<MichaelRaskin>:If you define Interfaces with C++, you are *not allowed* to define private members. They are for derived Classes.
<dtz>
took me a good while to convince peers that magically converting things the way allmux was something super unheard of/impossible for software as commonly used on linux/C/etc.
<dtz>
shlevy: *although* TBH I've wanted to come up with a good answer for actual dynamic extensibility
<dtz>
I just have pushed really hard on "no no let's see when we really need it"
<dtz>
"librsvg.so must be dynamically loaded using binding rules and lookup semantics that <no one understands> because I want to build it separately but then I'll 100% of the time have it available" is not a good reason lol
<dtz>
errr
<dtz>
i think there are exciting possibilities if we free ourselves from how we do things today, :3
<shlevy>
:D
<shlevy>
dtz: moving to #nixos-chat...
<shlevy>
:o you're not there :'(
<zybell_>
shlevy: think abut the -H option of tar and cpio, pls?
<shlevy>
zybell_: Hm?
<dtz>
errr
<dtz>
nixos-chat?! haha
<shlevy>
dtz: It can get quite... off topic
<shlevy>
But that's the point I guess :D
<dtz>
FWIW #allvm over on OFTC exists... as does #nixos-allvm .. "just in case" (no one uses the latter lol I just blocked it)
<dtz>
indeed
<dtz>
but I find I rather like the sort of people who enjoy Nix, esp intersected with those who use IRC :D
<MichaelRaskin>
Yeah, sometimes it goes off its topic and on to Nix packaging discussions
<zybell_>
that option sets the format, could load a plugin that implements the format. What is an arch other than a format?
<shlevy>
I need to figure out how to get znc-erc to play nicely with multiple irc networks at once... As of now it seems like I have to explicitly switch back and forth :(
<zybell_>
so i envise 'compile -H c++ -H i386-elf-linux -o ... ...' where c++ is the frontend plugin (on load it calls frontend.register(this)) and the other is backend.
<zybell_>
argv[0]-shortcuts and configurable defaults could be mixed in.
<dtz>
I always used irssi+screen, I gave up on ZNC but tbh I was already too invested in ssh+irssi+screen lol
<shlevy>
dtz: I'm sure it's just me not knowing the right things to run
<shlevy>
It's not a znc issue I don't think, since I don't get disconnected
<shlevy>
It's just my emacs can only talk to one at once
<shlevy>
zybell_: I mean, clang already has the flag you want, just called -target
<shlevy>
But it's not a plugin today
<zybell_>
On tar its no plugin too. But it could be easily implemented as one.
<dtz>
lol let's put guile^WNix into every application! muahaha
<dtz>
btw did you know bash as commonly built/shipped (incl. nixpkgs) includes support for dlopen'ing native code and using it?! haha
<dtz>
similarly, make has support for native plugins >.<
<shlevy>
Amazing
<shlevy>
what's a bash plugin look like? :'(
<dtz>
builtins, IIRC
<zybell_>
xyz.so what else?
<dtz>
IMO prevalence of "open this code and jump to it" is a biiiiig problem. Remember that scary samba CVE re:dlopen? It shouldn't have been possible in the first place--no one ever actually runs samba without having in mind the exact set of plugins they're using (enabled via config or installed via package, w/e)
<dtz>
haha no now sorry
<shlevy>
Agreed. Plugins mostly (only?) make sense in dynamic user environments. They're kind of crazy for system services, except to work around rebuild costs :P
<dtz>
! yes
<zybell_>
not really. Its the number of options needed, vs the memory of those not used
<zybell_>
shell is a kind of plugins too.
coconnor has quit [Quit: Lost terminal]
<dtz>
err why is memory needed for unused functionality? modulo page boundaries :P any significant amount of code unused won't be in memory (demand paging)
<dtz>
although some software's architecture/organization might not be friendly in that regard
<shlevy>
dtz: nommu :P
<dtz>
but I think it's not unreasonable to suggest that's just bad code being bad :P (although more fairly it's likely folks designing and optimizing around current platforms)
<shlevy>
dtz: and in general there's areason you can turn off overcommit :)
<shlevy>
dtz: anyway disk space is certainly needed, where's my on-demand binary cache fetching the first time I target RISCV-128?
<zybell_>
how often do you restart your shell after update? and the apache should?
<dtz>
:D
<zybell_>
udate of progs,that is
<zybell_>
*update
<dtz>
idk I currently run allexe's for most of my CLI needs, terminal, etc.-- most of which are JIT'd with a cache as-needed.. with optional AOT (pre-populate cache) for things like my pdf viewer and whatnot lol
<dtz>
sorry that's not really an answer, I'm just excited :3
coconnor has joined #nixos-dev
<dtz>
what do you mean how often do you restart your shell? (why would it matter if that was high or low?)
<dtz>
I'd guess that I, and "most" folks, execute their shell many times per update to it
<zybell_>
In some areas plugins feel natural, nobody thinks twice about firefox plugin. Yet in the bash which is the ultimate plugin runner,a in-process plugin feels strange, but out-of-process plugins are not even recognized as such.
<dtz>
oh
<dtz>
oh yes indeed
<dtz>
and you're right-- processes communicating over pipes are themselves a distributed system of sorts
<zybell_>
dtz: if you update 'cat' you don't have to restart your shell. If it were compiled in?
Jackneillll has quit [Read error: Connection reset by peer]
Jackneillll has joined #nixos-dev
<zybell_>
The best plugin would be a double sided one. In one program it lets express that in a standardized format. In the other progs it makes that format available. Although the format is standard, its completely shielded.
<dtz>
^_^
davidlt has joined #nixos-dev
<dtz>
Sonarpulse: would it be hard to plumb "localSystem" through nixos bits? :D haha
<dtz>
ooo we have nixpkgs.crossSystem ?? haha
<dtz>
hopeful face
<Sonarpulse>
dtz: instead of system and platform?
davidlt has quit [Remote host closed the connection]