<jtojnar>
maye that's what is keeping it from building webkitgtk
<jtojnar>
though I cannot see it in any queue
<gchristensen>
killing
MichaelRaskin has joined #nixos-dev
<MichaelRaskin>
Hm. Apparently I had lost the connection. Profpatsch: I guess I missed your reply
<ekleog>
here is a copy-paste:
<ekleog>
MichaelRaskin: These tests I’d put into leaves, yes.
<ekleog>
I don’t want to install zathura to build xelatex.
<ekleog>
MichaelRaskin: There’s two levels here: when I talk about leaves, I mean symbolical leaves, but the files could be anywhere.
<ekleog>
So for example the xelatex test could still have a file inside pkgs/zathura/tests/xelatex.nix
<ekleog>
And then be imported in pkgs/tests/default.nix.
<MichaelRaskin>
Thanks
<Profpatsch>
MichaelRaskin: These tests I’d put into leaves, yes.
<Profpatsch>
I don’t want to install zathura to build xelatex.
<Profpatsch>
MichaelRaskin: There’s two levels here: when I talk about leaves, I mean symbolical leaves, but the files could be anywhere.
<Profpatsch>
So for example the xelatex test could still have a file inside pkgs/zathura/tests/xelatex.nix
<Profpatsch>
And then be imported in pkgs/tests/default.nix.
<Profpatsch>
^ was my reply
<MichaelRaskin>
Right now nixpkgs doesn't have symlinks…
<Profpatsch>
What I’d like to see: each package should come with its leave tests in a meta attribute, automatically run by ofborg
<Profpatsch>
MichaelRaskin: No symlinks, just imports.
<MichaelRaskin>
Well, there is this problem of attributing to one side or another
<Profpatsch>
Most tests are pretty clear, right?
<MichaelRaskin>
Not sure where it will actually go…
<Profpatsch>
xelatex/zathura tests whether xelatex rendering works by opening with zathura if I understand correctly?
<MichaelRaskin>
Well, it is hard to say which side it tests.
<MichaelRaskin>
Actually, I ended up treating it the other way round.
<Profpatsch>
But the important thing is the leaves/edge distinction.
<Profpatsch>
Where the files are is secondary really.
<MichaelRaskin>
First, ghostscript/imagemagic is used to check if xelatex/pdflatex builds a PDF that can be tesseract-recognised
<Profpatsch>
Except it’s harder to go from a nix symbol to the file where it originates than the other way around.
<Profpatsch>
MichaelRaskin: Yeah, that’s what I reckoned this test would do.
<MichaelRaskin>
Then Evince/Zathura can be tested on files from pdfLaTeX/XeLaTeX/ImageMagick
<Profpatsch>
So it’s really two tests with one common preparation step.
<MichaelRaskin>
(as renderers; this time tesseract is given the screenshots)
<MichaelRaskin>
It's worse — it's 3 tests that are also used for preparing the data for 6 further tests
<Profpatsch>
So there could be a test file in pkgs/xelatex and one in pkgs/zathura, containing their tests and linking to the preparation steps in /tests/latex or something.
davidlt_ has quit [Ping timeout: 260 seconds]
<Profpatsch>
Or all the files are in /tests/ and zathura and xelatex link from meta.tests to their respective tests.
<MichaelRaskin>
That last approach is actually better, in my opinion
<Profpatsch>
Yes, it sounds more reasonable.
<MichaelRaskin>
And it will be a natural evolution of what I started now.
<MichaelRaskin>
Also, for helpers that shouldn't be considered stable for external use, it is convenient when all the tests are relatively localised and can import directly via relative paths
<gchristensen>
it is very very annoying that stable NixOS can't build the next stable NixOS at all
<MichaelRaskin>
Anyway, I think that discoverability is important _but_ can start with actually linking the existing NixOS tests to the packages.
<Profpatsch>
Exactly.
<MichaelRaskin>
For independent nixpkgs tests I would say that discoverability solution won't change anything about themselves (as we agree they will live in /tests/)
<Profpatsch>
And a small library to generate those descriptions attrsets.
<MichaelRaskin>
I guess that library would be a requirement to actually establishing the links,
<gchristensen>
this is sounding overcomplicatede
<Profpatsch>
Should everything eventually be pkgs.tests?
<Profpatsch>
gchristensen: Should I mock something up?
<gchristensen>
probably
<Profpatsch>
Okay, then give me an hour.
<MichaelRaskin>
Profpatsch: in any case, I think that the first merged versions of nixpkgs tests will not be stable enough for linking it in such a way
<MichaelRaskin>
(whatever I do with it, it just doesn't matter)
<Profpatsch>
MichaelRaskin: Just try not to expose anything by the tests attribute for the time being. :)
<Profpatsch>
s/by/but/
<MichaelRaskin>
Have you seen my today's version?
<Profpatsch>
one sec
<Profpatsch>
MichaelRaskin: btw optCall is a code smell, meaning the data is not uniform enough.
<MichaelRaskin>
Well, it is like with overrides — right now it is just never needed, but in general, you could want to pass a list of fonts either as a list of packages from some nixpkgs, of as a function taking the packages from the nixpkgs providing the tested package (say, zathura)
<Profpatsch>
Hm, I’d not try to abstract before it’s needed.
<Profpatsch>
Everything runtime-testing types feels bad, the overlays are quite principled from a statically typed perspective.
<MichaelRaskin>
This is why we still have weird restrictions on what can be sanely overridden in Nixpkgs, though…
<MichaelRaskin>
Of course, a clean way to do runtime type-testing is via multiple-dispatch generic functions…
<MichaelRaskin>
But we don't have that. Yet.
<Profpatsch>
It’s the same thing from a semantics point of view.
<Profpatsch>
Maybe I’m too much of a ML-kind of guy.
<Profpatsch>
MichaelRaskin: should the tests attribute set be flat?
<Profpatsch>
Or fixed to two levels?
<MichaelRaskin>
I am not completely sure yet.
<gchristensen>
please be flat
<gchristensen>
well ... I don't know
<Profpatsch>
gchristensen: Would tests.program.hellotest be okay?
<gchristensen>
pkgs.tests.p.rogram.hellotest?
<MichaelRaskin>
I think ofborg handles arbitrary depth in NixOS tests well?
<gchristensen>
yeah it doees
<MichaelRaskin>
p.pr.program !
<Profpatsch>
Doesn’t mean arbitrary depth gains anything. :)
<Profpatsch>
tests.program-hellotest is just a nice.
<Profpatsch>
*as
<MichaelRaskin>
Well, arbitrary depth might make sense like pkgs.tests.view-pdf.xelatex.zathura
<Profpatsch>
pkgs.tests.view-pdf-xelatex-zathura
<Profpatsch>
:)
<MichaelRaskin>
I would say it's worse
<MichaelRaskin>
Also, _obviously_ viewpdf entries can be generated automatically
<gchristensen>
yeah ... what?
<Profpatsch>
Ah okay.
<gchristensen>
the only thing I have a stake in is making it easy to get from a specific attribute name to tests for that attribute name
<MichaelRaskin>
A top-level attrset being a mix of hand-defined entries and multiple sets of autogenerated combinatorial explosions sounds scary
<gchristensen>
yes
<Profpatsch>
gchristensen: meta.tests = …
<MichaelRaskin>
gchristensen: I would say that phase0 is ~ what there is now.
<Profpatsch>
In your derivation for that attribute name.
<ekleog>
why not pkgs.tests.view-pdf { generator = "xelatex"; viewer = "zathura"; }, if you really want auto-generation?
<MichaelRaskin>
Maybe with some naming cleanup — like this view-pdf thing
<MichaelRaskin>
ekleog: because @ofborg nixpkgs-test
<ekleog>
and then in the meta.tests just put whichever are pleasant, with attributes pre-filled
<Profpatsch>
gchristensen: meta.test: list of tests or attrset of tests?
<Profpatsch>
In this case list of sounds okay.
<Profpatsch>
On the other hand there shouldn’t be ordering …
<MichaelRaskin>
ekleog: What you say is more or less what is supported on the lower level, but it would be nice to pre-generate sane-ish things referenceable by bot
<ekleog>
(or even do like xelatex.meta.tests = map (view: pkgs.tests.view-pdf { generator = "xelatex"; viewer = view; }) pkgs.tests.all-pdf-viewers; or similar)
<MichaelRaskin>
Well, ordering by cost could actually sound sane
<gchristensen>
personally, I round that down to don't-care
<MichaelRaskin>
I just need to make a test that meaningfully uses both Firefox and Chromium. A systemd test — what was it about http logs?
<Profpatsch>
updateWalker
<Profpatsch>
If set to true, the package is tested to be updated correctly by the update-walker.sh script without additional settings. Such packages have meta.version set and their homepage (or the page specified by meta.downloadPage) contains a direct link to the package tarball.
<gchristensen>
jesus
<MichaelRaskin>
You will start caring immediately
<Profpatsch>
What, when did that meta attribute crop up?!
<MichaelRaskin>
I can auto-update quite a lot of packages that I use.
<MichaelRaskin>
Sometimes I need a real update metadata file (default.upstream), sometimes it is enough just to mark the package as safe for update.
<Profpatsch>
Oh hell, that’s a boolean attribute fixed to one single script.
<Profpatsch>
Whoever thought that’s a good general solution for a meta attribute? oO
<MichaelRaskin>
I want to mark a few tens of packages that I added and that are good and simple.
<MichaelRaskin>
And I wanted to mark just them and not put it into the manual
<gchristensen>
Profpatsch: remember to be kind
<MichaelRaskin>
But no, I was told it is OK to add but not OK to add without global description
<Profpatsch>
Ah, no offense.
<Profpatsch>
But fixing it to a single script seems a bit … weak.
<MichaelRaskin>
Well, the attribute means it is safe to update by a really stupid algorithm, so its semantics is somewhat separate from the script
<gchristensen>
with checkMeta being validated on every PR, it must be globally defined
<Profpatsch>
Ah, I didn’t know it was.
<gchristensen>
yeah, it has found a lot of bugs
<Profpatsch>
Nice.
<MichaelRaskin>
The story about the attribute is older than ofborg, though
<gchristensen>
indeed
<gchristensen>
oh
<gchristensen>
right I see
<Profpatsch>
But really nice that it reuses the type system.
<Profpatsch>
These unknown meta attributes should probably go into passthru instead.
<MichaelRaskin>
Well, I think the valuable check is that known attributes are OK, extra attributes in meta being wrong is a problem of whoever tries to fly under radar?
<gchristensen>
many of the problems we found were people trying to set one field but ... missing
<MichaelRaskin>
Does checkMeta even verify there are no extra arguments?
<gchristensen>
yeah
<Profpatsch>
gchristensen: Wait, isn’t the checkMetaAttr function just the submodule type?
<gchristensen>
I don't remember
<gchristensen>
it has been some time :')
<MichaelRaskin>
Pushed an example implementation of view-pdf
<MichaelRaskin>
I dunno if it is a good idea to merge what I have now. Pros: if nixpkgs-test is added to ofborg, I can say `@ofborg nixpkgs-test youtube-dl` on weekly updates of youtube-dl that we seem to get in PRs.
<Profpatsch>
Err, gchristensen; I don’t want to spoil the fun, but checkMetaAttrs in not correct.
<gchristensen>
oh?
<Profpatsch>
(for nested types)
<Profpatsch>
It does foo.check
<gchristensen>
oh well, we better make it better then
<Profpatsch>
(listOf str).check [ 3 4 ]
<Profpatsch>
true
<Profpatsch>
check only goes one level deep:
<Profpatsch>
(listOf str).check { a = 3; }
<Profpatsch>
false
<gchristensen>
gotcha
<gchristensen>
no fun spoiled here
<Profpatsch>
I have my … gripes with the type system. :)
<gchristensen>
sure
<gchristensen>
I'm sure we all do
<gchristensen>
Profpatsch: perhaps you could send a PR fixing it?
<Profpatsch>
gchristensen: I’m at it.
<gchristensen>
* please don't push it right to a branch
<shlevy>
Anyone around know how I can override libc just for one derivation?
<gchristensen>
peti: master is broken :(
<gchristensen>
anyone around familiar with haskell who can determine how to fix master's evaluation?
<Profpatsch>
Argh, I’m sure the type system needs to be copied over.
<Profpatsch>
gchristensen: Have the error message?
<Profpatsch>
gchristensen: Just remove the override.
<gchristensen>
that is acceptable?
<gchristensen>
I don't know what is okay and what isn't okay in haskell land
<Profpatsch>
gchristensen: Yeah, worst case it will break.
<Profpatsch>
Not sure how peti let it slip though.
<gchristensen>
he doesn't do PRs for it
<Profpatsch>
Should have been caught by his automation.
<gchristensen>
ah
<Profpatsch>
I *hope* he evaluates all packages against master before committing, anyway.
jtojnar has joined #nixos-dev
jtojnar2 has joined #nixos-dev
<Profpatsch>
The problem with the type system is that it’s completely entangled with the module system.
<Profpatsch>
So it should be quicker & easier to whip up a small type checker for the meta fields instead.
<shlevy>
If we're not able to move to branch protection soon (seems hopeful that we will!), I think at the very least we may be able to move to a no-questions-asked revert-and-ping for breakages that ofborg would have caught.
* gchristensen
vigorously +1's, after an email describing that you the ml with a feedback period
<shlevy>
Definitely
<shlevy>
gchristensen: would you like me to just send that now?
<gchristensen>
That would be really wonderful
<shlevy>
OK will do
<gchristensen>
Thank you :)
<dtz>
+1
<shlevy>
Sent.
<shlevy>
gchristensen: is 'next' the right branch to base feature work off of for ofborg?
<shlevy>
Also, I think it's time for @NixOSOfBorg :)
<gchristensen>
Yeah and agreed
<disasm>
lol, pruning my home directory, I still have this hanging around: nixos-graphical-15.09.784.7b85b8a-x86_64-linux.iso -- I think that can go!
mbrgm has quit [Ping timeout: 240 seconds]
<disasm>
oops, that was meant for normal nixos channel, sorry for cluttering dev :)
mbrgm has joined #nixos-dev
<shlevy>
gchristensen: Any pointers for the event model you're using in ofborg? E.g. I'm looking at the workflow for the overall_status check, it looks like it just iterates through all the eval checks and waits for them to finish one-by-one, is that correct or is there some implicit queuing/threading/event looping I'm missing?
<shlevy>
(this is my first real foray into Rust)
jtojnar has quit [Quit: jtojnar]
jtojnar has joined #nixos-dev
jtojnar2 has quit [Ping timeout: 240 seconds]
jtojnar2 has joined #nixos-dev
Lisanna has joined #nixos-dev
Lisanna has quit [Quit: Lisanna]
disasm has quit [Quit: WeeChat 2.0]
disasm has joined #nixos-dev
disasm has quit [Client Quit]
disasm has joined #nixos-dev
disasm has quit [Client Quit]
disasm has joined #nixos-dev
disasm has quit [Client Quit]
disasm has joined #nixos-dev
<thoughtpolice>
shlevy: perlPackages.JSON fails to build on latest HEAD for me somehow, due to e9ddfb789238c590e60cd76bbdabe046d58ee2d5 it appears
ma27 has joined #nixos-dev
davidlt_ has joined #nixos-dev
<Profpatsch>
gchristensen: Okay, whipped up a pretty cool type system. :P
<Profpatsch>
{ bar = { baz = { should = "string"; val = 4; }; }; foo = { }; }
<Profpatsch>
Message would be:
<Profpatsch>
bar.baz should be a string, is 4
<Profpatsch>
checkType (product { bar = product { baz = string; }; }) { bar = 4; }
<Profpatsch>
{ bar = { should = "{ baz: string }"; val = 4; }; }
<Profpatsch>
bar should be a { baz: string }, is 4
<Profpatsch>
checkType (product { bar = string; baz = unit; }) { bar = "abc"; }
<Profpatsch>
{ should = "{ bar: string, baz: unit }"; val = { bar = "abc"; }; }
<Profpatsch>
Now for some tasty sum types …
<Profpatsch>
(this is <150 LOC btw)
jtojnar has quit [Quit: jtojnar]
jtojnar has joined #nixos-dev
jtojnar2 has quit [Ping timeout: 260 seconds]
<MichaelRaskin>
Profpatsch: re: tests — maybe I even believe in a more PAM-like approach; in the sense that a library could be exercised by many unrelated tests, there is a recommended minimum to always run, there may be multiple «usually-sufficient» sets, then there are all the relevant tests.
<MichaelRaskin>
Profpatsch: Just like `nox-review wip` often builds too much to be an always-useful check
<Profpatsch>
Or "required" or something like that.
<MichaelRaskin>
I am not sure we will not want multiple usually-sufficient sets
<MichaelRaskin>
(you pick depending on what else you already need to build anyway)
<Profpatsch>
Hm?
<Profpatsch>
Hm, that’s a bit too many variables for my taste.
<MichaelRaskin>
I dunno, if something can be equally well-tested by doing a simple task in either Chromium or Firefox
<Profpatsch>
Then just pick one.
<Profpatsch>
And make a browser ? firefox flag or something.
<Profpatsch>
Where firefox is an attribute set that describes how to invoke firefox to run the test.
<Profpatsch>
then chromium can get one, elinks can get one, everbody can get one!
<Profpatsch>
But I’d abstract out of necessity, not prematurely.
<Profpatsch>
(says the guy who just wrote a type system based on recursion schemes)
<MichaelRaskin>
Well, right now I wonder if what I have to _define_ tests is in a good enough shape _for now_ to remove [WIP], merge after a day or two, then submit a PR with support to ofborg… — for manual application according to human judgement
vdemeester_ is now known as vdemeester`
<Profpatsch>
+1
<MichaelRaskin>
You don't see any blockers in the current state, right?
<Profpatsch>
Before ofborg starts running these, I’d like to fix the interface.
<Profpatsch>
But other than that, I don’t see why we can’t merge these first tests and see where it goes from there.
<Profpatsch>
With the unstable disclaimer of course.
<MichaelRaskin>
You can read README.md
<Profpatsch>
Maybe a few more people will chime in.
<Profpatsch>
Yes, I like that disclaimear.
<Profpatsch>
-a
<Profpatsch>
Since we are in a monorepo, we can just change the API on the fly.
<Profpatsch>
As long as we make clear that it’s unstable.
<MichaelRaskin>
Re: ofborg — I think it is reasonable to promise «importing tests/default.nix gives an attrsets with some tests»
<Profpatsch>
Nested attribute sets with tests at the leaves.
<MichaelRaskin>
Yes
<Profpatsch>
Where a test is an attribute set with these <insert list> attributes.
<MichaelRaskin>
test = derivation
<MichaelRaskin>
Something nix-buildable
<MichaelRaskin>
I reserve the right but make no obligation to add passthru
<Profpatsch>
Nah, test should be an attrset with metainfo and one field that points to a drv
<sphalerite>
couldn't the meta be in the derivation's meta field?
<MichaelRaskin>
What would be the metainfo for view-pdf.xelatex.evince?
<MichaelRaskin>
And yes, meta and passthru of the derivations are fair game.
<Profpatsch>
Test description, the name, the derivation, additional tags.
<Profpatsch>
derivation meta is kind of restricted and has a different scope.
<MichaelRaskin>
What useful tags can you put on the test on its own?
<Profpatsch>
"vm-test", "long-runtime", …
<MichaelRaskin>
meta.description and meta.platforms can be used as-is. Even meta.broken!
<Profpatsch>
Did they remove the strictness problem with derivations?
<Profpatsch>
Last time I checked drv.name wouldn’t evaluate if it was broken.
<MichaelRaskin>
I think so, and I hope it doesn't get reintroduced
<Profpatsch>
The other problem is that evaluating derivations is not guaranteed to be as fast as evaluatin an attribute set.
<Profpatsch>
*evaluating
<Profpatsch>
And they are not as first-class as attrsets are.
<MichaelRaskin>
Profpatsch: well, accessing meta doesn't evaluate the derivation itself
<Profpatsch>
Ah, it doesn’t evaluate the derivation, but that doesn’t mean the derivation object is not created in the nix C++ code & memory.
<Profpatsch>
You’d have to ask niksnut or maybe aszlig about that one.
<MichaelRaskin>
Wait, cna you do it without reading the dependecies??
<Profpatsch>
gchristensen could also know, since he watches the evaluation time closely.
<MichaelRaskin>
Let me put it another way — nixos tests are derivations with passthru/meta on top, and I do not want to be too different
<MichaelRaskin>
If we ever start caring, we can manually-override meta so that mkDerivation is not even used
<Profpatsch>
Might not matter, right.
<MichaelRaskin>
Ouch
<Profpatsch>
Might increase the evaluation time by a minute in the long run. :)
<MichaelRaskin>
I thought overrides were more eager
<Profpatsch>
But I’m the wrong person to ask, I just relay bits I learned from following the talk, so take it with a grain of salt.
<MichaelRaskin>
Well, you raised the (good) question of interface non-commitment, so I use you for a sanity check on that point
<MichaelRaskin>
As for tests-as-derivations I have a stronger opinion that I want to mimic NixOS tests
<Profpatsch>
We can always change it afterwards.
<MichaelRaskin>
And they return derivations with extra stuff on top
<Profpatsch>
We can also rewrite the nixos tests or read the meta.description into a wrapping attrset.
<MichaelRaskin>
Evaluating all tests and picking the relevant ones doesn't sound like a thing we know we will do.
<MichaelRaskin>
I will probably add a note about name instability.
<Profpatsch>
+1
<Profpatsch>
least guarantees for starters
<MichaelRaskin>
Should I write that anyone who wants to add descriptions to tests should just set the derivation meta?
<MichaelRaskin>
(or at least that .meta.description _will_ be the description if it is ever written…)
<Profpatsch>
I’d like to enforce a bit of description.
<MichaelRaskin>
I don't want the machinery to enforce anything _right now_, but I will try to set the style by adding the descriptions. And commit to meta.description — if it ever becomes a bad place for descriptions, the other Nixpkgs changes will make the test restructuring a negligible problem
<Profpatsch>
+1
<Profpatsch>
Okay, now I’m at the point where I’ve written a type checker and it works and I have no idea how really.
<MichaelRaskin>
Heh
<Profpatsch>
e.g. how the value is destructured on the deeper levels.
<MichaelRaskin>
I once have written an inference engine that managed to fool me by being too good.
<Profpatsch>
I just kept with the (imaginal) types, copying stuff from Haskell modules.
<MichaelRaskin>
It got mislabeled input and manged to do 90% of the job, I _assumed_ this is too good to be true and looked for this 10% bug everywhere
<Profpatsch>
in t: v: cata alg t v;
<Profpatsch>
cata is # cata :: (Type a -> a) -> Fix Type -> a
<Profpatsch>
But I’ve given it three types.
<Profpatsch>
*arguments
<Profpatsch>
So alg becomes # alg :: Type (Value -> Errors) -> (Value -> Errors)
<Profpatsch>
or better: a becomes (Value -> Errors)
<Profpatsch>
And I just know that this follows from inference (at least in Haskell it would).
<Profpatsch>
And it just works™, even in nix, and I have no idea why or how really.
<Profpatsch>
Which is spooky.
<Profpatsch>
frigging lambda calculus, gets me every time.
<Profpatsch>
And that is not even particularly strong recursion-fu
<Profpatsch>
Compared to e.g. histomorphisms.
<Profpatsch>
Or zygohistomorphic prepromorphisms for that matter. :P
<Profpatsch>
“Used when you really need both semi-mutual recursion and history and to repeatedly apply a natural transformation as you get deeper into the functor.”
<MichaelRaskin>
OK, sent an email to the mailing list about nixpkgs-test, I guess I will get zero replies, silently merge, then use ofBorg invocations to actually spread the message.
__Sander__ has joined #nixos-dev
<MichaelRaskin>
Bye for now.
MichaelRaskin has quit [Quit: MichaelRaskin]
JosW has joined #nixos-dev
davidlt__ has joined #nixos-dev
JosW has quit [Ping timeout: 260 seconds]
davidlt_ has quit [Ping timeout: 264 seconds]
davidlt has joined #nixos-dev
davidlt__ has quit [Ping timeout: 264 seconds]
JosW has joined #nixos-dev
orivej has quit [Ping timeout: 260 seconds]
Synthetica has joined #nixos-dev
disasm has quit [Quit: WeeChat 2.0]
<gchristensen>
peti: thank you, I'm glad it was the right fix :)
disasm has joined #nixos-dev
<clever>
shlevy: i did get the cross-compiled riscv64 busybox to build, but it claims the kernel is too old with qemu-riscv64
<shlevy>
Are you on staging?
<shlevy>
I have a lot of cross-fixes, some riscv-specific, there
<shlevy>
Also would be good to see exactly what syscall is failing
<shlevy>
Maybe your kernel is too old :P
<clever>
does qemu-user support syscall tracing?
<shlevy>
clever: Yeah, I don't remember the flag offhand
<sorear>
qemu-riscv64 -strace /path/to/binary
<shlevy>
I also like stracing with the host sometimes
<sorear>
yes
<shlevy>
sorear: Does qemu-user support the riscv-specific cache flush call?
<sorear>
I'm not sure how __LINUX_KERNEL_VERSION gets set in glibc
<shlevy>
clever: are you building against master or staging?
<clever>
831ef4756e3 from nixos-unstable
<shlevy>
Hmm so we pass linux kernel headers at build time
<clever>
for the busybox
<clever>
the busybox was cross-compiled on Linux amd-nixos 4.9.75 #1-NixOS SMP Fri Jan 5 14:46:36 UTC 2018 x86_64 GNU/Linux
<shlevy>
And I bumped those to 4.15 so we could compile against RISC-V
<sorear>
I think you need to reboot with a 4.15 host kernel or patch qemu to fake the kernel version
<shlevy>
Maybe glibc also has a runtime check?
<sorear>
glibc very much has a runtime check
<clever>
[1/3/545 built, 0.0 MiB DL] building binutils-2.28.1 (buildPhase): checking whether declaration is required for free... yes
<clever>
the laptop is in the middle of an armv7 cross-build as well, and id rather not interupt it at this point in time
<shlevy>
It's kind of interesting, since any situation where that check successfully runs kind of indicates the actual thing the check is looking for is there :D
<shlevy>
clever: So can you wait?
<clever>
yeah
<shlevy>
qemu-system works if you really need to run something. You can either cross-compile NixOS on staging or use the fedora image
<sorear>
sysdeps/unix/sysv/linux/riscv/configure.ac # riscv glibc wants 4.15.0 or newer
<clever>
i'll try things again from a point closer to master, and on 4.15, when i have time
<shlevy>
sorear: Right, but that's at build time, where it has 4.15 headers
<sorear>
shlevy: nope, it's a runtime check too
<shlevy>
It's confusing what that runtime check hopes to accomplish :D
<shlevy>
both with qemu-user and with a custom backport I could make it fail, but if it runs and fails it morally succeeded :D
<clever>
i also just noticed, the qemu-riscv in nixpkgs is dynamicaly linked
<shlevy>
Why shouldn't it be?
<clever>
so ldd wont do the right thing in the guest
<clever>
ldd sets magic variables, that cause ld.so to dump the libraries and never run main()
<clever>
a dynamic qemu-user leads to qemu-user's deps being dumped
<clever>
then you wind up with x86 libraries in your arm initrd
<shlevy>
Bleh
<shlevy>
I don't think we should statically link for that use case. Just call qemu-user ldd or whatever
<shlevy>
Also, cross-compilation support is really nice these days ;)
<clever>
ldd is just a bash script that will set variables and execve() the target
<shlevy>
right
<shlevy>
So call the bash-script from within qemu-user is my point
<clever>
and when it execve()'s the target, binfmt-misc calls qemu-user
<shlevy>
Or with a riscv-shell
<sorear>
still won't work
<shlevy>
Oh I see the issue
<clever>
ldd itself would have to be patched to detect that its under a qemu-user, and know what the host os is
<shlevy>
So the specific issue of the initrd is fixed on staging
<shlevy>
Where we don't use ldd any more
<clever>
ah
<clever>
and i dont know of any other ldd's in nixpkgs
<shlevy>
And I bet the solution there could be extended to other use cases that need automated ldd
<Dezgeg>
maybe in pax-utils? it has lddtree at least (but in the worst case it's a shell script that parses ldd's output)
<clever>
i'm guessing it builds the initrd from x86, and cant run the remote ld.so to perform the normal ldd'ing
<shlevy>
clever: Exactly. So we have a script that uses patchelf to get the NEEDEDs and iterate through the RPATH recursively to satisfy everything
<Dezgeg>
actually, lddtree might work
<shlevy>
clever: Doesn't ld.so ignore the env vars when uid!=euid? Just make your binfmt wrapper setuid and drop privs immediately :P
<clever>
[clever@amd-nixos:~]$ LD_TRACE_LOADED_OBJECTS=1 ls
<clever>
this causes an ldd output and no ls output
<clever>
i suspect ld.so aborted when it detected an attack
<shlevy>
Probably the easiest path is qemu-user-ldd which sets the env vars inside the qemu-user call
<shlevy>
But qemu-user itself is inherently hackish, and ld.so being controlled through env vars like this is inherently hackish, and ldd is especially hackish, so I'm personally not especially inclined to try to get it perfect :D
<clever>
static seems simpler to me
<shlevy>
I mean, as long as you don't change things for mainline nixpkgs that works
<clever>
a staticly linked qemu-user just ignores those vars
<niksnut>
I think some of the macs may have an old nix version
<gchristensen>
quite likely
<gchristensen>
I don't think we have a good practice in place of deploying updates to it
* gchristensen
fires up the "nixops but for nix-darwin" signal
<shlevy>
Hmm.... copumpkin the newer macOSes come with .tbl or whatever files listing symbols of libraries like libsystem, right?
<shlevy>
So we can build a proper cross-compiler? :D
<gchristensen>
niksnut: copumpkin ping?
<Sonarpulse>
shlevy: hi
<gchristensen>
oops, just copumpkin / contrapumpkin:)
<Sonarpulse>
started working on cleaning up ios cross compilation yesterday
<Sonarpulse>
so good timing
<shlevy>
Sonarpulse: Feel free to rip out whatever you don't like of what I had there
<shlevy>
It was super hacky just to get something working
<thoughtpolice>
Just throwing it out there, but has anyone here looked at/known of any works on dynamic MOTD for pam? I had a case where it would have been kinda useful lately but was wondering if anyone had tried it/been opposed to it.
<Sonarpulse>
shlevy: well, I'm very open to suggestions on where to better get the tbd files
<Sonarpulse>
i have no idea
<thoughtpolice>
(I found out surprisingly that dynamic MOTD is actually a patch by the Debian/Ubuntu fellows and not a real PAM feature!)
<Sonarpulse>
there is that old bit-rottted xcode thing
<shlevy>
Sonarpulse: Honestly I'd just tar them up for each release
<shlevy>
And then slap the appropriate license on them
<Sonarpulse>
shlevy: yeah that would be nicer on our users
<Sonarpulse>
can make an impure built-on-darwin derivation to allow anyone to make them
<gchristensen>
niksnut: could you take mac1 out of hydra for a while? I'll start working on an updater
<Sonarpulse>
shlevy: similarly while the android sdk is free to download, hydra seems to choke on the size
<thoughtpolice>
shlevy: While you're around, did you get my earlier message about perlPackages.JSON by chance?
<shlevy>
thoughtpolice: No, what's up?
<thoughtpolice>
shlevy: perlPackages.JSON fails to build on latest HEAD for me somehow, due to e9ddfb789238c590e60cd76bbdabe046d58ee2d5 it appears
<shlevy>
Checking
<thoughtpolice>
shlevy: Just try 'nix-build -A perlPackages.JSON'
<thoughtpolice>
Should fail on latest HEAD due to not finding the 'new' method or something in the testsuite? The patch looks totally innocuous but I admit I'm clueless here
<thoughtpolice>
(I just backed it out locally to upgrade my machine)
<shlevy>
Ugh and backing it out fixes it?
<thoughtpolice>
Yeah, just a revert. Want me to push it for now?
<shlevy>
No, I'm digging it
<niksnut>
gchristensen: done
<gchristensen>
thanks niksnut! I'll start working on an update
<shlevy>
digging in*
<thoughtpolice>
This prevented a nixos-rebuild for me although admittedly I didn't track down *what* brought it into my derivation
<thoughtpolice>
shlevy: Thanks!
<shlevy>
I needed it for cross-NixOS originally
<thoughtpolice>
Yeah ISTR you doing this for RISC-V, right
<shlevy>
Yep
<shlevy>
I need to make some time to do real perl cross-compilation
jtojnar2 has joined #nixos-dev
<gchristensen>
ok, here we go /nix/store/isrf97h8jqyx9p51frjdmz2609a88252-nix-2.0/bin/nix-daemon
<gchristensen>
niksnut: ^ can you add mac1 back in, and take mac2 out?
JosW has quit [Quit: Konversation terminated!]
<niksnut>
done
<gchristensen>
great
<gchristensen>
for the second one I think I'll try switching it to nix-darwin, since lnl has given me a nice script to help with it, then we can put the deploy script / configs in to the nixos-org-configurations repo
pie_ has quit [Ping timeout: 246 seconds]
<shlevy>
Hmm xz on glibc-2.27 is failing in sandbox on hydra but not outside of sandbox locally... Wonder if it's a /bin/sh thing
<shlevy>
Do all of the hydra builders have a "good" /bin/sh now?
<gchristensen>
are you able to identify a machine not operating properly?