<adisbladis>
cole-h: There is an upcoming blog post on this on the Tweag blog ;)
* cole-h
really hates the "..." shorthand in Go
<cole-h>
I always think it's a typo
* adisbladis
checks notes
<adisbladis>
Yeah, that was the major showstopper
<adisbladis>
Everything else we could possibly work with
<adisbladis>
And find decent workarounds
<cole-h>
ugh
<cole-h>
:(
orivej has quit [Ping timeout: 260 seconds]
kalbasit_ has quit [Ping timeout: 256 seconds]
kalbasit_ has joined #nixos-dev
kalbasit_ has quit [Ping timeout: 256 seconds]
Jackneill has joined #nixos-dev
saschagrunert has joined #nixos-dev
{^_^} has quit [Remote host closed the connection]
jonringer has quit [Ping timeout: 264 seconds]
orivej has joined #nixos-dev
__monty__ has joined #nixos-dev
<das_j>
Just noticed: Can someone with the appropriate permissions move the 21.05 milestone? I highly doubt it will be reached in march ;)
<qyliss>
das_j: looks like it's been done already
<das_j>
Huh, it was March just a couple of minutes ago
<das_j>
Well thank you, $ANONYMOUS
<das_j>
Oh no it was me. It says May but I read "March". Time to fetch some coffee…
JJJollyjim has joined #nixos-dev
<JJJollyjim>
those two months are the same imo
<das_j>
Is anyone (probably the release manager but I don't know who they are) opposed to adding
<das_j>
#76184 to the milestone?
<das_j>
It'd be nice to finally have this cleaned up
ky0ko1 has quit [Quit: killed]
jdnixx-M1 has quit [Quit: killed]
jdnixx-M has joined #nixos-dev
ky0ko1 has joined #nixos-dev
Jackneill has quit [Ping timeout: 246 seconds]
Jackneill has joined #nixos-dev
saschagrunert has quit [Remote host closed the connection]
saschagrunert has joined #nixos-dev
Jackneill has quit [Ping timeout: 264 seconds]
Jackneill has joined #nixos-dev
pmy_ has quit [*.net *.split]
edwtjo has quit [*.net *.split]
julm has quit [*.net *.split]
cransom has quit [*.net *.split]
adisbladis has quit [*.net *.split]
euank has quit [*.net *.split]
edwtjo has joined #nixos-dev
edwtjo has joined #nixos-dev
edwtjo has quit [Changing host]
cransom_ has joined #nixos-dev
pmy has joined #nixos-dev
julm has joined #nixos-dev
euank has joined #nixos-dev
Jackneill has quit [Ping timeout: 256 seconds]
adisbladis has joined #nixos-dev
cole-h has quit [Ping timeout: 240 seconds]
teto has joined #nixos-dev
Jackneill has joined #nixos-dev
WilliButz has quit [Ping timeout: 240 seconds]
WilliButz has joined #nixos-dev
<siraben>
Hm, we relied on ofborg to merge the stdenv.lib → lib PR
<siraben>
but ofborg doesn't eval everything
<supersandro2000>
siraben: automatically
<supersandro2000>
going to fix the issue with tangranm asap
<siraben>
supersandro2000: automatically?
<supersandro2000>
how ryan said
<gchristensen>
what do you mean ofborg doesn't eval everything, siraben?
<siraben>
#109679 and #109954 came up recently
<gchristensen>
are beets plugins evaluated in hydra?
<gchristensen>
they should be
<gchristensen>
no
<gchristensen>
it seems weird to have beets plugins be a passthru of beets itself, I think we need a pkgs.beetsPlugins = lib.recurseIntoAttrs pkgs.beets.plugins ;
{^_^} has joined #nixos-dev
justanotheruser has quit [Ping timeout: 260 seconds]
<supersandro2000>
please add something like that
<supersandro2000>
kaoune had a similar problem
<supersandro2000>
no one can know this when doing treewide PRs
<gchristensen>
hopefully someone involved with the stdenv.lib refactor or the beets packaging does it :)
<gchristensen>
ofborg's evaluation is to avoid hydra failing to evaluate, and then best-effort for everything else -- it isn't a guarantee that all things can be evaluated in every way
<supersandro2000>
siraben: ^
<siraben>
Ok, I can try to look into this
<siraben>
Maybe with the Emacs packaging which was affected
<supersandro2000>
please everyone keep calm and do not jump out of the window
<supersandro2000>
please keep calm!!!
<edef>
can you like, ever so briefly, act like a constructive adult
<edef>
i don't want to be an asshole about this but this stuff really gets on my nerves, and i doubt i'm the only one
<edef>
(i have now been informed i am in fact not the only one)
<V>
wasn't that the case as of several days ago
<edef>
yes, but i have the memory of a goldfish
<V>
true
<supersandro2000>
I just did a joke.
<edef>
right, it's more about the pattern
evanjs- has quit [Read error: Connection reset by peer]
<edef>
i could have a biased sample set here, but my model of you mostly consists of having very strong opinions on things ranging from the inconsequential and mundane to the fairly complex, that don't appear very thought through, and in general making noise disproportionate to how much useful content that noise conveys
<edef>
my best guess is that you're 12 years old and have the particular kind of uninhibited enthusiasm the years rid one of, which is fine, but feels like something for a different venue perhaps
evanjs has joined #nixos-dev
<gchristensen>
edef: I'm not sure this is a very appropriate way to deliver the message you have to deliver
<edef>
relatedly, i struggle to believe that the rate at which you merge PRs involves rigorous testing or review, and involves such a diverse range of software that i would be very impressed if you have the domain knowledge of each to verify that they are functioning correctly
<edef>
gchristensen: i genuinely mean no harm here. i'm not sure how else to deliver it. would you prefer we keep gossipping about our degree of annoyance in back channels?
<gchristensen>
edef: the part I mean to specifically refer to is bringing age in to it
<edef>
i think that reveals your ageism more than anything else
<gchristensen>
okay
<edef>
i too was once 12 years old and on the internet, contributing to F/OSS
<gchristensen>
I don't agree, though
<edef>
if you think that was bad of me, then you are welcome to air that view
<gchristensen>
would it have felt good and constructive to have been 12 years old and have someone say you're not doing good work because you're 12?
<V>
I assumed that everyone who put a year in their username was born then
<supersandro2000>
I can assure you that I am not 12 and more like 20
<edef>
gchristensen: no, i think that people whose workspace i was wading into were reasonable to ask me to behave with a certain degree of maturity perhaps not usually becoming of people my age
<gchristensen>
right, and I think you can ask for that without making any statements of age
<supersandro2000>
you could have said something along the lines to keep the jokes in #nixos-chat
<V>
this isn't about jokes, though
<V>
just yesterday people were complaining about how you go about reviewing PRs, etc
<edef>
the intended meaning of the phrasing was "the most charitable interpretation i have is that this is simply youthful enthusiasm"
<edef>
the joke was just the straw that broke the camel's back, the general signal-to-noise ratio is my actual issue here
<gchristensen>
edef: I totally understand where you're coming from, and also remember being 12 and being involved in foss communities, and I understand your intended meaning there. however, the reason I am pointing this out is that being of a certain age is literally unfixable by community members except by waiting, whereas behavior and involvement is fixable now... and being young, doing something exciting
<gchristensen>
like being part of a project, and then seeing someone make a comment of age ... I remember it being all about my age and being utterly discouraged thinking I had to just wait until I wasn't too young anymore. that is why I say that
<edef>
i only caught up from being the youngest contributor to most projects i worked on less than, say, 8y ago, i really don't think we're about to resolve the semantic argument you're making here
<gchristensen>
okay
<edef>
i'm fairly happy to believe your interpretation of my sentiment is the One True Parse, and i am simply wrong about my own experiences and what i believe
<gchristensen>
please keep the unrelated topic of age out of it, when other factors are the actual issue
<adisbladis>
siraben: #109954 merged
<supersandro2000>
edef: anything you want to tell me how to improve the situation for you?
<V>
supersandro2000: I'm ostensibly not edef, but could you try being less strongly/harshly opinionated? I feel that would help generally
<edef>
i try to stick to "strong opinions held weakly" where i find myself having strong opinions, and try considering when input will have useful effect
<edef>
which is not really a magical rule, but it's the closest to a reusable guideline i have
<supersandro2000>
I can try that but I can't promise anything
<edef>
i really am enthusiastic about nixpkgs gaining new, active contributors with genuine enthusiasm, and i hope you do stick around <3
<edef>
not sure that was clear and i definitely failed to state it as obviously as possible
<supersandro2000>
if it will continue like the last days I think it will be a hard time
<edef>
currently my biggest actual fear is that i find some commit with underconsidered blast radius that got merged among hundreds of other a day
<edef>
*others
<edef>
while figuring out why something in a complex deployment fails due to some basic aspect of its distant dependencies being broken
<supersandro2000>
you mean like stdenv.lib to lib which broke many plugins in packages which are not evaluated by ofborg?
<adisbladis>
"Slow is smooth and smooth is fast"
<edef>
that one i'm less concerned by, since a straight up build break is the least concerning failure mode
<edef>
the scary failure cases for nixpkgs produce binaries, the ones that just produce build/eval errors never make it to production
jonringer has joined #nixos-dev
<edef>
hence "if it builds, merge it" almost gives me the shivers more than "roll dice, merge"
<edef>
and it's not super clear what methodology you're using to merge things
<supersandro2000>
understiid
<supersandro2000>
understood
<edef>
at some point mass-merging stuff on a continuous basis looks more like a nixpkgs policy change than an individual maintainer choice in effect
<ekleog>
my personal philosophy is “if it shouldn't be merged when it builds, then the package was missing some passthru.tests in the first place” — though that doesn't prevent actually doing some tests
<edef>
i agree with that from an ideological, utopian standpoint
<edef>
but practically, few packages have tests for their entire dependency closure
<edef>
if we were at google and everything was obsessively tested because "if you liked it, you should have put a CI test on it" is agreed-upon policy, then i would wholeheartedly encourage merge-if-the-tests-pass
<edef>
and to some degree, i would consider a policy of purposeful chaos acceptable, if we are actively tracking and analysing what kinds of failures are most common
<edef>
but we're in the weird spot where tests aren't even necessarily *run* consistently, because we lack a global, common CI system with sufficient build and storage capacity that is run on every commit
<edef>
we're using Nix, "builds on my machine" is less of a vacuous statement than it is for most F/OSS for sure
<siraben>
adisbladis: yay
<edef>
but … basically, my criticism of mass-merging is that it's a policy change with significant blast radius, implemented on individual initiative, but which requires collective effort to make safe as a strategy and policy choice
<edef>
i really don't want to rag on the obvious degree of enthusiasm and energy in play
<edef>
legit kudos for showing up to a project that in many ways is tiny and in many ways constantly ossifying due to contributor turnover
<edef>
egh
<edef>
…eh, nevermind rewording that. the sentiment is in there
<edef>
love y'all, sorry for the wall of text <3
<adisbladis>
<3 edef
<{^_^}>
edef's karma got increased to 37
<gchristensen>
I agree with you, edef
<ekleog>
edef++
<{^_^}>
edef's karma got increased to 38
<lukegb>
I wish package authors were better at writing tests so, y'know, doCheck=true; was a sensible default 😢 (don't get me started on "the release tarball doesn't ship with the tests so if you want the tests you have to get them from github but then you also have to make sure you get the right commit because they don't tag them")
<ekleog>
(There's just the part about tests not being run consistently that I'm not sure about, IMO we don't need to build the full reverse-dep closure to ensure things work, something like “just” adding some representative sample of the reverse-dep closure to passthru.tests would make ofborg build them (hopefully it wouldn't be too much for it), and ofborg in many ways is our global, common CI —
<ekleog>
so in a way I feel like rather than tempering the enthusiasm merging PRs, changing our policy to make merging PRs easier might be the best way forward :))
<edef>
i think if we elevate maintainership to involve clearer responsibilities with more of an SLO / responsiveness objective of sorts, we could probably make "if the tests pass, breakage is on the maintainer" work
<edef>
and add significantly more CI capacity
<lukegb>
I mean, a FreeBSD-style maintainer timeout would be interesting at least
<lukegb>
(in general I'm in favour of giving maintainers more power at least over their own packages... but that's really a separate discussion)
<ekleog>
well, if we start going down the “breakage is on the maintainer” road, we're going to come back to the problem of breakage following changes to eg. stdenv :/
<edef>
mmh, this circles back to not having very much well-distributed, shared meta-knowledge about what creates breakage
<adisbladis>
ekleog: A transitive dependency breaking doesn't mean it's on the maintainer of the leaf package
<adisbladis>
But indeed, it is a problem
<ekleog>
well, my recollection of last we discussed this was something like “we don't want stdenv changes to land breaking 80% of reverse deps, but we don't want stdenv changes to have to fix the last percentage of reverse deps either”
<edef>
mm
<lukegb>
This problem is only going to get worse if nixpkgs is manyrepoed, fwiw :p
<edef>
oh, for sure. i *think* i have made my opinion on that abundantly clear already
<edef>
ekleog: mm
<edef>
ekleog: do you have a particular set of changes in mind?
<ekleog>
edef: I don't, sorry, it's not a domain I personally touch that much :/ it's just my recollection of the arguments against an all-grean hydra merge-to-master policy based on ofborg or similar (which were then answered by “so we would need to define some set of packages that should be merge-blocking” and then I think the discussion more or less stopped there)
<ekleog>
uhh s/ofborg/bors/
<edef>
currently having any package be merge-blocking is infeasible, unless you are a maintainer for that package
<edef>
because a package maintainer could in principle take an unbounded amount of time to respond
<edef>
and we don't have a notion of "core" packages vs "contrib", like some distro/package communities have
<ekleog>
Well, in a way saying that eg. gcc should be merge-blocking would IMO make sense, because even if the package maintainer never responded, someone else should just do it or hydra will never advance again anyway — so the in-parenthesis argument was actually arguing for creating the core vs. contrib split :) (that said I was saying this just to expose where I took the argument about stdenv
<ekleog>
breakage from, not really to litigate this other point ^^)
<puck>
siraben: yeah, because the stack overflows,
<edef>
the stack overflow error message is just a SIGSEGV handler
<puck>
which is why it can't print a stack trace :p
<siraben>
yeah, hm, isn't β-reduction of that expression supposed to not lead to it exploding?
<siraben>
omega → omega → omega ...
<V>
as we all know, using the C runtime's stack for anything at all is definitely sane and robust software engineering
<puck>
V: C++, tyvm :p
<siraben>
does Nix use a graph-reduction model underneath?
<V>
same thing IMO
<siraben>
à la lazy functional languages
<edef>
siraben: no, it is a tree-walking interpreter
<siraben>
oh
<edef>
siraben: it is implemented more like a scheme interpreter without tail recursion
<edef>
siraben: it has Boehm GC tacked on in an attempt to not use unbounded memory
<siraben>
oh I see, well
<siraben>
hmm, tail recursion would improve a lot of things
<puck>
technically the interpreter tries to do tail recursion, but it is at the mercy of the compiler, and C++ destructors don't work well with that
<edef>
siraben: it's at an interesting local equilibrium of undefined behaviour, where if you perturb the codebase in seemingly fine ways by refactoring, it will instantly spring ten new memory safety bugs
<siraben>
hmm
<edef>
siraben: the GC really pushes up the amount of entropy injected into that, because any failure to include a pointer in the GC trace instantly creates a use-after-free
<edef>
siraben: it's completely impossible to use ASan on without ripping the GC out
<siraben>
> lib.fix (f: x: if x == 0 then 1 else f (x - 1)) 10000
<{^_^}>
1
<siraben>
> lib.fix (f: x: if x == 0 then 1 else f (x - 1)) 100000
<edef>
siraben: address sanitiser, a clang/LLVM tool that … tracks pointer validity, kinda
<siraben>
so many partial guarantees
<edef>
for sure, yeah
<edef>
i've considered doing a fairly literal port of the existing code to a memory-safe language
<siraben>
any drop-in replacement evaluator for Nix in Rust/Go/C etc.?
<siraben>
Haskell even
<puck>
there's a few (partial) implementations (hnix, rnix), but neither are drop-in
<siraben>
I see
mkaito has joined #nixos-dev
mkaito has joined #nixos-dev
mkaito has quit [Changing host]
teto has quit [Ping timeout: 244 seconds]
<edef>
there are grand, ambitious rewrites amounting to little, and there's a(n) (somewhat abortive?) attempt to aggressively refactor the existing codebase at https://code.tvl.fyi/about/third_party/nix/README.md
<siraben>
I haven't read the semantics of the language, but it would be different from implementing yet another lazy purely functional language right?
<siraben>
or is these some subtleties in the evaluation model
<siraben>
s/right//
<edef>
there are some subleties and bugs that nixpkgs relies on in practice, puck can describe in more detail
<siraben>
something like lists being strict threw me off when i was experimenting with lazy streams
<puck>
yeah
<puck>
my favorite bug is
<edef>
but overall, i think anyone willing to dive into the codebase, and playing around in a REPL to exercise some of the more surprising/subtle behaviours, can build up a solid understanding
<puck>
> let a = x: x; b = { a = a; }; in [ (a == a) (b == b) ]
<{^_^}>
[ <CODE> <CODE> ]
<puck>
err
<puck>
> :p let a = x: x; b = { a = a; }; in [ (a == a) (b == b) ]
<{^_^}>
[ false true ]
<edef>
personally i found it a bit of a struggle to maintain enthusiasm, and don't really have all the domain knowledge for building a more well-considered interpreter design from scratch
<tazjin>
once we get a second implementation off the ground, my view is that the semantics should be semi-formalised and anything in nixpkgs that depends on something crazy should be fixed
<edef>
so i mostly tinker with parallel implementations of the protocols and file formats that surround the evaluator
<tazjin>
and then we get rid of those
<tazjin>
as well as all the crazy stuff that's just an implementation artefact
<gchristensen>
and add the second implementation to ofborg
<siraben>
puck: wait what
<siraben>
dang
<siraben>
you can compare functions for equality? kind of like eq? in Scheme I guess
<puck>
siraben: only when they're in an attrset!
<siraben>
¿¿¿??
<siraben>
wow
<edef>
nixpkgs relies on this particular one.
<puck>
siraben: functions cannot be compared, officially. but there's a legacy bug behavior that attrsets compare the contents by pointer value first
<siraben>
wait, how?
<sorear>
[what causes the memory safety issues? is nix doing known-boehm-problematic things (xoring pointers, etc), does modern gcc break boehm, has boehm just never worked?]
<siraben>
edef: ^
<tazjin>
my favourite is <..>
<adisbladis>
I seem to recall some lib.unique call where this was important
<tazjin>
> let __findFile = _: builtins.getAttr; in <a> { a = "hello"; }
<{^_^}>
"hello"
<siraben>
Nix lang is even more cursed than I thought
<puck>
all enums depend on this
<siraben>
what kind of enums?
<adisbladis>
> lib.types.enum
<{^_^}>
<LAMBDA>
<puck>
siraben: it's a nixpkgs library feature
<adisbladis>
Presumably
<siraben>
Oh I'm not familiar with lib.types.enum, interesting
<edef>
sorear: i believe we're setting Boehm to a mode where it does not take over the system allocator
<siraben>
No wonder Guile makes a lot of sense, heh
<edef>
sorear: and you have to explicitly help it trace pointers
<tazjin>
in theory we could have actual function equality :p :p
<edef>
sorear: some pointers are not specified
<edef>
tazjin: that is not decidable :)
<sorear>
i see, ew
<tazjin>
edef: it is if you constrain what "equal" means
<edef>
now you have PartialEq, not Eq :p
<tazjin>
sure, but that's fine
<Taneb>
There's no rule that not equal means distinguishable
<tazjin>
that's what people want in practice, usually (see e.g.: Rust)
<sorear>
above we have a failure of reflexivity, not just extensionality
<edef>
i think opportunistic function equality is a hazard that'll produce all kinds of unintended coupling
<tazjin>
well it's either that or fixing nixpkgs
<tazjin>
the latter is probably a better approach
<adisbladis>
This ^
<tazjin>
as long as we still have a monorepo
sterni has joined #nixos-dev
teto has joined #nixos-dev
<gchristensen>
I don't want to derail the conversation, it looks like it has somewhat ended. Is it okay if I change the topic?
<lukegb>
SGTM
<edef>
seems fine by me — to some degree the nix evaluator convo already derailed the convo i was most interested in (merge policy)
<gchristensen>
should it be possible for a company to use Nix to build their proprietary tool with Nix, then distribute their pre-built, proprietary build closure to users with Nix via flakes?
<adisbladis>
gchristensen: Why not?
<adisbladis>
Or, hm. I see what you mean now.
<gchristensen>
well, because right now you can't :)
mkaito has quit [Quit: WeeChat 3.0]
<puck>
oh huh, builtins.storePath doesn't work in pure mode, i guess
<gchristensen>
correct
<adisbladis>
gchristensen: Hmmmm... I think you kind of could with CAS.
<gchristensen>
for non-flake use cases, "Fake derivations" work fine -- these use builtins.storePath
<lukegb>
Is a Nix flake a good interchange format for proprietary software? Should it be?
mkaito has joined #nixos-dev
<puck>
gchristensen: hrm, recursive nix could reasonably do this
<adisbladis>
I'd probably end up with some codegen machinery if I were to solve this
<gchristensen>
well flakes specifically no, but people who manage their system's software with flakes, and also want to include proprietary software in their system configuration, have no choice but to also the fetch proprietary software with flakes
zarel has joined #nixos-dev
<puck>
gchristensen: hrm, does builtins.toPath do the right thing?
<gchristensen>
pretty sure it is prohibited also
<puck>
it is .. not, in my checkout at least
* gchristensen
'll have to look up exactly what toPath does
<puck>
it just runs coerceToPath and canonPath, without the checkSourcePath, and without trying to ensurePath, but it might not set the context correctly
<gchristensen>
I imagine that'd be a killer
<gchristensen>
yeah it appears to skip context, and I think the context is critical to force substitution
<siraben>
(forall x, f x = g x) → f = g, IMO
<gchristensen>
that is true but x is really big
<siraben>
but this is undecidable :(
<Taneb>
Ah, functional existentionality
<siraben>
it's strange now that Nixpkgs depends on weird bugs in Nix, hmm
<puck>
gchristensen: hrm, builtins.appendContext exists, but context keys have to exist in the store
<gchristensen>
siraben: it is not *so* strange that a ~20 year old project depends on the exact behavior of the only tool capable of interpreting it
<puck>
siraben: there's no real spec, so it could be considered a non-bug
<gchristensen>
okay but so, just to circle back ...
<siraben>
Oh right, 20 years is a long time
<gchristensen>
(oh god, circle back ..)
<gchristensen>
do we agree it should be possible? :P
<Taneb>
The proprietary flakes thing?
<siraben>
gchristensen: what should be possible?
<puck>
gchristensen: good question; i'm not entirely sure, tbh
<gchristensen>
that a company should be able to use Nix to build proprietary software, and distribute it to its users via Nix, in flakes
<adisbladis>
gchristensen: It also depends on how much information leakage you're OK with
<adisbladis>
You could distribute the plain expression with a hashed source
<adisbladis>
But nowhere to actually download the source from
<adisbladis>
Trying to obfuscate the expression itself I don't think is a good idea
<gchristensen>
why is that?
<adisbladis>
At least to me it goes against the entire spirit of Nix
<adisbladis>
Nix is radically transparent
<gchristensen>
yeah, that is a big part of what I like about Nix
justanotheruser has joined #nixos-dev
<Taneb>
I guess we can already do the equivalent with not-flakes, I can supply a nix file which describes a derivation which just fetches a tarball and sticks it in $out
<sphalerite>
but that doesn't account for dependencies.
<Taneb>
True
<gchristensen>
and with not-flakes you can make fake derivations which fetch closures with just
<gchristensen>
builtins.storePath
<gchristensen>
here is how Hydra produces a channel of "fake derivations" using builtins.storepath
<gchristensen>
:GUje
<gchristensen>
SIGH. this gnome-terminal bug is killing me.
<gchristensen>
well I can't paste, so src/lib/Hydra/Views/NixExprs.pm#L30
<ekleog>
That may sound dumb, but for proprietary software, what about having a bunch of FOD and then layer on top of it a derivation that adds the other required derivations in nix-support?
<ekleog>
that makes two public derivations for each actually wanted derivation, one which just has the derivation contents and one which lists dependencies
<gchristensen>
maybe, yeah, but you'd have to do binary-patching to fixup all the references
<ekleog>
oh right that doesn't work because FODs have a different hash scheme than normal drvs so it's not possible to make them collapse
<sphalerite>
As for the political side of the question: IMHO it's ok to allow it, but I wouldn't want nix to bend over backwards on a technical level to enable this.
<gchristensen>
I'd agree with that
<edef>
i can see an argument for mechanisms that outsource evaluation wanting to rely on the same
<gchristensen>
adisbladis: it strikes me that without obscuring the expressions, the distribution would be all of nixpkgs + their expression on top, shipped in a tarball, making it a pretty heavy way to go
<edef>
eh, you could cull your nixpkgs by eval-tracing
<gchristensen>
"could" yeah but can you actually? lol
<gchristensen>
like, that isn't a thing I'm going to do any time soon
<edef>
i have a reference implementation as a perl script, using a similar approach to lorri
<gchristensen>
whoa fancy
<edef>
tvix also has more explicit support for tracing what files it reads, contributed with the aid of yours truly
<gchristensen>
nice!
<gchristensen>
edef: were y'all looking at using grpc for the socket?
<edef>
yes
<gchristensen>
did you do that?
<edef>
i am not presently particularly involved, but i suspect grfn can tell you more about the current state of that effort
<edef>
on the fake drv side, i can see a case for eg shipping attrset → store path mappings from a shared company-wide CI without needing to do full evaluations constantly
<gchristensen>
that would be nice
<edef>
i have intent to implement this internally at Mutable when time allows, mostly so i can leave the evaluator out of most tooling
<gchristensen>
yeah
<edef>
since memory-safety issues make it a fairly hazardous thing to embed
<edef>
that said, i believe i'm in less of a position of secrecy-towards-users than your project is
<edef>
so it's quite plausible that our actual requirements here diverge
<gchristensen>
I actually don't think they do
<gchristensen>
but this isn't necessarily about any one project, so maybe the total set of requirements do diverge
<puck>
i feel like a hydra-CI flake type could reasonably exist, especially as there's already an evaluation cache; worst case, you literally ship the evaluation cache :p
<gchristensen>
hehe
kalbasit_ has joined #nixos-dev
<grfn>
gchristensen: the grpc stuff is done and working
<edef>
mfw i spent time at work today adding more defensive programming to my `nix-store --serve` protocol client
<gchristensen>
heyyy that was my yesterday
<infinisil>
"cached failure of attribute 'devShell.x86_64-linux'"
<infinisil>
But it doesn't tell you the actual failure :/
<catern>
is there any deduplication on the cache.nixos.org artifacts stored in S3? we're just storing the raw NARs in S3, right?
<gchristensen>
the NARs are content addressed like any other cache, but yea
kalbasit has quit [Ping timeout: 246 seconds]
kalbasit has joined #nixos-dev
<catern>
are there any stats about the cache.nixos.org store size? or rate of change?
<gchristensen>
I think it is 212T
<gchristensen>
I forgot its growth rate
<catern>
interesting, thanks for the statistic
<catern>
($DAYJOB is afraid that building everything in Nix and keeping the artifacts forever will create huge storage burdens)
<gchristensen>
a reasonable concern
<gchristensen>
probably want to keep some amount of history and gc the rest
<catern>
212TB is decidedly non-huge though, easily manageable
<gchristensen>
it is also a significant amount of history
<gchristensen>
and probably way more than you'd actually be building
<gchristensen>
(cross, aarch64, macos, x86, ...)
<puck>
at work iirc we had the nix cache GC'd after 30 days, which is a bit of trouble, as removing artifacts from the cache in that way isn't quite supported
<catern>
yeah, I'm regarding 212TB as an upper bound (at least upper bound when doing things right)
<domenkozar[m]>
puck exactly why cachix exists :)
<puck>
eh, we just dropped the rule, it's not like it'll use that much space
<catern>
does cache.nixos include everything built by Hydra, or just things merged to master of nixpkgs?
<catern>
(at $DAYJOB our all-artifacts-for-all-time storage is at 774TB, growing at 1TB/week, so 212TB for Nixpkgs is nothing in comparison)
<gchristensen>
everything
<catern>
which includes like, PRs built by gchristenbot?
<gchristensen>
no
<gchristensen>
those are totally separate for securixty reasons
<ajs124>
seems like #96536 broke the tarball job on unstable-small
<ajs124>
not sure how/why, but that's what git bisect tells me
saschagrunert has quit [Remote host closed the connection]
rajivr has quit [Quit: Connection closed for inactivity]
<andi->
that commit..
<andi->
doesn't even conform to contribution.md
<andi->
ugh, GH Actions logs time out after <1 day?!
<andi->
ajs124: have you tried reverting it?
<ajs124>
andi-: just now, yes. it's still building though. maybe I should find a less busy machine…
<ajs124>
last time I tried it on my desktop, it was killed by the oom killer, though
<supersandro2000>
it is pretty close to contributing in my defense
<puck>
ajs124: indeed it is caused by haste-client
<puck>
at least, i ran a custom evaluation of the find-tarballs script, which logged the derivation missing drvPath
<ajs124>
how can a derivation now have a drvPath?
<puck>
or, wait a moment, that isn't a derivation
<ajs124>
lolwhat
<puck>
wait, no, it has to, otherwise isDerivation wouldn't work?
<supersandro2000>
Can someone please tag me when you did find the solution? I would like to know what caused it to not merge such PRs in the future.
<ajs124>
revert also seems to fail
<puck>
oh, interesting
<puck>
ajs124: wvait, the outPath is ... in my nixpkgs checkout?
<puck>
oh, i see
<ajs124>
ah, oh no. my checkout is broken somehow…
<puck>
it has bundledByPath = true; set
<puck>
ajs124: bingo
<puck>
ajs124: it recursed into the propagatedBuildInputs, which is the bundlerEnv in the top of the file; one of those is the processed gemset.nix's "haste" attr, which, because it uses a path-type source, generates a fake invalid derivation
<puck>
(pathDerivation in development/ruby-modules/bundled-common)
<puck>
i'm not well-versed into ruby packaging infra in nixpkgs, but i suspect that drvPath usage in derivationsIn{,'} in find-tarballs.nix should be error-checked, and probably the derivation be rewritten to .. not depend on a bundler env?
<ajs124>
I don't know much about the ruby stuff either, so idk what should be done about it.
<puck>
supersandro2000: the package seems to be written in a sorta non-canonical way for bundler based gem packages, which someone knowledgable on the ruby infrastructure probably should've been able to catch
<sphalerite>
just revert it? It's not like we don't have any other pastebin clients in nixpkgs…
<sphalerite>
And it can be added back sanely
<puck>
sphalerite: i'm going to put in a PR right now to fix it
<puck>
oh, nvm, the author seems to be on it
kalbasit_ has quit [Ping timeout: 264 seconds]
teto has quit [Ping timeout: 260 seconds]
tdeo has joined #nixos-dev
tdeo has quit [Remote host closed the connection]
<V>
I've tried doing stuff with the ruby infra before and it's horribly confusing
<V>
my best experience was with .withPackage, which I try to use if I can
<V>
but I often couldn't get the bundler stuff to work. especially if dependencies had native extensions
<supersandro2000>
gchristensen: I merged a pr earlier today which still had pkgconfig in it
<gchristensen>
jtojnar: and swapping the position is not annoying of a workaround with `dconf write`! thank you
<gchristensen>
jtojnar: do you know if I can change the current keyboard layout with dconf, or another cli tool? that would make the transition very simple
<cole-h>
supersandro2000: (going through the backlog of nixos-dev) For future reference: please don't remove the ofborg-internal-error labels unless one of us (ofborg team members) tells you to. Usually, it's fine, but it makes it difficult to determine if it was GitHub serving a broken API for the Nth time or something on our end that needs to be fixed.
<cole-h>
(I haven't read the entire log yet, so you might have gotten the go-ahead from someone, but just in case)
evils_ has joined #nixos-dev
* cole-h
also needs to set up notifications for that label again...
evils_ has quit [Client Quit]
<supersandro2000>
I just removed it from PRs that did a successful eval
<supersandro2000>
but noted for the next time
<cole-h>
(This is mostly because I have a tab open with all internal-error labeled PRs that I check every so often, and removing the label, well... removes them from that query)
<cole-h>
Usually triggering a new ofborg evaluation fixes it, but it's impossible to know if the underlying issue is something more sinister without access to logs