<adisbladis>
gchristensen: That test suite is so sexy
<adisbladis>
<3
<manveru>
i'd like to cover nixos and darwin only for now
<adisbladis>
Testing stuff for darwin is always so annoying
<gchristensen>
yeah it is :(
<gchristensen>
I wish we could have a community mac like we have a community aarch64
<adisbladis>
That would be <3
<gchristensen>
I don't think we have a mac which we can allocate like that
<adisbladis>
I wonder if we should set up patreon/opencollective for nix community resources
<__monty__>
I know the GHC team runs one or multiple. They could probably lend some expertise if not resources.
<gchristensen>
we have 11 macs already :P
<manveru>
is the nixos opencollective not good for that?
<gchristensen>
yeah, the nixos opencollective is for exactly that, adisbladis
<adisbladis>
gchristensen: I was always under the impression that the opencollective was for build machines, cache & the like
<manveru>
with the money we have right now we can buy... maybe one notebook :P
<adisbladis>
And not for community resources
<adisbladis>
manveru: Well, there is macincloud and similar vendors
<adisbladis>
We don't need to buy the hardware outright
<gchristensen>
adisbladis: it should definitely support the community, too
<manveru>
nixos is the community for me...
<manveru>
but i don't know about the inner workings :)
<adisbladis>
gchristensen: Btw, I think we should remove `The rest of the funds go to organizing community events (Nix conference, sprints, etc).` from the opencollective page
<adisbladis>
It's not really true :)
<__monty__>
gchristensen: All those macs are busy running hydra or something?
<adisbladis>
__monty__: Yeah, and there is good reason to not let a large set of people have access to those
<gchristensen>
adisbladis: they have in the past, and still could
<gchristensen>
adisbladis: but maybe we should take that chat to private?
<__monty__>
Do the linux caches require 10+ build machines each too?
<adisbladis>
gchristensen: Sure
<gchristensen>
__monty__: we have 800-900 core of aarch64, and almost 1,000 cores of x86_64.
<__monty__>
o.O
<manveru>
heh
<__monty__>
arewebitcoinscaleyet.com
<gchristensen>
no, our build cores are useful
<joepie91>
is that just a static 'yes'
<joepie91>
:P
<__monty__>
What's the community build machine for btw?
<gchristensen>
building and testing things
<manveru>
any info about that?
<__monty__>
"Anything goes," or "only things which directly benefit the nixos community *may* be allowed?"
<gchristensen>
well firsty ou have to agree to the very unpleasant safety warning
<manveru>
thx
<adisbladis>
New cryptocurrency idea: The proof of work are nix builds
<gchristensen>
hehehe
<adisbladis>
Who wants to pre-buy nixcoin? :)
<adisbladis>
I'm selling
<__monty__>
One obvious issue I can see is that veriying the proof isn't any easier than generating it.
* adisbladis
has done ICOs beofore, this is how it works
<etu>
adisbladis: So if at least 10 users take the same build inputs and produce identical outputs they get rewarded a fraction of nixcoin in exchange for the output?
<etu>
adisbladis: And then 100% of nix is reproducible :p
<etu>
(at least the cache)
<adisbladis>
etu: That sort of thing could work\
<adisbladis>
For reproducible builds
<__monty__>
How'd you verify the proof?
<adisbladis>
etu: But now we're talking blockchain things. So the product doesn't have to make sense
<etu>
__monty__: If enough people produce the same outputs...
<etu>
or something
<__monty__>
etu: Then the network would be vulnerable to an attack by "enough" people.
<etu>
__monty__: Yes?
<__monty__>
You can hardly require over half the network to produce identical builds.
<adisbladis>
It's a classical 51% attack
<adisbladis>
Tbh PoS would make more sense
<adisbladis>
For this application
<__monty__>
adisbladis: Only if you require half the network to produce the same output.
<__monty__>
If you say 10 reproductions is enough then you're vulnerable to a 10-people-equivalent-build-power attack.
<__monty__>
Even if you require over half the network to reproduce you still have a weaker model than bitcoin. Because you have to rely on the network for verification.
<gchristensen>
(actual numbers on Packet.com are we have 888 (how lucky) x86 cores and 592 arm cores)
<__monty__>
I'm not sure why I thought it'd be like two or three 64 core machines.
<__monty__>
I guess I have no sense of scale.
<gchristensen>
I don't know why either :P Hydra builds like 1,250,000+ derivations a month
<__monty__>
Are these all to populate cache.nixos.org?
<gchristensen>
yes
<averell>
does that stuff expire?
<__monty__>
So this doesn't even include CI for nix development?
<adisbladis>
averell: Nope
<adisbladis>
__monty__: It does not
<averell>
crazy
<gchristensen>
this confusion about the size and scale of hydra might help explain why sometimes channels get "stuck"
<__monty__>
Then there's serving the site. Maybe a repository server? Do those machines actually serve the cache or only populate it?
<gchristensen>
the cache is stored on AWS S3, and the cache is served by Fastly
<gchristensen>
at last check a couple months ago, the cache was 180T
<__monty__>
gchristensen: No, the reverse. Three buildmachine -> yeah of course channels can get stuck. Over a 1000 cores -> how can these channels even get stuck?
<averell>
probably 700 of those only build chrome all day, like me :)
<__monty__>
Probably still ties in to not understanding the scale of things though.
<gchristensen>
if your workload can be handled by 3 build machines, it costs $50 to double your build capacity
<gchristensen>
if your workload uses 1,000 cores, it is significantly expensive to meaningfully increase the build farm capacity
<__monty__>
Ah, hadn't put on my bookkeeper's glasses.
<gchristensen>
does that make sense?
<__monty__>
Yes, lots.
<gchristensen>
especially when the channel is stuck on macos, where we only have like 40 cores (4cores x 10macs) -- any sizable queue there is very hard to catch up on
<gchristensen>
another article to write up :P ("size and scale of hydra")
<__monty__>
+1
<averell>
yes, you should
drakonis_ has joined #nixos-chat
<__monty__>
Would more eyes on this help? I wouldn't mind learning about ops.
drakonis has quit [Read error: Connection reset by peer]
<gchristensen>
not sure -- at this point we have pretty good team and setup. there is definitely work to be done, but in a lot of ways it feels like the fires are out. hydra.nixos.org was recently moved to a larger server with more memory, the x86 and arm builders are auto-scaling
<gchristensen>
there has been a lot of improvement in that area in the last 6mo
<manveru>
averell: i thought chrome just downloads a .deb and patches it up?
<das_j>
gchristensen: So the Hydra builds 1.25 million derivations per month on around 800-900 aarch64 cores, ~1000 x64 cores and 40 macos cores?
<adisbladis>
manveru: We build chromium from source though
<manveru>
ah, ok
<adisbladis>
And that takes forever even on a super beefy machine :/
<manveru>
yeah...
<__monty__>
das_j: 600 aarch64, 900 x86_64, but yes.
drakonis has joined #nixos-chat
drakonis_ has quit [Ping timeout: 276 seconds]
drakonis1 has joined #nixos-chat
<adisbladis>
So :) Who's going to ccc camp this year?
* lassulus
<lassulus>
also organizing a NixOS village
<adisbladis>
lassulus: I didn't see it on the wiki
<lassulus>
I should add it to the camp wiki, we are only on the assembly page for now
<__monty__>
Does CCC like nixos? I'd expect many people to mention the rebuild-the-world-for-security-patches problem.
<lassulus>
if someone wants to create the wiki page: feel free, I'm having no time for it until monday
<joepie91>
__monty__: well, there was a successful ad-hoc install party at the last congress
<joepie91>
__monty__: though in fairness, the success was probably at least in part due to somebody's genius idea to put up posters in the toilets
<gchristensen>
rebuilding the world for security problems is not really a problem
<joepie91>
(I forgot who it was)
<__monty__>
That doesn't keep people from going on about it, gchristensen : )
<gchristensen>
yeah
<__monty__>
Same with with curl ... | sh
<__monty__>
*thing
<gchristensen>
if someone is going on about it, say we can do a full world rebuild in less than 24 hours, which in no way puts us at a disadvantage at the 7-14 days members of the embargoed linux distro list try to hit.
<__monty__>
What's this embargoed linux distro list?
<gchristensen>
the list where severe security bugs are announced ahead of time
<averell>
does that mean they start rebuilding before the patches are public?
<gchristensen>
yes
<averell>
and you can't just look at that on hydra?
<gchristensen>
distros with access to those patches build it on private infra so they can push it live at the coordinated release time
<das_j>
gchristensen: Well you *could* build them on a second private hydra on the same store as the public one. This way, no hydra would build what is already there and people wouldn't be able to check the public hydra for the build inputs
<gchristensen>
that is true
<gchristensen>
there are definitely solutions :)
<das_j>
do multiple hydras on the same store work tho?
<das_j>
sounds like a big lock mess
<gchristensen>
it'd just be sharing the cache, not a store
<das_j>
ah yes that would work
<__monty__>
I'm sure ML would ruin your day though. You'd have to argue that distributing binaries is not equivalent to disclosure, no?
<gchristensen>
that is true
<gchristensen>
we have ways of solving these problems. the technical "how do we do it" is not the reason we're not on it :P
drakonis has quit [Ping timeout: 264 seconds]
<__monty__>
What's the buss factor on nixos-security btw?
<gchristensen>
nixos.org/nixos/security.html
drakonis has joined #nixos-chat
<aanderse>
yeah it was a couple years ago now you gave that talk where you were mentioning that list and how nixos should try to get on it
<aanderse>
right?
drakonis1 has quit [Quit: WeeChat 2.4]
<gchristensen>
yeah
endformationage has joined #nixos-chat
__monty__ has quit [Ping timeout: 245 seconds]
__monty__ has joined #nixos-chat
<pie_>
i wonder if we could pre-distribute encrypted patches
* pie_
goes to other channel
<pie_>
ok maybe thats pointless though
<Church->
Alright time for a backup and a nixos reinstall
<Church->
This should be exicitng
<cransom>
i don't think i've ever had an exciting nixos reinstall.
<averell>
and the more you try the less likely it gets, sad.
<averell>
what's the joyous occasion? moving to ZFS?
<ashkitten>
gchristensen: wow, does it really take other distros that long to rebuild everything? what gives?
<ashkitten>
and here i've been complaining about unneeded dependencies inflating my build times ;p
<Church->
Huh, so nixos has a grapical installer but no installation wizard
<Church->
Odd.
<ashkitten>
dark installation wizzerd
<mgdm>
Patches are welcome? :)
<Church->
Yeah I might write a shell script at least.
<ashkitten>
i came from arch so 😓
<mgdm>
I've never used Arch though its wiki is awesome
<ashkitten>
the arch wiki is a treasure trove of info about how to get shit working
<mgdm>
absolutely
<ashkitten>
not super useful on nixos since most of the time things either just work or don't build at all
<samueldr>
Church-: the graphical install media is more about having a DE, to have a graphical browser, network-manager, than having a graphical installer (for now)
<mgdm>
I often find out what I want to do from that wiki, and then translate that into persistent config on whatever distro I am actually using
drakonis_ has quit [Ping timeout: 264 seconds]
<Church->
samueldr: Nod
<ashkitten>
i've used the arch wiki to help with pretty much all the distros i've used. i think the main difference is that a lot of the "tips and tricks" sorta stuff in nixos just get incorporated back into nixpkgs whereas arch only provides the packages not other adjacent stuff to complement them
<ashkitten>
in nixos it's not "how do i make this thing work" usually, it's "how do i do this weird thing with nix that's not in the manual"
<ashkitten>
or "how do i package x"
<Taneb>
ashkitten: I used the arch wiki to get NixOS installed on my laptop
<Taneb>
(it needed a newer kernel and some paramaters set)
<ashkitten>
sure, that's totally reasonable
<ashkitten>
i'm just saying for a lot of the cases i'd see myself using the arch wiki for in other distros, there's an option for it in nixpkgs
<ashkitten>
"how do i get zeroconf discovery working in pulse?" -> arch wiki: "install avahi, start/enable avahi systemd service, load zeroconf discovery module in pulse"
<ashkitten>
it's one option in nixpkgs
<Taneb>
Yeah, that's pretty cool
<Taneb>
But it's a thing someone's got to put in nixpkgs in the first place
<ashkitten>
yeah
<ashkitten>
and a *lot* of stuff is there, but sure there's plenty of stuff that isn't
<ashkitten>
i'm just mostly saying that in general a lot of these things get incorporated back into nixpkgs rather than sit in a wiki waiting for users to have issues with it
<Taneb>
Yeah
<averell>
and the AUR is also good when creating nix packages. it's good to be mainstream.
<adisbladis>
What's the variable in nixpkgs to get the location of nixpkgs again (i)
<adisbladis>
I'm not looking for <nixpkgs>
<Church->
nix.nixpath I think?
<Church->
Let me check
<adisbladis>
Church-: No that's how you set NIX_PATH
<adisbladis>
I mean I know I can use builtins.unsafeGetAttrPos
<adisbladis>
But iirc there is something in nixpkgs that provides this in a nicer way
<pie_>
qyliss: i ran off to the store to buy some earbuds cause all my audio hardware is garbage suddenly :I
<qyliss>
I'm going to be using their video conferencing think to talk to nlnet next week. Had never heard of it previously. Wonder if it might work for office hours.
<samueldr>
never heard of it beforehand
<samueldr>
interesting
<qyliss>
it's apparently nlnet-funded
<zimbatm>
qyliss: I've packaged the sylk client for NLNet
<qyliss>
oh nice
<zimbatm>
the server is half-packages but I need to finish it
<zimbatm>
packaged
<qyliss>
are there public servers like jitsi has?
<zimbatm>
too tired..
<samueldr>
if you can, open the half-finished PR for the server so others can help?
<zimbatm>
not sure, nlnet wants to host their own I think
<talyz>
eyJhb: so the next station was the right one?
<__monty__>
I'd assume for uploading to a streaming site a reencoded version'd be better? No clue about video things thoguh.
<adisbladis>
__monty__: Oh yeah, the second one there is the same method :)
drakonis_ has joined #nixos-chat
<gchristensen>
so kdenlive dumps core a lot, and it turns out I do want to do more than just mush two together. any third favorites? :)
drakonis has joined #nixos-chat
<samueldr>
there's this one neat thing I'm doing for a $CLIENT where I can't assume they want to use nix, but they can use random docker artifacts
<samueldr>
where I just build a docker image, using the nixos/nix image
<gchristensen>
nice!
<__monty__>
gchristensen: Blender has been used effectively to edit video in the blender foundation projects. I'd give that a go if ffmpeg wasn't good enough.
drakonis_ has quit [Ping timeout: 250 seconds]
<ashkitten>
i've used blender for video editing before
<gchristensen>
opening up blender for this feels like smashing an ant with the NASA crawler
<pie_>
:D
<adisbladis>
Using a nix-shell shebang is there a way to load a nix file relative to the script itself?
<samueldr>
./the-script ?
<PyroLagus>
"import ./blah.nix"?
<samueldr>
in the shebang itself or in the script?
<adisbladis>
In the shebang itself
<samueldr>
because the shebang IIRC it all acts relative to the script, though I may be wrong
<adisbladis>
That doesn't work, that's relative to the pwd
<samueldr>
hm
<PyroLagus>
ah
<adisbladis>
I can restructure things a bit to make it workable, but I want to know anyway :)
<qyliss>
The hashes can change and you don't even notice
<qyliss>
Stuff gets built once and cached on Hydra forever, and it's very difficult to ever reproduce
<pie_>
oh right you dont control the input source code
<pie_>
thats what was meant by "like fetchurl"
<infinisil>
Also, with rust at least it's really bad to have to wait for everything to build, then get the hash and build it again.. And this for every minor derivation change
<pie_>
infinisil: thats why i want __impure
<infinisil>
There's no intermediate caching
<qyliss>
pie_: the difference with fetchurl is the hash of a tarball won't change when you upgrade curl
<infinisil>
pie_: __impure?
<pie_>
with __impure you could do some automated pinning, (unless im confusing stuff)
<pie_>
infinisil: its a pr from eelco that seems to have never gotten anywhere uhh...
<pie_>
i was screwing with go2nix and wanted to to use it with IFD
<pie_>
i got it to work but would have to change the hashes all the time
<pie_>
well, something something i wasnt using pinned lock files
<pie_>
its all a bit murky
<gchristensen>
getting rid of arbitrary fixed output derivations could kill nix, since Nix can't predict all the legit ways to do fixed output fetches
<pie_>
basically go2nix generates lock files and instead of using the lock files directly i was having them generated in a sandbox with fixed outputs so i could access the network
<gchristensen>
this might be better for #nixos-dev actuall y:P
<pie_>
time to copy paste several pages of scrollback? :P
<pie_>
(IRC needs thread split/merges :P)
<gchristensen>
nah
<pie_>
might be going off on a tangent here though
<samueldr>
IRC needs users to not dump all in the offtopic channel! >:|
<samueldr>
:)
<pie_>
conversation flows naturally, whatcha gonna do :P
<pie_>
basically the tl;dr for me is i think it would be cool if nix could handle imputrity well and then you could do even more orchestration with it?
<pie_>
or am i just lacking enough insight to see why that wouldnt work
<infinisil>
orchestration as in ?
<pie_>
well i only have my previous use case re: running go2nix, maybe that was a bad choice of words
<pie_>
i like semantically describing my processes instead of actually running them ;D
<samueldr>
don't you just like it when you end up writing what is a pile of hacks, but a reproducible, self-contained pile of hacks?