<{^_^}>
#56115 (by matthewbauer, 1 minute ago, open): nixos-rebuild: get Nix from new channel
<matthewbauer[m]>
Hopefully that could fix the upgrade issue in the future
drakonis_ has quit [Ping timeout: 264 seconds]
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Quit: Updating details, brb]
emily has joined #nixos-dev
emily has quit [Client Quit]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
pie___ has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
pie__ has quit [Ping timeout: 250 seconds]
drakonis has quit [Quit: WeeChat 2.3]
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
__Sander__ has joined #nixos-dev
drakonis1 has quit [Remote host closed the connection]
lassulus has quit [Ping timeout: 250 seconds]
lassulus has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
asymmetric has joined #nixos-dev
asymmetric has quit [Ping timeout: 244 seconds]
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
lassulus has quit [Ping timeout: 246 seconds]
init_6 has joined #nixos-dev
lassulus has joined #nixos-dev
emily has quit [Remote host closed the connection]
emily has joined #nixos-dev
orivej has quit [Ping timeout: 245 seconds]
init_6 has quit []
lassulus has quit [Ping timeout: 246 seconds]
lassulus has joined #nixos-dev
rsa has joined #nixos-dev
<samueldr>
zimbatm: saw your tweet; chances are we'll never really see the store being brought forward across steps in github actions (though I didn't ask tbf)
<samueldr>
my solution, to allow other actions to use the outputs (results are dangling symlinks otherwise) is to unravel them from the store, implemented as a first draft here https://github.com/samueldr/action-nix-build
<domenkozar>
can't you use cachix?
<etu>
Hmm, would anyone be opposed to enable audio in the iso's?
<etu>
I don't see why it wouldn't be enabled
<etu>
Probably takes extra space, but that can't be much extra space
<gchristensen>
like the graphical iso?
<srhb>
I feel like we need to ship a third iso..
<srhb>
Well, not *need*, but it would be nice to have a medium, nongraphical one..
<gchristensen>
what for?
<srhb>
The above, among other things. But yeah, the graphical one isn't that bad..
<gchristensen>
I've been dreaming of a build-a-<strike>bear</strike>iso service where you put in some config and it builds it and spits out an iso or iPXE image
<etu>
gchristensen: yeah
<gchristensen>
etu: yeah, I think sound should probably be enabled for the gui one
<etu>
gchristensen: of course the graphical one :)
<etu>
Someone I chatted with and mentioned nix tried it out and didn't get sound to work in the live environment and was like: "I can't get sound working"
<etu>
I didn't know that we didn't have it enabled, simply because I've never cared. But yeah. I think a graphical system without sound is a bit odd.
<thoughtpolice>
I'm having an issue with dockerTools.buildLayeredImage in master, anyone seen this?
<thoughtpolice>
^ Confirmed to fix this regression
<thoughtpolice>
(On top of latest `master`, which is de2bb1094b7213d0604dc1d978dd1edd209c653e for me)
<gchristensen>
I guess substituteAll doesn't preserve +x
<thoughtpolice>
I think we just need to set isExecutable = true
<thoughtpolice>
after some further looking
<thoughtpolice>
Confirmed, I'll submit a patch
<thoughtpolice>
Well, actually, I don't think that patch that broke this originally works either lol
<thoughtpolice>
bcf54ce5bbc8c325cfd2b6bcc5cec7661ef49183 was supposed to change scripts to not use /bin/sh, but for me, /nix/store/jixivxhh3c8sncp9xlkc4ls3y5f2mmxh-store-path-to-layer.sh in fact starts with '#!/bin/sh'
<{^_^}>
#56151 (by thoughtpolice, 14 seconds ago, open): dockerTools: mark store-path-to-layer.sh as executable
<gchristensen>
thanks
<thoughtpolice>
samueldr: BTW, I ran into a similar issue when using GitHub Actions for my RISC-V emulator. I actually worked around it by just using nix copy to just export a store, then re-import it
<thoughtpolice>
Because the Docker container used by GH is always executed as root, you're trusted, so re-importing the store without a signature/key is alright. Also, this only works because the workspace is persistent between actions. IIRC you get about 100GB for every workflow, which is more than enough for probably 99.9% of all closures
<thoughtpolice>
domenkozar: Also, I considered using Cachix for this part, but I didn't really want to abuse it by constantly pushing closures in and out, for every single intermediate build step. It also bloats execution time.
<thoughtpolice>
Especially for my case, my closures can be very very large (~1GB), so re-importing from local disk is significantly faster than a full push/pull cycle. This speeds up sequential actions that occur 'downstream' of the one that needs the store, which is important to keep the total time of the workflow low
<thoughtpolice>
This also ties together nicely so that I only ever update my Cachix repository on *successful* builds to master, so A) You always get a recent cache as a user and B) it doesn't contain broken or intermediate closures from other people
<thoughtpolice>
Anyway this strategy works very nicely for me. I am quite happy with how well it all turned out in my case.
<thoughtpolice>
zimbatm: ^^ relevant to you as well, perhaps
<samueldr>
thoughtpolice: yeah, I was thinking maybe the community should write-up a convention for export/import the closure for "actions" like things
<samueldr>
e.g. gitlab CI also could benefit
<samueldr>
domenkozar: yeah, cachix could be used, but that's not useful if the next step is from the marketplace and doesn't know about nix
<samueldr>
e.g. you'd want to use a pre-made "copy to s3" action
<samueldr>
not saying most users here would, but some users "light nix users" might
<samueldr>
after all, better integrate in the ecosystem (github actions) than working against
<thoughtpolice>
Right. I do an S3-like copy for some files but I also copy them out
<thoughtpolice>
Actually, I tar up the html/webassembly files, and the next step untars/uploads them. So in my case I have a custom action for that, too...
<thoughtpolice>
On that note, it would be nice if, when importing from a file:// store, you could just say "import every path, no question asked"
<thoughtpolice>
Right now I copy it to a file:// and separately serialize the roots, but there's no real need for that in this case...
<zimbatm>
I just caught-up with the thread and basically agree with everything in it
<zimbatm>
a further optimization would be to use `--store $PWD/nix` but the containers don't support user namespaces
<thoughtpolice>
zimbatm: Also, thank you for pointing me to skopeo a while back (via Twitter) -- I use it in my project as well, and it is exceedingly useful with actions
<thoughtpolice>
Because they don't allow docker import
<zimbatm>
container team :)
<zimbatm>
yeah and it's much faster
<thoughtpolice>
It also works extremely well with dockerTools.buildLayeredImage + pkgs.pkgsMusl
<zimbatm>
if you are pushing to docker hub I have another trick
<thoughtpolice>
My emulator closure is something like 6mb
<zimbatm>
nice
<zimbatm>
investigating pkgsMusl is on my todo list, the last time I tried most of the closures were broken
<thoughtpolice>
Well, my case is lucky and very special. My emulator is written in a language that can be compiled directly to C that is nearly standalone. So actually I just run the compiler, spit out all the C files and then put them into an output containing only the source. Then I use pkgsMusl to compile *only* those C files, with no other dependencies.
<thoughtpolice>
So I don't need almost anything other than the bootstrap closure -- except for gmp/zlib, which both work.
<thoughtpolice>
I use this same trick for compiling to WebAssembly, too -- just put the C source into its own output and re-use it as an input elsewhere in a very austere derivation
<emily>
thoughtpolice: what language, out of curiosity?
<emily>
yeah, I've seen rv32-sail before in fact ^^
<thoughtpolice>
It's basically a first-order version of OCaml, with some extra features to make bit-slicing etc easier. Pretty neat.
<emily>
really cool stuff!
<thoughtpolice>
emily: I definitely would not have been able to write it so easily otherwise! The whole project uses a fair amount of tricks I need to write up...
<zimbatm>
impressive stuff
<srhb>
Who manages bigmac?
<srhb>
(The builder, not the burger...)
<gchristensen>
I do
<srhb>
gchristensen: Do you know if running into max open file limits happens commonly on it?
<gchristensen>
I don't know :)
<srhb>
Alright. :)
<gchristensen>
let me ask you a differentq uestion
<gchristensen>
do you know if running in to max open file limits happens ever on it?
<srhb>
gchristensen: I think so, because: [...] /share/go/src/cmd/go: pipe: too many open files
<gchristensen>
interesting
<srhb>
But I'm rerunning now to see if it really is transient.
<srhb>
A bit before 2019/02/20 03:34:28 if you are tracking open files.
<gchristensen>
I have no idea, maybe you can poke around the prometheus instance?
<srhb>
Sure.
<gchristensen>
<3 that transparency, don't have to do it all myself :)
<srhb>
gchristensen: I think the problem is actually the per-process limit of only 256 (compared to 4096 in my linux build) -- the go test cases (especially net/http) try to accept a number of parallel connections, and if they are not finished fast enough, it can easily hit accept() enough times to hit the per-process limit.
<gchristensen>
interesting
<gchristensen>
LnL: can nix-darwin set these limits?
<LnL>
gchristensen: sysctl? I don't think so
<srhb>
If it's a soft limit maybe we can just increase it in the build itself.
<srhb>
I don't think I can tell from the log.
averell- has joined #nixos-dev
averell has quit [Ping timeout: 252 seconds]
averell- is now known as averell
averell has quit [Quit: .]
disasm has joined #nixos-dev
ixxie has joined #nixos-dev
__Sander__ has quit [Quit: Konversation terminated!]
<{^_^}>
#56115 (by matthewbauer, 17 hours ago, open): nixos-rebuild: try to get Nix fallback from new channel first
<matthewbauer[m]>
I think it solves the main problem we had with bumping min-ver.nix in 18.09 and should make it easier to make breaking changes in Nix
<LnL>
I don't recall how the (re)exec behaviour of nixos-rebuild works, but I like the idea
<LnL>
there can be some tricky stuff because of a nix <-> nix-daemon version mismatch
callahad has quit [Quit: Ping timeout (120 seconds)]
callahad has joined #nixos-dev
callahad has quit [Quit: Ping timeout (120 seconds)]
callahad has joined #nixos-dev
<gchristensen>
this is fine: $ nix-instantiate --eval --strict --json -E '{ a = 1; b = 2; }' -A a
<gchristensen>
1%
<gchristensen>
("%" means no new-line in my shell)
<gchristensen>
this is hilariously not: $ nix-instantiate --eval --strict --json -E '{ a = 1; b = 2; }' -A a -A b
<gchristensen>
12
<emily>
:D
<samueldr>
amazing
<matthewbauer[m]>
LnL: yeah when trying this on 17.09 -> 19.03pre, there was some weird issues from the progress reporter not working right. but it still seems like a good thing to at least try vs. using NixOS 17.09's Nix which is 1.11
<LnL>
gchristensen: lol
<LnL>
matthewbauer[m]: yeah, but I think if you add a fetchTarball it will fail in a weird/confusing way
<LnL>
also, isn't there something that tries to evaluate with a newer version already?
<matthewbauer[m]>
yes that could be an issue with fetchTarball... I don't think we want to upgrade the nix-daemon before the entire nixos system though.
<matthewbauer[m]>
i guess nix could be smart and check what the nix-daemon version is before exposing fetchTarball + any others that require specific daemon versions
<matthewbauer[m]>
But, the thing that tries the newer version will fail due to the default.nix's min-ver
<matthewbauer[m]>
This approach avoid that entirely by importing fallback-paths.nix instead - so evaluation is almost guaranteed to work
<matthewbauer[m]>
As long as fallback-paths.nix is a simple attrset
<matthewbauer[m]>
and exists at nixos/modules/installer/tools/nix-fallback-paths.nix
<{^_^}>
#50504 (by symphorien, 13 weeks ago, open): nixos: add preferLocalBuild=true; on derivations for config files and closureInfo
<rycee>
To avoid remote building for many tiny files…
<LnL>
yeah, you want preferLocalBuild for things like that
<LnL>
otherwise you end up copying all the build time dependencies to another host just to create a text file if distributed builds are configured
ixxie has quit [Ping timeout: 268 seconds]
<rycee>
Yeah. I thought it was a very nice change.
<rycee>
So if symphorien resolves the conflict I'll merge it.
phreedom has quit [Ping timeout: 256 seconds]
phreedom has joined #nixos-dev
<matthewbauer[m]>
@LnL Hmm, I hadn't seen the _NIXOS_REBUILD_REEXEC. I think that stuff predates the minver stuff because nix-build would always fail to build nixos-rebuild with older nix
drakonis has joined #nixos-dev
<LnL>
yeah could be, might be best to move it to that part or replace it entirely