gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html https://r13y.com | 18.09 release managers: vcunat,samueldr | 19.03 RMs: samueldr,sphalerite | https://logs.nix.samueldr.com/nixos-dev
pxc has quit [Ping timeout: 246 seconds]
kgz has quit [Ping timeout: 250 seconds]
kgz has joined #nixos-dev
<samueldr> hm! there wasn't a news entry for the 2.2 nix release it seems on nixos.org
callahad has quit [Quit: Ping timeout (120 seconds)]
callahad has joined #nixos-dev
ericnoan has quit [Ping timeout: 246 seconds]
ericnoan has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
drakonis has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
drakonis has quit [Quit: WeeChat 2.3]
drakonis has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.3]
drakonis has joined #nixos-dev
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 252 seconds]
noonien has quit [Quit: Connection closed for inactivity]
orivej has joined #nixos-dev
teto has quit [Ping timeout: 244 seconds]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
kgz has quit [Ping timeout: 245 seconds]
kgz has joined #nixos-dev
drakonis_ has quit [Ping timeout: 244 seconds]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
drakonis_ has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
ajs124 has left #nixos-dev [#nixos-dev]
ajs124 has joined #nixos-dev
orivej has quit [Ping timeout: 246 seconds]
jtojnar has joined #nixos-dev
orivej has joined #nixos-dev
coconnor has quit [Ping timeout: 255 seconds]
coconnor has joined #nixos-dev
drakonis has joined #nixos-dev
<globin> ekleog, aszlig: up for shepherding https://github.com/NixOS/rfcs/pull/41?
<{^_^}> rfcs#41 (by e-user, 4 weeks ago, open): [RFC 0041] SELinux Support
drakonis_ has quit [Ping timeout: 246 seconds]
dtz has quit [Ping timeout: 240 seconds]
thefloweringash has quit [Ping timeout: 250 seconds]
bennofs[m] has quit [Ping timeout: 250 seconds]
philipp[m] has quit [Ping timeout: 240 seconds]
Ericson2314 has quit [Ping timeout: 252 seconds]
vaibhavsagar has quit [Ping timeout: 265 seconds]
rycee has quit [Ping timeout: 252 seconds]
worldofpeace has quit [Ping timeout: 250 seconds]
Moredread[m] has quit [Ping timeout: 250 seconds]
matthewbauer has quit [Ping timeout: 250 seconds]
sphalerit has quit [Ping timeout: 252 seconds]
<ekleog> globin: right, I've been waiting to have a better view on my schedule before answering
nocent has quit [Ping timeout: 268 seconds]
yegortimoshenko has quit [Ping timeout: 268 seconds]
timokau[m] has quit [Ping timeout: 268 seconds]
<aszlig> globin: unfortunately no... the last time i was using selinux was ~9 years ago and i'm also a bit time-constrained at the moment :-/
<ekleog> I think in the following weeks I won't be able to shepherd
roberth has quit [Ping timeout: 264 seconds]
<ekleog> (plus my only knowledge of selinux is theoretical, I never actually tried to use it)
<ekleog> I wonder if we shouldn't nominate between e-user and etbe the one who's a selinux dev (or maybe fishilico?), if they're agreeing to it -- they aren't necessarily nixos contributors but would provide valuable insight I think -- or at least invite them to the meeting
drakonis has quit [Read error: Connection reset by peer]
drakonis_ has joined #nixos-dev
<globin> sphalerite just tagged 19.03 \o/
<makefu> partey!
<gchristensen> niksnut: could Nix exit a different code if a fixed output build failed due to hash mismatch?
<niksnut> yes
<niksnut> even better though would be if we had a `nix build --json` that outputted a structured build result
<gchristensen> that would be darling
<niksnut> basically the BuildResult data structure
<gchristensen> I could try my hand at writing a patch to exit a different code if it is approachable -- could you send me in the right direction?
<niksnut> see Worker::exitStatus in build.cc, where it returns a different exit status for temporary or permanent failures
<gchristensen> neat, thanks!
<gchristensen> aszlig: I'm on it!
<aszlig> gchristensen: no need to hurry :-)
<gchristensen> good, b/c I have many meetings between now and then :P
<aszlig> gchristensen: it's just that i regularily build a large subset of nixpkgs master and thus get breaking builds usually before they hit hydra.nixos.org
<aszlig> and i'm not even using wordgrinder =)
<gchristensen> :)
<aszlig> to add some context in case others reading this are confused: https://github.com/NixOS/nixpkgs/commit/6b2bd330fa39376fe5143d8095b9a1b925255110
* LnL dislikes nix-prefetch-url for anything other than automated use
<gchristensen> +1
<Taneb> I ran into something the other day for which tofu didn't work... fetchZip, maybe?
<LnL> except for nix-prefetch-url -A that is
<aszlig> LnL: yeah, especially if you have postFetch or similar stuff
jtojnar has quit [Remote host closed the connection]
<gchristensen> sphalerite: here too?
sphalerite changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | NixOS 19.03 released! https://discourse.nixos.org/t/nixos-19-03-release/2652 | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html https://r13y.com | 19.03 RMs: samueldr,sphalerite | https://logs.nix.samueldr.com/nixos-dev
<gchristensen> it is kind of hilarious that our release notes don't say *how* to upgrade..!
<tilpner> The nixos.org news entry links to the upgrading manual section though
<gchristensen> ah nice
<tilpner> So it's fine as long as the release notes are not your entry point
<gchristensen> +1
<gchristensen> thanks!
<aminechikhaoui> https://nixos.org/nixos/manual/index.html#sec-upgrading should be modified to `nixos-rebuild switch -I nixpkgs=channel:nixos-19.03` probably :)
<aminechikhaoui> one command
<tilpner> That's not permanent though
<tilpner> You still need to set NIX_PATH or modify channels
<tilpner> Or do you just want to update 17.03 to 19.03?
<aminechikhaoui> right, I've been using channel:nixos-<release> in NIX_PATH as well
<aminechikhaoui> better than dealing with nix-channel I find
<gchristensen> niksnut: is it possible to know a function's name in libexpr's forceFunction ?
<tilpner> aminechikhaoui: But doesn't that mean you always need a working internet connection for rebuilds?
<tilpner> (Which might be fine)
<aminechikhaoui> tilpner yeah
<gchristensen> (function-position nanoseconds-spent number-of-calls) (top 3 sorted by nanoseconds spent)
<gchristensen> /nix/store/xnjvl3lnbl729hsi585v5075fqsgijl9-nixos-19.03.172138.5c52b25283a/nixos/pkgs/stdenv/generic/make-derivation.nix:108:33 437582 8981
<gchristensen> /nix/store/xnjvl3lnbl729hsi585v5075fqsgijl9-nixos-19.03.172138.5c52b25283a/nixos/pkgs/stdenv/generic/check-meta.nix:22:5 503557 14230
<gchristensen> /nix/store/xnjvl3lnbl729hsi585v5075fqsgijl9-nixos-19.03.172138.5c52b25283a/nixos/pkgs/stdenv/generic/check-meta.nix:214:9 2817229 5093
<infinisil> Oh boy..
<infinisil> That's the function that checks for the platform
<gchristensen> top three slowest functions: (last column is the average # of nanoseconds per call)
<gchristensen> /nix/store/xnjvl3lnbl729hsi585v5075fqsgijl9-nixos-19.03.172138.5c52b25283a/nixos/pkgs/development/compilers/ghc/8.6.4.nix:20:27 3309 3 1103
<gchristensen> /nix/var/nix/profiles/per-user/root/channels/nixos/pkgs/top-level/release-lib.nix:83:9 272303 195 1396.43
<gchristensen> /nix/var/nix/profiles/per-user/root/channels/nixos/lib/systems/parse.nix:174:12 10180 6 1696.67
<gchristensen> my instrumentation is probably not good, but it was only +5loc to get this level of info
<tilpner> gchristensen: During what evaluation did you measure this?
<gchristensen> nixpkgs' release.nix unstable
<tilpner> Ahh, of course
tilpner has quit [Quit: WeeChat 2.4]
tilpner has joined #nixos-dev
<{^_^}> nix#2761 (by grahamc, 28 seconds ago, open): nix: trivial instantiation timing experiment
<tilpner> gchristensen: You could check if the counts match with NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1
<gchristensen> yeah, it doesn't
<gchristensen> whatif we didn't re-evaluate check-env a half a million times: https://github.com/NixOS/nixpkgs/commit/9eedb72455f3c65a9cd757e89b9b07f937512926
ckauhaus has quit [Quit: WeeChat 2.2]
<Profpatsch> gchristensen is asking the right questions
<Profpatsch> gchristensen++
<{^_^}> gchristensen's karma got increased to 100
<Profpatsch> whoooo
<gchristensen> I misread the stats :|
<Profpatsch> gchristensen--
<gchristensen> LOL
<Profpatsch> He, no, of course not
<gchristensen> 8,000 to 150 is pretty good though
apaul1729 has joined #nixos-dev
pxc has joined #nixos-dev
srk has quit [Ping timeout: 250 seconds]
srk has joined #nixos-dev
<gchristensen> what if ofborg produced this table on each PR? https://github.com/NixOS/nixpkgs/pull/59323#issuecomment-482253971
<gchristensen> also samueldr relevant stats in there that I think you'll like ^
<gchristensen> hrm, maybe not. heap went up :o
<samueldr> are executions of nix deterministic?
<samueldr> or uh, evals
<gchristensen> I think so, let's find out
<infinisil> gchristensen: The non-deterministic stats at least should be done multiple times to get a standard deviation. Such statistics otherwise are more misleading than useful
<samueldr> because I wouldn't think so considering how the too many heap sections issue seems to not be deterministic in hydra evals
<gchristensen> hydra's evaluator is fancier
<samueldr> yeah, but I think it should be deterministic if nix evals are :/
<samueldr> going from memory, but hydra's evals while fancier are simple, it restarts under some conditions only, and those conditions I assume from memory should be deterministic
<gchristensen> no, it isn't deterministic
<samueldr> "it" being "only hydra's evaluator" or "all nix evals"?
<gchristensen> nix-instantiate's stats
<samueldr> right
<gchristensen> knowing this, would it be helpful or confusing to have this table on each eval?
<samueldr> without determinism it might not be as useful :/ though maybe there's a ballpark estimate that can be made from those (with possibly multiple evals)?
<sphalerite> mumble mumble confidence intervals I can't do statistics
<samueldr> I won't pretend I know how to, I'll just pretend I understand them :)
<gchristensen> I certainly don't understand
<sphalerite> but I do know enough about statistics to know that there are tools that can help us work out how useful the numbers are
<sphalerite> just not how to do it :D
<infinisil> Yeah confidence intervals are important
<infinisil> I do know some statistics
<gchristensen> great!
<gchristensen> yeah pretty sure these numbers are not actually very useful :|
<gchristensen> just kidding, I re-ran the stats export for the same branch instead of switching to another branch I wanted to test with :)
<gchristensen> okay, so I think the delta-percent is pretty reasonably useful
<gchristensen> for b54bce24cf4..0a80d3dc822 https://gist.github.com/grahamc/b12a720685459fa03173734feff8af95
<gchristensen> I'm inclined to put it in a Check result.
<gchristensen> a change of +/- a small percent is not going to raise any eyebrows, but for future big changes? probably a good idea to expose this info
lopsided98 has quit [Quit: Disconnected]
lopsided98 has joined #nixos-dev
drakonis_ has quit [Ping timeout: 252 seconds]
drakonis_ has joined #nixos-dev
drakonis has joined #nixos-dev
drakonis_ has quit [Ping timeout: 264 seconds]
timokau has joined #nixos-dev
timokau has quit [Client Quit]
orivej has quit [Ping timeout: 246 seconds]
drakonis has quit [Ping timeout: 264 seconds]
orivej has joined #nixos-dev
drakonis has joined #nixos-dev
phreedom_ has joined #nixos-dev
phreedom has quit [Ping timeout: 256 seconds]
callahad has quit [*.net *.split]
lassulus has quit [*.net *.split]
page has quit [*.net *.split]
ekleog has quit [*.net *.split]
Enzime has quit [*.net *.split]
MoreTea has quit [*.net *.split]
<gchristensen> I decided this is important information, even if it isn't statistically useful in every case
ekleog has joined #nixos-dev
<gchristensen> I'm also thinking I'll publish the same stats for NixOS on each PR
<gchristensen> it feels this sort of info is important for keeping an eye on things, and could be really helpful for future big PRs
<samueldr> I think that as long as it's relatively in the same ballpark, it'll be useful for the big ones
<samueldr> yeah
<gchristensen> yeah
<samueldr> and that small variations shouldn't be scary or needed to be optimized
<gchristensen> could put up a label if there is a 10% change in anything, but I don't think I'll do that yet
page has joined #nixos-dev
MoreTea has joined #nixos-dev
lassulus has joined #nixos-dev
Enzime has joined #nixos-dev
callahad has joined #nixos-dev
<gchristensen> and a 30s decrease in eval time is no joke
<gchristensen> samueldr: any feelings about backporting?
<samueldr> no need to label, just that if people ask, numbers are mostly informational, unless the change hits hard
<gchristensen> yea
<samueldr> that's a no-op as far as eval goes?
<gchristensen> yea
<samueldr> sounds good to me
<gchristensen> cool
<ekleog> wow 30s in eval time? what did I miss with this freenode netsplit?
<gchristensen> I moved a function definition down 200 lines, and an import up 200 lines
<{^_^}> #59323 (by grahamc, 6 hours ago, open): check-meta: don't execute check-meta.nix 15,000 times
<gchristensen> a less cute explanation is the check-meta function is pretty complicated, and it defined in an environment which included each package's parameters
<gchristensen> despite not using any of the package's parameters
<ekleog> wow
<gchristensen> so I moved the package-specific environment parts to a tiny part of it https://github.com/NixOS/nixpkgs/pull/59323/files#diff-096a3f0fd04e519accb1cde6c483e72bR248 and then moved the import outside of the package environment too: https://github.com/NixOS/nixpkgs/pull/59323/files#diff-ef38de73175ae8487d0f4b36b75d2af7R2
<samueldr> more context over here ekleog :) https://logs.nix.samueldr.com/nixos-dev/2019-04-11
<gchristensen> nix caches thunks, but doesn't automatically lift thunks out of environments when the thunk doesn't depend on the environment (maybe that is really hard? I don't know)
* gchristensen is on the edge of his specific knowledge
<ekleog> thanks :)
<gchristensen> this came from this extremely janky and bad experiment https://github.com/NixOS/nix/pull/2761
<{^_^}> nix#2761 (by grahamc, 8 hours ago, open): nix: trivial instantiation timing experiment
<gchristensen> oh that reminds me, nixos's release-combined.nix doesn't evaluate cleanly. it *almost* does, but not quite
<gchristensen> breaks this being automated
<gchristensen> nor does nixpkgs :) much more noisy, that one