<gchristensen> Profpatsch: it was all vcunat's and nbp's doing :) so much of ofborg's smarts is based on https://github.com/NixOS/nixpkgs/blob/master/maintainers/scripts/rebuild-amount.sh
adisbladis has quit [Ping timeout: 240 seconds]
mbrgm has quit [Ping timeout: 248 seconds]
mbrgm has joined #nixos-dev
orivej has joined #nixos-dev
<Profpatsch> Do we have a way to display a “this package is unmaintained” warning?
<Profpatsch> We can’t just remove stuff, but at least a warning should be displayed, right?
<Profpatsch> Also, is there a story for removing packages?
<Profpatsch> .shit Latschige Chips in Drachenblut einlegen -->> unendlich haltbar
<Profpatsch> Ah, wrong channel
Lisanna has quit [Quit: Lisanna]
Lisanna_ has joined #nixos-dev
FRidh has joined #nixos-dev
Guest97638 has joined #nixos-dev
JosW has joined #nixos-dev
Guest97638 has quit [Ping timeout: 276 seconds]
Guest97638 has joined #nixos-dev
vcunat has joined #nixos-dev
<flokli> :-D
<vcunat> Since a couple days ago, most nixos evaluations on Hydra end up in out-of-memory. We'll need to do something soon. Probably try to find a commit that noticeably increased memory usage around the point it stopped working.
<vcunat> Or if it's possible to give it more memory, that certainly wouldn't hurt. (We want to avoid significant usage increases anyway.) https://hydra.nixos.org/jobset/nixos/trunk-combined#tabs-errors
Guest97638 has quit [Ping timeout: 276 seconds]
Guest97638 has joined #nixos-dev
<andi-> vcunat: is tere some built-in way to figure out the memory requirement for an evaluation? I would probably be trivial to plott the memory usage for each commit in order
<LnL> things like this is why I was concerned with changing the ofborg gc options
<vcunat> andi-: we do this already
<vcunat> exactly
<vcunat> I'm just doing too many things at once, like retpoline for our kernels ATM.
<LnL> oh I didn't notice before but the lapp graphs went way up
<andi-> vcunat: nice, did you see that recent PR regarding exposing the kernel features? I could imagine implementing retpoline a a feature flag
<vcunat> it is a standard feature flag
<vcunat> but first you need gcc >= 7.3.0 to have the option
<andi-> yeah
<vcunat> so I'm doing that for 17.09 first
<vcunat> (while still allowing you to explicitly override the compiler)
<LnL> it's either 34778 or 29440
<vcunat> What's that?
<LnL> the huge spike in the metrics, possibly related to the memory issues
<vcunat> And those two numbers mean what?
<FRidh> PR's
<vcunat> (I'm sorry if I'm being obtuse.)
<LnL> oh pull requests :)
<vcunat> Oh :-)
<vcunat> of course
<vcunat> Hmm, I think so. Pushed gcc7 for kernel and its modules to 17.09.
orivej has quit [Ping timeout: 260 seconds]
vcunat has quit [Ping timeout: 260 seconds]
vcunat has joined #nixos-dev
<gchristensen> hmm yeah ofborg was blocking a PR or two because of memory, and then master stopped evaling with the default initial heap
<gchristensen> so I increased the initial heap on the evaluators to match what hydra has (4g, an increase from the default of 384MiB)
<gchristensen> ouch
<LnL> almost doubles the evaluation time of the nixos metric
<gchristensen> oxij won't be impressed by a second reverting of their PR :P
<MichaelRaskin> I wonder if this can be made optional in the sense of a manual having an extra parameter to switch this on or off
vcunat has quit [Ping timeout: 255 seconds]
<LnL> I'm guessing it's something related to module strictness
<gchristensen> LnL: the spike before was the same PR actually
taktoa has quit [Remote host closed the connection]
vcunat has joined #nixos-dev
<gchristensen> LnL, vcunat: I'm thinking I should revert that PR, then. agreed?
<vcunat> gchristensen: I'm missing the context. (I saw just two PR candidate numbers.)
<gchristensen> oh https://github.com/NixOS/nixpkgs/pull/33898 this PR added all the bump
<gchristensen> https://hydra.nixos.org/job/nixpkgs/trunk/metrics#tabs-charts scroll down to lapp.* and both thee bumps in the history are from this PR, and a previous (reverted) version of this PR
<MichaelRaskin> The problem is that this is a meaningful addition of functionality that has a reason to use more memory during evaluation…
<vcunat> Oh, this one again :-D
<vcunat> reverting a re-revert
<gchristensen> I agree, MichaelRaskin
<MichaelRaskin> There should be some way of discussing the tradeoffs involved and making a collective decision. Ah, sorry, that's Nixpkgs, forget it.
<vcunat> It is sad
<LnL> true, but this impacts _every_ nixos eval
<gchristensen> but 5,000,000 -> 200,000,000 values is a big jump for related packages
<vcunat> That's strange.
<gchristensen> and 200,000,000 allocations -> 1,000,000,000 allocations
<vcunat> I can't immediately see anything expensive in the diff.
<vcunat> When we pin-point the source of this, we can do more, e.g. just make it optional to _manual_ build, as it's probably useless elsewhere.
<LnL> yep
<LnL> it's back to normal with optionsList = optionsListDesc;
<LnL> the configuration.nix manpages are part of every system so you pay the sorting cost every time if it's done at evaluation time
<vcunat> Ah :-D
<vcunat> Maybe xslt would be a better tool.
<vcunat> or possibly just select the few special cases (moving them to the beginning) and don't touch the rest
<LnL> yeah
<vcunat> Actually, it might be possible to use toJSON at eval-time, and during runtime use the same code via a nix call.
<vcunat> but I expect there will be some obstacles, so it might not be worth it at all.
<vcunat> It would be nice if we had some nice generic way of moving some processing to runtime "seamlessly". IIRC this is a big advantage of guix.
<gchristensen> what does that mean?
<vcunat> Which part?
<gchristensen> moving some processing to runtime seamlessly
<LnL> oh, how do they do that?
<MichaelRaskin> Well, when you evaluate using Guile and run builders using Guile, just cut-and-paste of code starts being an option for a lot of things
<vcunat> I think you can quote an expression tree and pass it to the builder.
<gchristensen> I guess I'm not sure where runtime is?
<gchristensen> ohh I see
<gchristensen> that could be nicely used for IFD... no?
<vcunat> Hmm, I expect it might cover some use cases.
<LnL> ah right they have guile in their stdenv
<vcunat> guix uses guile for both eval-time and for builders (instead of our bash)
<gchristensen> exciting
<LnL> yeah, so you wouldn’t have the nix runtime part but everything else like the stdlib is available
<gchristensen> that is pretty cool!
Guest97638 has quit [Quit: WeeChat 2.0]
ma27 has joined #nixos-dev
ma27 has quit [Client Quit]
zarel has joined #nixos-dev
ma27 has joined #nixos-dev
vcunat has quit [Ping timeout: 256 seconds]
<fpletz> domenkozar: niksnut: could one of you please give me push rights to NixOS/systemd? I've prepared the 234 -> 237 update :)
<fpletz> this is the branch I would like to push: https://github.com/mayflower/systemd/tree/nixos-v237
<pie_> when i changed the rev for a fetchFromGitHub but I didnt change the hash....nix-shell didnt cause a rebuild...??
<MichaelRaskin> Well, if the name of checkout didn't change, why Nix would refetch?
<MichaelRaskin> That's why I always change one character in hash when doing stuff like that
<clever> pie_: you gave it the same hash as before, so nix believes that the source is the same as before
<clever> pie_: increment one of the digits in the hash to force it to re-download
<fpletz> pie_: the fetch* functions are fixed output derivations, that means that the store path is basically identified by the content hash. if you already have the store path to the corresponding hash in your store, that will be used (in your case, the old tarball)
<pie_> clever, yeah thats what i did but shouldnt this be considered a usability issue?
<pie_> or at least a bad thing
<clever> pie_: its generally solved by putting the rev into the name of the derivation, but i think fetchFromGitHub is broken in that reguard right now
* pie_ checks if an issue exists for this
<pie_> oh dear github seems a bit slow
<pie_> meh, search isnt working really well for me, anyone know if theres already an issue for this?
<clever> dont know of an issue, but the fix is simple
<clever> 210 fetchFromGitHub = {
<clever> 211 owner, repo, rev, name ? "source",
<clever> pie_: i think the default name here should be "source-${rev}" in all-packages.nix
<clever> that should fix the problem
<pie_> hmm makes sense
<clever> then even if the hash is the same, the name attribute is different
<clever> so it goes from /nix/store/hash1-source-rev1 to /nix/store/hash1-source-rev2
<clever> and that 2nd path will never exist, so it re-downloads, and fails because the hash is wrong
<pie_> yeah
<pie_> ill be using that now thanks :)
<pie_> version = "${src.rev}";
<LnL> that was renamed to source to avoid rebuild :)
<clever> LnL: ?
<pie_> im not actually sure whats going on with the name ? "source" thing there?
<LnL> clever: hydra uses that for inputs IIRC, this way caches are compatible between hydra/builtins.fetchTarball/fetchFromGitHub
<clever> LnL: ah
<clever> so you would also have to patch the nix-prefetch-git that hydra uses
<clever> so they both use rev
<clever> but that wont always match up, because you can use a short rev, long rev, tag, or branch
<clever> simpler to just leave it as it is
<pie_> somehow i thought stuff like rev would be part of the "inputs" that get hashed when running an expression
<pie_> well even if you can use different names for a rev, shouldnt you canonicalize it to the original git hash?
<clever> pie_: fixed-output derivations only use the hashmode, hash, and name
<LnL> yeah, if an upstream url breaks you can rename the url and won't need to rebuild everything
<LnL> it's not part of the hashing for fixed outputs
<clever> assuming the name at the end of the url is the same, or you set the name attribute to ensure its unchanged
<LnL> indeed, the name is special because it ends up in the path
<pie_> LnL, well that makes sense i guess
* pie_ scratches head
<pie_> ehh
<LnL> I generally run nix-build -A hello.src --check to verify an update
<pie_> this sounds like something that would bite people in the butt a lot
<pie_> prevention vs treating the symptoms, i dunno
<pie_> but ill stop poking at it for now, gotta do other stuff
<LnL> I'm not disagreeing with you, just explaining how it is right now :)
aminechikhaoui has quit [Ping timeout: 264 seconds]
<LnL> a while back I found a package that uses the apple fork of a project but had the same hash as the linux version
<LnL> so it used the wrong sources resulting in weird build errors
<clever> ive also seen some new users causing similar problems when trying to package an app with 5 deps, and copying the fetchurl between each
<LnL> heh
<clever> and i think it was all github, so they all used the name="source"; via fetchFromGitHub
zarel has quit [Remote host closed the connection]
ma27 has quit [Ping timeout: 255 seconds]
aminechikhaoui has joined #nixos-dev
aminechikhaoui has quit [Ping timeout: 240 seconds]
aminechikhaoui has joined #nixos-dev
vinymeuh has joined #nixos-dev
vcunat has joined #nixos-dev
<pie_> so im packaging an application that breaks if you dont pass it a config file with a valid font path, it doesnt have a default config or anything so im thinking id wrap it so that if you call bonzomatic it runs Bonzomatic $HOME/.bonzomatic.json
<pie_> and have the nix script also create a default config in the home directory (if it doesnt exist)
<pie_> hm actually i guess i should make the wrapper script do that
<pie_> anyway, how would i go about making such a wrapper?
<pie_> for the default config id need to get a path from fontconfig somehow
<pie_> would be nice if i could get nix to help with that or something
<vcunat> like pkgs.makeFontsConf ?
<FRidh> The following is an error from Nix, and not from inside the nix-build, right? Error: mkdir('/homeless-shelter/.cache/bazel/_bazel_freddy'): :X
<pie_> vcunat, ooh that looks helpful
<vcunat> IIRC you can find some use cases in nixpkgs
<pie_> yeah i dont se eit documented so poking around in the source...
<vcunat> it was relatively rare
<pie_> whatever fontDirectories is getting set to might be sufficient for me
pie_ has quit [Ping timeout: 248 seconds]
yegortimoshenko has joined #nixos-dev
orivej has joined #nixos-dev
yegortim1 has quit [Ping timeout: 255 seconds]
vinymeuh has quit [Read error: Connection reset by peer]
pie_ has joined #nixos-dev
pie__ has joined #nixos-dev
pie_ has quit [Read error: Connection reset by peer]
pie___ has joined #nixos-dev
pie__ has quit [Ping timeout: 264 seconds]
ma27 has joined #nixos-dev
vcunat has quit [Quit: Leaving.]
<pie___> is it possible to have an optional argument or only a default argument
<MichaelRaskin> What is an optional argument?
<MichaelRaskin> What should happen when you try to use it?
<pie___> basically im trying to think of a nice/acceptable way to have the config point to a default font file https://bpaste.net/show/4dca803bca40
<pie___> thats very WIP
<pie___> the config should be dropped by the wrapper script if it doesnt exist
<pie___> if theres no default font, it should just not use the wrapper
<MichaelRaskin> Default to null and use if
<pie___> yeah i was thinking you might suggest that, ok.
<pie___> thanks
<pie___> ok and for the wrapper how would I make it so that the original executable doesnt show up in the path so I can use the same name for the wrapper?
<MichaelRaskin> I guess you need to set package priorities.
<pie___> you mean make another package for the wrapper?
<MichaelRaskin> In the same package you could rename the original file if you generate the wrapper
<MichaelRaskin> Which is what makeWrapper does
<pie___> oh huh ill check that out
<pie___> also wrapProgram
<pie___> adding an arg is exactly what I want to do
<pie___> ok I think I know what I'll do.
FRidh has quit [Quit: Konversation terminated!]
<pie___> what would the appropriate no-op be for postBuild? null or ""?
<gchristensen> :
<gchristensen> bash's noop is :
<pie___> so postBuild = if defaultFont then wrapScript else ":";
<pie___> look good?
<pie___> hm...so nix is actually dependent on bash not sh?
JosW has quit [Quit: Konversation terminated!]
<pie___> at the least make-wrappers.sh is "The command can push extra flags to a magic list variable extraFlagsArray, which are then added to the invocation", no arrays in sh afaik
ma27 has quit [Ping timeout: 276 seconds]
<pie___> ugh looks like --run takes an actual command man this is kind of annoying
<pie___> or rather im not sure if --run is evaluated at build or runtime...
* pie___ tries to find an example
orivej has quit [Ping timeout: 256 seconds]
<pie___> so how secure is nix anyway
<pie___> could an exploit in the evaluator lead to privesc?
<gchristensen> pie___ #nixos might be a better place for most of your questions
<pie___> oh oops :D
<gchristensen> :)