gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.09 release managers: vcunat and samueldr | https://logs.nix.samueldr.com/nixos-dev
wladmis has joined #nixos-dev
<makefu> reboot and boot a previous revision?
<samueldr> they just got it resolved
init_6 has quit [Ping timeout: 250 seconds]
jtojnar has quit [Read error: Connection reset by peer]
drakonis1 has joined #nixos-dev
pie__ has joined #nixos-dev
pxc has quit [Quit: WeeChat 2.3]
pie_ has quit [Remote host closed the connection]
<ekleog> hmm… mass-rebuilds that solve important security issues (CVSS3 8.8) should go in master directly, right?
<ekleog> (a heap buffer overflow in libtiff, in the current case)
<gchristensen> I think that policy might need revisiting with the staging-next workflow, but that was my policy before
<ekleog> oh indeed the staging RFC states that fixes are supposed to go in `staging-next`
drakonis1 has quit [Quit: WeeChat 2.2]
<gchristensen> I think only fixes for problems preventing staging-next from merging, though, is the complicating factor
<gchristensen> ekleog: is there a PR?
<ekleog> hmm, “staging-next is branched from staging and only fixes to stabilize and security fixes shall be delivered to this branch”
<gchristensen> ah nice
<{^_^}> #51105 (by andrew-d, 15 hours ago, open): libtiff: 2018-11-04 -> 4.0.10
<gchristensen> ekleog++
<{^_^}> ekleog's karma got increased to 3
<gchristensen> ekleog: why don't you just pick that commit to staging-next by hand?
<ekleog> good point, was thinking about the educative aspect but I guess for security fixes speed is more important
<ekleog> kind-of relatedly, there's release-18.09, staging-18.09 but no staging-next-18.09… should I assume that staging-18.09 is also staging-next-18.09?
<gchristensen> how recent is staging
<gchristensen> how recent is staging-18.09?
<gchristensen> maybe just apply to release-18.09 direct, unless staging-18.09 is recent
<samueldr> there normally wouldn't be major changes needing stabilization in -stable, so it may not need a separate staging-next, right?
<ekleog> hmm I can't find a job for staging-18.09
* ekleog kind of wonders why there is even a staging-18.09
<ekleog> let's go for release-18.09, then :)
pie__ has quit [Ping timeout: 268 seconds]
layus has quit [Remote host closed the connection]
layus has joined #nixos-dev
hedning has quit [Quit: hedning]
<yl[m]> why the eval failed here https://github.com/NixOS/nixpkgs/pull/51163 ?
<{^_^}> #51163 (by kalbasit, 30 minutes ago, open): aminal: init at 0.7.4
<yl[m]> I can eval & build on my machine
<simpson> called without required argument 'xlibs' https://gist.github.com/GrahamcOfBorg/768c794ccc71e6a3a779f2c089c08083
<yl[m]> I just figured it out
<yl[m]> yea /facepalm
jtojnar has joined #nixos-dev
<yl[m]> thx simpson
orivej has joined #nixos-dev
jtojnar has quit [Quit: jtojnar]
clever has quit [Ping timeout: 252 seconds]
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 244 seconds]
<ekleog> is there any reason why hydra sending mails to maintainers when the packages break hasn't been turned on agian?
<ekleog> anyone available to re-trigger https://hydra.nixos.org/build/84806551 ? same as clang last time, it being down blocks tests on ofborg-darwin
worldofpeace has quit [Ping timeout: 246 seconds]
orivej has joined #nixos-dev
<yl[m]> LnL: I'm building a package on Mac that erroring out `./glfw/src/cocoa_joystick.m:38:10: fatal error: 'Kernel/IOKit/hidsystem/IOHIDUsageTables.h' file not found` I found that file installed at `Kernel/IOKit/hid/IOHIDUsageTables.h`instead in `darwing...IOKit`. known or bug?
<LnL> moin
<LnL> darwin.IOKit or darwin.apple_sdk.frameworks.IOKit?
<yl[m]> darwin.apple_sdk.frameworks.IOKit
<yl[m]> LnL: I tried `darwin.IOKit` as well. I got the same issue
<yl[m]> LnL: I'm working on https://github.com/NixOS/nixpkgs/pull/51163 can you take a quick look if you have time?
<{^_^}> #51163 (by kalbasit, 3 hours ago, open): aminal: init at 0.7.4
<LnL> yeah no, if it was the other way it could have been the case
<LnL> did you look in Kernel?
<LnL> yl[m]: It's in Kernel.framework but I guess those headers are not copied into the output for some reason
<Mic92> the first buildRustPackage cross-compiled!
__Sander__ has joined #nixos-dev
<qyliss> Mic92++
<{^_^}> Mic92's karma got increased to 5
<Mic92> The cargo cross-compilation looks quiet sane. I don't think we have to fix many rust packages in future for cross compilation.
init_6 has joined #nixos-dev
<Mic92> Ericson2314: do you have an idea how we could deal with all these $dev/bin/$name-config packages?
<Mic92> i.e. freetype-config pcre-config
<Mic92> A hack would be to take buildPackages version and replace the strings in it. This would work at least for scripts.
<Mic92> for binaries this will not work since the store paths for cross-compiled packages are longer
<Mic92> bennofs[m]: ^
<LnL> cool!
<ekleog> up @ restarting https://hydra.nixos.org/build/84806551 so that builds can be ofborg-tested on darwin again :)
<srhb> ekleog: up? :P
<ekleog> (the whole latest evaluation appears to have been cancelled somehow)
<ekleog> srhb: did a poke something like 8 hours ago :p
<srhb> ekleog: Ah. Restarted it..
<ekleog> thank you! :)
<srhb> I bet someone cancelled the eval since trunk will be pretty useless until staging-next hits master, at least for most channels.
<ekleog> hmm that makes sense, but for ofborg-testing both clang and rustc were cancelled and thus blocked builds… I guess gcc hasn't been cancelled, though, otherwise there'd be more complaints about ofborg being borken
<srhb> ekleog: Broken because of timeouts?
<ekleog> yup
<srhb> I see, yeah.
<ekleog> sometimes it can be worked around by triggering the build multiple times and ofborg will reuse its own cache, but rustc or clang both don't fit in a 30min slot, so…
<srhb> ekleog: Right right.
<bennofs[m]> looks like there is great interest in cross compilation right now. so much activity (PRs and issues) related to that right now
<bennofs[m]> Mic92: so actually, xxx-config is related to target I think: xxx-config should be built to run on host, but produce the store path for the derivation that is cross compiled for host = target
<Mic92> bennofs[m]: I think we are on good track to get nixos isos cross-compiled.
<Mic92> cross-compiling to macOS from linux is also intersting.
<bennofs[m]> btw, is there a shorter way to write `ix build --arg crossSystem 'with import <nixpkgs/lib>; systems.examples.aarch64-multiplatform'` ?
<tilpner> Isn't that nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform.*
<bennofs[m]> oh didn't know about pkgsCross
<bennofs[m]> thx that's better
<bennofs[m]> Mic92: I created an issue https://github.com/NixOS/nixpkgs/issues/51176
<{^_^}> #51176 (by bennofs, 13 seconds ago, open): cross compilation: xxx-config binaries
* ekleog hoping for the day one will be able to have an armv7 nixos transparently using the binary cache from a cross-compiling hydra
<Mic92> bennofs[m]: https://nixos.wiki/wiki/Cross_Compiling I also try to condense important stuff here.
<bennofs[m]> ekleog: if you want to dream... deterministic cross compilation where it doesn't matter if you cross built or built natively
<ekleog> :DD
<ekleog> that would require patching like so much upstream stuff, though
* ekleog wonders whether the idea wouldn't already be achievable by having a lying hydra
<ekleog> like hydra cross-compiling but saying it did the build natively
<bennofs[m]> i think you need to rewrite hashes for that as well?
<bennofs[m]> or at least lie about hashes at build time
<ekleog> hmm yeah that'd need actually nix support
<ekleog> so that the things get placed at the “right” (lying) location
<ekleog> … :(
* ekleog wonders whether there could be a `nix-build -A foo --lie-and-say-it-was-build-with bar --override-dependency baz.drv baz-crosscompiled-path`
<ekleog> that starts getting really ugly, though…
<ekleog> actually rather `nix-build -A pkgsCross.arch.foo --lie-and-say-it-was-built-with "with import <nixpkgs> { system = arch; }; foo" --override-dependency baz.drv previously-crosscompiled-baz-path
<bennofs[m]> is the "cross triple" of a pkgs set accessible through attributes somewhere? probably stdenv has it right?
<ekleog> ugh. srhb: care to restart https://hydra.nixos.org/build/84955192 too? it got failed due to no space left on device, hopefully the builder has been garbage-collected since :/
<ekleog> bennofs[m]: stdenv.system has something and stdenv.{build,host,target}Platform other things, depending on which one you're looking for
<gchristensen> ekleog: ooh yay we can test the new mac deploys! also not sure why -guest metrics aren't showing up here, but https://status.nixos.org/grafana/d/hkRCcV0mk/instance-metrics?orgId=1&var-instance=mac7-guest&var-instance=mac7-host&var-machine=All&from=now-7d&to=now
<srhb> ekleog: Trade you for a thumbs up :P https://github.com/NixOS/nixos-org-configurations/pull/61
<{^_^}> nixos-org-configurations#61 (by srhb, 3 weeks ago, open): Automatic GC if we drop low. Limits debatable...
<srhb> Oh, won't help on macs...
<ekleog> srhb: added a heart as a bonus :p
<gchristensen> sorry I haven't deployed that yet
<ekleog> gchristensen: oh is this graph linked somewhere? :)
* ekleog doesn't exactly understand what you mean by “new mac deploys”, though
<gchristensen> half the macs are macos installed on macs. the other half are macos running in a VM on NixOS on macs, and each reboot of the mac is a clean slate
<srhb> Wait what. NixOS on mac hardware running OS X in vms? Didn't even know that was possible :P
<gchristensen> the VM gets 95% of the host's resources, so minimal slowdown, but in exchange when things go wonky because macos is weird, we can just restart the VM and it isn't wonky anymore
<srhb> gchristensen: Thanks, interesting!
<ekleog> nice :D
<gchristensen> the config of the mac vm is configurable -- not stuck in stone , in the disk image: https://github.com/grahamc/nixos-org-configurations/blob/macs-on-linux/macs/guest/apply.sh
pie__ has joined #nixos-dev
drakonis has joined #nixos-dev
sir_guy_carleton has joined #nixos-dev
ckauhaus is now known as ckauhaus[afk]
ckauhaus[afk] is now known as ckauhaus
<gchristensen> and good thing, too, they were ... erm ... full
<srhb> gchristensen: Nice! :D
hedning has joined #nixos-dev
<Ericson2314> @Mic92 congrats! This $name-config thing is a rust ideom?
<Ericson2314> Hmm that sucks
<Ericson2314> Yeah Rust is great for cross
<gchristensen> that is so good to hear
<Ericson2314> Cargo OTOH is quite stupid
<gchristensen> I made a rust fstab parser this weekend while I was sick, thinking about the activation script. https://gist.github.com/grahamc/d7dc77dfef592c1dbfc5e562be8fff64
<Ericson2314> It pains me how much less thought went into Cargo
<Ericson2314> The design is both evolved piecemeal and off-the-shelf
<Ericson2314> @gchristiansen oh nice
<samueldr> sick
<Ericson2314> *gchristensen heh I never noticed that spelling before
<Ericson2314> Will be good to purge that Perl
<gchristensen> :)
<gchristensen> (this rust fstab parser, btw, contains all a copy of the tests the util-linux fstab parser has...)
clever has joined #nixos-dev
<globin> Is there any perl-knowledgable person here, who could help me deduplicate the nativeRequires/requires code here: https://github.com/NixOS/nixpkgs/pull/51180/commits/2d0fad757ab26d943c1f0f4edf078c6cd8670861#diff-b5d56e101656e4fa727b963e1f9d80e2
__Sander__ has quit [Quit: Konversation terminated!]
<Mic92> Ericson2314: no. this is more related to c libraries
<Mic92> Ericson2314: rust is quite good for cross. They seperate there build code in build.rs and compile it for the native machine automatically.
<Mic92> globin: did you now also drunk the cross-compile Kool-Aid?
<globin> Mic92: quite a while ago :)
jtojnar has joined #nixos-dev
<yl[m]> LnL: thank you, the Kernel did provide the header I'm looking for.
<yl[m]> LnL: I'm seeing another problem now. `Undefined symbols for architecture x86_64: "_NSDefaultRunLoopMode"` https://gist.github.com/kalbasit/cb1c82393d2861b86762ce41afd18caf
<yl[m]> I found the symbol part of the Foundation framework https://developer.apple.com/documentation/foundation/nsdefaultrunloopmode so I added `darwin.apple_sdk.frameworks.Foundation` but it did not work.
<yl[m]> interesting, I did find the symbol in /nix/store/g2kv3rqzlmwfn4f2s2kgz3hxq51ra6w8-apple-framework-Foundation/Library/Frameworks/Foundation.framework/Headers/NSRunLoop.h this leads to believe it's an ld issue instead. I must be doing something wrong for access gcc/clang for this package
<yl[m]> adding `darwin.cf-private` worked!
<LnL> yl[m]: yeah there are a bunch of symbols missing from our CF
<LnL> I want to look into fixing that, but for now cf-private can be used as a workaround
<yl[m]> cool
<yl[m]> LnL: I have the PR ready, this is my first PR that has specific work for darwin. Can you give me a review on it? https://github.com/NixOS/nixpkgs/pull/51163
<{^_^}> #51163 (by kalbasit, 14 hours ago, open): aminal: init at 0.7.4
<LnL> yl[m]: could you annotate the change similar to how I've been doing, that will make it much easier to remove the references again once that's fixed
<yl[m]> sure, let me do that
<yl[m]> LnL: I pushed the change
orivej has quit [Ping timeout: 246 seconds]
<domenkozar> nixos weekly going out tomorrow, anyone has something to add? :)
<{^_^}> nixos-weekly#71 (by domenkozar, 1 week ago, open): Call for Content: 2018/13
<gchristensen> uhhh ZFS support is coming to the install ISOs
<yl[m]> LnL: for buildFlags you mean `buildFlags = "-ldflags=-X ${goPackagePath}/version.Version=${version}";`?
<yl[m]> (fixing missing quotes)
orivej has joined #nixos-dev
<LnL> yeah
<LnL> oh dear, it's used incorrectly all over the place :/
<yl[m]> yes
<yl[m]> I can 't seem to get buildFlags to work
<yl[m]> it's buildFlagsArray everywhere
<LnL> ^ buildFlags doesn't support flags that include spaces while buildFlagsArray does
<yl[m]> I see
<LnL> buildFlags = [ "MOZ_APP_DISPLAYNAME=Tor Browser" ]; would result in make "MOZ_APP_DISPLAYNAME=Tor" "Browser"
<Ericson2314> __Sander__: hi
<yl[m]> LnL: oh I see why it's broken now thx
<LnL> once structured attrs are a thing we won't need the distinction anymore and we could even enforce the types
<yl[m]> that'd be awesome
<yl[m]> LnL: I updated the PR, PTAL.
<LnL> huh, you actually need a space in this case
<LnL> did that first thing work?
<yl[m]> LnL: both worked for me
<yl[m]> I see the version correctly set
<yl[m]> on the binary
<kgz> domenkozar: could I add a call for sticker ideas? :P
<domenkozar> sure, under RFC
<LnL> yl[m]: I don't really understand, maybe because it's only a single flag?
<yl[m]> LnL: I looked up an ldflags in a Makefile at work and I see this `go install -ldflags "-X github.com/owner/repo/internal/helper.version=$(VERSION)"`
<yl[m]> let me try with a space instead of the first equal
<yl[m]> LnL: no it requires the equal in this case
<yl[m]> LnL: what format did you expect it to be?
<kgz> domenkozar: I commented, feel free to re-word however you want
<LnL> yl[m]: equivalent to preBuild = '' buildFlagsArray=("-ldflags=-X" "foo=bar") '';
<yl[m]> LnL: that does not work actually. I tries to build the "foo=bar" as a go package
<yl[m]> `can't load package: package github.com/liamg/aminal/version.Version=0.7.4: cannot find package "github.com/liamg/aminal/version.Version=0.7.4" in any of: `
<yl[m]> s/I tries/it tries/
<LnL> yeah exactly, so it does result in the correct flags somehow
<LnL> ah I get it, $fooFlags vs "${fooFlagsArray[@]}"
<yl[m]> yea it prints them as elements quoted
<LnL> so it kindof works, but only for a single flag
<yl[m]> yea
<yl[m]> LnL: what's left to get the structured data to builders?
<LnL> thought I was making up stories there for a second :p
<yl[m]> lol
<gchristensen> yl[m]: deciding nix1 is toast
<yl[m]> so `buildFlagsArray = []` gets converted to bash somehow. Maybe you missed something that recently made it possible to have it as an array?
<yl[m]> gchristensen: what do you mean by nix1? did this commit not make it to nix2?
<LnL> first we need to wait for nix 1.11 to be deprecated
<yl[m]> gchristensen: oh
<yl[m]> gchristensen: nvm
* yl[m] needs coffee lol
<LnL> once we do that the stdenv has to be converted to load data from that file instead of using environment variables
<LnL> and that's pretty tricky because 1. bash 2. compatibility
* LnL looks at Ericson2314
<yl[m]> so we have to have a json parser in pure bash?
<gchristensen> please no
<LnL> aah, I didn't even realise that
<yl[m]> gchristensen: we can't have binaries in stdenv can we? If we can, we should just use jq
<yl[m]> the project above uses awk actually so might not be exactly a fit but should be doable to convert that part as well
<cransom> my world got all topsy turvy... a json parser written in shell, but it has a `setup.py`.
<yl[m]> cransom: lol yea
<LnL> btw, I wonder if we should introduce eg. stdenv2 at some point for large changes like structured attrs, strict deps, etc.
<gchristensen> probably yeah
<yl[m]> LnL: I'm not sure about that. That would just increase confusion. stdenv should be smart enough to know if it's nix1 or nix2 or whatever (maybe gated features?)
<LnL> this is separate from nix versions, it's about expected behavour
<yl[m]> so when you would use stdenv2 instead of stdenv1?
<yl[m]> (also is https://github.com/NixOS/nixpkgs/pull/51163 ready to merge on your side LnL?)
<{^_^}> #51163 (by kalbasit, 16 hours ago, open): aminal: init at 0.7.4
<LnL> eg. there are lots of cases in (and outside) of nixpkgs that use incorrect types stdenv arguments like buildFlags
<yl[m]> yea mostly used as an array attribute which we should support transparently
<yl[m]> I'm not opposed to having like a basic stdenv for building initial dependencies for stdenv (like jq or similar to parse the JSON)
* yl[m] lunch break
drakonis has quit [Quit: WeeChat 2.2]
__Sander__ has joined #nixos-dev
<aminechikhaoui> I got a `nix ls-store --store <s3_bucket> <store_path> --json` that takes 45s on my machine but is taking more than 10minutes on another without finishing (still hanging now) but not sure why it's stuck
<aminechikhaoui> here is the stacktrace http://ix.io/1uLh
<aminechikhaoui> running strace shows that it's reading from the fd corresponding to the s3 connection so it's doing something but insanely slow
<aminechikhaoui> a flamegraph which is not very helpful to me https://www.dropbox.com/s/rr5s8u5dwu2zu91/perf-nix-ls-store.svg?dl=0 :D
<aminechikhaoui> gchristensen thanks for the best website everr
<gchristensen> haha :D
<Ericson2314> LnL: I will happily rewrite everything I've ever touched to use structured atttrs
<Ericson2314> LnL: stdenv 2 would also be nice
<Ericson2314> (and a layered stdenv as niksnut , __Sander__ , and many others etc have proposed)
<Ericson2314> also would be nice to do the propagation logic in nix
<Ericson2314> less bash period
<gchristensen> stdenv is so slow already, I'm not sure niksnut would be thrilled about adding more evaluation time to stdenv
<gchristensen> he was saying: "To give some numbers: `nix-instantiate channel:nixos-18.09 -A geeqie --dry-run` takes about 0.4s, but with nixos-16.03 it was < 0.2s. Also half the memory. Or more extreme: Firefox is 0.75s vs 0.20s"
<bennofs[m]> not sure if that's stdenv eval time?
<bennofs[m]> I think all the new bootstrapping logic takes its tool on performance
__Sander__ has quit [Ping timeout: 250 seconds]
<gchristensen> nix-flamegraph :)
<Ericson2314> gchristensen: the bash stuff is also slow
<Ericson2314> which hurts smaller incremental derivations
<Ericson2314> we're approaching an incremental evaluator there
<Ericson2314> I hope to have some good data to motivate changes from bash->nix nixpkgs, and whatever in C++ Nix is needed to support it
<{^_^}> C's karma got increased to 1
<Ericson2314> fundamentally, it's a tradeoff between mass rebuilds and slow evals
<Ericson2314> given the current implementation
__Sander__ has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
<yl[m]> it'd be awesome if we can get the `Reviewed points` as a comment in the PR https://nixos.org/nixpkgs/manual/#reviewing-contributions-sample-package-update
<yl[m]> gchristensen: maybe we ofborg can do that?
<gchristensen> yes-not-yet
<gchristensen> ofborg needs to stop leaving build comments first, which is on its way with the Checks API
<gchristensen> I need a personal task manager so I can focus on tasks and defer other ones... I'm getting badat this.
<yl[m]> gchristensen: I use taskwarrior, it works pretty well
<yl[m]> if you want to share tasks then I guess Trello
worldofpeace has joined #nixos-dev
sir_guy_carleton has quit [Quit: WeeChat 2.2]
__Sander__ has quit [Ping timeout: 246 seconds]
<gchristensen> Mic92: https://github.com/NixOS/nixpkgs/pull/49470 can you take a look-see at this one? I'd like to backport, too.
<{^_^}> #49470 (by grahamc, 4 weeks ago, open): kernel: enable MLX5_CORE_EN when possible
<gchristensen> ("guess what I'm doing")
<gchristensen> :o I want 72 cores...
<gchristensen> what do you have there?
<Mic92> Some university machine I got access today: Intel(R) Xeon(R) Gold 6154
<Mic92> The upside when working at a university
<gchristensen> that thing has $7kUSD of CPU in it :) nice
init_6 has quit []
<Mic92> Yeah many times higher than I would ever spent on a computer.
<gchristensen> certainly for personal use
<gchristensen> for comparison I'm compiling the same kernel with 1 core at 2.40GHz
<Mic92> 1 core? How that?
<gchristensen> a budget AWS instance I happen to be deploying from
<infinisil> Arghh I also need a better machine for my nixbot. I tried to automatically update its nix-index, but that needs more memory than it has (2GB + 2GB swap)
<gchristensen> a shame it takes so much ram :(
<Mic92> is it the evaluation?
<infinisil> Mic92: Yeah
<gchristensen> Mic92: mine is almost finished compiling, so yours must be close :)
<infinisil> Ah
<infinisil> (Specifically nix-index does a `nix-env -qaP --out-path --xml --arg config '{}' --file path/to/nixpkgs`, which is the RAM eater)
pie__ has quit [Remote host closed the connection]
pie___ has joined #nixos-dev