2016-12-16

<clever> your making it behave like a weaker system, as viewed by a subset of the processes
<clever> and i think using cgroups on a more powerful machine, would just recreate that problem
<clever> yeah
<clever> cgroups wont really help that kind of thing
<clever> or i -j 1 and wait 4 times as long for the build
<clever> so i either -j 4, and let 3 gcc's fight over the ram with that 1 thing
<clever> but there is no way to set that
<clever> so i dont want any parallelization on just that step
<clever> there is a step in the glibc build, that takes a crap-ton of ram
<clever> and there is a related problem ive run into on my arm builders
<clever> and i saw a bug a few months ago where a derivation ignored that entirely, and used /proc/cpuinfo to max the system out on its own
<clever> rly: so nix-daemon doesnt know how many cores a given derivation is actualy going to use
<clever> rly: another issue related to what you said, build-cores simply sets $NIX_BUILD_CORES when a build starts, and the stdenv may run make -j $NIX_BUILD_CORES, or it may just run make
<clever> rly: it lines up perfectly
<clever> rly: you probably added it to extraConfig
<clever> each derivation will only use 1 core
<clever> rly: but on the other hand, build-max-jobs=10, build-cores=1 will waste resources when you have a series of derivations that depend on eachother
<clever> rly: the issue, is that build-cores doesnt always help derivations (some ignore it), so if you go purely with build-max-jobs=1, build-cores=10, it will waste resources
<clever> correction, 8, and 8, so 64 things
<clever> rly: so it will run up to 40 things at once
<clever> rly: line 5 says it can do up to 5 derivations at once, and line 16 says each of those can use 8 cores on make
<clever> rly: can you pastebin the contents of /etc/nix/nix.conf?
<clever> rly: thats not the 16.09 channel
<clever> rly: which branch of which git repo are you using?
<clever> rly: nixpkgs=/path/tp/nixpkgs
<clever> rly: can you pastebin the contents of /etc/nix/nix.conf?
<clever> rly: did you use -j on nixos-rebuild?
<clever> gchristensen: so i had to cheat, and only add 1 child to every visible folder
<clever> gchristensen: but pre-adding all children to all visible folders costs 8 seconds
<clever> gchristensen: i have to pre-add children to every visible folder, to make the + option beside them
<clever> gchristensen: and there, much better, 0.29 seconds!
<clever> gchristensen: down to 8 seconds load time
<clever> pie_: the server is QT (c++), the client is javascript
<clever> i should probably defer that until you expand the folder
<clever> looks like a decent chunk of the cpu is spent pre-adding all the nodes to everything
<clever> pie_: yeah
<clever> during which time, the UI is unresponsive, and chrome wont even let you reload the page
<clever> gchristensen: with a minor hicup, generating that entire tree takes 32 seconds at 100% cpu load
<clever> gchristensen: http://imgur.com/a/jZo7m and i have partial docs!

2016-12-15

<clever> and segfault!
<clever> for the config, i can pull it up from the nixos options docs
<clever> or a way to edit it without popups
<clever> next thing i should do is something about all of the dead space on the right
<clever> the 1st is more, here is everything you want, all at once
<clever> gchristensen: it will also have prev/next buttons to cycle thru it
<clever> gchristensen: the 2nd one is more wizard style, you fillout everything on the page, then continue to the next one
<clever> gchristensen: which of these do you think work better? http://imgur.com/a/02dr3 or http://imgur.com/a/AnRhK
<clever> LnL: without any cost of building things
<clever> LnL: nix-instantiate '<nixpkgs/top-level/release.nix>' -A unstable should reproduce the same error hydra has
<clever> LnL: to test for given jobs like unstable
<clever> LnL: oh, and it should be possible to add a few calls to nix-instantiate to the travis stuff
<clever> acowley: and it sounds like that assert will prevent those issues
<clever> acowley: there have been issues in the past, where non-cross stuff recompiles when you add crossSystem
<clever> acowley: i have used crossSystem to get windows builds out of nixpkgs, but i'm not sure how the internals work fully
<clever> :O
<clever> lol
<clever> and the other side wants to ensure crossSystem is null
<clever> LnL: acowley: it looks like one side wants to make sure nothing depends on crossSystem's value
<clever> but it cant assert that it is null, because the assert false; null; is bailing before it can return null, lol
<clever> which asserts that it is null
<clever> it is an argument to that default.nix
<clever> while evaluating anonymous function at /home/clever/apps/nixpkgs/pkgs/stdenv/linux/default.nix:6:1, called from /home/clever/apps/nixpkgs/pkgs/stdenv/linux/make-bootstrap-tools.nix:176:21:
<clever> and that was added in the most recent change to that file
<clever> i think
<clever> something is reading the crossSystem attribute, when it shouldnt be
<clever> 183 crossSystem = assert false; null;
<clever> and because i can reproduce it localy, i can track it down
<clever> and that fails
<clever> error: assertion failed at /home/clever/apps/nixpkgs/pkgs/stdenv/linux/make-bootstrap-tools.nix:183:19
<clever> [clever@amd-nixos:~/apps/nixpkgs]$ nix-build pkgs/top-level/release.nix -A unstable
<clever> LnL: in this case, its pkgs/top-level/release.nix in nixpkgs
<clever> LnL: everything else, is defined in the nix expressions
<clever> LnL: the jobset is defined as a path to a nix file, and a git thing to checkout
<clever> so hydra cant generate a derivation for it
<clever> acowley: unstable is still in the jobset, but the nix expression is throwing an assertion
<clever> acowley: on the evaluation errors tab of the jobset
<clever> assertion failed at [1m/nix/store/r826773xw8qyvi5pi9w8z8zck2sckk5b-git-export/pkgs/stdenv/linux/make-bootstrap-tools.nix[0m:183:19
<clever> in job ‘unstable’:
<clever> gchristensen: it is now running on my laptop!
<clever> lewo: then whatever <nixpkgs> resolves to, will be packaged up and added as the nixos channel at bootup
<clever> lewo: if you add that to your imports, like imports = [ <nixpkgs/nixos/modules/installer/cd-dvd/channel.nix> ];
<clever> lewo: oops, wrong l name
<clever> you want the nixpkgs docs, ctrl+f for packageOverrides
<clever> jazzencat: packages can be changed with packageOverrides, its explained in the docs
<clever> jazzencat: nixos-rebuild will use the channel called nixos by default, but you can force things with -I nixpkgs=something
<clever> fpletz: if this makes its way into the buildInputs, and a v6 job makes v7 binaries, it will fail hard at postinstall
<clever> fpletz: let me pastebin what i had
<clever> fpletz: the arm build of readelf can detect v6 vs v7
<clever> fpletz: i was mainly working on a bash script you can add to the stdenv, that would cause the builds to fail if they contain the wrong opcodes
<clever> LnL: i plan to nixos it
<clever> LnL: i have an arm based router on the way thats open-source-ish
<clever> and it detects a v7 capable cpu
<clever> yep
<clever> fpletz: the only reason i noticed, is because i had a mix of v6 and v7 in my cluster, the v7 would make bad v6 builds, then the v6 would barf on them
<clever> fpletz: it might have been the hand-rolled assembly in openssl, that would ignore -march
<clever> fpletz: perl and openssl where affected
<clever> fpletz: i didnt look into that side of it in much depth, and only a few packages did it, not all
<clever> gchristensen: related, armv6 builds on an armv7 arent pure, the compiler detects a v7 host and makes borked builds
<clever> Baughn: sent some info in PM
<clever> Baughn: yeah, let me dig thru this more
<clever> Baughn: but there is also the odd delay between each of these
<clever> 0.49 seconds for this process to run
<clever> 1481823394.129914 +++ exited with 0 +++
<clever> 1481823393.632137 getpid() = 1 <0.000019>
<clever> this looks like the nix-store child that is creating the sandbox
<clever> -rw-r--r-- 1 root root 518378 Dec 15 13:34 logfile.8381
<clever> [root@amd-nixos:~/Baughn]# strace -p 6288 -ff -o logfile -T
<clever> reading it over
<clever> 91 forks for a single urlencode, more logging this time
<clever> Baughn: 180 forks just to build 2 of the things
<clever> let me strace on this end
<clever> its got a pretty big delay between each derivation
<clever> just applied that change
<clever> Baughn: the instantiate took 1.9 seconds on my zfs system with no changes to the config
<clever> Baughn: https://github.com/cleverca22/nix-tests/blob/master/efi-notes.md and on to the next issue!
<clever> got it mostly figured out
<clever> ok, done messing with efi in vbox
<clever> heh
<clever> ah, ive seen that pack in the listings, didnt know it used nix
<clever> can you gist the nix expressions so i can profile them on this end?
<clever> ow
<clever> for no overhead, and assuming the above is an average, thats 4 seconds
<clever> how many URL's are you trying to convert?
<clever> there is also a hefty amount of overhead before that gets ran, which may be to blame
<clever> ..." | time python ...
<clever> Baughn: simplest thing i can think of is to put the "time" command at the start, before the python, and see how fast it is
<clever> Baughn: so now the rpm of your disk comes into play
<clever> Baughn: nix also wants to sync things to disk between every build
<clever> Baughn: but its still a bit costly, several forks per conversion
<clever> Baughn: you can, and as long as python doesnt change, it could cache these conversions
<clever> Baughn: if you can find a nix expression that returns the ascii code for a given character, you could potentialy use map to split a string up, and then escape anything outside a given range
<clever> Baughn: if you can stop the inputs it depends on from changing
<clever> Baughn: you could split that up, and reuse the forest every time
<clever> yeah, i believe the evaluator will just block waiting for that derivation
<clever> import from derivation will cause it to build things at eval time
<clever> once that step is done, it no longer has to touch any nix expressions
<clever> Baughn: that will fully evaluate everything bar references, without building anything
<clever> Baughn: you can test things by just manualy running one of the steps, nix-instantiate foo.nix -A bar
<clever> Baughn: i had to boot with efi off, then do an install with boot.loader.grub.efiInstallAsRemovable = true;
<clever> Baughn: aha, if virtualbox cant find an efi system partition, it fails so hard that the video output doesnt even come up
<clever> Baughn: the second step can be made parallel, and happens after the entire nix expression has been evaluated
<clever> Baughn: then you have nix-store -r, which turns each .drv into an output
<clever> Baughn: the first step is nix-instantiate, its single threaded, and it turns nix expressions into .drv files
<clever> Baughn: there are 2 main steps to building things in nix
<clever> Baughn: i believe nix is entirely single threaded
<clever> Baughn: trying something different in vbox first
<clever> Baughn: id prefer qemu if that option worked
<clever> Baughn: not currently installed
<clever> Baughn: if i flip efi on in virtualbox, it just fails to launch entirely, it cant fallback to legacy pxe booting
<clever> jazzencat: it should always use the build from the last 'nixos-rebuild switch'
<clever> Baughn: it would help if i was able to boot efi under a vm and experiment with it more, just to eliminate user-error from the equation
<clever> always had to enable the CSM and use legacy booting
<clever> and ive had bad luck with efi, every install has failed
<clever> and it has more options
<clever> mpickering: i'm guessing somebody just forgot to put options on it, ive just been sticking to grub because i know how it works more
<clever> mpickering: that sounds like the simplest option
<clever> you will either want to chainload nixos's systemd-boot from the ubuntu EFI, or switch to grub
<clever> so there is no way to configure it further
<clever> and systemd-boot has 1 option, enable, nothing else
<clever> so its ignoring all of the entries you put into boot.loader.grub
<clever> yeah, that would explain it
<clever> mpickering: can you pastebin the configuration.nix?
<clever> so you cant just paste nixos stuff into the ubuntu files
<clever> nixos needs to update the bootloader config every time you do "nixos-rebuild switch"
<clever> that will break nixos
<clever> mpickering: can you pastebin the configuration.nix?
<clever> mpickering: which bootloader are you using?
<clever> mpickering: now look in /boot/grub/grub.cfg and see if they are added
<clever> mpickering: did you run nixos-rebuild switch?
<clever> and the old jobs will keep using old expressions
<clever> gchristensen: changes in the nix expression should cause the job to restart on its own
<clever> mpickering: and the entire 8-13 section should be copy/pasted from debian's grub config
<clever> mpickering: some parts like line 6 will need tobe modified to suit your setup
<clever> mpickering: in configuration.nix
<clever> but it hasnt been used in nearly a year
<clever> mpickering: http://pastebin.com/FSG9pBjG this is how i was booting win7 on my machine
<clever> mpickering: one sec
<clever> i cant remember exactly, but i think this XFS was last mounted on a sparc machine, which has since died
<clever> [ 239.210651] XFS (sdc): dirty log written in incompatible format - can't recover
<clever> and now ive run into XFS problems
<clever> mpickering: nixos should probably be last, it needs to control grub to boot correctly
<clever> randalltux: its probably something weird with the dual-gpu thing then, havent played with that much yet
<clever> randalltux: what attribute did you set within configuration.nix?
<clever> randalltux: can you pastebin the configuration.nix?
<clever> rardiol and randalltux
<clever> randalltux: how did you install the gpu drivers?
<clever> rardiol: oh oops, too many ra names in here
<clever> yes
<clever> rardiol: how did you try to install it?
<clever> rardiol: and also stop wpa_supplicant before, and start it after
<clever> rardiol: you could rmmod the driver and then reload it with modprobe
<clever> rardiol: yes
<clever> arianvp2_: should be as simple as replacing the first line with { writeText, oh-my-zsh, vim }:
<clever> rardiol: id start by checkign journalctl -f
<clever> it has to be simp_le
<clever> yeah
<clever> arianvp2_: and then model it like every other package
<clever> arianvp2_: in users.nix, you want to use pkgs.callPackage ./zsh-config.nix
<clever> arianvp2_: that second nixpkgs will obey the ~/.nixpkgs/config.nix of whatever user ran nixops
<clever> arianvp2_: there is also an unrelated problem in your zsh-config, you import <nixpkgs> again, so it will ignore all config set in nixpkgs.config
<clever> arianvp2_: that currently says simple_le, which isnt right
<clever> arianvp2_: can you pastebin the whole config?
<clever> LnL: in my case, i have an old kdenlive in ~/.nix-profile, and the new kdenlive in nix-shell gets upset about incompatible kde stuff in the env
<clever> ixxie: it can
<clever> LnL: kdenlive is the most recent example i came across
<clever> LnL: some naughty packages use propagated-user-env packages and refuse to work under nix-shell
<clever> ixxie: ah, what you have would work then
<clever> ixxie: no need to change any system settings
<clever> ixxie: if you nix-shell -A linuxPkgs.arcane-chat in this folder, you get all of the stuff you need to develop with it
<clever> ixxie: i generaly put all of the shared stuff into a default.nix, one sec
<clever> did you read the new docs?
<clever> wait, thats the xml for the docs
<clever> arianvp2_: an xml file nearby got changed, to put the challenge as a subdirectory
<clever> ah
<clever> arianvp2_: can you pastebin that service file?
<clever> arianvp2_: is there a service file with the same name as the timer?
<clever> if its ignoring the typo i gave above, then nothing is using this simp_le attribute
<clever> lassulus: good point, it is simp_le
<clever> arianvp2_: what acme related files do you see in /etc/systemd/system/ ?
<clever> ah
<clever> nixpkgs.config.packageOverrides = pkgs: { simp_le = pkgs.callPackage ../nixpkgs/pkgs/tools/admin/simple_le {}; };
<clever> no, change the override to simp_le
<clever> try changing it over
<clever> the module is using simp_le
<clever> path = [ pkgs.simp_le ];
<clever> arianvp2_: nixpkgs.config.packageOverrides = pkgs: { simp_le = pkgs.callPackage ../nixpkgs/pkgs/tools/admin/simple_le {}; };
<clever> arianvp2_: oh, and the override has to be called simp_le
<clever> arianvp2_: can you check the service file in /etc/systemd/system and confirm its doing this?
<clever> arianvp2_: the nixos module is probably what makes the directory, and modules are harder to override
<clever> arianvp2_: nixpkgs.config.packageOverrides = pkgs: { simple_le = pkgs.callPackage ../nixpkgs/pkgs/tools/admin/simple_le {}; };
<clever> arianvp2_: it has to go inside a packageOverride
<clever> jazzencat: i havent used connman before, so all i can do is help setup bare wpa_supplicant
<clever> arianvp2_: ah, then its a bit more complex, id (import ./nixpkgs> { config = {}; }).pythonPackages.foo
<clever> arianvp2_: if you give it the path to a directory, it will load the default.nix within it
<clever> ive got a measly 3 on my radar, and 2 of them are mine
<clever> heh, you have nearly 10 times as many networks as me
<clever> and thats a lot of networks near you
<clever> jazzencat: yep, the drivers are fully working
<clever> that also works
<clever> s/ti/to/
<clever> arianvp2_: you can copy the foo dir to where your nixops stuff is, and use a relative path, callPackage ./foo {};
<clever> arianvp2_: and ti make things more stable (things will change if you checkout something else in nixpkgs)
<clever> arianvp2_: packageOverrides make that easy, foo = pkgs.callPackage /home/clever/nixpkgs/pkgs/bar/foo {};
<clever> jazzencat: what does "iwlist scan" give?
<clever> jazzencat: and it has created wlp3s0
<clever> jazzencat: looks like the rt2800pci driver has already claimed the device
<clever> jazzencat: can you pastebin the output of "lspci -v" and "ip link"
<clever> nix-shell -p pciutils
<clever> jazzencat: lspci is in the pciutils package, and nixos uses dhcpcd most of the time
<clever> LnL: mostly going on how other nix tools work, havent had a chance to play with nixops yet
<clever> though if your doing -I $HOME/.nix-defexpr/channels '<nixos>' you might as well just put in an absolute path
<clever> with this, it should also find channels by name
<clever> export NIX_PATH=nixpkgs=/home/arian/.nix-defexpr/channels/nixpkgs:/home/arian/.nix-defexpr/channels
<clever> then it can only ever find the channel called nixpkgs
<clever> and in $NIX_PATH ?
<clever> arianvp2_: what exactly is it saying in nix-channel --list?

2016-12-14

<clever> Mic92: gchristensen: http://imgur.com/a/xM4vS
<clever> yeah
<clever> Mic92: could be used, but then your basicaly running a ~300mb executable
<clever> Mic92: gchristensen: http://imgur.com/a/EJVcK
<clever> Mic92: in my case, i reference it directly via nix-build with -A config.system.build.kexec_tarball
<clever> Mic92: its mainly used for internal nixos things, and you then reference it from elsewhere
<clever> Mic92: the entire system.build section lacks a type, so the module framework just lets you go wild and do whatever you want
<clever> kier: sudo likes to wipe the env, and nixos-rebuild requires a nixos-config entry in the path, you left it out, and it didnt fail, so sudo must have blocked things
<clever> kier: try with -I instead
<clever> close enough
<clever> oh newline, nix-instantiate --eval -E 'with import <nixpkgs> {}; lib.concatMapStringsSep "\n" (x: "${x}") [ zlib hello ]'
<clever> gchristensen: done!
<clever> "/nix/store/9jwwgvd5cc2g3cyq3bwdcjb71znw9x05-zlib-1.2.8 /nix/store/7663fv58jvwww5km6ldrm3qi1rfz2c9z-hello-2.10"
<clever> [clever@amd-nixos:~/apps]$ nix-instantiate --eval -E 'with import <nixpkgs> {}; lib.concatMapStringsSep " " (x: "${x}") [ zlib hello ]'
<clever> gchristensen: use map inside a nix expression
<clever> gchristensen: nix-instantiate --eval -E 'with import <nixpkgs> {}; "${zlib.out}"'
<clever> gchristensen: nix-instantiate
<clever> Mic92: what about just nix-build '<nixpkgs>' -A zlib --no-out-link ?
<clever> Mic92: i can implement that in my automated stuff
<clever> Mic92: sfdisk has similar, if you omit the size, it just fills the area up
<clever> Mic92: and it can handle types
<clever> Mic92: ive added mode nodes to the tree, and it can now automaticaly add things
<clever> kmicu: how does this look as a basic config editor? http://imgur.com/a/GqdJL
<clever> kexec<-systemd<-util-linux<-world
<clever> heh, yeah
<clever> Mic92: one idea is that on startup, it will query an http url thats programed into it, like http://example.com/online?id=foo, which will then either dump a config file back, or pass the IP on for a user to control
<clever> Mic92: i do plan to have it serialize all of this config, so it can be ran in a more automated fasion
<clever> but i didnt want to bother with trying to support all of the variations
<clever> Mic92: the "proper" way to do it, is to have the init system (systemd or the older sysinit stuff) to a full shutdown, and then "kexec -e" rather then reboot/halt/poweroff
<clever> Mic92: and here is a related project i have been working on: http://imgur.com/a/GqdJL
<clever> Mic92: not sure, and it cant hurt to throw an extra sync in