2018-07-12

<clever> infinisil: it will cache the result of `import <nixpkgs>` but not the result of calling that with {}
<clever> infinisil: it will use roughly twice the ram and take roughly twice as long, assuming its using identical parts of a and b
<clever> tobiasBora: if your channel is called nixos, then you need nix-env -iA nixos.raspberrypi-tools instead
<clever> logzet: my laptop has 3 proper usb ports, and the webcam in the display is also usb based, lsusb only claims 2 root hubs
<clever> tobiasBora: the vcgencmd binary should allow reading the current clocks
<clever> tobiasBora: i dont think it causes any underclocking
<clever> Dezgeg: there are some overclock flags in config.txt that mess with the uart clocks, but linux isnt aware of that, so the uart can run slow or fast
<clever> 115200 is fairly normal
<clever> tobiasBora: thats still managed by config.txt in the same way
<clever> its already tied to the bluetooth, the error is because its used by both bluetooth and console=ttyAMA0,115200n8
<clever> try removing the ttyAMA0 entry
<clever> tobiasBora: you are using all the serial ports as serial consoles, so the bluetooth one is being messed with
<clever> console=ttyS0,115200 without any other console='s, is enough to get a shell when under qemu
<clever> i would expect the above to at least print something, due to the loglevel=7
<clever> tobiasBora: for each console= on the cmdline, the kernel will print dmesg to it, and the last one that appears will be /dev/console, which is used for a login prompt if systemd enables that
<clever> the wifi chip in the rpi can transmit arbitrary IQ signals on 2.4 or 5ghz
<clever> i think devicetree stuff in config.txt can change the mappings some
<clever> Dezgeg: one of the uart's got repurposed for the bluetooth
<clever> tobiasBora: https://github.com/cleverca22/nixos-configs/blob/master/qemu.nix and then just set qemu-user.arm = true; in configuration.nix and you can magically run any arm binary, then just nixos-enter like you normally would
<clever> tobiasBora: if you enable qemu-user, you can also just chroot into the image and change whatever you want and nixos-rebuild boot
<clever> benley: yeah, sessionCommands is ran shortly before it sources .xprofile, check lines 122-127
<clever> benley: oops, wrong name tab-completed
<clever> makefu: that is the script responsible for launching the login prompt and starting the session
<clever> thoughtpolice: for example, if i import 2 different files, and both happen to contain { a = 5; }, how hard would it be to ensure both parse to the same Value* ?
<clever> thoughtpolice: the tricky part is ensuring every creation of a value does that, and updating the references
<clever> thoughtpolice: a secondary idea i had, but it would be a decent amount of work, is to store every single Value in a giant hash table, and after a thunk has been forced, check the hash table to see if you just duplicated the result, and optionally merge
<clever> thoughtpolice: rather then re-computing a giant attrset and duplicating its values (50 times), it can just reuse it
<clever> thoughtpolice: the memoise branch was able to shave 3.5gig off one of the evals ive been fighting
<clever> thoughtpolice: but it has to perform a deep equality comparison between lambdas, and that results in infinite recursion in the stdenv
<clever> thoughtpolice: the change turned out to not work right anyways, i was trying to modify the import function to always call memoise on everything
<clever> thoughtpolice: it was simpler to reuse the code for parsing a string (used by -E) and take the Expr* it made
<clever> thoughtpolice: i tried making some "simple" changes with ExprVar and ExprApp, and it just segfaulted endlessly
<clever> ah
<clever> tazjin: pkgs.fetchFromGithub ?
<clever> tobiasBora: yeah, thats a nixos option, seperate from nixpkgs and nix
<clever> tobiasBora: that is a list of strings containing ssh pubkeys, and it will add them when you re-deploy
<clever> tobiasBora: users.extraUsers.clever.openssh.authorizedKeys.keys = with keys; [ clever.amd keys.ramboot clever.laptop ];
<clever> gchristensen: maybe it could be a bit more strict, so <space> only finds a directory with the exact name space
<clever> nschoe: nix-shell has a non-interactive build of bash in the inputs by default
<clever> > add 1 2
<clever> logzet: you could probably even put that module into the imports section of a laptop booting from the liveusb, and nixos-rebuild switch
<clever> logzet: yeah, i had originally made that config to run on a laptop, to spread the infection that is nixos :P
<clever> logzet: and that pc needs 2 network interfaces
<clever> logzet: this module turns a laptop into a router+netboot server, treating the wifi as WAN and the ethernet as LAN
<clever> logzet: or look into network booting over ethernet
<clever> logzet: put the hdd into another machine, install, then move it, use the removable efi option i mentioned for the usb stick
<clever> logzet: ah, that could explain why nixos also cant boot
<clever> the bot also can do it
<clever> > lib.optional true "foo" "foo"
<clever> > optional true "foo" "foo"
<clever> Enzime: that will attempt to call "foo" on "foo", and fail
<clever> Enzime: no
<clever> Enzime: vs optionals true [ 2 3 ]
<clever> Enzime: there are examples where i linked, optional true "foo"
<clever> Enzime: the first argument to both must be a boolean
<clever> Enzime: optionals doesnt wrap the arg in a list, so you can pass it a list of several optional things to include
<clever> even pid 5 is missing!
<clever> and i see gaps as low as 20
<clever> root 234 0.0 0.0 0 0 ? I< Jul05 1:31 [kworker/0:1H]
<clever> root 164 0.0 0.0 0 0 ? I< Jul05 0:00 [scsi_tmf_9]
<clever> the gap is the result of a userland process that was started, prior to the kernel thread, implying the kernel thread could be optional
<clever> a major hint with them, is to look for a gap
<clever> some of them are zfs related
<clever> samueldr: i see over 110 kernel threads
<clever> samueldr: `ps aux | head -n20` and good luck stopping all of those from starting in the first place :P
<clever> samueldr: even with that cheating, it wont be 1 :P
<clever> samueldr: but you will still have kernel threads from drivers, which appear as unique proccesses
<clever> samueldr: this project gives you a haskell based "shell" as pid 1, where you can then implement `ps aux` as a forkIO, no new processes
<clever> and only in very very special cases can you see exactly 1 process
<clever> so its literally not a running system until after pid 1 has been made
<clever> that only occurs after pid 1 has been created :P
<clever> samueldr: line 1068, system_state = SYSTEM_RUNNING;
<clever> samueldr: in that state, ps cant be running, and just the act of starting ps will create 1 process
<clever> samueldr: prior to line 408, pid 1 doesnt exist, and there are zero running processes
<clever> samueldr: which is just an execve, that replaces the current context
<clever> samueldr: and line 1073 is where it tries to run the initrd init process
<clever> samueldr: kernel_init is defined on line 1057
<clever> prior to that, only pid 0 exists, which is invisible to nearly all tools
<clever> samueldr: line 408 is responsible for creating pid 1
<clever> samueldr: one min
<clever> samueldr: 0 :P
<clever> i dont think any of my 2.6 machines are still running
<clever> Linux c2d 3.8.13-gentoo #3 SMP Thu Dec 15 10:33:05 AST 2016 x86_64 Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz GenuineIntel GNU/Linux
<clever> 658
<clever> samueldr: `ps aux` outputs about 205kb
<clever> samueldr: i suspect its an actual buffer overflow exploit, my process list is too big for a fixed-size buffer
<clever> samueldr: oh, i have chrome open
<clever> samueldr: your version fails for me
<clever> iqubic: the initrd can still use busybox without having it in systemPackages
<clever> samueldr: ive got /nix/store/5r3alnn6v25fi7nav1sxww76kd0sr3i6-busybox-1.28.4
<clever> iqubic: its used by the initrd
<clever> `nix run nixpkgs.busybox -c valgrind top` reveals that its caused by a call to vfprintf
<clever> iqubic: thats not a stack overflow
<clever> *** buffer overflow detected ***: top terminated
<clever> Aborted (core dumped)
<clever> samueldr: `ps aux` is missing half the expected columns, `top` segfaults, and `mount` fails to mount things
<clever> commonly, busybox is wrongly given as an answer, because it includes nearly every program
<clever> but it shouldnt be installed in a normal system
<clever> its used by the initrd, so its required to boot
<clever> but if you saved your configuration.nix file, you would have recreated the identical system, with identical problems, upon reinstalling
<clever> as for why reinstalling typically fixes other distros, everything is lost, including busybox, and the problem just vanishes :P
<clever> we have identified the problem
<clever> no real need
<clever> it has dumber versions of mount, top, and ps
<clever> iqubic: remove busybox from your systemPackages
<clever> iqubic: yeah, thats the cause of all of your problems
<clever> iqubic: ls -lh /run/current-system/sw/bin/{ps,top,mount}
<clever> iqubic: try which instead
<clever> iqubic: and even `type mount`!
<clever> iqubic: and `type top`
<clever> iqubic: what does `type ps` return?
<clever> iqubic: busybox, that explains a lot of things
<clever> iqubic: fsck /dev/sdb
<clever> iqubic: pastebin?
<clever> iqubic: file -s /dev/sdb
<clever> iqubic: anything new in dmesg|tail ?
<clever> iqubic: mount /dev/sdb /home/iqubic/usb -v
<clever> iqubic: dmesg | tail
<clever> iqubic: blkid /dev/sdb
<clever> iqubic: fdisk -l /dev/sdb
<clever> iqubic: what does `ls -l /dev/sdb*` say?
<clever> iqubic: read the README file inside the tar
<clever> iqubic: the memtest image can only be put on the usb when it is NOT mounted
<clever> iqubic: https://xkcd.com/1168/
<clever> sir_guy_carleton: those are available by default, but nix-shell --pure will exclude them
<clever> iqubic: thats a tar file, so you have to unpack the tar file first
<clever> sir_guy_carleton: yes
<clever> yes
<clever> iqubic: check the memtest pages i linked, there is a download page
<clever> nix-shell is just a shell
<clever> nothing is stopping you from running vim inside nix-shell
<clever> iqubic: run the vim inside the shell
<clever> iqubic: yes
<clever> yeah
<clever> iqubic: give you a shell with the things defined in shell.nix
<clever> iqubic: shell.nix is just the default file that nix-shell reads when given no args
<clever> and if you need to use python3, use nix-shell -p 'python3.withPackages (ps: [ ps.tkinter ])'
<clever> 2018-07-11 23:53:17 < clever> iqubic: thats why you dont install python or libraries, thats why you put that expression into a shell.nix for each project
<clever> because you installed python, you are able to run the installed one, which lacks tkinter
<clever> dje4321_: run python, not python3
<clever> iqubic: thats why you dont install python or libraries, thats why you put that expression into a shell.nix for each project
<clever> iqubic: that expression automates it for you, and keeps things pure
<clever> iqubic: it creates a python that has tkinter in PYTHONPATH
<clever> dje4321_: nix-shell -p 'python.withPackages (ps: [ ps.tkinter ])'
<clever> dje4321_: you need to use the python binary made by `python.withPackages (ps: [ ps.tkinter ])` instead
<clever> samueldr: for that, you want to look at the `nix-store --query --tree` of the .drv files
<clever> if top is breaking for weird reasons, it cant hurt to run a memtest
<clever> yes
<clever> yes
<clever> yes
<clever> 2018-07-11 23:37:09 < clever> iqubic: wait a few hours
<clever> memtest just tests the mem
<clever> that would be called memfix
<clever> no
<clever> iqubic: i suspect your memory is bad
<clever> iqubic: wait a few hours
<clever> iqubic: then boot from it
<clever> correct
<clever> iqubic: the memtest binary doesnt boot with efi
<clever> iqubic: yes
<clever> iqubic: the usb image works without grub
<clever> iqubic: or from here http://www.memtest.org/
<clever> iqubic: download a disk image and flash it to a usb stick
<clever> infinisil: thats also why you where missing all of the functions
<clever> infinisil: which is wrong
<clever> infinisil: -p creates a new derivation, that depends on the env
<clever> infinisil: what args did you run it with?
<clever> infinisil: the .env derivation only works in nix-shell
<clever> iqubic: enable it, nixos-rebuild, and then reboot and select memtest
<clever> infinisil: you need to use the .env attribute of the derivation
<clever> iqubic: is memtest enabled?
<clever> iqubic: then your store is not corrupt
<clever> the nix-store command also has options to fix them
<clever> no
<clever> wait for the verify to finish and see if it found anything
<clever> though zfs should have also caught it
<clever> the last cmd i gave is checking for corruption
<clever> something may be corrupt from an improper shutdown
<clever> iqubic: that shouldnt break anything
<clever> it has to read every single file in /nix/store/
<clever> thats normal
<clever> iqubic: nix-store --verify --check-contents
<clever> iqubic: just use top
<clever> iqubic: half the columns are just missing for some strange reason
<clever> iqubic: this line
<clever> PID USER TIME COMMAND
<clever> iqubic: oh, strange, i can see ps aux, and sort -nk5 in the ps listing
<clever> you can also just run `top` and hit `M` to sort by memory usage
<clever> iqubic: the process with the highest RSS value
<clever> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
<clever> there are several columns missing
<clever> iqubic: i said to run `ps aux | sort -nk6`
<clever> iqubic: it may need 3gig of ram, check the ps output again with the cmd i gave above
<clever> or do the "windows thing" and reboot :P
<clever> iqubic: probably a memory leak within firefox, try just closing it and then re-running the nix cmd
<clever> gchristensen: id say that its a bug for https://github.com/NixOS/nixpkgs/blob/release-18.03/nixos/modules/programs/dconf.nix to not configure dbus
<clever> infinisil: so i threw another 64gig swap partition at it
<clever> infinisil: i sometimes ran out of ram with 32gig of ram and 32git of swap
<clever> iqubic: ps aux | sort -nk6
<clever> iqubic: you have 8gig of ram, just close a few programs!
<clever> iqubic: free -m ; df -h /boot/
<clever> iqubic: how much total ram?
<clever> iqubic: not even /boot?
<clever> iqubic: is there a non-zfs FS?, how much ram does the machine have?
<clever> iqubic: thats what i expected, i think your only choice is to add some swap temporarily, and dont put the swap on zfs
<clever> iqubic: what path is it downloading when it fails?
<clever> iqubic: if its nix-daemon that runs out, --cores isnt at play
<clever> gchristensen: https://nixos.org/nixos/options.html#dbus.packages
<clever> iqubic: i have 95gig of swap..., but its not on any zfs volume
<clever> Dezgeg: yep
<clever> > unrar.meta.license
<clever> > unrar.meta
<clever> probably just pkgs.unrar
<clever> ah
<clever> ,locate unrar
<clever> ,locate bin/unrar
<clever> iqubic: unrar x
<clever> gchristensen: xfce has an option in its settings gui for that, dont know about i3
<clever> gchristensen: the gconf daemon has to be running
<clever> pie_: shell.nix
<clever> thats what mine are at
<clever> GeometryHeight=853
<clever> GeometryWidth=1073
<clever> then my xfce directions wont help much
<clever> gchristensen: what desktop manager are you using?
<clever> i think the electron side is the broken one
<clever> chromium
<clever> one anoying thing ive noticed, is that chrome has a good gtk file diaglog, with previews for images, but electron based apps have the identical dialog, without previews
<clever> gchristensen: alt and left drag?
<clever> gchristensen: try alt+space ?
<clever> 2018-07-11 20:18:18 < infinisil> static languages with compilers wouldn't have allowed this
<clever> infinisil: it would also resolve symbols sooner on, and become a less dynamicaly typed language
<clever> infinisil: yeah
<clever> infinisil: so i can just edit configuration.nix, re-compile it, then run it to produce the .drv files
<clever> infinisil: i dont mean a programming language that looks like nix, but turning nixpkgs into a dynamic library, and then turning configuration.nix into an actual executable
<clever> infinisil: i'm wondering, could hnix and http://www.stephendiehl.com/llvm/ be jammed together, to translate nix files into ELF binaries?
<clever> infinisil: the shear volume of data, and the need to perform api queries and deal with github rate limiting to get further data
<clever> infinisil: i think the mongodb is just to store the raw json of the events, and then another program goes over that, and makes mysql rows
<clever> description The Impact of Continuous Integration on Other Software Development Practices: A Large-Scale Empirical Study
<clever> infinisil: https://cmustrudel.github.io/papers/ase17ci.pdf is the paper that used it
<clever> infinisil: that has all github activity, going back to 2012, totalling 4tb of json in mongodb, and 1.5 billion rows of mysql data
<clever> infinisil: aha, http://ghtorrent.org/
<clever> infinisil: actually, now that i think about it, the talk wasnt about the database, but it used data from that database
<clever> the data that exists outside of git
<clever> infinisil: its more of an index of every repo, comment, branch change, pr, and other such events
<clever> infinisil: i saw a talk at ase2017 about a github scrapper database
<clever> what was it called...
<clever> infinisil: one min
<clever> bgamari: ive memorized the patterns of URL's github uses, and just type out what gchristensen linked above :P
<clever> bgamari: that is searching for a commit with that hash in the commit msg
<clever> bgamari: this is the old name of the attribute
<clever> [root@amd-nixos:~]# nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion
<clever> trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
<clever> bgamari: in my case, its a channel managed by root called nixos
<clever> /nix/var/nix/profiles/per-user/root/channels/nixos
<clever> bgamari: you can also use `nix-instantiate --find-file nixpkgs` to see where nix thinks <nixpkgs> is
<clever> "18.09pre144959.be1461fc0ab"
<clever> [root@amd-nixos:~]# nix-instantiate --eval '<nixpkgs>' -A lib.version
<clever> bgamari: yes, one sec
<clever> infinisil: yeah
<clever> infinisil: you must remove it from your files
<clever> infinisil: nixops defines the option itself, so you cant rename it in your own files
<clever> joehh: what if you run the failing nix command with -vvvvvv, what does it output?
<clever> joehh: i dont expect nix to prefer v6 in that case, and the test script only hung because the last step forced v6 usage
<clever> joehh: does `ip addr` show a v6 address not starting with fe80?
<clever> joehh: and nix is prefering v6 when it shouldnt
<clever> joehh: it looks like you have ipv6 enabled, but not working
<clever> CouperinGTRameau: you have to make the derivation with haskellPackages.mkDerivation instead of stdenv
<clever> joehh: what is the issue you are seeing?
<clever> just edit configuration.nix on a different machine and nixos-rebuild
<clever> you can more easily customize the initrd if it still fails

2018-07-11

<clever> s/oot/boot/
<clever> logzet: then you will have a non-live usb stick, that has a fully self-contained nixos that can freely be modified with nixos-rebuild, and have all changes persist
<clever> logzet: partition and format the usb stick as-if you where doing a normal UEFI install, then mount the USB stick partitions under /mnt/ and generate some config, then set oot.loader.grub.efiInstallAsRemovable=true; and finish the nixos-install
<clever> logzet: do you have nixos on anything else?
<clever> to get anything more, you would need a proper lspci, which is tricky at this stage
<clever> logzet: if you check lsmod, does it show ehci and xhci modules being loaded?
<clever> logzet: they all look pretty standard
<clever> 1e31 7 Series/C210 Series Chipset Family USB xHCI Host Controller
<clever> 1e26 7 Series/C216 Chipset Family USB Enhanced Host Controller #1
<clever> 1e2d 7 Series/C216 Chipset Family USB Enhanced Host Controller #2
<clever> logzet: it sounds like your missing usb drivers, confirming what devices you have
<clever> less /nix/store/m2bwx92zkkjm1iwh1as2p23ih4gv0y1a-pciutils-3.5.6/share/pci.ids
<clever> lol, 8086 is the VID for intel!
<clever> logzet: what are the vid:pid pairs after each one?
<clever> logzet: hmmm, and on closer inspection, 0c03 is 1.0, 2.0, and 3.0
<clever> one is busybox lspci, the other is `lspci -nn` from pciutils
<clever> 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31)
<clever> yeah
<clever> like: 00:14.0 Class 0c03: 8086:a12f
<clever> logzet: do you see a class of 0c03?