2017-02-27

<clever> srhb: "strace -s 5000 -ff -o logfile xdg-settings get default-web-browser"
<clever> srhb: strace is of value for this
<clever> Heffalump: fixed-output derivations can use any hash, md5, sha1, sha256, sha512, md5 is already being phased out, and sha1 can easily be removed as well
<clever> yeah
<clever> Heffalump: ive heard something about the attack working by appending large amounts of specialy crafted junk, which changes the size in that header, causing the state of the sha to become scrambled
<clever> which destroys any pre-existing collisions
<clever> git does its object storage and de-dup after prefixing the files with the above mentioned header
<clever> Havvy: i believe they used svn
<clever> tilpner, Havvy: but of note with git, all files have a header of "blob %d\0" prepended to them, including the size of the data
<clever> maybe 512
<clever> Havvy: nix will package the entire directory tree up as a NAR, then hash that with sha256 i think
<clever> Havvy: i think the bigger issue is going to be the git nixpkgs is stored on

2017-02-26

<clever> sphalerite: blanking out the start of the drive silences all warnings, so they wont ask you to confirm things
<clever> sphalerite: some tools like mkfs.ext4 will warn you that your trying to format an existing filesystem
<clever> that kind of thing can be hard to debug, because the debug tools are also broken
<clever> let me see
<clever> yeah
<clever> sphalerite: i fixed that bug, but i dont think it hit nixos-16.09
<clever> sphalerite: oh yeah, you need to use the nixpkgs from nixos-unstable
<clever> gnome2.scrollkeeper
<clever> where did you set that, can you pastebin your nix file?
<clever> unlmtd[m]: unquoted relative path should work
<clever> ekleog: i had to include these modules to make not-os boot under qemu
<clever> and it was never listing the -drive you gave
<clever> that may be an entirely unrelated drive

2017-02-25

<clever> and in this case, it overwrites the db on each boot, so any changes the vm may have done to the store cease to be valid
<clever> ekleog: this is what loads it on startup
<clever> ekleog: which the vm will import on bootup, to convince nix that the things in /nix/store are valid
<clever> ekleog: the reginfo file is just a partial dump of nix's db.sqlite
<clever> mount --bind is similar to a hardlink
<clever> ekleog: the sandboxes for nix builds work in a similar way
<clever> ekleog: if you have root on the host, you could use mount --bind to copy parts of the store to an area like /tmp/vm1/nix/store/ and then expose that
<clever> ekleog: this allows the guest to mount the host /nix/store without having to use any block devices or filesystems
<clever> -virtfs local,path=/nix/store,security_model=none,mount_tag=store \
<clever> ekleog: also of note, nixos qemu doesnt copy the /nix/store about at all, it uses 9plan to share it
<clever> ekleog: its using this arg to set the boot media
<clever> -drive index=1,id=drive2,file=$TMPDIR/disk.img,media=disk
<clever> yeah
<clever> ekleog: you can just add that to the kernel cmdline and it will give a shell without having to edit configuration.nix
<clever> ekleog: there is also boot.shell_on_fail
<clever> ah hmmm, actualy, i'm not sure if nix.conf applies under that chroot
<clever> because under the chroot, it will stop obeying the ocnfig the netboot was built against
<clever> i believe you also need to add it to the configuration.nix that nixos-install runs against
<clever> sphalerite: and for removing, just delete it from configuration.nix after you install
<clever> sphalerite: cache.nixos.org is always added to the cache list, and it takes extra work to remove it
<clever> so you can do simple things like adding a binary cache or a install script, or crazy things like turning on xfce, lol
<clever> sphalerite: this netboot.nix lets you build the netboot images from any configuration.nix you want, so you can freely configure whatever you want
<clever> sphalerite: you add a binary cache to its nix.conf via the same gist, and nix-serve can act as a binary cache
<clever> sphalerite: if you customize the nixos in the netboot using the gist i linked, you could even add xfce to it, if you where feeling crazy enough
<clever> sphalerite: :D
<clever> i think the only way to unfork it, would be to delete the entire project (and all issues/pr's), then remake it as a not-fork, and upload
<clever> and the fork is now the active branch of the project and has the changes i want
<clever> i sometimes have to clone a repo to grep it, because github refuses to search a fork
<clever> Unode: i just grep a nixpkgs checkout
<clever> a gimp plugin
<clever> pkgs/applications/graphics/gimp/plugins/default.nix: name = "wavelet-sharpen-0.1.2";
<clever> and that has nothing to do with hydra
<clever> about all i can think of then is that the binary cache didnt respond in time
<clever> do you have any packageOverrides in your config?
<clever> that channel will only update once hydra has tried to build every single job
<clever> Unode: which channel are you on and what is compiling?
<clever> ah, nice
<clever> gchristensen: how did you activate hibernation?, gui or cli?
<clever> in the middle of me using a voice chat program
<clever> i had closed the lid because it was in the way, and it went into standby on its own
<clever> gchristensen: and i wound up testing standby by mistake already, lol
<clever> gchristensen: that reminds me, i havent tested hibernate at all on my laptop
<clever> sphalerite: you can also test a lot of this out under VM's, qemu lets you execute a kernel+initrd pair easily, and virtualbox can record the screen to video
<clever> LnL: and sanboot just hooks the drive, then executes sector 0, triggering the normal boot process
<clever> LnL: in theory, this might even work for dos and windows 3.11
<clever> LnL: so any OS that uses the bios to read the hdd, will go over iscsi
<clever> LnL: ipxe will hijack the legacy hdd API in the bios, and reroute hdd access to iscsi
<clever> LnL: yeah
<clever> sphalerite: i used the kexec method on both of those
<clever> sphalerite: 2 so far
<clever> sphalerite: i have had to convert a few ubuntu servers to nixos, without any physical access to the machines
<clever> lol
<clever> viric: but its currently not in use right now
<clever> viric: i have used it to netboot nixos on my laptop before, even the MBR for grub was on the iscsi drive
<clever> sphalerite: and there is no way for the anti-virus to detect it from within the vm
<clever> sphalerite: this also has security implications, in theory, i could put a hypervisor into the NIC boot rom, that then boots the hdd under a vm
<clever> sphalerite: probably more likely to have it omited, but you can sometimes just stick a crappy 100mbit card onto the pci bus, purely for the bootrom
<clever> sphalerite: but sometimes, the NIC vendors just omit the boot rom entirely, and you must supply your own
<clever> so you can reflash it with a utility that knows the NIC
<clever> most of the time, its actualy flash memory
<clever> 3 would require coreboot support, which is harder to find
<clever> 2 should be possible with anything that has an ethernet card
<clever> some motherboards have ISP, but its not always documented
<clever> either pull it out of the socket or de-solder it
<clever> in the case of 3, you need to rip the bios chip off the board and reflash it, if you want to bypass the signing requirements
<clever> 3, as a payload in coreboot, directly in the bios
<clever> 2, directly into the boot rom of a NIC
<clever> 1, put it into the hdd via grub
<clever> 3 main options i was thinking of
<clever> and even if somebody does get into the network, they cant inject custom code
<clever> sphalerite: the design idea behind that boot method, is that you cant brick the hardware, you can always swap files out on the tftp server, hard-reset the machine, and it will come back to life
<clever> i use that any time i need an example that will be publicly visible
<clever> yep
<clever> sphalerite: so you can just make your own self-signed cert, and use that key to sign things
<clever> sphalerite: a certificate is embeded into ipxe at build-time
<clever> sphalerite: this one compiles down to a ~47mb squashfs (1/4th the size of the netboot image), and all files it executes must be signed with a certificate
<clever> sphalerite: i also have custom distro, based on nixos, that boots from a cryptographicaly signed rootfs
<clever> i think its using grub1
<clever> the bootloader part is all i would need to run the netboot image
<clever> sphalerite: at a glance, id say that just configured ubuntu to loopback mount c:\ubuntu.img as the rootfs, and then modifies the bootloader
<clever> if you can get ring0 via a "malicous" driver, you can just turn all irq handlers off, and execute a linux kernel
<clever> in theory, it could also be done against modern windows, but i havent seen a program that does it
<clever> though i dont think it will be happy over a 200mb initrd
<clever> sphalerite: and a 9x era windows, can reboot to raw dos
<clever> sphalerite: this is a raw dos program, that will kexec linux from dos
<clever> sphalerite: it would need to be a windows 9x era windows, loadlin.com
<clever> sphalerite: if i had root on anything, i could have nixos infect it :P
<clever> sphalerite: but the main upside of the kexec method, is that you can do it 100% remotely
<clever> main downside of the kexec method, is that you have no way to recover a mistake
<clever> you can then ssh into that, format the hdd, and install nixos
<clever> sphalerite: and within 60 seconds, you will have nixos running purely from ram, with no changes to the hdd
<clever> sphalerite: this script, will generate a tarball, you just unpack it to /, and /kexec_nixos
<clever> sphalerite: and because that netboot kernel+initrd runs nixos entirely from ram, i have also thrown them at kexec
<clever> and you can obviously customize that to match the partition layout you want, and fs types
<clever> sphalerite: oh, and line 11 of justdoit, i believe if you remove the size= field, it will just take the entire disk automaticaly
<clever> it still needs some nixos modules to add iscsi support to the initrd after that step
<clever> that only gets you far enough to load the kernel+initrd
<clever> sphalerite: due to how sanboot works, grub thinks its booting from a normal hdd in the local box, but ipxe is redirecting it over iscsi transparently
<clever> sphalerite: line 6 of boot.php is another fun test i did, that laptop was booting with the rootfs and grub on iscsi
<clever> Unode: and you need to fix the uuid for / in either hardware-configuration.nix or configuration.nix, so it can find the new root
<clever> Unode: you need to fix the boot.grub.device in configuration.nix, so it knows what to grub-install against
<clever> sphalerite: the ipxe embeded into virtualbox doesnt handle http, so i have to chainload a better ipxe, and then detect the difference
<clever> sphalerite: oh, and the version thing on line 12 is just to work around weirdness in virtualbox
<clever> sphalerite: and boot.php is something i have to allow each machine to boot into a different image, so i can dynamicaly configure things without having to touch the dhcp server
<clever> sphalerite: gistfile1.txt is how the modified netboot gets built
<clever> and it needs to enable ssh and setup an ssh key so you can get in
<clever> target-config.nix will need to exist when you build that configuration.nix to make the netboot image
<clever> so you can just type "justdoit" into the root shell, and it does it
<clever> sphalerite: this is fragments of some config from an example i made 3 months ago, it modifies the netboot image to include a script called justdoit
<clever> sphalerite: that will generate a directory containing an ipxe script, along with a kernel+initrd, and if you boot those, you get the same env as if you had booted from a cdrom
<clever> sphalerite: the nix-build command is in the youtube description
<clever> sphalerite: ah, there is a good pxe target in nixos
<clever> sphalerite: depends on where the machine is and what kind of access you have
<clever> sphalerite: nixops does have a way to deploy to baremetal, as long as the target machine has sshd and nixos already on it
<clever> and of course, /boot has to be mounted to the dir you chroot'd to
<clever> Unode: you want nixos-rebuild boot, which only touches /boot and the boot sector
<clever> Unode: switch is bad in a chroot, because it shouldnt be messing with the active systemd
<clever> Unode: what did you run nixos-rebuild with?
<clever> hyper_ch: and i believe the signatures on the git history are signing the sha1 of the commit
<clever> hyper_ch: all objects in git are identified by sha1 hashes

2017-02-24

<clever> and there goes the entire trust system
<clever> and because nixos can rebuild those with patches, the end-user must have the signing keys
<clever> the bootloader and kernel would have to be signed by MS, or a cert MS has approved
<clever> gchristensen: so any box that trusts the M$ certs pretty much doesnt even have secureboot enabled
<clever> gchristensen: there is a bug in a binary MS signed, that allows it to run unsigned code from the EFI bootloader context
<clever> lol
<clever> contrapumpkin: and if C > c, it needs a check
<clever> and capitol C sets the current count
<clever> contrapumpkin: i think tune2fs -C 0 would be equal to the fsck
<clever> only change i see is one i helped with back in dec, it was fudging the number before it umounted
<clever> that fakes fsck having been ran
<clever> oh right
<clever> the ami image generation scripts had an fsck in them
<clever> contrapumpkin: its paranoid about disk errors
<clever> contrapumpkin: resize2fs always wants an fsck, even if the filesystem was shutdown cleanly
<clever> heading off to bed now
<clever> how are you requesting new units?
<clever> bendlas: so i'm thinking its not systemd doing it
<clever> bendlas: the message that appears is part of switch-to-configuration.nix
<clever> copumpkin: it appears to be tied to the @new variable, on lines 419, 436, and 441
<clever> the following new units were started: snmpd.service
<clever> stopping the following units: snmpd.service
<clever> copumpkin: checking something on this end...
<clever> bendlas: line 176 appears to write to it
<clever> bendlas: and if you ctrl+c the script, it can do sane things upon the next run
<clever> bendlas: i'm guessing the script records its state to those files as it runs
<clever> copumpkin: my first guess, systemd polls /etc/systemd/system/ and starts any units that appear?
<clever> gchristensen: neat
<clever> yep :)
<clever> gchristensen: i did a quick test in a build-vm, and /etc/modprobe.d/nixos.conf does update after a switch
<clever> so it depends on if dccp has been used previously or not, "lsmod | grep dccp"
<clever> but because i tested dccp sockets earlier, the module got loaded and is now stuck in ram
<clever> shouldnt need a reboot to take affect
<clever> there is no tv plugged in and it messed things up
<clever> and i happen to need the other line to stop alsa from trying to output audio on HDMI
<clever> double-single is mainly of use when you want to put more then 1 line
<clever> you can also use a normal doublequote, "..."; and put it all on one line
<clever> yeah, that looks good
<clever> nh2: gchristensen: https://gist.github.com/cleverca22/21ce2134bd507af19996eac5613d2fb3#file-gistfile1-txt-L21 adding this line leaves my system unable to load dccp.ko
<clever> nh2: yep, that works
<clever> nh2: and now for my next trick!, messing with kernel module stuff, without rebooting!
<clever> [root@amd-nixos:~]# nixos-rebuild build-vm
<clever> boot.extraModprobeConfig = "install dccp /run/current-system/sw/bin/false";
<clever> ah yeah
<clever> ah, yeah, thats what it does
<clever> nh2: not sure how it got there
<clever> and turn all dccp support off
<clever> in theory, you can also just "IP_DCCP n\nINET_DCCP n\nNF_NAT_PROTO_DCCP n\nNETFILTER_XT_MATCH_DCCP n"
<clever> nh2: https://gist.github.com/cleverca22/a2e6e79a041d1f7d256a9f183d1681b7 i have used this in the past to enable things
<clever> and now that i have made a single dccp socket, the module is in-use, and i cant unload it
<clever> so it just stops auto-loading, because a driver claimed support at compile-time
<clever> but does nothing to actualy stop loading
<clever> nh2: acording to the man page, blacklist makes the kernel ignore the special pci:123 aliases on a module
<clever> nh2: gchristensen: hmmm, boot.blacklistedKernelModules did nothing to stop a python script from loading dccp_ipv4 and dccp
<clever> i'm not that good with formal things like that
<clever> you simply loose dccp support, though ive never seen anything that used it
<clever> nh2: if the kernel cant load the module, then the buggy code will never be in ram
<clever> nh2: i believe you can also use boot.blacklistedKernelModules as a short-term fix
<clever> gchristensen: how has try#2 gone?
<clever> 2017-02-23 19:49:09 < gchristensen> back in a bit, clearing my head, then going for try 2
<clever> 2017-02-23 19:45:41 < gchristensen> *sigh* I did this wrong.
<clever> 2017-02-23 19:45:13 -NixOS_GitHub:#nixos- nixpkgs/master 1d68edb Graham Christensen: linux kernels: patch against DCCP double free (CVE-2017-6074)
<clever> grep confirms
<clever> 2016-12-14 18:46:18< LnL> gchristensen: nixpkgs manual is pretty simple nix-build '<nixpkgs/doc>'
<clever> gchristensen: i'm guessing
<clever> nix-build '<nixpkgs/doc>'
<clever> but i do try avoiding entering anything secret on the site anyways
<clever> greymalkin: i also check it, but my browser doesnt remember the exception for that long, and i sometimes skip checking the next time it comes up
<clever> and if it already has ssl warnings, is anybody going to notice a self-signed cert?......
<clever> greymalkin: and judging by what i saw in tcpdump when debuging network issues, i suspect i can mitm any box in the datacenter, potentialy including the main website
<clever> greymalkin: the main website for a datacenter i use, has an ssl cert that expires 130 days ago
<clever> xcmw: is there anything near the end relating to illegal instructions?
<clever> xcmw: is dmesg a valid command, what happens when you try to run it?
<clever> xcmw: does osx have anything like dmesg, which may say which process exactly is failing?

2017-02-23

<clever> pikajude: the paste he previously linked: http://pastebin.com/TEYCzesS
<clever> suolrihm: ah, what was it?
<clever> it appears to be absolute, the openssl that nix was compiled against
<clever> 1.11.6 is using an external program to check signatures on nar files!
<clever> that code has changed massively
<clever> things may be different in nix master
<clever> copumpkin: and further reading shows that with nix 1.11.6, there doesnt appear to be a way to --import into a store mounted at the "wrong" location, you will need to keep fakerooting for now
<clever> copumpkin: you could maybe use nix-store --register-validity and NIX_STATE_DIR to alter db.sqlite, after having rsync (or --restore'd) something in, i'll read some more related source...
<clever> yeah
<clever> copumpkin: so this variable is to modify a store that will always be at a weird place, not one that is temporarily at a weird place and will become /nix/store later
<clever> copumpkin: and its expecting the --export to contain absolute paths, starting with /tmp/mnt/nix/store, that where compiled against that path
<clever> copumpkin: because of NIX_STORE_DIR, nix believes the store will be at /tmp/mnt/nix/store, at runtime!
<clever> copumpkin: oh wait, i think i see the problem
<clever> NIX_STORE_DIR=/tmp/mnt/nix/store ... error: path ‘/nix/store/kk71vkqipf30qc165718jmp0s8cggn2y-glibc-2.24’ is not in the Nix store
<clever> copumpkin: aha, its complaining that /tmp/mnt/nix/store isnt a sub-dir of /nix/store
<clever> copumpkin: aha, the 1.11.6 source of nix is fairly different in this region
<clever> it will be much much much harder to collide 2 or 3 hashes at once
<clever> shlevy: it appears to be using sha256, sha512, and whirlpool, for all of its "fixed-output style" downloads
<clever> shlevy: the approach ive seen in gentoo, is to put several hashes on the object at once
<clever> shlevy: if something in nix gets compiled to take advantage of that opcode, it will just not run at all on other cpu's, and now the binary cache needs 2 copies of every build
<clever> copumpkin: i still need to figure out why this doesnt work!, lol
<clever> copumpkin: looks like i'll need to build nix to debug this further
<clever> copumpkin: aha, so nix-store --import goes thru this code path, one step closer to finding out why it fails: https://github.com/NixOS/nix/blob/master/src/libstore/local-store.cc#L912-L923
<clever> ive been using sha256 and 512 on things since 2 years ago
<clever> shlevy: i hear there are ways to check files to see if they have signs of being used for a collision, but those signatures may change in the future
<clever> shlevy: pretty much
<clever> viric: same reason all modern servers reject ssl 3.0 connections, you can perform a downgrade attack via mitm, before either end has been verified with certs
<clever> viric: enless you keep the old sha1 hashes for backwards compat, and then somebody can just insert a sha1 object and exploit away
<clever> viric: in theory, you could, but it would invalidate every git commit hash out there, and also invalidate all of the existing signatures in the git history
<clever> viric: yeah
<clever> viric: you would have to re-clone the entire project, and compare every blob in the history
<clever> viric: yeah, in that event, they can swap things out, and you cant notice by doing a git pull
<clever> so it should stick to whatever version it got first
<clever> and similarly, github shouldnt accept a new version of an object being uploaded
<clever> viric: so an old git clone and a new git clone can produce 2 different trees for the exact same commit
<clever> viric: but anybody who already downloaded that version wont re-download it, because git assumes that if the sha1 matches, it already has a copy
<clever> viric: yeah, if you can collide one file in the tree, then you can swap out its contents in every commit that references that exact version of the file
<clever> viric: to keep the chain of commit hashes intact, and to subvert the signatures, you need to hash collide against the sha1 of the commit
<clever> this is also something i noticed that gentoo did differently, every file portage can download has 3 hashes on it, a sha256, a sha512, and a Whirlpool hash
<clever> viric: but in all of those cases, it has the same sha1 as its ID, so the local git client will think its the original you had to begin with, and wont download the modified one
<clever> viric: it could be anything from a modified commit, directory, or file
<clever> viric: so you can only see it on the web ui, or with a fresh git clone
<clever> viric: yeah, if github was somehow hacked, you would have a hard time noticing this issue, because your own git client wont re-download the modified blob
<clever> viric: but the same applies to github, you cant upload a blob that github already has in the project, so i cant see it being abused easily
<clever> viric: so if an attacker replaces an object on the remote git server, your git client wont download it, because you already have an "identical file" on your machine
<clever> viric: one thing i can see as making the sha1 stuff harder to exploit, "git pull/fetch" wont re-download an object you already have
<clever> copumpkin: so if i run nix-store --import and set the right vars, i can see it unpacking to /tmp/mnt/nix/store/nix-22852-0/unpacked, but it then fails to move it to the store for unspecified reasons
<clever> suolrihm: and if you look in /dev/usb/, what are the permissions and user/group of the node for that usb device?
<clever> suolrihm: and your in the wheel group on this new machine?
<clever> suolrihm: can you pastebin that nix file?
<clever> suolrihm: the simple fix is to just find its bus and device number in lsusb, then give yourself read permissions to its entry under /dev/bus/usb/, though that will have to be repeated each time you plug it in or reboot
<clever> suolrihm: udev rules would automate fixing the permissions every time it gets plugged in
<clever> suolrihm: but i no longer have access to it, and forgot to make any PR's
<clever> suolrihm: i had a chance to mess with it a bit, i had to chmod the usb dev node under /dev/bus/usb/ so the user has r/w perms
<clever> k0001: revCount is the total number of commits i believe, and outPath allows you to treat that nixpkgs attrset as a normal path
<clever> k0001: so if your release.nix has { nixpkgs }:, you can get the revision at nixpkgs.shortRev
<clever> k0001: every input you list in hydra is passed to the main nix file you set in the jobset config, as an attribute set like this
<clever> k0001: hydra passes that in as an argument
<clever> k0001: do you just need the git revision its from, or are you trying to get the git logs?