2017-08-05

<clever> thblt: which means you dont need a stable swap encryption, leading to swapDevices.*.randomEncryption
<clever> thblt: so hibernation is out of the question if you have zfs
<clever> thblt: the reason i was asking about FS earlier, is that zfs doesnt support suspend to ram, or disk
<clever> ah
<clever> srhb: i tried to run hydra-eval-jobs on btrfs a year ago, it had to create ~20,000 400 byte files in /nix/store, btrfs timed out on writing, and went into read-only mode
<clever> so there are multiple copies of it
<clever> and even that i think is a ring-buffer
<clever> where it has to change some pointers into the immutable data
<clever> the only mutable part is the root directory area
<clever> but if snapshots are on, it will keep them around
<clever> and if you have no snapshots, it will GC the old versions fairly quickly
<clever> any time you write to the disk, it creates a new variant of the file (and every directory above it), refering to a combination of the unmodified blocks, and the new blocks
<clever> nearly
<clever> zfs is a bit like nix, all blocks on the disk are immutable
<clever> ive been using zfs on most of my new installs
<clever> that will play into these choices some
<clever> thblt: also, what filesystem do you want for the rootfs?
<clever> i havent done any testing to verify the speed, but its a decade old laptop, so its not likely to make a difference to me
<clever> then the rootfs and swap live on lvm
<clever> in my case, i only have a single luks, and then i ran pvcreate + vgcreate on the luks device, followed by 2 calls of lvcreate
<clever> i just used LVM to do the same thing
<clever> so you only enter 1 pw by hand, for that 3mb partition
<clever> thblt`: he is using the 3mb partition as a "password", to unlock 2 more luks, the swap and rootfs
<clever> thblt`: aha, i see why he did that
<clever> i would just skip that
<clever> weird
<clever> thblt`: weird, can you link the guide?
<clever> Infinisil: my dell with a half-dead battery gets ~40 minutes
<clever> thblt: i think thats to let it boot without the user entering a pw, which sort of makes the luks pointless
<clever> he often picks it up by the display, and my dells wouldnt last a week if i did that
<clever> dash: taktoa has a laptop from there and i hear its pretty solid
<clever> dont think so
<clever> ive been thinking of https://system76.com/laptops
<clever> also needs nix-channel --update
<clever> thblt: something like this, and to find the full path, nix-store -qR /run/current-system | grep kbd
<clever> setfont /nix/store/l8szcakw2yyx37dnnqm9x9pxzjb6353n-kbd-2.0.3/share/consolefonts/Lat2-Terminus16.psfu.gz
<clever> thblt: trying to find a path...
<clever> thblt: via a call to setfont i think
<clever> Olgierd: run "passwd username" as root to change it
<clever> Olgierd: that option only applies when first creating the user, and is otherwise ignored
<clever> Olgierd: was initialPassword used when you created the user?, or did you add it afterwards?
<clever> Olgierd: did you set a password on the user?
<clever> Olgierd: isNormalUser sets home for you
<clever> pierron: that sets things like createhome automatically
<clever> pierron: and also, isNormalUser = true;
<clever> pierron: yeah, thats why it fails horridly when you just symlink the root directory of this "overlay"
<clever> timclassic: which is the one i mentioned
<clever> timclassic: do any other files contain the word overlay?
<clever> because <nixpkgs> loads the overlay, which loads <nixpkgs>, which loads the overlay ....
<clever> but that can also cause infinite recursion if configured wrong
<clever> default.nix will import <nixpkgs> and apply the overlay for you
<clever> timclassic: you want rust-overlay.nix
<clever> timclassic: that points to default.nix, which is not an overlay
<clever> timclassic: what overlay did you symlink?
<clever> and that was enough to make the list work in android
<clever> the java core gave an api to query the row count, and to fetch a given row#
<clever> it was an sqlite database
<clever> and i just remembered, the chat history wasnt an in-ram array
<clever> so i just had to give it a function to turn a single row into a UI element, and plug the 2 together
<clever> and android has some powerfull tools to turn an array into a low-ram usage list on-screen
<clever> chat for example was handled almost entirely in core, which kept arrays of history for each channel
<clever> the ui was then just a matter of calling the right rpc methods (via the wrappers), gathering values, and presenting them
<clever> deltasquared: the java core was 80% wrapping the http based RPC in java functions, and 10% tracking state via polling for events
<clever> deltasquared: both of those shared the java core, so there was a lot less rewriting going on, and i could test the java core by just writing simple desktop based bots for the game, then quickly throw an android UI over it
<clever> deltasquared: http://imgur.com/a/mC4Rz
<clever> deltasquared: finding screenshots...
<clever> deltasquared: yeah
<clever> deltasquared: https://github.com/google/j2objc
<clever> deltasquared: then i used that core, in a SWT desktop app, an android app, and i had plays to transpile the java to objective-c for iOS
<clever> deltasquared: i have done that in the past, i wrote the core as a java library
<clever> but i didnt feel like rewriting the entire android behaviour
<clever> at the minimum, it needs a custom mobile UI written in QT
<clever> and then the mobile versions look like crap
<clever> but it also looks almost identical (except for host theme differences)
<clever> nearly identical codebase, shared between windows, linux, mac, android, and iOS
<clever> my thought when i had started, was cross-platform to the extreme
<clever> yeah
<clever> deltasquared: but making the controls work in the exact oposite way is just bad UI
<clever> and you drag it the oposite direction from normal android
<clever> for example, a list, has a small tab in the scrollbar, that you have to drag to scroll
<clever> Infinisil: the main problem with QT on android, is that it doesnt look like android, it looks&behaves like a desktop
<clever> Infinisil: in the past, i have used QT on android, and it did "work"
<clever> Infinisil: lol
<clever> obadz: combine both of those, and you can survive long enough to go 2 or 3 minutes between GC cycles
<clever> obadz: a second major tool, turn automatic GC off, and then force a GC between levels, when the user wont notice jitter
<clever> it cant leak like pointers in c, its just moved up a layer
<clever> and just reset them to defaults upon each use
<clever> mainly, avoid local variables in functions, always keep things as instances under this
<clever> because even if you can get a solid 60 fps, if a single GC cycle takes 5 frames of time, you get jitter
<clever> ive also seen guides on how to avoid garbage creation in java, to get better FPS in android games
<clever> the next GC cycle?
<clever> kk
<clever> evangeline: another example of how to build drivers: https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/acpi-call/default.nix
<clever> *tries*
<clever> oh, there
<clever> where is it in nixpkgs?
<clever> ah, thats just building a piece of the kernel
<clever> evangeline: as an example, this is the cpupower driver: https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/cpupower/default.nix
<clever> nixpkgs will automatically pass you the right kernel headers
<clever> then read the others drivers listed in that area for examples
<clever> you probably want to create a open-vm-tools attribute, pointing to a custom nix package
<clever> some of it
<clever> could be docker messing with things
<clever> read any of the files listed there for examples
<clever> evangeline: so we need to add an attribute here, for the vmware drivers
<clever> evangeline: we have now found the source for the kernel modules
<clever> gchristensen: ping
<clever> lol
<clever> then things might work, once the host loads the modules
<clever> evangeline: what does "lsmod" say under the docker?
<clever> evangeline: possibly
<clever> and if you then have no swap left, you cant use suspend to disk
<clever> and if your swap is on a zvol, you have to swapoff first
<clever> you must export all zfs pools before you can suspend
<clever> zfs doesnt support suspend to disk, period
<clever> evangeline: and the modprobe we see everywhere says its kernel
<clever> evangeline: i would expect the username stuff to be pre-built
<clever> but we need a copy of the source, not to make vmware build its own garbage
<clever> the other vmware-modconfig
<clever> the info we want is in the vmware-modconfig binary or module
<clever> run strings on that and see if there are any paths to tars or source
<clever> strange
<clever> evangeline: check line 77 of the last paste you linked
<clever> evangeline: so we have to find this
<clever> ${tools_BINDIR}/vmware-config-tools.pl --modules-only
<clever> you need to find the driver source, and build thru via nix
<clever> but even then, they cant load
<clever> yeah, i see, it tries to modprobe the drivers, and if that fails, it tries to build the drivers
<clever> same general set of files, no trace of any source that could use linux headers
<clever> evangeline: i'm seeing files under /etc/vmware-installer, /usr/lib/vmware and /usr/lib/vmware-installer
<clever> if your using find, thats -mount
<clever> should also limit it from entering /proc and /sys
<clever> and try uploading to gist this time, gist -p ccc.txt
<clever> otherwise, the order will cause a false difference
<clever> you may want to sort aaa and bbb before you diff them
<clever> these look like false positives?
<clever> +Wed May 20 05:10:20 2015 /bin/bzip2recover
<clever> i dont see that helping that much
<clever> they made this as un-friendly as they could possibly make it
<clever> that elf helper then re-opens the bundle, and extracts moar!!
<clever> all it can do is extract an elf helper
<clever> evangeline: ah, i see my problem, this bash script is just unable to extract everything
<clever> and it still only extracts 25mb, lol
<clever> evangeline: edit the bundle, in replace mode (not insert mode!) and put an exit; on the 2 rm's in on_exit, and the install at the botton of main()
<clever> vim silently added a single \n to the end of the binary content
<clever> aha, thats why vim broke everything
<clever> evangeline: it even contains iso files for vista, lol
<clever> 225018118 0xD698106 gzip compressed data, maximum compression, has original file name: "winPreVista.iso", from Unix, last modified: Mon Jun 19 23:57:36 2017
<clever> evangeline: do you see a file like this anywhere in your docker?
<clever> 68359398 0x41314E6 gzip compressed data, maximum compression, has original file name: "vmnet.tar", from Unix, last modified: Mon Jun 19 23:54:39 2017
<clever> 19951200 0x1306E60 Windows Script Encoded Data (screnc.exe)
<clever> evangeline: if i run binwalk over the bundle, i can even see windows executables!!!
<clever> and the tar unpack is failing after 39mb, the tar is 456mb!
<clever> evangeline: the tar i am able to extract contains no source code, and makes no mention anywhere of kernel headers
<clever> there is no nix enforcing where it can make a mess
<clever> and due to the FHS, finding what it modified is just a matter of guesswork
<clever> we need to look at the files vmware added to the system
<clever> evangeline: nearly all of those are unrelated to vmware, and those samples/examples arent required to make it work
<clever> "output88 output89 output9000 output9001", nope, bash failed to re-assemble the split!
<clever> evangeline: any .c files
<clever> evangeline: even if you can get the kernel module to build, docker will never let you load it
<clever> evangeline: to confirm if its a kernel module or a userland component
<clever> then you can eval texlive within that repl
<clever> kainospur[m]: like nix-repl '<nixpkgs>'
<clever> kainospur[m]: you must give it the path to a file with nix expressions
<clever> evangeline: https://gist.github.com/cleverca22/6253abf2cc45fefbda34386e6ff23a12 but the tar still seems to be truncated!
<clever> and :r to reload
<clever> evangeline: so with the unmodified bundle, the above split command cuts it up, and outputaa contains the bash script, the rest are fragments of a tar.gz
<clever> kainospur[m]: nix-repl '<nixpkgs>' then use tab completion
<clever> kainospur[m]: userland components are far less fussy about what version of the kernel headers are used
<clever> ah, because i edited it, lol
<clever> no, size seems off now
<clever> `split -b 15163 VMware-Workstation-Full-12.5.7-5813279.x86_64.bundle output` maybe
<clever> oooo, the 'split' program looks handy
<clever> but vmware isnt making it easy to even confirm it has source
<clever> if you have the source for the module, it can easily be built
<clever> kainospur[m]: nixpkgs makes that easy, when you integrate the source into the linuxPackages tree
<clever> lets just see what happens if i edit the bash!
<clever> and if i use an ibs of 1, with skip, it reads 1 byte at a time, and will take hours to convert
<clever> the problem, is that the value of ibs causes it to skip the last 15kb of data, because its not an even multiple of ~15kb
<clever> the bash script uses a bunch of shell magic to count its own size and compute the answer
<clever> and adjusted it until i saw no more bash script
<clever> i just piped the dd into |hexdump -C | head
<clever> brute-force
<clever> and docker doesnt allow loading kernel modules
<clever> its likely to just never work under docker, because it needs a kernel component to run right
<clever> i'm still trying to get the entire tar, i'm loosing the tail end of it
<clever> but they missed /lib64/ld-linux-x86-64.so.2
<clever> and they basicaly tried to make a crappy clone of nix, to ensure all libs it needs are available
<clever> ./lib/libconf/lib/gtk-2.0/2.10.0/loaders/libpixbufloader-pnm.so
<clever> evangeline: i can confirm why the normal install fails, its expecting an ld.so at /lib64/ld-linux-x86-64.so.2, as it usual with ELF's from outside of nix
<clever> ah, needs an obs also
<clever> though its missing the last 15kb
<clever> evangeline: i now have a normal tar file that i can just unpack
<clever> evangeline: [clever@amd-nixos:~/vmware]$ dd if=VMware-Workstation-Full-12.5.7-5813279.x86_64.bundle skip=1 ibs=15163 > foo.tar.gz
<clever> evangeline: just plain old vim
<clever> evangeline: are you trying to get the vmware-workstation?
<clever> evangeline: and reading the bash code, it has a --extract option
<clever> evangeline: the bundle is just a bash script, prefixed to what is probably a tarball
<clever> and i cant download until i register, lol
<clever> it must have source, thats the only way it can use kernel headers
<clever> evangeline: can you link the dl you used?
<clever> evangeline: and where did you get this source from?
<clever> it prefers .hi files over .hs!
<clever> vim does the same thing, and cyces in a weird order, so i often open the wrong files
<clever> evangeline: what part of vmware needs the linuxHeaders, can you pastebin the error?
<clever> what are you building that needs the linux headers
<clever> evangeline: is this the headers for building a module or a userland component?
<clever> ah yeah, 4.9 is missing, let me see
<clever> evangeline: also, to build a kernel module, you must use nix-build via the linuxPackages tree, nixos will fight you at every step if you try to build it by hand
<clever> evangeline: and to answer your other question, linuxHeaders_4_9
<clever> yeah, the nix-build also fetches it as well
<clever> but the nix-build also prints it
<clever> evangeline: this prints it out
<clever> nix-instantiate --eval -E 'with import <nixpkgs>{}; "${linuxHeaders}"'
<clever> evangeline: so they should just work
<clever> evangeline: the shell automatically added the headers to the gcc search path
<clever> evangeline: that breaks a lot of what nix does to make building simple
<clever> evangeline: you must never install things like headers
<clever> kainospur[m]: it just sets a bunch of env variables to change the search paths for things
<clever> nix-shell spawns a shell with it just added to PATH, so its isolated
<clever> nix-env installs things into your users profile
<clever> yeah
<clever> kainospur[m]: into ~/.nix-profile/bin/ which should already be in $PATH
<clever> evangeline: nix-shell -p linuxHeaders
<clever> kainospur[m]: nix-env -iA nixpkgs.nix-repl
<clever> Infinisil: oh, weird, havent seen it work like that
<clever> kainospur[m]: it will only ever use what is defined within nixpkgs
<clever> kainospur[m]: nix will go out of its way to ignore things already on the host at all costs
<clever> anything else isnt considered valid
<clever> it must either start with: ./ ../ or /
<clever> any path starting with a . is relative to the directory the nix file is in
<clever> kainospur[m]: nix-env -iA nixpkgs.nix-repl
<clever> kainospur[m]: one example i had was just pdfit = latex2pdf "CT" ./input3.tex;
<clever> Display all 3097 possibilities? (y or n)
<clever> nix-repl> texlive.<tab><tab>
<clever> you run it on a string (a name), and a path to a latex file (can be a derivation), and it spits out a pdf
<clever> https://gist.github.com/cleverca22/e411a205908cfd14a012d931f1dbdaf7 this is a nix function i wrote a while back
<clever> i dont see mactex anywhere in nixpkgs
<clever> the stdenv expects buildInputs to be a list
<clever> and this entire thing, is already inside another "with pkgs"
<clever> yeah
<clever> you can also do buildInputs = with rPackages; [ R ggplot2 knitr ];
<clever> so where you normaly would have done buildInputs = <value>;, for example, buildInputs = [ rPackages.R rPackages.ggplot2 rPackages. knitr ];
<clever> and then the value that follows, can access any attribute within <set>
<clever> any place you can have a value, you can also do: with <set>; value
<clever> for everything after that
<clever> kainospur[m]: so just 'pandoc' is enough
<clever> kainospur[m]: the "with pkgs" already brought all of nixpkgs into scope
<clever> so it never sets a version in anything
<clever> kainospur[m]: the build script never reads that, so absolutely nothing happens
<clever> kainospur[m]: all it does is set an env variable called "version" to 1
<clever> heh, was doing the same
<clever> all it serves to do is to mislead people into thinking it does something useful
<clever> the version="1"; also does absolutely nothing
<clever> very good question, that example is just wrong
<clever> kainospur[m]: try instead using src = ./.;
<clever> kainospur[m]: the compiled version of nix lacks a Makefile
<clever> kainospur[m]: you have set your source directory to be the compiled version of nix
<clever> src = if pkgs.lib.inNixShell then null else nix;
<clever> yeah
<clever> login with gist --login
<clever> i prefer this over most pastebins, because you can edit things, and github shows a history, you can also clone a gist with just "git clone"
<clever> nix-env -iA nixpkgs.gist

2017-08-04

<clever> kainospur[m]: https://gist.github.com
<clever> kainospur[m]: if you gist your default.nix i can look at it
<clever> kainospur[m]: it will look at the directory specified by the src attribute
<clever> kainospur[m]: only nix-shell will have trouble
<clever> kainospur[m]: yeah, nix-build is much more isolated and wont care
<clever> so you have to install the nixpkgs bash
<clever> kainospur[m]: nixpkgs relies on some bash features that are missing in darwin's bash
<clever> kainospur[m]: nix-env -iA nixpkgs.bashInteractive
<clever> kainospur[m]: darwin?
<clever> and my qemu/xen changes, are having trouble going from 16bit to 64bit mode
<clever> qemu has a gdb server, but the gdb as a client, gets rather upset when you switch between 16bit, 32bit, and 64bit modes, so its difficult to debug early boot stuff
<clever> what i'm doing is 100% self-contained to qemu, and can even be ran without root and kvm
<clever> so you need a real xen hypervisor to use that
<clever> and then qemu emulates the motherboard
<clever> xen emulates the cpu, and will proxy all IO requests to a cpu-less qemu
<clever> there is xen support in qemu, but its radically different
<clever> so xen-only guests can run under qemu
<clever> that emulates a xen hypervisor
<clever> ive also modified qemu some, adding support for a custom x86 platform
<clever> and acording to #osdev, bochs has a better debugger then qemu
<clever> evangeline: i have heard in #osdev that bochs has a powerfull built-in debugger
<clever> evangeline: how different is virtualbox from vmware?
<clever> so its recomended to just use the package in nixpkgs if it exists
<clever> evangeline: everything you download must be patched by a nix expression, or it will fail like that
<clever> i'm not sure how its doing it
<clever> ive also noticed how nix-shell will download the docs for the bash in the stdenv
<clever> nixos-install is just a script that runs nixos-rebuild under a chroot
<clever> and as always, you can just boot the installer, re-mount the root and /boot to /mnt and /mnt/boot, and run nixos-install to repair it
<clever> i think it was reformat /boot, "nixos-rebuild test" with the right uuid, and systemd will mount the right thing, then "nixos-rebuild boot" to update /boot
<clever> seequ: nixos can still be fixed
<clever> so you cant mount the right one (because systemd'd uid is wrong)