2017-07-02
20:38
<
clever >
boomshroom: builtins.fetchurl allows you to fetch things in an impure fasion, but it will re-download it every time you eval the nix file
20:30
<
clever >
where you make a nix file that pre-builds everything it will need, before launching the vm
20:30
<
clever >
the build-vm stuff in nix is mostly meant for read-only use of the store
20:27
<
clever >
but when you start doing weird things, you might break it in weird ways
20:26
<
clever >
so it wont delete that closure
20:26
<
clever >
and the host knows thats in use via the cmdline in /proc
20:26
<
clever >
for the normal guests, only stuff that the guest should know about is registered into the db.sqlite
20:24
<
clever >
just make sure you enable kvm, or performance will be horid
20:23
<
clever >
just delete the xen bits
20:23
<
clever >
it will work for your needs
20:23
<
clever >
but if your fine with impure stuff like persisting the disk image, and using nix-copy-closure to import new builds
20:22
<
clever >
who would be crazy enough to run a vm inside a vm?
20:22
<
clever >
it only detects xen, so it turns off qemu features
20:22
<
clever >
and the linux is inside xen, inside qemu
20:21
<
clever >
its all happening inside a nix-buid
20:21
<
clever >
the host lacks root
20:21
<
clever >
say goodbye to your free space :P
20:21
<
clever >
so i need a new 1gig disk image file for every test
20:21
<
clever >
but the critical problem, is that the 9plan/virtio stuff to share /nix/store/ from the host, doesnt work
20:20
<
clever >
so i had to run xen under qemu
20:20
<
clever >
and i wanted the entire test to run without root
20:20
<
clever >
i wanted to test xen unikernels, but they only run under the xen hypervisor
20:19
<
clever >
and makes use of xen commands inside the guest
20:19
<
clever >
then boots it under qemu
20:19
<
clever >
spinus: this creates a disk image containing a full nixos install, with grub, and xen
20:18
<
clever >
spinus: ive got an example in a gist that does some whacky things
20:17
<
clever >
catern: does tsnix own everything in the store?
20:16
<
clever >
tests/common.sh.in: export NIX_IGNORE_SYMLINK_STORE=1
20:16
<
clever >
catern: there is an option to allow it to be a symlink
20:16
<
clever >
spinus: you would need to use make-disk-image.nix, and then nix-copy-closure or nixops to keep it up to date
20:13
<
clever >
but the tmpfs its building in might be mounted noexec
20:12
<
clever >
catern: the one you just posted
20:11
<
clever >
catern: can you pastebin the entire output of "mount"?
20:10
<
clever >
causing github to claim the build is pending
20:09
<
clever >
because it took 9 hours, it could say a build was queued, after the build passed
20:09
<
clever >
and that was just a few hours after i fixed a problem in the github status plugin, related to it
20:08
<
clever >
during which time, it cant do another eval of that jobset
20:08
<
clever >
eelco made a commit recently saying it takes 0.7sec for each, and ~9 hours in total
20:08
<
clever >
but also, the evaluator will run the build-queued perl hook, for every single build
20:07
<
clever >
gchristensen: i have seen it sit on single-threaded curl requests, and not fire new jobs up
20:07
<
clever >
gchristensen: as hydra loads each job from the queue, it will check for subs, before moving to the next job
20:07
<
clever >
gchristensen: i believe hydra checks for subs after the eval, inside the hydra-queue-runner daemon
18:13
<
clever >
curse is trying to kill ftb
18:13
<
clever >
and mcedit
18:12
<
clever >
Infinisil: i do have nix packages for multimc and ftb launcher
18:12
<
clever >
its just a server ran on nixos by one of the guys in here, ive played on it before
18:09
<
clever >
the server is heavily supported by nix
18:09
<
clever >
Baughn runs one
18:08
<
clever >
i play it every now and then, but the computer has trouble getting reasonable perf
18:05
<
clever >
boomshroom: and that java launcher they had?, lets deprecate it!
03:47
<
clever >
tilpner: check the journal
03:47
<
clever >
swflint: you need to add fetchgit to the function arguments on line 1
03:43
<
clever >
tilpner: did you use a revision of nixpkgs that wasnt from a nixos channel?
03:40
<
clever >
nix-prefetch-git might also give a rev
03:39
<
clever >
git will tell you the rev
03:39
<
clever >
nix-prefetch-git will tell you the sha256
03:38
<
clever >
git needs a rev, and nix needs a sha256
03:38
<
clever >
swflint: yes
03:37
<
clever >
otherwise, you would have to compile amd and nvidia versions of every gui app
03:37
<
clever >
so LD_LIBRARY_PATH is required, to sneak those in after things have been made immutable
03:36
<
clever >
and rpath and immutable outputs makes that tricky on nix
03:36
<
clever >
vendors like to make things difficult, and you need the right libGL for your card
03:36
<
clever >
pie_: yeah, thats where nixos puts the gl drivers for your gpu
03:33
<
clever >
src = fetchgit { url = "something"; rev = "something"; sha256 = "something"; };
03:32
<
clever >
swflint: your missing -lX11, check my gist
03:31
<
clever >
or relative to . since the default unpackPhase copies the source to the current dir
03:31
<
clever >
swflint: and then use a path relative to $src in the script
03:31
<
clever >
src = fetchFromGitHub { owner = "foo"; repo = "bar"; sha256 = "baz"; rev = "something"; };
03:26
<
clever >
joepie91: ive moved 10-15mb of strace logs like that before
03:26
<
clever >
joepie91: gist only limits what can be seen on the site, you can still git clone the logs
03:26
<
clever >
swflint: then you can just nix-build '<nixpkgs>' -A x11idle, or nix-env -iA nixos.x11idle
03:25
<
clever >
swflint: in the packageOverrides, do x11idle = pkgs.callPackage ./x11idle.nix {};
03:23
<
clever >
joepie91: gist is also a command in nixpkgs, and it just uploads right to gist
03:22
<
clever >
swflint: from which command?
03:20
<
clever >
swflint: and to make it work without that incanation, add it to either your config.nix, or nixpkgs and open a PR so it works for all
03:20
<
clever >
swflint: oh right, have to add unpackPhase = ":";
03:19
<
clever >
pie_: nixos has to change it at runtime for the gpu driver you need
03:19
<
clever >
pie_: LD_LIBRARY_PATH provides the real libgl
03:17
<
clever >
swflint: oh right, need to add { stdenv, xlibs }: to the start of the file
03:16
<
clever >
eacameron: yes
03:16
<
clever >
swflint: nix-build -E 'with import <nixpkgs> {}; callPackage ./yourfile.nix {}'
03:15
<
clever >
eacameron: --no-out-link
03:14
<
clever >
swflint: what error does it give?
03:14
<
clever >
eacameron: what about the nix-build i gave above?
03:13
<
clever >
pie_: its in glibc
03:13
<
clever >
glibc.out 0 s /nix/store/izxnyg94352qxa4a4783dzgnpy5cwazj-glibc-2.25/lib/libdl.so.2
03:13
<
clever >
[clever@amd-nixos:~]$ ./apps/nix-index/result/bin/nix-locate lib/libdl.so
03:12
<
clever >
pie_: the library that gives you dlopen
03:12
<
clever >
eacameron: nix-build -E 'with import <nixpkgs> {}; callPackage ./. {}'
03:12
<
clever >
joepie91: i have months worth of undo's waiting :P
03:12
<
clever >
joepie91: i prefer zfs snapshots
03:11
<
clever >
eacameron: nix-env -i -E 'with import <nixpkgs> {}; callPackage ./. {}' i think
03:06
<
clever >
just need to scroll up past the error handling logic
03:06
<
clever >
pie_: usualy, it will stop when it fails, so the last thing it did is the important one
03:06
<
clever >
joepie91: reflog?
03:02
<
clever >
dont worry, bash does the same thing with file handles, lol
03:02
<
clever >
ever heard of readdir?
03:01
<
clever >
any files its trying to open are a good place to stat
03:00
<
clever >
dlopen is a library function, strace wont find it
03:00
<
clever >
pie_: while you wait, try strace -e open
03:00
<
clever >
but dlopen isnt covered
03:00
<
clever >
i think LD_DEBUG only handles things flagged with DT_NEEDED
02:58
<
clever >
joepie91: mouse/keyboard not working after patchelf i believe
02:58
<
clever >
pie_: what is LD_LIBRARY_PATH set to?
02:56
<
clever >
pie_: ah, is that unity based?
02:55
<
clever >
fresheyeball: do you really want to install something, or just test that it builds and runs?
02:54
<
clever >
joepie91: ah
02:54
<
clever >
swflint: stdenv.mkDerivation { name = "x11idle"; installPhase = "mkdir -p $out/bin; gcc -lXss ${./x11idle.c} -o $out/bin/x11idle"; buildInputs = [ xlibs.libXScrnSaver ]; }
02:52
<
clever >
if it ever hits a black hole, it will know its turned back and hit its own stack trace
02:52
<
clever >
and while running, it will temporarily change the type to black-hole
02:52
<
clever >
joepie91: and if you try to use that value for anything, nix will run it on the spot, and replace the thunk with the return value
02:51
<
clever >
joepie91: basicaly, any time nix parses a non-trivial expression, it just generates a value of type thunk, that basicaly contains a function pointer and some args
02:51
<
clever >
joepie91: it can help to know how thunks work in nix
02:49
<
clever >
pie_: so if you do everything with nix-store, then nix will never delete something in-use
02:49
<
clever >
pie_: nix can only know about things being in-use by other things in /nix/store
02:49
<
clever >
swflint: and also, installing gcc doesnt work, you must nix-shell -p gcc
02:49
<
clever >
pie_: that is why you and swflint should always build things with nix-build, and never try to do things by hand
02:48
<
clever >
swflint: what does "type gcc" say?
02:46
<
clever >
try gcc.cc first
02:45
<
clever >
pie_: gcc.cc.out i think
02:44
<
clever >
can you copy/paste everything in the terminal to a pastebin and link that?
02:43
<
clever >
swflint: try without the --run, and then manualy do the gcc inside of that
02:40
<
clever >
swflint: what is the exact nix-shell commant you ran?
02:40
<
clever >
swflint: -p xlibs.libXScrnSaver
02:40
<
clever >
swflint: you must add that to the nix-shell args
02:39
<
clever >
swflint: xlibs.libXScrnSaver
02:37
<
clever >
it has to go into the RPATH, via makeLibraryPath in my gist
02:37
<
clever >
oh right, and buildInputs is 100% useless when patching things
02:36
<
clever >
it gets the version that is the default in the current nixpkgs
02:36
<
clever >
mesa_noglu
02:36
<
clever >
and typicaly, you use mesanoglu for this part
02:35
<
clever >
pie_: it must be a name without the version
02:31
<
clever >
swflint: what is the new nix-shell command, the command your running inside that, and the error?
02:24
<
clever >
you must tell nix-shell to include pkgconfig
02:24
<
clever >
swflint: that shell wont have a working pkgconfig in it
02:23
<
clever >
swflint: what nix-shell command did you use?
02:23
<
clever >
pie_: the patchelf one?
02:22
<
clever >
you must nix-shell -p pkgconfig libfoo libbar
02:22
<
clever >
installing libraries wont make them visible to pkg-config either
02:22
<
clever >
and installing pkg-config will never make it work right
02:22
<
clever >
pkg-config only works inside nix-shell and nix-build
02:20
<
clever >
swflint: are you using nix-shell?
02:10
<
clever >
noobly_: environment.systemPackages = [ (pkgs.callPackage ./racket.nix {}) ];
02:01
<
clever >
eacameron: sounds neat
02:00
<
clever >
hmmm, but that might not work with <nixpkgs/nixos> now that i think of it, would need some experimentation
02:00
<
clever >
example b: { ... }@args: import (builtins.fetchtarball { url=...; sha256=...; }) args
01:59
<
clever >
example a: { ... }@args: import /home/clever/nixpkgs args
01:59
<
clever >
tilpner: something like this
01:18
<
clever >
and by giving both fetches a hash, it will only do the network access once
01:18
<
clever >
it would still need to import 2 nixpkgs, but it would exist within $NIX_PATH, so all tools use it by default
01:16
<
clever >
and as long as that returns a nixpkgs set, and obeys the config arg, it will work
01:16
<
clever >
tilpner: what if you just do NIX_PATH=nixpkgs=/bootstrap.nix
01:16
<
clever >
ah, i think i see what your doing
01:15
<
clever >
what do you keep in $NIX_PATH?
01:13
<
clever >
and some parts like nixos and nix-build cant easily be told to fetch from somewhere dynamicaly
01:13
<
clever >
the version of nixpkgs is typicaly controlled with nix-channel
01:12
<
clever >
why cant you fetch nixpkgs over http?
01:11
<
clever >
if you want to fetch a copy of nixpkgs
01:10
<
clever >
(import <nixpkgs>{}).fetchgit
01:10
<
clever >
thats usualy done like this
01:09
<
clever >
but you will need to know the hash of the file
01:09
<
clever >
tilpner: fetchurl
01:03
<
clever >
then systemd-askpass blocked
01:02
<
clever >
so it burns the cpu for 5 minutes, uncompressing all ram
01:02
<
clever >
oh, and it wants to restart the zram service
01:02
<
clever >
tilpner: nixos-rebuild wanted to restart the dhcpcd and wpa_supplicant services, so it shut the network off
01:01
<
clever >
tilpner: yikes, i just lost all network for over 5 minutes from my nixos-rebuild!
00:39
<
clever >
noobly_: what nix-shell command are you using, and what do you want different?
00:39
<
clever >
noobly_: you generally avoid installing things in nix
00:24
<
clever >
just open both nix files it makes, and paste the list of inputs in one into the other
00:23
<
clever >
cabal2nix will generate a nix file, then you could manualy merge the 2 nix files
00:22
<
clever >
you would need to get a list of haskell deps from both, concat them, and make a new env using ghcWithPackages
00:19
<
clever >
luigy: so it might allow building servant, but it may also complicate which servant will be loaded
00:18
<
clever >
luigy: i'm guessing that in addition to providing servant, it will also provide the libs servant uses
00:14
<
clever >
luigy: its not really simple to join the deps of 2 things
00:13
<
clever >
luigy: ah yeah, that would provide servant and servant-server, not their deps
00:11
<
clever >
luigy: nix-shell -p '(haskellPackages.ghcWithPackages (ps: with ps; [ servant servant-server ]))'
00:09
<
clever >
yeah, the - tells it to read stdin
00:08
<
clever >
noobly_: nix-build -E 'with import <nixpkgs> {}; callPackage ./racket.nix {}'
00:08
<
clever >
also, it sounds like your making that way too complex
00:07
<
clever >
noobly_: you can also nix-build .... 2>&1 | gist -
2017-07-01
23:56
<
clever >
you would need a recursive function that checks the string length and prepends a "0"
23:55
<
clever >
ah, yeah, nothing to do that
23:55
<
clever >
spinus: not really, but you can just do "foo ${bar} baz" most of the time
23:38
<
clever >
hodapp: so you can then setup the container on the same port, and give it a unique ip
23:37
<
clever >
hodapp: nixos containers get their own instance of the networking stack
23:22
<
clever >
that will kill performance, but it only happens once
23:22
<
clever >
hodapp: oh fun, lol, it reads the php-fpm config one byte at a time!
23:20
<
clever >
and if your running php-fpm manualy, you can copy the php.ini, edit the copy, then point php-fpm to the new php.ini
23:19
<
clever >
hodapp: we may also want to change display_errors = on in the php.ini
23:17
<
clever >
so gettingn the entire trace, from startup to first request, will give more details
23:17
<
clever >
and line 14 of the last paste is just confirming the file hasnt changed
23:16
<
clever >
and one theory, is that the first time it loaded the php file, it cached the contents, and possibly the error
23:16
<
clever >
it might also be doing other weird things at startup
23:15
<
clever >
hodapp: no real change, but i dont see the initialization from running php directly under strace
23:14
<
clever >
hodapp: lets see...
22:55
<
clever >
ah, i suspected that, but didnt see anything in the module i previously linked
22:48
<
clever >
hodapp: can you gist the entire strace output when you manually run it?, i can check it for odd errors that explain the 502
22:37
<
clever >
ok, go back to tracing the nginx process, what does that show?
22:35
<
clever >
what is the full command you ran?
22:35
<
clever >
what about the strace for php-fpm?
22:34
<
clever >
anything new in the logs?
22:33
<
clever >
what is an example url?
22:32
<
clever >
did you stop phpfpm-main first?
22:32
<
clever >
hodapp: did you use strace -f?
22:31
<
clever >
hodapp: ah, read the phpfpm-main.service then
22:29
<
clever >
hodapp: and then run the thing in ExecStart with strace -f
22:29
<
clever >
hodapp: read the systemd unit file in /etc/systemd/system/phpfpm.service
22:28
<
clever >
hodapp: what does the strace look like if you stop the systemd unit, and run the full php-fpm command under strace?
22:24
<
clever >
id have to cross-reference the php source to see whats going on
22:24
<
clever >
yeah, it doesnt really make much sense
22:19
<
clever >
a nixos container would have nearly identical config
22:16
<
clever >
then you can see what its doing on startup
22:16
<
clever >
hodapp: it may also help to set it to 1 worker, then stop the systemd unit, and manualy run the command defined in /etc/systemd/system/phpfpm.service under strace -f
21:59
<
clever >
they should land in the journal for the phpfpm service
21:59
<
clever >
hodapp: ah, i think the 2 lines gchristensen linked will help a lot
21:58
<
clever >
hodapp: i saw this in another irc room about an hour ago
21:58
<
clever >
2017-07-01 18:15:54 < mst> PHP isn't a language, it's a malignant perl templating engine that metastasised
21:54
<
clever >
hodapp: all i can think of is to mess with the error_log option in the php-fpm config file, via the nixos config
21:53
<
clever >
nothing special
21:51
<
clever >
let me double-check what i had done
21:51
<
clever >
yeah, that aint working
21:49
<
clever >
and the strace output when you add "-s 5000" ?
21:47
<
clever >
thats how nixos handles it 80% of the time
21:47
<
clever >
its usualy on the commandline passed to the daemon
21:47
<
clever >
ps aux | grep php-fpm
21:46
<
clever >
can you do that again with "-s 5000" and also pastebin the php-fpm config file?
21:45
<
clever >
that is odd
21:42
<
clever >
what does the fpm-worker strace look like now?
21:39
<
clever >
hodapp: it may also help to add "-s 5000" to the strace args
21:38
<
clever >
so now repeat the same thing agaist the php-fpm workers
21:37
<
clever >
which is the php-fpm socket
21:37
<
clever >
hodapp: it read a 404 from fd 12
21:37
<
clever >
[pid 281] recvfrom(12, "\1\6\0\1\0Z\6\0Status: 404 Not Found\r\nX"..., 4096, 0, NULL, NULL) = 120
21:37
<
clever >
323 enableDebugging = pkg: pkg.override { stdenv = stdenvAdapters.keepDebugInfo pkg.stdenv; };
21:37
<
clever >
322 # intended to be used like nix-build -E 'with <nixpkgs> {}; enableDebugging fooPackage'
21:35
<
clever >
boomshroom: read the code for stdenvAdapters, that shows one way to mutate a stdenv and modify its args
21:35
<
clever >
323 enableDebugging = pkg: pkg.override { stdenv = stdenvAdapters.keepDebugInfo pkg.stdenv; };
21:34
<
clever >
322 # intended to be used like nix-build -E 'with <nixpkgs> {}; enableDebugging fooPackage'
21:34
<
clever >
hodapp: try passing all of the nginx processes to strace, with strace -p <pid1> -p <pid2>
21:32
<
clever >
hodapp: what pid's do you see in "ps aux | grep nginx" ?
21:31
<
clever >
boomshroom: nix goes out of its way to remove march=native from the flags
21:31
<
clever >
hodapp: throw it all in a pasetbin
21:28
<
clever >
hodapp: i would throw strace at the nginx process, and then see what it says
21:22
<
clever >
boomshroom: you would need to make an override to stdenv.mkDerivation
21:18
<
clever >
containers/user_ns would give good performance without needing root
21:14
<
clever >
and once you have root, you could just install full nixos, even remotely
21:07
<
clever >
boomshroom: give what Mic92 linked and see what error it says
21:07
<
clever >
its possible that they just didnt install the tools
21:06
<
clever >
ah, then you cant fire up your own container
21:05
<
clever >
boomshroom: is nsenter in $PATH?
21:05
<
clever >
Mic92: he wants to use nix, but lacks root
21:03
<
clever >
if storedir isnt set to /nix/store/, you loose all binary cache help
21:03
<
clever >
boomshroom: there is a builtins.storeDir, that could detect it
21:02
<
clever >
yeah, i think using NIX_STORE_DIR, you can force a nix to build into a dir other then where it lives itself
21:02
<
clever >
and the stateDir as well
21:01
<
clever >
so you will want to set nix.storeDir in config.nix, to enforce it staying right
21:01
<
clever >
i only looked at nix/default.nix
21:00
<
clever >
boomshroom: so you will need a config.nix that overrides it back to the right value, or the next time oh, one sec
21:00
<
clever >
boomshroom: the store directory for the expression defaults to /nix/store/
20:59
<
clever >
so that never actually works
20:59
<
clever >
tilpner: the root default.nix in nixpkgs has a version check, which stops you from building the new nix
20:59
<
clever >
tilpner: ah, ive found that that thing has never really worked
20:58
<
clever >
tilpner: --fast will skip that step
16:52
<
clever >
then copy the result over
16:51
<
clever >
you just need to use a machine where you have write to both /nix/store and /home/foo/.nix/store/
16:51
<
clever >
if you use nix to build nix, it will handle that for you
16:51
<
clever >
but i forget the details of how it was done
16:51
<
clever >
so that nix was built by nix, on a machine that did have /nix/store/
16:51
<
clever >
boomshroom-away: i showed osmebody how to build nix with .override before, to change the storepath
16:50
<
clever >
boomshroom-away: one last thing before you go
16:50
<
clever >
yeah, that would require building it from scratch
16:50
<
clever >
the curl script will skip sudo if you can write to /nix
16:49
<
clever >
all you need, is ownership of /nix/
16:49
<
clever >
you dont technicaly need root to install nix
15:41
<
clever >
anything you set in _module.args gets passed to all modules