<boomshroom>
Do any of the Minecrafers here know how to speed up launch time when using a modpack?
magnetophon1 has quit [(Remote host closed the connection)]
<joepie91>
boomshroom: depending on the modpack you may need to increase the RAM allocated to the JVM; eg. the default setting for the kitchen-sink FTB packs is way too low and leads to the launcher spinning its wheels endlessly garbage-collecting uncollectable stuff
<clever>
boomshroom: you can also run jvisualvm from the oraclejdk to debug the java process
<clever>
boomshroom: it works even on the non-oracle jvm
<joepie91>
boomshroom: beyond that, there's not much you can optimize that I know of (without actually modifying the modpack or its contents in some way) aside from some minor speed gains by loading from an SSD instead of a HDD
<clever>
most of my time is spent in the cpu anyways
<clever>
and i think its heavily single threaded
<joepie91>
yeah, most of MC initialization is just bursting a single CPU thread
<joepie91>
so short of getting an obscenely expensive CPU there's not much to improve there
<joepie91>
:p
<clever>
or fixing the initialization code in every mod
<joepie91>
the SSD will only help with the I/O-heavy phases; the process isn't perfectly parallelized so there are some parts of the loading process where it'll bottleneck on I/O instead of CPU and vice versa
<joepie91>
but the bulk is still CPU-bound
jtojnar has quit [(Ping timeout: 260 seconds)]
<joepie91>
I'm still amazed at how long eg. FTB packs take to initialize
<joepie91>
there's really no excuse for needing 5 minutes for such a relatively limited amount of content
<joepie91>
sure, it's a lot of content, but not 5-minutes-a-lot
<clever>
i remember it being more like 10mins
<joepie91>
varies by system, it's about 5 for me
<clever>
the fml logs may say which mod is to blame
<joepie91>
for whatever is the canonical FTB kitchensink pack at the time :P
<clever>
joepie91: do you see a loading-log.log file?
<joepie91>
clever: I *suspect* that it's some sort of exponential lookup thing caused by mod 1 interacting with mod 2 which leads to mod 2 interacting with mods 3 and 4 etc.
<joepie91>
plus crappy autoupdater code
<joepie91>
clever: not sure what you're referring to with that, but I've never seen anything particularly weird in the FTB launch log other than lots of mod interactions
<clever>
joepie91: try this cmd, find $HOME -name loading-log.log
<joepie91>
clever: isn't that in ~/.ftblauncher?
<joepie91>
normally
<joepie91>
because I don't seem to have any logs there
<joepie91>
and my $HOME is pretty big :)
<clever>
yeah, you can limit the search to there if your using ftb
s33se_ has joined #nixos
<joepie91>
nothing found with that name
<clever>
mine was in ~/.MCUpdater/instances
<clever>
your forge might be too old to make it
k0001 has joined #nixos
<clever>
what version of MC?
<joepie91>
not a clue, haven't played MC for a while
<joepie91>
whatever the FTB kitchensink packs use
k0001_ has quit [(Ping timeout: 240 seconds)]
<clever>
they are split between the modern mc and the old 1.7 stuff
<joepie91>
I think I've played newer as well
<joepie91>
but not certain
vaibhavsagar has joined #nixos
<clever>
tconstruct: 722ms
<clever>
my log file says things like this
<joepie91>
oh, it times mod loads?
<clever>
integrateddynamics: 3251ms
<clever>
yeah
s33se has quit [(Ping timeout: 260 seconds)]
<joepie91>
perhaps I've only played older then, no idea
<joepie91>
:p
<joepie91>
anyway, brb
ertes has quit [(Remote host closed the connection)]
aneeshusa has joined #nixos
<nh2>
with what command can I fetch a store path from a binary cache?
<clever>
nix-store -r /nix/store/foo
k0001 has quit [(Ping timeout: 248 seconds)]
ertes has joined #nixos
<clever>
nh2: i aslo need to test our your nginx nixos cache when i get home, ive got gigabit inside the lan so it should help with the latency of common things
marantz has joined #nixos
<nh2>
clever: I've just updated the gist, to contain some improved caching settings for cloudflare. Before I made it cache 404s which isn't good
<clever>
ah, i see a diff
<nh2>
clever: so far it brought a 2x speedup for my nixos stuff vs cache.nixos.org
<nh2>
so I'm very happy, thanks for your initial gist
<clever>
nh2: i also noticed you use 8.8.8.8, but i recently had issues with 8.8.8.8 claiming some servers i use dont exist
<clever>
i just run my own recursive cache
<gchristensen>
nh2: you put it behind cf? :D
<nh2>
clever: yes, you can put everything there, IIRC the only reason I put it was that one needs to set it to _some_ IP that's a nameserver, so I just picked that
<clever>
ah
<nh2>
gchristensen: yes, it's pretty fast
Maxdamantus has quit [(Ping timeout: 268 seconds)]
<gchristensen>
fancy
<nh2>
clever, gchristensen: if you also want to try this with CloudFlare, you need to set their settings to "Cache Everything" (in contrast to expectations, this doesn't force caching, it still respects upstream caching headers; the "cache everything" just says "cache all file types", including the nar stuff)
<clever>
ah
<clever>
i would expect cloudflare caching and ec2 stuff to not be that much better then cache.nixos.org, other then it being an isolated instance
<clever>
and maybe being in your region
hellrazo1 has joined #nixos
<clever>
but i do have ~5 nixos machines at home, and a local cache to share between them would help
<nh2>
clever: I would also not expect it but in my measurements from earlier today it showed that cache.nixos.org Cloufront performs very badly even when accessed from AWS Frankfurt
<nh2>
for me it's almost always a crawling slow few-MB/s-at-max
<clever>
almost sounds like cloudfront needs more work?, do we know anything about the internals behind how it works?
cpennington has quit [(Remote host closed the connection)]
<nh2>
clever: only thing I know is that even when I curl the same .nar.xz from CloudFront from the same machine multiple times in a row, it will cache-miss every now and then
<nh2>
CloudFlare is much more predictable in that regard, if you hit the cache once from one location, you can be pretty sure it'll always hit the cache from that location for the next couple hours (have never observed it otherwise so far)
<nh2>
clever: I'm currently trying to add an extra feature to this config so that I can put my own built packages (generated with `nix-push`) into /var/public-nix-cache (thus my question about force-fetching a path), but somehow it doesn't work yet
<clever>
ah
<clever>
another thought, is to write a custom client that will recursively browse the narinfo tree and fetch every nar via the cache
<nh2>
clever: the setup for that is already in the gist but it doesn't seem to work. Fetching the file with curl certainly does, but not with `nix-store -r`. I wonder if I have to do something special with that beyond putting the output of `nix-push` into that directory?
<clever>
but just throw out the bytes as it goes
<clever>
i think nginx has an index of what it has downloaded
<clever>
and nix-push will re-sign things, so the narinfo will be wrong
jtojnar has joined #nixos
<nh2>
clever: also, I generated it with `nix-push --none`, is that a problem (not xz-compressed, because that took too long to try it this evening)
<nh2>
?
<nh2>
clever: what will nix-push sign it with if I run it from my desktop?
<clever>
you have to pass it a key with --key-file
<clever>
and if you dont, the narinfo files just wont be signed
<clever>
nix-daemon may ignore them entirely
<clever>
checking the http access logs can reveal what its doing
<nh2>
clever: hmm, OK. I guess I should first enable access logs so I can see if if it's even trying
<clever>
and since you also have nginx acting as a mitm for cache.nixos.org, you can see it trying again on another cache
<gchristensen>
clever: is there any reason I need to be trusted to seed torrents for the binary cache?
<clever>
also, when i check the A records for cache.nixos.org, i can see 8 ipv4 addresses behind the domain
<gchristensen>
/ generate the .torrents ?
<clever>
gchristensen: as long as you serve the original .narinfo files from cache.nixos.org, the signatures stay intact, but you would need to host the unpacked torrent on http, and set that as a binary cache
<clever>
i dont think nix can read narinfo+nar files from a local dir
<nh2>
gchristensen: wait, nix can already fetch stuff via torrents?
<gchristensen>
no it can't, nh2
<nh2>
ah that'd be cool
<hodapp>
gchristensen: did you catch what I said earlier about why the database wouldn't matter for this PHP issue on ttrss?
<gchristensen>
hodapp: I did, but I'm not sure what to say :9
<gchristensen>
sorry
<hodapp>
gchristensen: me neither, but I'm hoping someone who knows more about nginx configuration can weigh in or at least reproduce the issue\
<clever>
hodapp: did you ever try it with display_errors = on in the php.ini?
<hodapp>
clever: I don't think I tried that. At a certain point, I just gave up trying to diagnose anything, and instead just wanted to see if as vanilla of a configuration as possible reproduced the issue
<clever>
hodapp: if you read the php-fpm command from the systemd unit, youll find the current php.ini path
<nh2>
clever: is there some standard way to enable access_log for nixos nginx?
<clever>
copy it somewhere, change display_errors, and then manualy run php-fpm with the right end, and the new php.ini path
<hodapp>
clever: out of curiosity, if you check out the configuration in that issue I filed, do you get the same problem?
<clever>
nh2: dont see one, youll need to use one of the many extraConfig options to insert it at the right place
<hodapp>
thanks. please make me feel stupid by finding the one obvious thing I ignored.
<gchristensen>
clever: a major problem with bittorrent is distribution of the .torrents. heh.
<hodapp>
magnet links!
<hodapp>
my friend says it worked for a site called pirate bay, where I guess he buys all his movies and stuff
<nh2>
clever: `nix-store -r /nix/store/abcabc...-cudatoolkit-8.0.44` doesn't hit the access_log at all, no idea why not
<clever>
nh2: what is the contents of nix.conf on that machine?
<gchristensen>
then you have to distribute the magnet links, hodapp
<hodapp>
they're tine
<hodapp>
...tiny*
<clever>
so are the narinfo files
<gchristensen>
whether or not that is a problem depends if your goal was to eliminate the central cache or not
<nh2>
clever: ah, I ran this from the nginx server machine, which hasn't actually got itself as binary-caches entry, will fix that
<hodapp>
hmm. maybe I should just use a Python-based RSS client so that I have a much better chance of it actually working, and of being able to hack it
<hodapp>
as much as folks swear by ttrss
drakonis has joined #nixos
lambdamu has joined #nixos
mbrgm has quit [(Ping timeout: 260 seconds)]
mkoenig has joined #nixos
<clever>
hodapp: i can confirm the same problem in a vm when i use the config from the gist
<gchristensen>
I think there are cool ways to experiment with alternatives to the binary cache by making an http prooxy
<clever>
hodapp: now i can just debug it locally
* hodapp
throws a dart at the wall
<hodapp>
let's use... IPFS
<clever>
nh2: you need to generate a keypair (nix-store man page), then use --key-file on nix-push so it can sign things
<clever>
nh2: and ensure the public half is in the nix.conf file
<gchristensen>
sure, go make an http proxy and get playing :)
<clever>
nh2: and it currently has a bug, where it doesnt save the signatures from cache.nixos.org, so anything you get from nixos, is re-signed by your key
lambdamu_ has quit [(Ping timeout: 240 seconds)]
<hodapp>
nah, I didn't want to actually do the work, I just wanted to pretend I had a clue
<hodapp>
the IPFS paper has been on my to-read list for embarrassingly long now
<gchristensen>
simpson doesn't like it
<hodapp>
why not?
<clever>
ipfs is mainly a hash(value) = value database
<clever>
so you need to know the hash of the content to ask for me
<clever>
for it*
<clever>
and the binary cache stuff isnt based on the hash of the content
<clever>
its more of a hash(source) = compiled_binary database
<clever>
and your "lookup" function is gcc
<nh2>
so in short: IPFS is hash(output)=output, Nix is hash(input)=output
<clever>
yeah
<clever>
and the narinfo files act as a map between things
<clever>
and the naming of the nar file depends on the server
<clever>
for example, hydra uses a narpath that is identical to the storepath, without even a .nar at the end
<catern>
also I heard that IPFS is not particularly competent
Nycatelos has quit [(Ping timeout: 240 seconds)]
<clever>
so when it gets the nar request, it knows exactly what path to pack up on-demand
<hodapp>
catern: competent as far as what?
<clever>
catern: another limitation of IPFS, is that you only store the things you asked the network for
<clever>
catern: so if nobody asks for a given object, it just doesnt get mirrored, and it dies off
ij has quit [(Ping timeout: 255 seconds)]
<clever>
and related, the object must remain in the ipfs storage dir, in blocked form (not usable by nix) to share with others
<clever>
so you need 2 copies of everything you install (one in /nix/store, one in the ipfs store)
<catern>
hodapp: eh I dunno
<clever>
that just encourages people to delete it from the ipfs store, and then nobody benefits from it
<catern>
yeah
<clever>
its the same as serving .zip files in a torrent
<joepie91>
it took me a full 7 months to realize that I can work around lack of left recursion support in a PEG parser, by always recursing on the right side
<joepie91>
...
CustosLimen has quit [(Ping timeout: 255 seconds)]
<clever>
now i have to unpack it to use the content, and then i just delete the zip
<catern>
Nix is kind of fundamentally different from CA storage, right?
<hodapp>
hmmm
<catern>
CA (content addressed) storage is, y'know, content addressed
<clever>
catern: CA storage?
<clever>
ah, yeah
<joepie91>
well, yes and no
<catern>
the Nix store is a cache for a *function*
<joepie91>
you could consider the binary cache to be content-addressed
<joepie91>
in a way
<joepie91>
that is, you request the hash of the build configuration from the binary cache to get a build
<hodapp>
yeah, sorta
<catern>
yes, that's fundamentally different from CAS
<nh2>
catern, joepie91: see above, IPFS is hash(output)=output, Nix is hash(input)=output
<catern>
they both use hashes of course :)
<catern>
nh2: right
<joepie91>
that seems like a needlessly strict definition of "content-addressed" :P
<clever>
and the output is generated via output=f(input)
<clever>
and f isnt always going to give the same out for the same in
<nh2>
clever: OK, so that narinfo files are mutable (given that I just added a signature), we should probably reflect that fact in the nginx config by not caching those forever
boegel has joined #nixos
<clever>
nh2: at least on cache.nixos.org, the files are immutable, but when you take different cache providers into account, they may each have different narinfo contents
<catern>
CAS is this function: hash(x) -> x; Nix is this function: x -> y, and an associated cache which is cheaper to perform looks up in, which is hash(x) -> y
<clever>
nh2: something ive been thinking of, is a custom daemon, that can query several binary caches, then cache them all in one spot, and serve them up
<catern>
it's like memoization, basically
<clever>
nh2: and will be configured to prefer a cache.nixos.org signature, when one exists
roconnor has quit [(Ping timeout: 248 seconds)]
Nycatelos has joined #nixos
Supersonic112 has quit [(Disconnected by services)]
ij has joined #nixos
Supersonic112_ has joined #nixos
Supersonic112_ is now known as Supersonic112
<nh2>
clever: that would be useful; this CloudFlare thing is already doing wonders for me but the eventual even better solution would be a machine in the same datacenter that has it all on the disk
<clever>
nh2: it may also help to mix the 2, use a slower daemon with a full cache of everything, then a nginx over that, that can quickly serve recently used data
<clever>
then you can take advantage of the optimization ngingx has already had
<nh2>
clever: `nix-store -r` works now against the cache, thanks!
koserge has quit [(Ping timeout: 240 seconds)]
<nh2>
clever: yes, I've found that a couple things 404 on github that shouldn't. Recently also on a "History" page of a file, clicked the "<>" to show the file contents, got 404
<hodapp>
clever: I guess I'll add that gist to the GitHub issue
<nh2>
clever: though in this case it might not be github's fault
<_habnabit>
ok so this isn't nixos per se but i installed nixpkgs on alpine and it's having trouble running binaries that need glibc. how can i get nix-env or whatever to set the right environment cruft so that it uses nix's ld-linux.so and libc.so
<clever>
nh2: could be another force push, ive been having problems with those lately
mrcheeks has quit [(Quit: ERC (IRC client for Emacs 26.0.50))]
<clever>
_habnabit: are you using nix-shell?
<_habnabit>
clever, i could, if that would help
<_habnabit>
clever, i wasn't at the moment
<clever>
nearly everything compiling related breaks if not used in nix-shell
<_habnabit>
clever, ok. so how do i start nix-shell?
<clever>
and installing the compiler/libs wont help, and sometimes makes it harder to see the problem
<clever>
nix-shell -p gcc libfoo libbar
<clever>
that will give you a shell that can compile things using libfoo and libbar
<_habnabit>
ah ok
<_habnabit>
$ ldd rustup-init => /nix/store/7n45x3waczv1smsdkax2dy4j2zhbfk82-glibc-2.25-bin/bin/ldd: eval: line 1: ./rustup-init: not found
<_habnabit>
.. huh
<clever>
_habnabit: try with patcheld --print-interpreter
<_habnabit>
clever, at last! ldd is giving output!
<joepie91>
the only + there is string concat, and - is missing entirely
<_habnabit>
and now i can see i'm also missing libz
drakonis has quit [(Read error: Connection reset by peer)]
<clever>
_habnabit: lol, zlib is in my example
<_habnabit>
clever, so i'm looking at this code, but i'm not really sure what to do with it
<clever>
_habnabit: if you run nix-build on this file, you will get a result symlink pointing to a bash script
jgertm has quit [(Ping timeout: 276 seconds)]
<_habnabit>
clever, ah ok
<clever>
that bash script can be ran on an ELF file to patch it
<_habnabit>
clever, got it
<clever>
and as long as you keep that result symlink, nixos wont delete the libraries
<clever>
so the rustup binary will keep working
<_habnabit>
is there something like https://nixos.org/nixos/packages.html but shows the contents of files? i'm not sure if something other than the 'zlib' package has libz.so.1
hapticFeels has quit [(Read error: Connection reset by peer)]
<joepie91>
well, and command-not-found for executables, I don't think that includes libraries
<clever>
i have used nix-index, and it works well when you know the path
marantz has quit [(Quit: Leaving.)]
<_habnabit>
is `nix-env -i zlib` in nix-shell supposed to do whatever magic is required to make libz.so.1 dynamically loadable?
<clever>
nope
<_habnabit>
oh
<clever>
installing libraries with nix-env will never make them work
<clever>
you must edit the rpath with patchelf, to point to zlib
<clever>
the gist i linked can do that
<_habnabit>
ah ok
koserge has joined #nixos
ebzzry has quit [(Ping timeout: 240 seconds)]
eacameron has joined #nixos
ebzzry has joined #nixos
Infinisil has quit [(Ping timeout: 240 seconds)]
<hodapp>
"Could not find a version that satisfies the requirement celery-with-mongodb (from leselys==0.2.6) (from versions: )" I am kinda not sure what that means...
Infinisil has joined #nixos
garbas has quit [(Quit: WeeChat 1.7.1)]
<_habnabit>
clever, haha, nice.. another executable already has an rpath set, and --set-rpath clobbers it :c
<clever>
yeah
<clever>
there is a --print-rpath
<_habnabit>
this is getting complicated
<_habnabit>
i haven't even gotten rustc running yet and i'm not sure if it'll produce correct output
<clever>
its usualy simpler to use the rust thats already packaged into nixpkgs
DanC has left #nixos ["Hasta"]
<_habnabit>
i need rust nightly, unfortunately
<clever>
i think thats in the mozilla overlay
<_habnabit>
oh!
<_habnabit>
where's this?
<clever>
2017-06-26 21:45:59< Infinisil> You can however clone the nixpkgs-mozilla repo, then link to it with nixpkgs.overlays = [ (import path/to/mozilla/overlay/rust-overlay.nix) ];
jgertm has joined #nixos
<_habnabit>
nice
<clever>
but thats for configuration.nix
<_habnabit>
i'm not sure what that means
<clever>
for normal nixpkgs, its in ~/.config/nixpkgs/overlays i think
<clever>
lookup the nixpkgs overlays int he docs
<joepie91>
clever: do you know if there is any *complete* reference of operators in Nix? I've assembled a list of all the operators and bits of syntax I could find in the parser source, but I haven't figured out how to determine the precedence
<clever>
joepie91: there are also some edge cases, like set.a-b or set."a.b"
<joepie91>
clever: the former evaluates to set."a-b" if I'm not mistaken, I'm not sure what the edgecase for the latter is
<joepie91>
clever: can you elaborate on that last one?
<clever>
its an attribute names a.b
<clever>
which will give fun error messages, because set.a.b doesnt exist
<joepie91>
clever: is it possible to define an attribute (ie. not attribute path) "a.b" on a set?
<joepie91>
which would make set."a.b" a valid lookup
<clever>
yeah, { "a.b" = value; }
<joepie91>
then I'm not sure what's the edgecase, unless you mean it in the sense of: set.a.b != set."a.b"
<clever>
its just an unusual thing that not many will expect
<joepie91>
(which seems obvious to me, but may not be to others? idk)
<clever>
and the errors say set.a.b
<joepie91>
ahhhh
<clever>
so you dont notice that the . isnt part of the path
<joepie91>
okay, I get what you mean now
<joepie91>
clever: I'm a bit biased because writing parsers for invented and/or existing formats is pretty much a saturday evening hobby project for me, so I don't even notice some of the typical language design quirks
<joepie91>
I just kind of assume them to exist
<joepie91>
:p
<clever>
:D
<joepie91>
clever: anyhow, currently trying to implement a Nix parser, using a PEG generator
FRidh has quit [(Read error: Connection reset by peer)]
ogkloo has joined #nixos
mog has quit [(Ping timeout: 255 seconds)]
<lewo>
Hi. I have 2 jobs in hydra. One is a 'buildPythonApplication' which appears in the channel. The other is a dockerTools.buildImage (a runCommand derivation) which doesn't appear in the generated channel. How hydra decides packages to include in the channel, and how could I add the dockerImage also?
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] Mic92 pushed 1 new commit to master: https://git.io/vQEaM
<NixOS_GitHub>
[nixpkgs] FRidh pushed 2 new commits to master: https://git.io/vQEom
<NixOS_GitHub>
nixpkgs/master bf9e5d2 Ioannis Koutras: kdiff3: add dependency to kcrash
<NixOS_GitHub>
nixpkgs/master 680298e Frederik Rietdijk: Merge pull request #27039 from jokogr/f/kdiff3-kcrash...
NixOS_GitHub has left #nixos []
erictapen has joined #nixos
m0rphism has joined #nixos
stanibanani has joined #nixos
ertes-w has joined #nixos
mkoenig has quit [(Remote host closed the connection)]
thc202 has joined #nixos
Itkovian has joined #nixos
jgertm has quit [(Ping timeout: 260 seconds)]
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] FRidh pushed 2 new commits to master: https://git.io/vQEK7
<NixOS_GitHub>
nixpkgs/master 8f5cf68 Frederik Rietdijk: ktorrent: fix build
<NixOS_GitHub>
nixpkgs/master 43ca911 Frederik Rietdijk: kdevplatform: fix build
NixOS_GitHub has left #nixos []
zraexy has joined #nixos
pietranera has joined #nixos
Wizek has joined #nixos
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] FRidh opened pull request #27079: virtualenvwrapper: only add Python to $PATH, fixes #26982 (master...virtualenvwrapper) https://git.io/vQE6r
NixOS_GitHub has left #nixos []
Wizek_ has joined #nixos
<ij>
I tried doing `nix-env -i terraform` to see whether it has it and judging by strace, it's now crawling /nix/store now — why?
Filystyn has quit [(Read error: Connection reset by peer)]
k2s has joined #nixos
Filystyn has joined #nixos
Filystyn has quit [(Changing host)]
Filystyn has joined #nixos
koserge has joined #nixos
<LnL>
I think what you're seeing is that it's evaluating all of nixpkgs
<ij>
Why does it need to?
<LnL>
nix-env -f '<nixpkgs>' -iA terraform is a lot faster because it uses the attribute
<hyper_ch>
gchristensen: ktorrent and kdeveleop issues seem to have been fixed now :) at least on nixos the handling of bug reports won't take years without any feedback
<ij>
LnL, There's so much more I need to understand.
* ij
concludes
<LnL>
-i searches by name, so it needs to look at the name of each package to find it
<LnL>
while using an attribute is like a regular dict lookup
k2s has quit [(Quit: Leaving)]
garbas has joined #nixos
erictapen has quit [(Ping timeout: 255 seconds)]
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] Mic92 pushed 2 new commits to master: https://git.io/vQEPG
<NixOS_GitHub>
[nixpkgs] vcunat pushed 1 new commit to staging: https://git.io/vQuI6
<NixOS_GitHub>
nixpkgs/staging bb3c8a1 David McFarland: lib: Include darwin in isUnix
NixOS_GitHub has left #nixos []
stanibanani has joined #nixos
erictapen has quit [(Ping timeout: 260 seconds)]
jensens has quit [(Ping timeout: 240 seconds)]
indi_ has joined #nixos
<sorcus>
No ideas about snat?
ixxie has quit [(Quit: Lost terminal)]
Itkovian has quit [(Quit: My MacBook has gone to sleep. ZZZzzz…)]
indi_ has quit [(Ping timeout: 240 seconds)]
Itkovian has joined #nixos
mudri has joined #nixos
<hodapp>
blargh, sometimes I have no idea what these Python packages are doing when they try to install
<hodapp>
just used fetchPypi on one and the result was completely missing several modules
erictapen has joined #nixos
<hodapp>
so I did 'python setup.py build' from a nix-shell with the source, and the resultant script is trying to run completely a Python wrapper completely without the correct environment
jensens has joined #nixos
erictapen has quit [(Ping timeout: 260 seconds)]
erictapen has joined #nixos
Itkovian has quit [(Quit: My MacBook has gone to sleep. ZZZzzz…)]
<__Sander__>
I need to convert an SHA512 hash in base64 notation
<__Sander__>
to nix's notation
Itkovian has joined #nixos
<gchristensen>
why_
<gchristensen>
?
k2s has quit [(Ping timeout: 240 seconds)]
<gchristensen>
sha512 = "..." can take the standard notation
<__Sander__>
oh does it also understand base64?
<gchristensen>
hmm not sure, try it :)
erictapen has joined #nixos
mudri has quit [(Ping timeout: 276 seconds)]
<niksnut>
no, base16 and base32
<__Sander__>
yup does not work: error: hash ‘m+qzzcn7KUxEmd1gMbchF+Y2eIUbieUaxkWtptyHywrX0rE8QEYqPC07Vuy4Wm32/xE16NcdBctb8S0Xe/5IeQ==’ has wrong length for hash type ‘sha512’
<__Sander__>
so I need to convert it to nix's base32 notation
<__Sander__>
but I guess I have to implement such an encoder myself
Itkovian has quit [(Quit: My MacBook has gone to sleep. ZZZzzz…)]
peacememories has quit [(Quit: My MacBook has gone to sleep. ZZZzzz…)]
orivej has quit [(Ping timeout: 240 seconds)]
<__Sander__>
hmm or print the hash base16
Itkovian has joined #nixos
<__Sander__>
not sure if that's a good idea for SHA512 hashes
<srhb>
__Sander__: Silly question, why don't you just use the hash that nix will print for you once it determines that your bogus hash is wrong? (if you get the length right)
<__Sander__>
srhb: my use case is a generator
<__Sander__>
node2nix
<srhb>
Ah.
<__Sander__>
I want to take the registry's sha512 hashes
<__Sander__>
but they use base64 notation
<ikwildrpepper>
make nix support that :)
darlan has joined #nixos
orivej has joined #nixos
mog has joined #nixos
<__Sander__>
ikwildrpepper: that could be a nice addition
ThatDocsLady has quit [(Quit: Arma-geddin-outta-here!)]
ebzzry has quit [(Ping timeout: 268 seconds)]
earldouglas has joined #nixos
<catern>
is there a simple interface to make-disk-image.nix
<Guest32498723>
hello, I'm trying to set up an encrypted partition/boot partition. I've followed https://nixos.org/nixos/manual/index.html#sec-luks-file-systems, but get "error: will not proceed with blocklists" and "installation on /dev/mapper/c" failed when I try to `nixos-install`. Any suggestions?
<catern>
all I want for now is to just test whether my somewhat bespoke Nix setup is functioning enough to build a NixOS disk image
<catern>
any NixOS disk image at all
<catern>
so I don't want to have to make up values for the parameters
<gchristensen>
catern: what if you did nixos-rebuild build-vm?
Itkovian has quit [(Quit: My MacBook has gone to sleep. ZZZzzz…)]
<catern>
gchristensen: I'm not on a NixOS setup, I have Nix on Debian :)
<catern>
so I did successfully build a NixOS disk image
<catern>
(the installler CD, as instructed by the NixOS manual)
<clever>
build-vm can also be done with nix-build
<catern>
clever: indeed I used nix-build
<clever>
nix-build '<nixpkgs/nixos>' -I nixos-config=./configuration.nix -A vm
<catern>
now I want to build, say, a minimal disk image that just runs in QEMU
<catern>
there are lots of various configurations in nixpkgs, is there one that satisfies that?
<catern>
which is just: cd nixpkgs/nixos; nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix
<catern>
that is easy for a mere babe in arms like me
<catern>
but i'm not sure exactly... how to do that?
<catern>
oh wait
<clever>
you can also add any of these modules to the imports section of a custom one
<clever>
and then aim nixos-config at that
<clever>
which is exactly how configuration.nix works
<catern>
okay but, then I want to do the system.build.novaImage action
<clever>
use -A
<clever>
the example you pasted already has that
<catern>
sure, so when I do "nix-build nova-image.nix -A system.build.novaImage -I ..." it says it can't autocall a function without default params
<catern>
which is what I would expect
<__Sander__>
heh building with base16 notation works
<__Sander__>
but still
<clever>
you must put nova-image into the nixos-config, and build nixpkgs/nixos/default.nix
<clever>
not build nova-image.nix directly
<gchristensen>
it sounds ... long, __Sander__ :)
<__Sander__>
sha512 hashes in base16 are looong
<__Sander__>
yes
<catern>
clever: ohhhh :) thank you!
mkoenig has quit [(Remote host closed the connection)]
<catern>
I see now (well, I guess :) )
<clever>
catern: the default.nix under nixos will merge all of the core modules and the <nixos-config> module together, to create a single config attrset
<clever>
catern: and the modules are able to set eachothers options and inter-connect
erictapen has joined #nixos
<catern>
ah
<catern>
huh, neat
ldesgoui has joined #nixos
DeaDSouL has joined #nixos
<ldesgoui>
hey, how would I go if I wanted to import an expression from github in the imports field of my NixOS configuration.nix? "<https://url/file.nix>" doesn't seem to work that way
<clever>
ldesgoui: need to call fetchurl on it first
orivej has quit [(Remote host closed the connection)]
<ldesgoui>
I think what I actually want to do is load the repo like a channel and use that from there, because the file I'm using does local imports and fetchurl will definitely not work
orivej has joined #nixos
<clever>
fetchtarball would unpack it after fetching
<clever>
then it would need the .zip or .tar url from github
<schmits>
Hi guys! I'm here to ask one simple question: How can I determine an absolute path to a file with known relative location to the *nix file itself within a nix-shell file? More precisely, I need to add a CPLUS_INCLUDE_PATH and I know the path relative to the *.nix file...
pxc1 has joined #nixos
Itkovian has quit [(Quit: My MacBook has gone to sleep. ZZZzzz…)]
<clever>
schmits: add an attribute to the derivation, CPLUS_INCLUDE_PATH = "-I${./include}";
tmaekawa has joined #nixos
<clever>
but keep in mind, it will import a copy of the include dir when nix-shell is ran
<clever>
and it will be an immutable snapshot of the headers
<schmits>
Thank you clever, this seems to work! I don't know what I am exactly doing there but I guess this is something I could imagine if I only knew functional langs like haskell?
<clever>
i learned nix before learning haskell
<schmits>
:-) a dumb question then, is there a nix language documentation around here somewhere?
<schmits>
Hmm, this approach does not seem to work for LD_LIBRARY_PATH! Why is that?
<clever>
nixpkgs may already be doing things with that var
iyzsong has quit [(Quit: bye.)]
bennofs has quit [(Ping timeout: 248 seconds)]
dash has quit [(Ping timeout: 246 seconds)]
<schmits>
is there a way to workaround this? How can I tell my build system to append a bunch of long /nix/store paths to to search paths. (Without hardcorded /nix/store paths in the build system)
DeaDSouL_ has quit [(Quit: Linux, is a world where its limitation is your imagination..)]
DeaDSouL has joined #nixos
DrWaste has quit [(Quit: Run away!)]
<clever>
schmits: refer to the derivations, like LD_LIBRARY_PATH=${hello}/lib
<schmits>
oh sorry maybe I didn't express myself well but this is what I am doing but it doesn't work :(
<clever>
what error does it give, how does it not work?
<nh2>
clever: bummer, something stopped working. For some reason `nix-store -r` suddenly started preferring cache.nixos.org again, instead of my binary cache URL. Both are in `binary-caches`. strace shows that it reads nix.conf with those in `binary-caches`, but it chooses cache.nixos.org for some reason even though it is second in the list
<nh2>
and strace shows that it doesn't even try to connect to mine
MoreTea has quit [(Quit: Leaving)]
DeaDSouL has quit [(Remote host closed the connection)]
hotfuzz has quit [(Quit: WeeChat 1.7.1)]
<joepie91>
okay, sanity check. my compiled list of operators (https://gist.github.com/joepie91/c3c047f3406aea9ec65eebce2ffd449d) says that the precedence of the boolean negation operator is *lower* than that of numeric operations like addition; this would mean that `!1-1` would be interpreted as `!(1-1)`, not as `(!1)-1`
<joepie91>
did I get my documentation wrong, or is this really how things are supposed to work in Nix?
<joepie91>
well that was poorly and confusingly phrased
<joepie91>
s/predence is lower/binding is weaker/
<joepie91>
precedence *
<gchristensen>
clever: ping
<joepie91>
(I am aware that `!1-1` is not valid code, which is why I can't figure it out through the REPL - I'm just wondering how it's interpreted by the parser)
<Infinisil>
Damnit did all my messages not come through?
<Infinisil>
joepie91: Well anyways, this works: `2*-2` evaluates to -4
<joepie91>
Infinisil: yeah, but that's numeric negation, not boolean negation
<Infinisil>
Ahh
<joepie91>
it has a difference precedence
<joepie91>
Infinisil: and no, I saw no messages at all from you :p
ris has quit [(Ping timeout: 268 seconds)]
<joepie91>
Infinisil: if you look at https://gist.github.com/joepie91/c3c047f3406aea9ec65eebce2ffd449d you'll see that numeric negation is at precedence 3, so a stronger binding than multiplication (at precedence 6), but a *boolean* negation has precedence 8 (ie. weaker binding than multiplication)
koserge has quit [(Ping timeout: 246 seconds)]
<joepie91>
I'm not sure if this has something to do with it being inherently impossible to combine boolean and numeric operators in a single sequence, but this precedence seems difficult to implement
<joepie91>
and I'm not totally convinced that I can wing it :P
<joepie91>
(by ignoring the precedence)
<clever>
gchristensen: pong
<Infinisil>
Ah I see now
<joepie91>
Infinisil: the official manual is unfortunately of no help here, since it does not document numeric operators *at all* :)
<Infinisil>
joepie91: Yeah this doesn't apply when the expression isn't even valid, there's nothing to parsed correctly
<gchristensen>
clever: a while ago during that kernel bug w.r.t. that obscure protocol, you had a magical way of super-duper forcing a module from loading. do you remember what that was?
<joepie91>
Infinisil: problem is that type checking is a separate concern from AST parsing
<clever>
gchristensen: ah yeah, one min
<joepie91>
Infinisil: so even if the end result after evaluation would be the same, ignoring the precedence of boolean negation relative to numeric operators would yield a different AST than the reference implementation would
<clever>
gchristensen: this will run false any time the user wants to load dccp
<gchristensen>
yeah, perfect :D
<nh2>
clever: I suspect it's something with cache priorities, cache.nixos.org has 40, my cache has 50 apparently. Do you know what those priorities are?
<clever>
gchristensen: this is also how the fuse root exploit works
<gchristensen>
ohh
<clever>
gchristensen: fusermount is setuid root, and if it cant find fuse in /proc/filesystems, it will just system("modprobe fuse")
<Infinisil>
joepie91: The reference implementation? There is no result of an invalid expression
<clever>
gchristensen: but modprobe can load its config via an env variable
<noffle>
globin: is there an equivalent of 'nix edit' in stable? I'd just like to view the defn of a package's derivation. Right now I go to github nixpkgs, which works, but I'd much rather be able to view it offline and re what's actually running on my machine.
<clever>
gchristensen: and if you mess with ulimit some, fusermount can fail to even open /proc/filesystems
<gchristensen>
:|
<sphalerite>
noffle: local checkout of nixpkgs and grep >_>
<joepie91>
Infinisil: there's no *evaluation* result, correct. doesn't mean that there's no AST representation
<globin>
noffle: nope but you can always do nix-shell -p nixUnstable
<Infinisil>
joepie91: You said something about the reference implementaiton being different
jgertm has joined #nixos
<clever>
nh2: ah, so you may want to adjust that, if you want to take over for bandwidth/latency reasons
marantz has joined #nixos
<Infinisil>
joepie91: I assume you mean the implementation of nix?
Sigma has joined #nixos
<joepie91>
Infinisil: what I'm saying is that (!1)-1 is the logical interpretation I'd expect, and that's how it works in other languages... but the parser code for Nix suggests that Nix would interpret it as !(1-1) instead, which is difficult to implement in my current model
<joepie91>
Infinisil: so I'm trying to verify whether I'm correct that Nix would interpret it as !(1-1)
<joepie91>
if so: I have a problem :P
<Infinisil>
joepie91: Well, just look through the source code
<joepie91>
because then my parser would also have to interpret it as !(1-1)
<joepie91>
Infinisil: yes, that's how I've put together that operator table in the first place
<nh2>
clever: the reason it started working: Before I populated /var/public-nix-cache, there was no nix-cache-info in it, so it fetched it from upstream, and thus my cache URL also had priority 40 (simply copying what cache.nixos.org had). But once I rsync'd my `nix-push` output over, it suddenly didn't pull that nix-cache-info file from upstream, thus obtaining priority 50. That's why it stopped working in the middle of the night, when CloudFlare inv
<joepie91>
I am explicitly looking for third-party confirmation that I understand the behaviour of the parser correctly
<nh2>
clever: I mean "the reason it stopped working"
<sphalerite>
clever: #TIL
<clever>
nh2: ahhh
<clever>
sphalerite: also, <foo> will desugar into builtins.findFile at parse time
<Infinisil>
joepie91: Haha, well if you got it from the source then it must be right, you could try modifying nix to output the ast so you check though
<joepie91>
Infinisil: thanks. that seems to confirm my suspicion :(
<joepie91>
Infinisil: didn't know there was a --parse flag
<Infinisil>
joepie91: Man pages are quite nice if you take the time to read through them :)
<joepie91>
Infinisil: they're also totally useless if you're looking for a specific thing :)
<Infinisil>
Yes :)
<joepie91>
(and you don't know where it's going to be, that is)
<clever>
nh2: my idea with the cachecache project, is that you point it to many caches (you would have to nix-push to a normal http server), and then it would multiplex for you, and manage choosing which cache to get something from
<Infinisil>
Well, searching through manpages helps sometimes
<clever>
i cheated a bit with writeText, to make it easyer to read
zeus_ has quit [(Remote host closed the connection)]
<Infinisil>
bbl
<nh2>
clever: I've added now an nginx rule `= /nix-cache-info` so that my cachecache server always delivers that file from upstream
<nh2>
that fixes it
<clever>
nix-push will only create that file if its missing
<clever>
so you can always edit it and make it a higher priority
<clever>
then your cache will take over when there is a conflict
<clever>
which you may prefer, since you have the same content as cache.nixos.org anyways, but at a higher speed
hotfuzz has joined #nixos
jonte has joined #nixos
<nh2>
clever: I generate nix-push locally and then rsync it to the server, so it will always create the file, and there's no flag to set the priority in nix-push, so I'd have to make a script that edits or deletes the file and make sure that people always use the script instead of nix-push itself, that seems too risky for me, so I make nginx ignore it. Given equal priorityes, nix seems to choose the one that's first in the binary-caches list (probably
jgertm has joined #nixos
<clever>
sounds like rsync is the problem area, it needs to not overwrite
<clever>
but yeah, i can see how just staying at equal priority would work
stanibanani has quit [(Ping timeout: 240 seconds)]
stanibanani has joined #nixos
fabian_a has joined #nixos
<sphalerite>
Is there a way besides FUSE for a userspace program to provide a "view" of another file (e.g. with a particular offset, or decrypting an encrypted one for instance) in which other programs can seek and write?
<sphalerite>
Sort of like a FIFO but with operations you have on normal files too, like seeking
<hyper_ch>
if JRE is the answer, then you're asking the wrong question IMHO :)
<sphalerite>
haha
<sphalerite>
that sort of thing would be useful for making disk images
<sphalerite>
e.g. mkdosfs doesn't support an offset parameter like mke2fs does
<sphalerite>
Building disk images is generally a pain
orivej has joined #nixos
ertes-w has quit [(Ping timeout: 260 seconds)]
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] FRidh pushed 3 new commits to master: https://git.io/vQuHf
<NixOS_GitHub>
nixpkgs/master 54eeab4 Frederik Rietdijk: Python: fix update script in case no PyPI releases are available for a given version
<NixOS_GitHub>
nixpkgs/master 034c1a2 Frederik Rietdijk: python.pkgs: several maintenance updates
<NixOS_GitHub>
nixpkgs/master c64d8ea Frederik Rietdijk: python.pkgs.jupyter_client: 5.0.1 -> 5.1.0
NixOS_GitHub has left #nixos []
<sphalerite>
hyper_ch: I suppose it could be done with seccomp/ptrace hackery as well :D
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] vcunat pushed 1 new commit to release-17.03: https://git.io/vQuHW
<NixOS_GitHub>
nixpkgs/release-17.03 72c9ed7 Vladimír Čunát: Merge #26628: treewide: setuid/setgid fallout...
<NixOS_GitHub>
[nixpkgs] dtzWill opened pull request #27095: csmith: init at 2.3.0, pull in Sys::CPU as dependency. (master...feature/csmith) https://git.io/vQubw
NixOS_GitHub has left #nixos []
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] gnidorah opened pull request #27096: maxx: init at 1.0.0 (master...maxx) https://git.io/vQubo
NixOS_GitHub has left #nixos []
stanibanani has quit [(Ping timeout: 260 seconds)]
<clever>
ToxicFrog: if you fix the value of $NIX_REMOTE (copy it from the normal user end to the unit), nix-shell will work again
<ToxicFrog>
If I were putting this in configuration.nix, that would be fine, but I'm not.
<ToxicFrog>
Aah.
<clever>
but its better to generate the unit file from nix
<ToxicFrog>
It would be better, and if nixos ever adds the ability to generate per-user systemd.user unit files, that's what I'll do
<clever>
yeah, that would help
<clever>
ive only ever used system units with the User= on them
<ToxicFrog>
But as it is, I want this to be enabled by default only for a subset of users, which configuration.nix doesn't understand how to do, sooooo...
<clever>
it would need the script to read the user, and then nop itself
<sphalerite>
ToxicFrog: rycee's homemanager would probably be a more elegant solution
<sphalerite>
ToxicFrog: I haven't used systemd user units at all yet but I plan to eventually get home-manager set up and use that for them
siel has joined #nixos
<sphalerite>
clever: you probably know tricks to get a nixos image smaller... right? :D
<sphalerite>
currently I'm using just <nixpkgs/nixos/modules/profiles/base.nix> for the config
<clever>
sphalerite: start by finding the top-level derivation (before it became a .img), and run something like this on it
<clever>
this lets you cut out locales you dont want
DeaDSouL_ has joined #nixos
<sphalerite>
When did the format change from en_GB.UTF-8 to en_GB.UTF-8/UTF-8?
<sphalerite>
What does the second UTF-8 mean? :/
<clever>
no clue
<sphalerite>
ah well, not really that important I suppose
DeaDSouL has joined #nixos
<sphalerite>
Anyway, I've torn through make-disk-image.nix to make it usable for writing UEFI-bootable USB stick images
cpennington has joined #nixos
<sphalerite>
(but breaking BIOS boot)
<clever>
it should be possible to do both
cpennington has quit [(Remote host closed the connection)]
reardencode has quit [(Read error: Connection reset by peer)]
reardencode has joined #nixos
<clever>
if you set boot.loader.grub.efiInstallAsRemovable it should work in a usb stick, and if you also set device, it should with via legacy, as long as you have both a bios boot, and a fat32 efi system
goodwill has quit [(Ping timeout: 276 seconds)]
DeaDSouL_ has quit [(Read error: Connection reset by peer)]
<sphalerite>
I'm using systemd-boot
goodwill has joined #nixos
<clever>
ah
<clever>
grub supports both
<clever>
systemd-boot, i havent even seen a limit on how many generations it puts in /boot, and i'm not sure if it even has a rollback menu?
<clever>
sphalerite: and also with an scp option, i could build a nixos kernel+initrd that has a full rootfs embeded into the initrd
<clever>
sphalerite: then i could upload that to petitboot, and kexec it instead
<clever>
which lets me remotely reinstall or repair a nixos
<sphalerite>
haha
uwap has quit [(Client Quit)]
uwap_ has joined #nixos
indi_ has quit [(Ping timeout: 268 seconds)]
uwap_ has quit [(Remote host closed the connection)]
uwap has joined #nixos
simukis has quit [(Quit: simukis)]
acertain has joined #nixos
uwap has quit [(Client Quit)]
<sphalerite>
Just reading about IBM POWER
<sphalerite>
"
<sphalerite>
IBM Power E870 can be configured with up to 80 POWER8 cores and 8 TB of RAM."
uwap has joined #nixos
* clever
drools
<dash>
chubby
uwap has quit [(Remote host closed the connection)]
uwap has joined #nixos
<dash>
clever: i guess this means 8TB isn't Big Data
<sphalerite>
That sounds like it would be good enough for zfs dedupe :p
<clever>
lol
hapticFeels has joined #nixos
hapticFeels has quit [(Client Quit)]
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] knedlsepp opened pull request #27104: gogs: Fix cyclic dependency on darwin (master...fix-gogs-on-darwin) https://git.io/vQzUJ
<joepie91>
I seem to have found a solution for the iffy precedence of boolean negation too \o/
justbeingglad has left #nixos []
<sphalerite>
joepie91: hah. iffy because they're booleans. xD
<avn>
sphalerite: I believe it can be enough ;) I definelly no more try enable ddt with lesser RAM
<joepie91>
sphalerite: nah, mostly iffy because it has a totally different precedence from what it'd have in most languages :P
<sphalerite>
joepie91: so what's this parser for and in?
<joepie91>
sphalerite: in JS, using PEG.js, for parsing arbitrary Nix code
<joepie91>
goal is to have it produce equivalent ASTs to the reference implementation
<joepie91>
eventually my goal is to have better developer tooling for Nix
<sphalerite>
Ah
<clever>
joepie91: one of my past projects needed an AST for nix files, with the ability to apply arbitrary mutations and then serialzing it back out to nix
<clever>
joepie91: for example, reading a configuration.nix file (a function returning a set, possible with complex expressions)
<sphalerite>
joepie91: So why this approach and not emscripten + original nix?
<clever>
joepie91: and then adding a simple key=value entry to it, overwritting one, or appending
<joepie91>
clever: the 'serializing back' already sorta exists in JS, just the parsing doesn't yet :p
<joepie91>
sphalerite: because I want a maintainable codebase that I can actually work with in practice and design tools around
<joepie91>
emscripten is nice to get black-boxed things to run in a browser, but it doesn't produce maintainable code
<sphalerite>
fair enough
<sphalerite>
but why JS? xD
<joepie91>
(and there can be significant performance hits)
<joepie91>
sphalerite: JS is my default language for things without specialized requirements, because of 1) ease of building maintainable/usable abstractions, 2) size of the existing ecosystem, 3) sane package management, and 4) generally-pleasant-to-work-with-and-simple language
<joepie91>
it's the fastest way for me from an idea to a working implementation
<joepie91>
(for values of 'working' that go beyond 'it looks like it works')
<sphalerite>
sane package management? :D
<joepie91>
yep
<dash>
there's a sane package manager for js now?
<dash>
i mean yes you can use nix
<joepie91>
this sounds like bait I'm not going to engage with :)
uwap has quit [(Client Quit)]
<joepie91>
it's virtually always the same discussion, yadda yadda leftpad yadda yadda disk space yadda yadda bloat yadda yadda install time etc.
<dash>
joepie91: oh i guess there's yarn now
<dash>
joepie91: nah
<joepie91>
throw in some poorly understood complaints about semver and automatic updates
<joepie91>
and ta-da
<dash>
"deterministic install order"
<joepie91>
dash: congratulations, you're the first person I speak to who, of their own accord, manages to point out the one valid criticism about NPM
<joepie91>
:p
freusque has joined #nixos
uwap has joined #nixos
<joepie91>
(technical criticism that is)
<dash>
joepie91: well uh
<joepie91>
that having been said, there's a reason I said "package management", not "package manager"
radvendii has joined #nixos
<joepie91>
NPM itself is of very poor quality
<dash>
joepie91: what is this channel about, affter all
<sphalerite>
joepie91: I don't know enough to really know much about what other people hate/criticise about npm, if you have an article or something that explains and/or dispels them I'd definitely be interested
<joepie91>
but NPM implements the model set out by Node, and the *model* is sane
<simpson>
Maybe.
<joepie91>
dash: even in here I've seen a good amount of misdirected comments :P
<radvendii>
I need a patched version of pjsip for a package I'm building, what is the recommended way to do this?
<sphalerite>
My only reason to believe that node's package management isn't sane is "everyone seems to say so", and I know very well that it's not a good reason :)
<joepie91>
which, respectively, address a lot of misdirected NPM criticisms, and the concept of small modules and why it's beneficial
jgertm has quit [(Ping timeout: 255 seconds)]
<sphalerite>
Right
<sphalerite>
Thanks
<sphalerite>
I also need to go to bed!
<sphalerite>
Gnight all
<joepie91>
sphalerite: good call. very few people actually understand how NPM or modular/nested dependencies work, this is especially true amongst people who don't actually use these things
<joepie91>
sphalerite: I'd be happy to explain more details if there are particular things you're wondering about :p
<joepie91>
night!
<simpson>
joepie91: Why is there only one NPM?
<simpson>
Like, if you *really* wanna talk about technical issues, we can talk about technical issues, but it's not gonna be fun.
<joepie91>
simpson: can you rephrase that? there's a few different things called "NPM", the registry, the client, the company
<simpson>
joepie91: All of those things, why is there only one of each?
<joepie91>
simpson: that is a question I've been hammering on for a while now, and I'd prefer if there were zero of the company
<simpson>
joepie91: I can understand that preference.
<joepie91>
the thing with NPM is that the company does an extremely poor job, the client is terrible from an implementation POV, and the registry is proprietary; however, the concepts it implements are sound and, even with the terrible implementation, still blow virtually every other language package manager out of the water based on purely technical/economic metrics
<simpson>
Can you show us those metrics?
<joepie91>
which is why at the same time I get extremely tired of people complaining about things they either don't understand or that aren't a problem, but also very publicly criticize NPM and all its operations
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] LnL7 pushed 2 new commits to master: https://git.io/vQzTQ
<NixOS_GitHub>
nixpkgs/master 7f7b74d Josef Kemetmueller: gogs: Fix cyclic dependency on darwin...
<simpson>
Well, yes, but the people who complain about the right things are usually seen as opinionated cranks.
<joepie91>
no disagreement there
<simpson>
For example, I would like us to stop doing JS and the Web entirely.
<joepie91>
:p
<joepie91>
simpson: and show, no, but I can certainly explain them. most notably, the nested dependency model significantly reduces dependency conflicts (to nearly zero), resulting in adding a dependency being essentially free (ie. there's no per-dependency cost), which in turn results in a lot of smaller dependencies instead of a few big ones, which means different parts of functionality can be independently versioned and maintained, which significantly reduces
<joepie91>
the cost of maintenance, audits, replacement, and so on
Guest9783452398 has quit [(Ping timeout: 260 seconds)]
<joepie91>
simpson: the ecosystem grows around the dependency model, and the fact that it's /really hard/ to cause a conflict has caused a lot of other desirable traits to flourish in the ecosystem (one of those being nearly universal semver adherence)
hiratara has joined #nixos
<joepie91>
the cost is now only in the actual surface of the dependencies you add, not the amount of dependencies itself
<simpson>
semver might be a local maximum. Not sure yet.
<joepie91>
(and many small dependencies have a lower total surface than a single big one, since you can more granularly pick the specific bits and pieces you need)
<simpson>
Aside from that, meh. Nix can do all of this and lets you use non-JS additionally.
uwap has quit [(Client Quit)]
mudri has joined #nixos
<simpson>
Also I'm still not sure whether modules deserve to know their own names.
<joepie91>
simpson: why do you think I'm here writing development tools for Nix? :P
<simpson>
joepie91: So that you can escape JS, Kurt-Russell-style?
<joepie91>
right now it's not realistic to suggest use of Nix to the average developer, in part due to the state of its documentation
uwap has joined #nixos
<joepie91>
even for me personally it's still too much hassle to rely on it during development
<joepie91>
that doesn't mean I don't recognize the technical benefits; it's just not at a point yet for me where it's a net win
freusque has quit [(Quit: WeeChat 1.7.1)]
<simpson>
My solution was to stop worrying about the average developer directly. Instead, I'll build things, and show folks the ideas from my things. They won't *use* my things, but they'll learn my ideas.
<joepie91>
besides that, no matter whether you use Nix or not, you're still limited to the package model introduced by a language, and the ecosystem that has grown around that model
<joepie91>
which means that even if I were to replace NPM with Nix, it'd still be beneficial for me to write in JS since the ecosystem is generally saner
<joepie91>
as a result of the language-native package management being less crap
<simpson>
I still worry that it's solving a problem that didn't need to be solved. Monte doesn't *have* language-native package management; it relies entirely on Nix.
<joepie91>
right now, on the metrics that actually matter (development speed, reliability, work required for maintenance, auditability, etc.), JS + NPM score higher than any other language I've looked at
<dash>
simpson: it doesn't do a very good job of it though
<simpson>
dash: We have like three people writing Monte. We need more people and modules before solutions for managing those modules will become palatable.
<dash>
yet
<dash>
yeah
<simpson>
joepie91: We'll have to agree to disagree; I don't think that we use the same rubric.
uwap has quit [(Read error: Connection reset by peer)]
uwap has joined #nixos
<joepie91>
simpson: well, you didn't specify yours :P
jgertm has joined #nixos
<simpson>
joepie91: What I'm looking for in a general-purpose language is how much of a *platform* it provides. If we assume that DSLs are good, it follows that the platform language needs to be a good DSL host. Also, there's several properties that are hard to implement in DSLs without also implementing them in the host language, so we want to make sure that the host language is sufficiently restrictive to keep
<simpson>
footguns at arms' length.
<simpson>
I'm only willing to even *consider* JS because of the massive effort put forth by the rest of the object-capability community on making cap-safe JS a thing, with Caja and all the ES5/ES6 work.
<joepie91>
simpson: I'm not convinced that DSLs *are* good, as a general rule of thumb... and that seems like a very academic view that doesn't really take into account the practical tradeoffs
<simpson>
But otherwise it's a terrible language for readability and it offers very little in the way of upwards meta while being tied in its runtime to a legacy of rushed decisions.
<joepie91>
I'd strongly disagree with "terrible for readability"
<dash>
joepie91: well, what's readability
<simpson>
You're thinking of the code that you would write. You're not thinking of the code that is *possible* to write.
<joepie91>
dash: the ability to efficiently and accurately convey the semantic meaning of the code to the reader
<joepie91>
simpson: I *am* thinking of the code that is possible to write; however, I'm looking at the optimistic end, not the pessimistic end
<joepie91>
simpson: if your argument is "it's easy to write unreadable code in JS", then sure, I will agree with you there
<joepie91>
but that is something very different from "terrible for readability"
<joepie91>
because at the same time, it's *also* easy to write *highly readable* code in JS -- it's just entirely dependent on the priorities of the developer
<simpson>
joepie91: Well, that's a pattern in my rubric; I judge languages not just by their idioms and popular styles, but also by how underhanded, obfuscated, hard-to-read, and convoluted they can get.
<dash>
joepie91: reasonable answer, i'm dismayed how many people want to say readability is about conveying the intent of the code or somesuch
<joepie91>
simpson: while that can be *a* metric, you seem to be using it as a critical metric; overlooking the optimistic end -- and the problem with that is that that optimizes your judgment for poor and unmotivated developers
<simpson>
joepie91: I imagine code review as adversarial: One person wants their code in the codebase, and one person wants good code in the database. Thus, there is a moment where a person tries to convince another person that their code, which may not be good, is indeed good.
<simpson>
So, how do you tell if code is good? By reading it. How hard can the code be to read? Depends on the language.
<joepie91>
simpson: that is, if you treat the pessimistic end of what is possible as your primary metric, you end up analyzing "what kind of code would be written by an incompetent developer", which isn't a terribly useful metric, as an incompetent developer will produce substandard code *either way*. instead, you'd want to consider what a competent developer *could* write, what the ceiling is for their performance in a given language.
<joepie91>
optimizing for mediocrity is just not useful in the end.
thc202 has quit [(Ping timeout: 260 seconds)]
<simpson>
joepie91: So why do you use JS instead of Smalltalk? We should all write APL and Factor, shouldn't we!
indi_ has joined #nixos
<simpson>
joepie91: More seriously, I'm afraid that I don't see our peers and contemporaries as skilled engineers and craftsmen. I see us all as painters in caves, mixing paints by hand and ignorant of fundamental concepts like point perspective.
<joepie91>
simpson: because like I said, I have not run into a language where the tradeoffs work out better [for general purposes] than in JS. I'm always open to learning about one, *if* somebody can provide a solid rationale that doesn't totally ignore half of the metrics (eg. developer productivity, ecosystem health, and so on)
<joepie91>
(to be clear: yes, there are cases where JS is totally unsuitable)
<dash>
we are all in the gutter
<dash>
but some of us are looking at the stars
<DavidEGrayson>
linus is looking at the potholes
<joepie91>
simpson: right, but that comes back to what I said earlier - it's optimizing for mediocrity. my goal is to not just keep improving my own competence, but also that of others, and to end up with more reliable systems in the end. to that end, it makes sense to invest effort into something that has a high optimistic ceiling of what a competent developer could accomplish, both for my own purposes and that of other people.
<joepie91>
simpson: worth noting that I do a lot of work on the 'making people better developers' aspect too, btw - it's not just sitting and hoping
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] vbgl pushed 3 new commits to master: https://git.io/vQzIr
<NixOS_GitHub>
nixpkgs/master ba757e6 Vincent Laporte: ocamlPackages.pgocaml: requires OCaml ≥ 4.00
<NixOS_GitHub>
nixpkgs/master 9f9ca26 Vincent Laporte: ocamlPackages.lwt: requires OCaml ≥ 4.00
<NixOS_GitHub>
nixpkgs/master 91e8a6a Vincent Laporte: ocamlPackages.csv: 1.4.2 -> 1.5
NixOS_GitHub has left #nixos []
<joepie91>
I actively work to root out bad practices from the ecosystem, teach people how to write better code, not to cut corners, etc.
<simpson>
joepie91: I understand your position well; I've worked with many folks who have held positions like this. Unfortunately, I have not seen efforts to teach people work half as well as simply *changing the process*.
<joepie91>
simpson: I constantly measure the effect :)
<DavidEGrayson>
In javascript, can't you just write "foo.blah = 1;" for any object named foo, even if it doesn't support a property named blah? I'd prefer to get a runtime or compile-time error.
<simpson>
joepie91: In lines of code? In pages per week?
indi_ has quit [(Ping timeout: 240 seconds)]
<DavidEGrayson>
I prefer typos to result in an error.
<simpson>
DavidEGrayson: Sure, that's a way to look at it.
<DavidEGrayson>
ASAP
<DavidEGrayson>
So then something like Typescript could probably catch errors like that.
<simpson>
DavidEGrayson: I look at it and think, "Why does `foo` allow callers to alter its attributes like that? That's so rude! Languages shouldn't allow that."
<joepie91>
simpson: that's output, not effect. I measure effect by tracking habits in the ecosystem, tracking where my writings spread, how public opinions about things change, what proposals get added for ES, how module usage phases in and out, and so on.
<simpson>
joepie91: Okay. But do you have graphs? I'm an SRE; if you really want to shift my view, you need to show me graphs.
<joepie91>
simpson: a considerable part of my time is spent simply on talking to people and reading forum threads and chatlogs and whatnot, to understand how habits are changing.
<joepie91>
simpson: nope, nor do I need them
<simpson>
joepie91: I agree that we should be metrics-driven, that we should be empirical, and that we should measure stuff. I think that computer scientists are pretty bad at actually measuring stuff and that we are often too totemic.
<joepie91>
simpson: I do want to point out that not all metrics are numeric
<dash>
joepie91: that only reveals stuff about people who *discuss* programming
<joepie91>
or graph-plottable
<dash>
joepie91: Hm. I guess the other question that comes to mind is why JS and not, say, Elm or ClojureScript
<simpson>
joepie91: But they're at least *comparable* and obey some axioms.
<joepie91>
dash: not necessarily. I particularly focus on people who ask questions, *especially* beginners, what answers they get, as well as third-party information about what people talk about in closed social circles
<simpson>
dash: It's the argument by supremum; JS supposedly sits at the best intersection.
<joepie91>
dash: I've seen no compelling arguments to use either Elm or ClojureScript
<dash>
simpson: My best argument against is how they look on your resume
<joepie91>
dash: this is a (freelance) developer indicating that they have trouble selling MongoDB to clients, in part because of the article I compiled
<joepie91>
that's second-hand information about opinions of non-social-developers
<joepie91>
one of many, but most data points are less clear
<joepie91>
simpson: you're missing the point here.
<joepie91>
simpson: I'm measuring ecosystem impact, not lost reads.
<joepie91>
er
<joepie91>
writes*
<simpson>
joepie91: I'm not a sociologist.
<joepie91>
that article does not provide numbers on ecosystem impact, it provides numbers on lost writes
<joepie91>
therefore it measures a totally different thing from what I'm interested in knowing
<joepie91>
them being numbers doesn't make it any more relevant
<simpson>
joepie91: What's relevant is that numbers can be *compared*, and what you're trying to pass off as "metrics" aren't necessarily comparable.
<joepie91>
simpson: what I do lives on an intersection between technology, sociology, psychology, and pedagogy.
<simpson>
joepie91: It reminds me of https://www.quorumlanguage.com/ , which really is an aggressively mediocre language that caters to lowest-common-demoniator thinking. They measure, and they write papers, but they're failing to climb upward. I think that maybe they're not measuring the right things.
<joepie91>
simpson: this is a "perfect or nothing" argument.
<simpson>
Huh? No, it's a question of priorities.
<joepie91>
these things are simply not numerically measurable. you can say "well, no numbers is bad, so let's not measure", or you can say "let's try to get as accurate a measurement out of this, even if not numerical, in an attempt to track impact"
<simpson>
joepie91: Let's say two events X and Y happen and impact your community. Which one had more of an impact? How do you tell?
<joepie91>
simpson: the question is too vague to answer.
<simpson>
joepie91: Okay. Now, here's the thing. As an SRE, when I get paged, I have metrics: Time taken to respond, SLAs impacted and measured, SRE-hours consumed, revenue impact to the business, etc.
<joepie91>
simpson: human behaviour and the spread of concepts, habits and opinions isn't something that's as trivially quantifiable and generalizable as you are demanding, nor is it always clear how to measure things without a very well-defined idea of what you're looking to learn. most of the time, it's an iterative process of trial and error.
<joepie91>
sure, but how does that relate to my goal of getting people to write better software?
<simpson>
joepie91: Okay. Can we iterate on JS and maybe produce something better?
<simpson>
joepie91: I can measure things like *which language the erroring service is written in*.
<joepie91>
theoretically, yes. in practice, it seems that everybody is too mired in either petty biased language wars or an overly academic view of programming, to actually produce anything that holds up as better in the real world.
radvendii has quit [(Quit: leaving)]
<simpson>
joepie91: How do you feel about the fact that ECMAScript's committee is filled with people who hold my views about languages *but* think that JS can be saved? They've been porting *academic language features* into JS for a few years now.
<simpson>
Maybe "academic" isn't such a curse word!
<joepie91>
simpson: I am very unhappy with the recent additions to JS.
<simpson>
joepie91: Excellent! Why?
<joepie91>
simpson: some of the features were long missing, but many of the features had absolutely no valid reason to be added, and have resulted in undesirable consequences.
<joepie91>
simpson: eg. instead of capitalizing on JS' prototypical object model, class syntax was added, and it has further muddied the waters and confused people on how objects actually work in JS.
<simpson>
Ah, sure. Classes aren't the best.
<joepie91>
some of the more recent things, such as async/await and ES modules, also have significant design issues.
<joepie91>
and generally solve non-existent problems.
<simpson>
Do you have an opinion on WeakMap?
<joepie91>
the problem with the recent additions to JS is that they are often substitutes for better documentation and education.
<joepie91>
adding features instead of explanations.
<joepie91>
people *already* have an extremely poor understanding of how JS works as it is (and it's not like it's difficult! it's just not documented well)
<joepie91>
and the additional features are only making this problem worse.
<joepie91>
I will have to investigate it more before I can form a useful opinion on it :)
hiratara has joined #nixos
<avn>
joepie91: JS have too many bracec in code (but thanks gods not sigils)
<simpson>
joepie91: Personally I'm disappointed that template strings aren't very extensible.
ilyaigpetrov has quit [(Quit: Connection closed for inactivity)]
<joepie91>
simpson: I'm very skeptical about how complex they already are. I haven't really seen a valid rationale yet for having tagged template literals, for example.
<joepie91>
one of the big benefits of JS as a language, was that it was a very /small/ language, in the sense of having a relatively small amount of syntactic constructs that were largely generic in nature
koserge has quit [(Ping timeout: 260 seconds)]
<joepie91>
some mishaps (like constructors) aside, generally speaking things were only implemented in the language itself if it was impractical to not do so
<joepie91>
the recent additions have *significantly* increased the learning curve of the language, and this is causing people to (over)use language constructs where generic abstractions would have been more appropriate
<simpson>
joepie91: In E, the tags are used to implement DSLs. We've practically shown in Monte that you can have DSLs for parsers, tree transformers, Monte itself, HTML, and more.
<joepie91>
simpson: right, and I still feel that it's not a desirable thing to have that feature in a general-purpose language.
<simpson>
joepie91: Also in E and Monte, template strings can be used as *patterns*, which perhaps motivates the flexibility more.
NixOS_GitHub has joined #nixos
<NixOS_GitHub>
[nixpkgs] Fuuzetsu closed pull request #27090: sxiv: Add support for custom config (master...config-sxiv) https://git.io/vQukz
NixOS_GitHub has left #nixos []
<joepie91>
it significantly increases learning curve, also on a per-project basis, because every project will essentially be written in its own language.
<simpson>
joepie91: Well, yeah. That's the way that projects *already are*; they have dialects. Monte aims to have *standard syntax* for dialects and DSLs.
<simpson>
And then, eventually, *tooling* that can understand the DSLs.
<joepie91>
simpson: anyhow, I should get back to my project before I discuss my entire remaining weekend away :P
<joepie91>
I definitely want to finish my Nix parser tonight
<simpson>
joepie91: Are any of these well-written: Angular, Backbone, Dojo, Ember, GWT, jQuery, MooTools, React, SproutCore, Vue? I hear that these are all big mature projects with opinionated libraries that require non-standard idioms.
<simpson>
IOW they have dialects.
<joepie91>
simpson: many of them are extremely poorly designed, some as a consequence of the time during which they were designed and the available environment support at the time -- not all, however.
<joepie91>
simpson: that having been said, I'm very skeptical of the DSLs introduced by the non-poorly-designed libraries
<joepie91>
(these are also all frontend libraries, which isn't a good baseline metric for JS, because it's largely "working around browser capabilities")
<simpson>
joepie91: Sure, JSX is terrible, but what's worse: Writing "jsx`whatever`", or writing JSX in JS comments?
<simpson>
I'm thinking of JSX, right? Or was it something else that was JS+HTML mashed into JS comments?
<joepie91>
I don't recall "JS+HTML mashed into JS comments" at all
<joepie91>
JSX is an actual custom-ish format
<joepie91>
that is, it's an extended variant of JS
<joepie91>
(and a standard JS parser will not understand it)
<joepie91>
(I also don't really like JSX for a number of technical reasons)
<joepie91>
but I really should get back to my project :P
indi_ has joined #nixos
<simpson>
Yeah, I'm getting back to battlestation-building.
Khetzal has quit [(Ping timeout: 240 seconds)]
<joepie91>
simpson: btw, to clear something up: I'm not using "academic" as a curse word -- in fact, that's why I explicitly said "*overly* academic". the problem is that when you only focus on the theoretical design and quantifiable metrics, you're going to miss things, and I often find that people and projects with a strictly academic view fail hard on the 'softer' factors, such as (driving/analyzing) human behaviour, documentation, economic tradeoffs, and so
<joepie91>
on. sure, theoretical aspects and quantifiable metrics are an important part when considering something, but they are not the *only* part.
<joepie91>
(and it's extremely difficult to discuss a topic when every argument about a softer factor is met with something along the lines of "yeah well, but the numbers/theory say...")
<hodapp>
On the other hand, calling a language "practical" is the sort of thing that can be used to dismiss literally any aspect of its design or any failing from a usability standpoint.
<joepie91>
hodapp: calling something "practical" doesn't somehow remove the requirement to provide a rationale supporting the claim :)
<joepie91>
any word can be used to dismiss any failing, that doesn't mean that it's any more valid to do so with one word than with another
<catern>
hey #nixos, I'm incrementally setting up NixOS on my personal machine (currently running Arch), here is my plan, can you tell me if this is crazy in any ways? 1. install multi-user Nix on top of Arch, putting /nix into a separate btrfs subvolume, 2. use Nix and install all the packages I want, 3. install NixOS in another btrfs subvolume, sharing the /nix subvolume with Arch, 4. completely switch from Arch to NixOS
<joepie91>
hodapp: ie. if somebody claims there to be 'practical benefits' in something, I'd just as much expect them to provide a rationale as to what those benefits are and why they are beneficial over the other options, all things taken into account
<hodapp>
It's certainly curious the sort of blindness that using a language extensively can grant a person, though.
<joepie91>
which is why experience with multiple languages is important :)
ertes has joined #nixos
hamishmack has quit [(Quit: hamishmack)]
earldouglas has quit [(Quit: leaving)]
<gchristensen>
clever: what if the extraConfig kernel override barfed if you passed in a param starting with CONFIG_ or had an '=' in there?
<gchristensen>
to catch common problems
<gchristensen>
(where common problems = how I wasted my last 1hr :P)