gchristensen changed the topic of #nixos-chat to: NixOS but much less topical || https://logs.nix.samueldr.com/nixos-chat
<vika_nezrimaya> which part of NixOS is the most stable?
drakonis has joined #nixos-chat
drakonis_ has joined #nixos-chat
Guanin has quit [Remote host closed the connection]
vika_nezrimaya has quit [Remote host closed the connection]
vika_nezrimaya has joined #nixos-chat
<aleph-> vika_nezrimaya: Ask in #nixos
<aleph-> This is the offtopic chat mainly
vika_nezrimaya has quit [Quit: ERC (IRC client for Emacs 26.2)]
vika_nezrimaya has joined #nixos-chat
vika_nezrimaya has quit [Ping timeout: 245 seconds]
<drakonis> ok what's the deal with the darch guy?
<drakonis> its still unclear to me what's the benefits
<drakonis> it looks heckin clunky
<drakonis> manage everything using docker images
<drakonis> every time nixos comes up, he swoops in and props up his own thing
waleee-cl has quit [Quit: Connection closed for inactivity]
<gchristensen> darch?
<drakonis> this thing
<drakonis> manage your system with docker
<drakonis> with dockerfile
<drakonis> so, its... questionable at best
<drakonis> it was created after the developer thought nix was too hard
<samueldr> that's quite the statement there
<samueldr> anything to back that?
<drakonis> ah yes i have it
<drakonis> he also repeats that quite often on HN
<drakonis> he also does it to fedora silverblue
<samueldr> that does not support the claim of "created after the developer thought nix was too hard"
<samueldr> considering this is a tweet from close to two years after they introduced darch
<aleph-> Heh
<aleph-> Heya drakonis samueldr
<drakonis> hey
<samueldr> right, oof
<drakonis> he makes a note to bring that up nearly every time he plugs darch
pie_ has quit [Ping timeout: 252 seconds]
<aleph-> Hmm, I really wanna riff off this sentry module at work
<aleph-> Since I want to write an elixir/erlang module for our error reporting software
pie_ has joined #nixos-chat
<drakonis> this guy has twitter hot takes
<drakonis> hot yikes
pie_ has quit [Client Quit]
<drakonis> nevermind, i don't care
<ashkitten> i've never found nix difficult
<ashkitten> it's a learning curve, yes, but the dsl is actually quite intuitive to use both before you realize it's a functional programming language and afterwards
<ashkitten> (as in, it's easy to think of it as just a "configuration format" for most use cases)
<drakonis> there's a few typing systems for it now, right?
<drakonis> and a couple ways to run foreign language code
<Ralith> the difficult thing about nix is the sparse documentation of idioms
<aleph-> ^
<aleph-> I love it but the docs do kinda suck
<Ralith> the docs that exist are good
<Ralith> but nixpkgs mostly survives on tribal knowledge
<ashkitten> also utility functions are largely undocumented
<ashkitten> i have to grep nixpkgs quite often to figure out what arguments to give something
<ashkitten> or even where it's defined
endformationage has quit [Ping timeout: 246 seconds]
drakonis has quit [Quit: WeeChat 2.4]
drakonis has joined #nixos-chat
drakonis_ has quit [Ping timeout: 252 seconds]
drakonis has quit [Remote host closed the connection]
drakonis has joined #nixos-chat
cjpbirkbeck has joined #nixos-chat
cjpbirkbeck has quit [Quit: Quitting now.]
cjpbirkbeck has joined #nixos-chat
cjpbirkbeck has quit [Quit: Quitting now.]
umachan has left #nixos-chat [#nixos-chat]
<eyJhb> I feel like I have way too many .swp files just laying around... I should properly close my editor properly instead
cjpbirkbeck has joined #nixos-chat
jtojnar has joined #nixos-chat
<joepie91> welp, just realized that my tinc config changes aren't actually causing a restart of the daemon
<joepie91> because they're stored in /etc and therefore changing them doesn't change the systemd service and therefore doesn't cause a restart?
<joepie91> this seems like a bug in the nixpkgs module
<joepie91> and I wonder how many more modules suffer from this bug
<joepie91> elvishjerricco: did you ever find a solution to this? https://logs.nix.samueldr.com/nixos/2018-08-30#1522242;
<ajs124> joepie91: I've seen this every now an then, so I'm sure there are lots of them. That's what we have restartTriggers for, but they often aren't properly configured.
<joepie91> ajs124: given that they appear to be undocumented, that is unsurprising :)
<ajs124> systemd.services.<name>.restartTriggers is documented? or do you mean in the modules themselves?
<joepie91> ajs124: in the Nix{,OS,pkgs} docs
<joepie91> it's not obvious that this is necessary specifically in the special case of environment.etc.
<joepie91> problem is that it excludes host configs
<ajs124> There's apparently 77 of them in nixpkgs, but some of them look like the probably won't even work. Although I'm not 100% on that.
<joepie91> this would have fixed that: https://github.com/NixOS/nixpkgs/pull/27660#issuecomment-321419593 -- though it seems to have not been made part of the PR for some reason?
<joepie91> without further comment
<ajs124> joepie91: yeah, it's not too obvious. I'm not sure I ever read all the way through those manuals, but they do seem to be missing some things, indeed.
<{^_^}> #66431 (by joepie91, 12 seconds ago, open): tinc: Service fails to restart on host configuration changes
<ajs124> hm. didn't https://github.com/NixOS/nixpkgs/commit/ddd13e13756ac74a592b38569263a675cf72ba39 add that? or does that just not work, for some reason?
<andi-> Am I the only person that misses an `systemctl show-config-file <service>` to retrieve the actually configuration of a service? Maybe that is just because I am on NixOS and want to paste someone a non-nixos snippet.. not sure
<ajs124> andi-: isn't that what systemctl cat does?
<andi-> not exactly but yeah that is what I usually use then
<gchristensen> there is another one which dumps all of the settings
<gchristensen> `systemctl show` shows all props
<andi-> I am very often just interested in knowing where the config file / directory is. And that varies a log between services
<gchristensen> aah
<gchristensen> location-on-disk, not content
<andi-> e.g. the nginx -c ${config} parameter (the ${config})
<andi-> well whatever is easier
<andi-> i would do `less $(systemctl magic service)` most of the time
<joepie91> I usually just `ps aux`
<joepie91> or alternatively `systemctl show foo | grep ExecStart`
<gchristensen> | cut -d= -f2 ... blah blah blah. I get what you mean andi- :P
<andi-> yeah
<andi-> I am thinking that if we had that information in a structured way (single file, single dir, list of those, …) we could also handle service reloading better.
vika_nezrimaya has joined #nixos-chat
edef has quit [Read error: Connection reset by peer]
edef has joined #nixos-chat
<gchristensen> anyone want to invent a search engine for a git repo, but where the index is on the history and not the current revision
<vika_nezrimaya> so... my RPi went outta memory when compiling gcc
<vika_nezrimaya> lol
<ajs124> swap. more swap.
<vika_nezrimaya> on an SD card?
<ajs124> no, NFS
<vika_nezrimaya> I don't have NFS in my network
<vika_nezrimaya> wait
<vika_nezrimaya> swap on NFS?
<vika_nezrimaya> wouldn't it be slow?
<ajs124> sure. but it won't OOM :P
<vika_nezrimaya> what if kernel networking stack gets swapped out?
<vika_nezrimaya> what if packets get lost?
<vika_nezrimaya> ugh
<vika_nezrimaya> I think I'm better off trashing my SD card completely
<vika_nezrimaya> >.<
<ajs124> Idk, but apparently the linux kernel does support this. Although I'm not sure who would deploy it.
<ajs124> You could also swap to a USB device.
<vika_nezrimaya> ajs124: ddg search says swapping over network is A Bad Idea(TM)
<vika_nezrimaya> And swap on NTFS is not possible because... umasks, rights and all of that stuff
<vika_nezrimaya> yea I'm better off trashing my SD card completely to do that
<tazjin> I'm writing a design doc for improved container image layering with Nix, DM me an email if you'd like access (input very welcome) (cc: gchristensen, adisbladis)
<ajs124> lots of things are bad ideas. you could try to set up cross-compilation or distcc.
<vika_nezrimaya> wouldn't it change the hashes?
<vika_nezrimaya> I go through all of this just to get the hashes from native compilation
<vika_nezrimaya> so the Pi could self-update
<vika_nezrimaya> but thinking about cross-compilation as a last-resort measure since it seems to be practical
<vika_nezrimaya> except I'll need my laptop for every single update
<ajs124> I think it shouldn't. At least if you do cross-compilation with qemu.
<vika_nezrimaya> qemu as in qemu-binfmt setup?
<vika_nezrimaya> Never worked for me
<vika_nezrimaya> random hangs in the middle of the build
<vika_nezrimaya> usually sed does this
<Miyu-chan> When was this?
<gchristensen> what kind of rpi is this?
<vika_nezrimaya> gchristensen: RPi2 (a rather old one)
<vika_nezrimaya> Miyu-chan: Um... around a month ago? it happens all the time
<vika_nezrimaya> I never got to compile nix that way
<Miyu-chan> Ah. :(
<vika_nezrimaya> I'm kinda sad about it, to be honest
<vika_nezrimaya> 'cause that could've been an easy way
<vika_nezrimaya> well... good news is - at the end of the month I'll probably buy an RPi4 and will be able to run NixOS images from Hydra :3
<vika_nezrimaya> wow my new ERC face for my messages looks definitely better with the dark theme
<gchristensen> vika_nezrimaya: I have 5 rpi cm1s in a carrier I'd like to use to run builds for hydra. but I also can't get past glibc.
<vika_nezrimaya> gchristensen: wait, I should have armv7l glibc on my RPi
<vika_nezrimaya> I dunno how to check though
<gchristensen> o maybe it wasn't glibc. it was something though :)
<vika_nezrimaya> gcc fails for me because OOM
<gchristensen> maybe it was gcc
<vika_nezrimaya> It probably should've been GCC because it OOMs
<vika_nezrimaya> 1G of memory on an RPi isn't enough
<vika_nezrimaya> I need some swap space...
<gchristensen> tell me about it ... cm1 has 512m
<vika_nezrimaya> oof
<vika_nezrimaya> the same as old pis
<vika_nezrimaya> so... Internet, should I trash my SD card for the sake of resurrecting NixOS on armv7l efforts? :p
<gchristensen> maybe #nixos-aarch64 could help provide more guidance?
<gchristensen> / opinion
<vika_nezrimaya> #nixos-aarch64 is, well, aarch64 :3 and the only aarch64 machine I have here at home is my phone
<vika_nezrimaya> it's running Android
<gchristensen> its really #nixos-arm
<gchristensen> but aarch64 was the initial project to inspire it
<vika_nezrimaya> #nixos-arm didn't seem to exist. I'm its op right now it seems?
<gchristensen> I know. it is called #nixos-aarch64 but it could be called #nixos-arm
<vika_nezrimaya> Once I get an RPi4 I'll probably use it to compile images for armv7l
<vika_nezrimaya> and I promise to publish them (if I won't forget :3)
<vika_nezrimaya> so everyone could use them
<vika_nezrimaya> because armv7l is in really bad state now, the last image I've encountered is several months old
<vika_nezrimaya> and the fact that it's based on unstable doesn't help
<vika_nezrimaya> btw anyone ever launched NixOS on a Pi4?
<vika_nezrimaya> oh well not the right channel
<andi-> Has someone followed up on the linux OOM thread linked a few days ago? I just had my system shutdown after an hour of trying to work with the limited amount of RAM still available.
<gchristensen> that OOM therad seemed totally confused
<gchristensen> the very first post said "No swap" and all the replies I saw were assuming swap
<andi-> I kept telling people (face2face) for years that OOMkiller stopped working for me almost 10y ago.
vika_nezrimaya has quit [Read error: Connection reset by peer]
<gchristensen> is that an instruction, or a "I just read"
<andi-> just an I just read
<gchristensen> gotcha. I'll read! back in a bit
<andi-> it tries to shed some light into why it doesn't work and larelgy supports my theory
<andi-> meanwhile maybe running oomd wouldn't be that bad..
<andi-> Also I don't understand how `curl | mpv -` can have audio delays if no disk i/o is involved in the playback..
vika_nezrimaya has joined #nixos-chat
<joepie91> same here, oom-killer doesn't seem to actually work
<joepie91> and I've seen others have the same issue
<gchristensen> iirc facebook made their own?
<ajs124> gchristensen: that's oomd
<gchristensen> ah
cjpbirkbeck has quit [Quit: Quitting now.]
drakonis_ has joined #nixos-chat
drakonis has quit [Ping timeout: 258 seconds]
drakonis_ has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-chat
drakonis_ has joined #nixos-chat
drakonis has quit [Ping timeout: 268 seconds]
Remosi has joined #nixos-chat
<tazjin> gchristensen: question for you (or anyone else involved in operating the official cache) - I'm wondering if we could get per-package download stats somehow? (Just a count, nothing more, to determine relative popularity)
<eyJhb> tazjin: no access :(
<tazjin> eyjhb: that link (with the /pub) should work πŸ€”
<tazjin> let me double check
<eyJhb> It said that I don't have access to this published document :/
<tazjin> ah I see why, one minute
<eyJhb> Damn, looks much more thoughtout that most of my suggestions
<tazjin> it's combining a few ideas that people had
<eyJhb> `FROM nixery.dev/…) share this layer maximum with the created image`, never thought of that. Having a `/layers:10/` option would be nice.
<tazjin> is that "remaining layers" or "maximum layers for the image"?
<eyJhb> Maximum layers to consume, but could be both. I think the remaning layers would actually be a nice thing
<eyJhb> But I would call that `/free-layers:10/` or something like that instead, to indicate it
eyJhb is now known as eyJhb
<tazjin> the way it's set up atm there are about ~30 or so layers remaining
<eyJhb> I hate those DDoS attacks against my VPS...
<eyJhb> Hmm.... But, you can quickly come up and use quite some layers
<eyJhb> E.g. when you have many ENV, LABEL, HEALTHCHECK, etc.
<eyJhb> Would we worth checking how many I normally use
<tazjin> I suppose you don't deploy on k8s?
<eyJhb> Nope, I have never used k8s :)
drakonis has joined #nixos-chat
<tazjin> eyJhb: curious, if you have a lot of `ENV` set in your image builds, do you build separate images for different environments?
<tazjin> Or what are those used for?
drakonis_ has quit [Ping timeout: 268 seconds]
obadz has quit [Ping timeout: 268 seconds]
<eyJhb> tazjin: I might have been a little over the top with that one, it might just be 5... And might not be used if I used a nix way of building
<eyJhb> But seems like most of my images is around 10 layers, based on this `find . -name "Dockerfile" -exec bash -c 'echo $0 && grep -E "(RUN|COPY|ENV|VOLUME|HEALTHCHECK|ENTRYPOINT|LABEL|FROM)" $0 | wc -l' {} \;`
obadz has joined #nixos-chat
<tazjin> that's closer to what I would expect
<eyJhb> *sidenote* my bash skillz really need some finetuning
<eyJhb> But yeah, should be fine. However, if you only need 5 layers, then having it use the remaning layers for cache would be nice
<tazjin> I actually have a theory that it might not be beneficial to use all available layers
<eyJhb> How come?
<tazjin> as per the section about direct vs. indirect references, if you have say a tuple of derivations that always appear together then it wouldn't make sense to split those into separate layers if you still have an extra layer available
<eyJhb> But increasing buildtime on the server might be the most beneficial
<tazjin> because you'd bust both layers the nest time something else is added
<tazjin> *next
<tazjin> I'm trying to make these decisions based on what's likely to happen in terms of changes (hence also the extra consideration of the top-level packages in the last section)
<eyJhb> True.. I can see that
<eyJhb> Btw. the image under (Close to) ideal ... is misisng/not loading for me, just blank space
drakonis_ has joined #nixos-chat
<tazjin> I thought I fixed that, can you hard reload?
<eyJhb> Yeah, fixed!
<eyJhb> Hmm, you seem like you have a better a handle on it than me tazjin ;) I just like the idea of nixery. But yeah, you might be right
<eyJhb> Also... Don't know if you could construct some "master" layer, that almost always gets added.
<tazjin> turns out - so far - the closest to that `master` layer is just the `libc` layer :)
drakonis has quit [Ping timeout: 252 seconds]
<eyJhb> `libc` masterrace I suppose? :p
<eyJhb> Btw. random question, do you code in Golang tazjin ?
<tazjin> only when I can't avoid it πŸ˜‰
<eyJhb> It is going to be like that I see! ;)
drakonis has joined #nixos-chat
<eyJhb> Reading up on error handling and proper ways, is both.. Good, and.. Boring, but good. But.. Dear god kill me. But that is essentially the same you feel if you don't...
<eyJhb> If anybody have some good materials for how to do proper error handling (generic or Golang) I would appreciate it
drakonis_ has quit [Ping timeout: 264 seconds]
<tazjin> also, Common Lisp's error handling is very interesting
<tazjin> it essentially lets you attach code to exception handlers that can make decisions and "rewind" to certain states, modify something and try again
<eyJhb> Rust syntax hurts my eyes for some reason
<eyJhb> Will look at it!
<eyJhb> Currently just finished up on this very basic thing - https://dave.cheney.net/2016/04/27/dont-just-check-errors-handle-them-gracefully
<eyJhb> But just this, with wrapping errors etc. is SO MUCH BETTER than what I am currently doing... Which is.. A disgrace
<eyJhb> *nobody look at my public code
drakonis_ has joined #nixos-chat
drakonis has quit [Ping timeout: 264 seconds]
<gchristensen> tazjin: historically we've avoided that sort of thing for privacy, but we could possibly change that opinion
<ajs124> if my filesystem (xfs on a raid5) seems to have performance problems with lots of small files, e.g. building system-path derivations, how could I debug or tune that?
<gchristensen> I would google "Brendan gregg xfs performance"
<gchristensen> and see what happened
<ajs124> I don't see anything xfs specific, when I google his name, but he seems to have interesting writeups about ebpf and perf and stuff
drakonis has joined #nixos-chat
<Taneb> ...I was *just* talking about Greggs (UK fast-food-ish bakery chain) in another channel, now I'm really hungry and the only one in this city closed two hours ago
<drakonis> them shibboleths tho
<eyJhb> Taneb: That sucks.. :/ Go for pizza instead!
<infinisil> I just had some delicious prosciutto pizza, mmmm
<infinisil> Hm, just got the second 504 on GitHub today
<tazjin> gchristensen: I'm not interested in any per-user data, and not necessarily even the whole package set. I think the top 25% of packages or something in order would already be majorly helpful
<eyJhb> infinisil: stop taking down Github :D
ixxie has joined #nixos-chat
<gchristensen> hmmm I should get myself a live CD and rename my zfs datasets so I don't back up /nix anymore.
<drakonis> wouldn't that be a massive waste of bandwidth?
<drakonis> regenerating the store every time you spin up an install?
<gchristensen> on the other hand, I only install once each computer, and then back up 6 months of /nix/store's
<eyJhb> gchristensen: thanks for reminding me to cleanup Docker image space used
<yorick> gchristensen: you back up the nix stores?
<gchristensen> yeah but not on purpose
<yorick> I'm thinking of setting up a local cache and nix copy --all to it occasionally
drakonis1 has joined #nixos-chat
drakonis has quit [Quit: WeeChat 2.4]
drakonis_ has quit [Ping timeout: 250 seconds]
drakonis has joined #nixos-chat
drakonis has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-chat
drakonis1 has quit [Ping timeout: 272 seconds]
<Remosi> tazjan: Hmm. So the goal is to partition the graph into a subgraphs that are easily cachable units.
<Remosi> I wonder if a greedy recursive algorithm would work.
<tazjin> Remosi: the main problem is dividing them into units that are likely to be grouped in the same way if changes are made to the graph. For example in the image I have in the doc, lets assume that F is ncurses - we add a second program G to the image that also depends on ncurses, suddenly caching D,F together is not the optimal thing to do anymore because we now have a G->F dependency, too
<Remosi> Right exactly.
<tazjin> however inspecting the whole package tree would reveal that there are a lot of programs that depend on ncurses, so we might have been able to use that information during the earlier grouping decision
<tazjin> I have to do a quick run to a store before it closes, but I'd be interested to know what you're thinking - back in ~15!
<ashkitten> why does docker limit the number of layers?
<gchristensen> because layered filesystems have limits
<Remosi> So if you have D->F (eg foo and foo-data) then that is likely to be a useful grouping.
<gchristensen> also, each layer adds a performance hit
<gchristensen> since it has to look down the layers for if the file is in that layer or not, or if a later layer "erased" it (with a white-out file)
<ashkitten> gchristensen: this is relevant to me because my gf is working on a linux distro that will distribute packages as squashfs images to be overlaid on the rootfs
<gchristensen> neat
<Remosi> So any package that only has one incoming edge, then that entire tree can be grouped together.
<gchristensen> sounds a bit like Snap
<ashkitten> i'm pretty interested in the performance implications of such a system tbh
<ashkitten> if there's been research into optimization for it
<gchristensen> there are a lot of problems with overlay filesytsems
<gchristensen> it is very difficult to achieve posix compatability
endformationage has joined #nixos-chat
drakonis_ has joined #nixos-chat
jtojnar has quit [Ping timeout: 245 seconds]
<averell> maybe it would make sense to factor in layer size when chunking in derivations? in reality nobody would mind an extra curl or curses lib if it's doesn't add anything percentage wise, but anything that pulls in half the haskell ecosystem is different.
drakonis1 has joined #nixos-chat
drakonis has quit [Ping timeout: 250 seconds]
<aleph-> Sup folks
<tazjin> averell: interesting idea, but I'm not sure if we can access closure size in a reasonable way - I guess it could be precomputed, but lets put that in the parking lot - I'll make a note of it
<tazjin> Remosi: are you talking about one incoming edge in the entire package set, or in the specific image's graph?
<tazjin> I'm currently doing the latter, with the exception that if that package has popularity above some threshold (which I've yet to figure out) in the whole set, it should be separated out
<Remosi> Hmm.
<tazjin> that's just to catch the ncurses, openssls etc. of the world
<Remosi> Right.
<Remosi> Hmm.
<Remosi> I think you're trying to find the dominator of a set and partition along those edges.
<tazjin> agreed, that's step 1 - followed by singling out some of the leaves
<tazjin> averell: just randomly noticed that closure size is actually included in the export references graph
drakonis1 has quit [Quit: WeeChat 2.4]
<averell> cool. i'd definitely be interested in nixery request stats too :)
drakonis has joined #nixos-chat
drakonis1 has joined #nixos-chat
drakonis2 has joined #nixos-chat
drakonis_ has quit [Ping timeout: 272 seconds]
drakonis1 has quit [Ping timeout: 276 seconds]
<tazjin> Remosi: actually that's a really useful way to think about it, building a dominator tree out of the image tree makes it very convenient to do the rest!
<Remosi> I think there is a well defined way of expressing this.
<Remosi> But I'm not sure what it is.
<ixxie> of defining graphs?
<Remosi> Of partitioning graphs into multiple subgraphs that are related.
<tazjin> yeah, there is - I'm reading http://www.cs.au.dk/~gerth/advising/thesis/henrik-knakkegaard-christensen.pdf atm which explains various methods of constructing it
<Remosi> Something like cliques or graph labelling etc.
<tazjin> section 2.6
<ixxie> what are dominators?
<ixxie> never heard that term, and I studied at least basic graph theory
<tazjin> nodes in a tree that all paths to that node must path through
<gchristensen> oh cool a christensen
<gchristensen> nice to see the family name so tightly connected to the project
<ixxie> so the goal is to factor the full dependency graph into cachable chunks right?
<tazjin> yep, with the constraint that there is a maximum number of possible chunks
<tazjin> s/that node/another node
<ixxie> hmm
<ixxie> I think about two years ago I was looking at graph clustering algorithms
drakonis has quit [Ping timeout: 250 seconds]
<ixxie> a lot of clustering algos will have a parameter for the number of clusters but I don't know about in the case of graph clustering
<ixxie> a review paper
<ixxie> "Graph clusteringis the task of grouping the verticesof the graph into clusters taking into consideration the edgestructure of the graph in such a way that there should bemany edgeswithineach cluster and relatively fewbetweenthe clusters."
<ixxie> would this be an accurate description of the problem at hand, excluding special constraints such as restricting the number of clusters?
<tazjin> sort of, I think - the actual practical issue is that we'd like to have arbitrary container images (with different program sets in them) share as many layers (i.e. "clusters") as possible for any overlapping dependencies
<tazjin> so it's not just a grouping within the graph for a given image, but taking into account information from the whole package set to influence the decision
<gchristensen> another thing I wanted to try and do was have the images share as many layers as possible between upgrades of nixpkgs too
<ixxie> is this project writted out somewhere?
<ixxie> written*
drakonis has joined #nixos-chat
<tazjin> gchristensen: hmm, wouldn't they already do that?
<gchristensen> currently yes, but if grouping different packages together in to one layer, it reduces the chances
<gchristensen> anyway
<gchristensen> you have people way smarter than me working on this :P I'm glad you are
<tazjin> heh, don't undersell yourself, afterall you're the Nix MΓ€dchen fΓΌr alles πŸ˜‰
<ixxie> tazjin: hmm doesn't F have only 1 ref?
<tazjin> ixxie: one direct, two indirect
<tazjin> (second table)
<ixxie> aha
<Remosi> I think that there are some interesting graphs where counting edges isn't quite going to do the right thing.
<tazjin> Do you have an example?
<ixxie> dang this is a super facinating project, but I don't think I have time to dive into it
<ixxie> but I will say this: it may be helpful to put the full nixpkgs graph into a graph database to make querying it easier
<ixxie> at the office we use ArangoDB, but it seems Dgraph is also making some waves
<gchristensen> I've done something similar once as an experiment for visualizing build progress with neo4j
<gchristensen> I reached out to a hosted neo4j thing and they didn't want to sponsor the project.
<ixxie> well ArangoDB and Dgraph are both FOSS
<andi-> thats a pity. While in the train to Fosdem this year we sat with the Neo4J guys and fixed our nix expression with them. They seemed very supportive :/
<tazjin> andi-: the devs don't necessarily make sponsorship decisions though ;-)
<andi-> yeah, but the company seemed small enough
<andi-> I think they had the people in charge sitting next to us..
<tazjin> gchristensen: this was the build dependency graph though, right?
<ixxie> tazjin: what other graphs are being considered?
<tazjin> ixxie: for the container images we only care about the runtime dependency graph
<tazjin> i.e. build-time dependencies like compilers don't matter
<ixxie> right
<Remosi> ( some nodes ) -> A, B. A, B -> C, D
<Remosi> ^^ there should be no 1 edged nodes.
<Remosi> But a,b -> c, d is a sensible cluster.
<tazjin> right, hmm
<ixxie> so we have the local (image) graph, and a global (nixpkgs) graph - these are the primary data sources; using this data the goal is then to optimize the layering to reduce the rebuilds necessary in realistic use, thus accelerating image builds?
<gchristensen> right, tazjin
<tazjin> ixxie: yes, though the goal is maybe not the speed (afterall reshuffling into different layers won't actually rebuild) and more the amount of layers that need to be downloaded
<ixxie> aah hence the caching
<ixxie> right I see because Nixery is an image registry
<gchristensen> _woohoo, writing backups at 75M/s, over wifi_
<tazjin> hence also averell's suggestion to consider closure size
<ixxie> right so we can think of it as a graph with weighted nodes
<manveru> so i finally got the site generator into a presentable state: https://gitlab.com/manveru/euphenix
<tazjin> oh btw, I should point out that due to the filesystem overlaying it's possible to put something in a layer independently of its parents and children
<manveru> Will publish some more sites built with it tomorrow, but ran out of time today :|
<tazjin> actually that might not be as useful as I thought for a few seconds
<tazjin> because you wouldn't be independent of the children anyways
<tazjin> manveru++
<{^_^}> manveru's karma got increased to 19
<manveru> i think the readme is now longer than the source code :)
<tazjin> manveru: gonna add taking a look at that to the todo list! Do you actively use it for something?
<manveru> yeah, i'm building three sites with it at the moment
<gchristensen> :o
<manveru> finally decided to actually give it its own project :)
<gchristensen> I'll have to give it a go for by blog
<tazjin> how does it compare to styx? (apart from having more recent commits than 3 years ago πŸ™ƒ)
<manveru> well, it's very light on IFD
<manveru> and doesn't do the whole themes stuff
<manveru> it's also not 100% Nix, so you can actually write normal HTML/CSS/Markdown in your project
<manveru> i'm sure gchristensen will want to combine it with asciidoc or docbook or something, so that's also not hard to add :P
* gchristensen 's pretty happy for markdown for his bolg
<manveru> likewise
<manveru> i mostly just wanted something that only needs maybe 3-4 lines of actual nix code, the rest is inferred and/or at the place where you actually look for it
<manveru> anw, it's still all in the "make it work first" stage, i'll work on rewriting the ruby parts in something faster so initial compilation is a bit nicer
<tazjin> in something faster, like Nix πŸ˜›
* tazjin hides
<manveru> :P
<manveru> yeah... i tried that first, didn't work out so well ^^;
<tazjin> Remosi: wondering if the situation you pointed out might be detectable by a stark decrease in the number of references in to a cluster
<manveru> might still add the option to write templates in nix, so you get even more fine-grained builds, but not really a priority atm
<manveru> anw, gn8 folks
<tazjin> sweet dreams
<ixxie> gnight folks
<ixxie> tazjin: good luck with this project; I will have to try and force myself not to think of this facinating problem!
<tazjin> ixxie: if you do accidentally think about it, let me know the outcome πŸ˜› good night!
<ixxie> tazjin: one last thought is that it starts to seems to me like the problem you have can potentially be formulated as a giant nested graph, because you have the full nixpkgs graph, the registry's subgraph, the image subgraphs, data flow graphs to clients of the registry, etc
<ixxie> tazjin: and it may be that such a formulation will help elucidate the constraints you want to optimizing for, and the algorithm you wish to produce
<ashkitten> rip nixuser, long live home-manager?
<ixxie> or maybe it can also be seen as an index building problem; you put this super graph into a graph database, construct the right sorts of indeces, and query that to produce your layer partitioning
<ixxie> anyway
<ixxie> time to sleep
<ixxie> that's probably nonsense anyway
<ixxie> gnight
<drakonis> ashkitten: that has been the status quo for while now
<ashkitten> yeah, but the pr was just closed
<infinisil> ashkitten: It's been dead for a while, I just did the official funeral :P
<ashkitten> :p
<ashkitten> rest in pieces
<drakonis> i'd like to have it integrated with nixos
<drakonis> directly
<drakonis> but its not going to happen anytime soon
<ashkitten> yeah
ixxie has quit [Remote host closed the connection]
<infinisil> Why should that be integrated with nixos?
<ashkitten> i'd also like to fix a bunch of things in it
<drakonis> its still treated like a seperate entity
<drakonis> with separate modules
<ashkitten> and it reimplements a bunch of things nixpkgs already does
<drakonis> seperate
<ashkitten> like systemd services
<drakonis> ^
<drakonis> duplication of code you see
<ashkitten> and it doesn't even use the system nixpkgs config
nckx has quit [Ping timeout: 248 seconds]
<ashkitten> also the home-manager systemd module is inferior to the nixpkgs one
nckx has joined #nixos-chat
<infinisil> ashkitten: Is it?
<infinisil> How so?
<ashkitten> it doesn't do anything for you
<ashkitten> if you use it you're literally just writing a systemd service file but in a different format
<infinisil> What do NixOS services do for you that home-manager ones don't?
<ashkitten> the nixos module is much nicer to work with
<infinisil> Well I guess home-manager could use the Install section by default, but other than that I still don't see what NixOS modules do more
<infinisil> I guess the `script` thing yeah
<ashkitten> yeah
<ashkitten> idk, i like nixos's module better
<infinisil> But that's really just a convenience thing for `ExecStart = pkgs.writeScript "foo" "..."`, which is not much longer
<ashkitten> well if i was using nixos-19.03 it'd be `ExecStart = pkgs.writeScript "foo" "#!${pkgs.runtimeShell} ..."` but that's beside the point
<infinisil> Ah right
<infinisil> Yeah I you have a point
<ashkitten> since in nixos-unstable there's writeShellScript
<ashkitten> which is a wrapper
<infinisil> NixOS also has a bunch of specialized options
<infinisil> Which don't feel as alien as the UpperCaseSystemd names
<ashkitten> yeah
<ashkitten> like path
<infinisil> And since recently, systemd.service.*.confinement
<ashkitten> yup
<infinisil> From #57519
<{^_^}> https://github.com/NixOS/nixpkgs/pull/57519 (by aszlig, 21 weeks ago, merged): nixos: Add 'confinement' options to systemd.services
<ashkitten> i haven't looked at that yet, it seems nice
<ashkitten> the fact that there's a popular rust crate called "nix" with nothing to do with the package manager gets me every time
<aanderse> joepie91: ping