gchristensen changed the topic of #nixos-chat to: NixOS but much less topical || https://logs.nix.samueldr.com/nixos-chat
<clever> elvishjerricco: have you seen the recent nix profiling tools?
<clever> NIX_COUNT_CALLS=1 NIX_SHOW_STATS=1 NIX_SHOW_STATS_PATH=profile3.json hydra/bin/hydra-eval-jobs ....
<clever> it works on some nix binaries (including nix-instantiate)
<clever> but the json needs nix master i believe
lnikkila has quit [Quit: lnikkila]
<clever> ah, its also in nixpkgs master
<infinisil> If somebody has the energy, this thread needs a Nix mention: https://www.reddit.com/r/programming/comments/amgb1l/homebrew_200_released/eflsgbh/
<infinisil> Like, desperately
<gchristensen> https://lobste.rs/s/ogyn45/ansible_s_creator_on_state_contemporary this one too, but I'm in no mental state to write a coherent reply
<jasongrossman> infinisil: Done.
<infinisil> Nice :)
<clever> jasongrossman: up-voted!
jasongrossman has quit [Read error: Connection reset by peer]
jasongrossman has joined #nixos-chat
iqubic has joined #nixos-chat
drakonis has joined #nixos-chat
lassulus_ has joined #nixos-chat
lassulus has quit [Ping timeout: 272 seconds]
lassulus_ is now known as lassulus
pie__ has joined #nixos-chat
pie___ has quit [Ping timeout: 272 seconds]
endformationage has joined #nixos-chat
drakonis has quit [Quit: WeeChat 2.3]
<elvishjerricco> TIL Nix doesn't actually implement the maximal laziness paper. That's sad. Why is that?
<clever> before, { args }: { args2 }: let foo = constExpr; in ...
<clever> after, { args }: let foo = constExpr; in { args2 }: ...
<clever> elvishjerricco: just making this small change in nix-tools, has reduced the memory usage of release.nix by 2gig
<elvishjerricco> clever: Must because Nix lacks maximal sharing :P
<clever> i think the problem, is that functions (and the entire AST) is a tree of Expr objects
<clever> and that Expr returns a Value (which can be a lazy thunk, that runs an Expr)
<clever> but, when you eval that lazy Value, it only mutates the Value, and not the Expr it came from
<clever> so when you run the same constant-expr twice, you get different Value's
<clever> i think to fix that, we need 2 changes to nix
<elvishjerricco> Yea I've been poking around the Nix evaluation code for a minute. I'm not horribly surprised it's not extremely fast. It lacks lots of what Haskell does for speed, and doesn't have the maximal laziness I thought it had to make up for it.
<clever> first, detect when an expression is pure, (can be tricky?), and then store the result of an Expr inside itself?
<clever> another thing to note, is that an attrset, is a list of key/value pairs, and the keys must all be known
<clever> so you cant get lazyness in keys when you { key1 = foo; "${computeKey2}" = bar; }
<elvishjerricco> Even "impure" expressions should do that. Nix is much more pure if you pretend all readFile calls of the same path return the same data for a given session
<clever> what i mean by pure, is the scope chain
<clever> f = x: x * y
<clever> this is not "pure" and depends on what scope it gets ran in
<elvishjerricco> Oh. You mean no free variables?
<clever> although, the y cant ever change...
<elvishjerricco> I.E. combinatorial?
<clever> it depends on the scope its parsed/defined within
<elvishjerricco> Combinators*
<clever> and when you call f from different places, it always uses the y from where f was defined
<clever> not where f was called from
<clever> let me check some source...
<elvishjerricco> Yea there's a whole other theory accompanying lambda calculus about when you limit functions to only using their arguments
<elvishjerricco> Don't know much about it
<elvishjerricco> But it's supposedly kinda useful
<elvishjerricco> SK(I) is super important for it
<clever> all thunks in nix, are a pair of pointers, for the Env, and the Expr
<clever> the nix parser, will return an Expr
<clever> so if you parse, "5 * 5", it returns an ...
<clever> "5*5" turns into 5 AST nodes, the 2 inputs, ExprInt
<clever> then an ExprVar, that looks up __mul in the current scope
<clever> then an ExprApp, that partially applies the __mul function to the 1st int (__mul 5)
<clever> then a second ExprApp, to further apply it, (__mul 5) 5
<clever> elvishjerricco: you have permission to cry :P
<elvishjerricco> clever: Lol. "Current scope." Does that mean you can override the behavior of "*"?
<clever> if you then make a set, { x = 5 * 5; }, the value in the set, will be a thunk, conaining the current scope, and the root ExprApp from above
<clever> elvishjerricco: yes
<elvishjerricco> Ouch. I do not like overloading like that
<clever> elvishjerricco: this means that 5*5 == "boo!"
<elvishjerricco> More and more I wish Nix were just an embedded variant of Haskell :P
<clever> you need to either define __mul in a let block, or mess with scopedImport, to get such nastiness :P
<clever> but doesnt haskell allow you to redefine * as well?
<clever> infix * 1 ; * :: Int -> Int -> Int?
<elvishjerricco> The one thing it had going for me in terms of language theory was maximal laziness, which apparently it doesn't do.
<elvishjerricco> Yea Haskell can rebind syntax but you have to enable an extension
<elvishjerricco> And * only kind of uses this
<elvishjerricco> * is the kind of types
<clever> so, if i have a set, with { x = 5 * 5; }, then the value in the set, is a thunk, containing the scope for an Expr, and the ExprApp itself
<elvishjerricco> When GHC detects * in this kind, it treats it as such
<elvishjerricco> But otherwise it's a normal type operator
<clever> forceValue() is used any time nix wants something to cease to be lazy
<clever> and line 29 detects if it was a thunk, and will then extract the scope and expr out of it
<clever> it then runs expr->eval, and passes it a value to write things into
<clever> and ExprApp::eval, will just run eval on the 1st element (the (__mul 5)) and pass it the 2nd element (the 5), to create a vFun
<clever> so its turning an Expr + Value, into a thunk, i think
<clever> hmmm wait...
<clever> its not passing the __mul 5, the 2nd 5
<clever> its just running eval
<clever> which, winds up being recursive in this case, so we start over with the `__mul 5`, thunk
<clever> and, the __mul, is actually an ExprVar, that looks up __mul
<clever> so that will do a lookup of __mul in the current scope, and force it to be non-lazy, then COPY it from whereverit was found (so __mul only gets ran once, if it was lazy)
<clever> so now, on line 1044, vFun is a tPrimOp (which contains a c++ function pointer)
<clever> ahhhh!, maybeThunk is a function, that either turns an Expr into a thunk (laziness), or just runs the Expr!
<clever> elvishjerricco: this is likely where maximal laziness can be impacted
<clever> in this case, the 5 is an ExprInt, which bypasses thunk creation, and it increments nrAvoided
<clever> state.callFunction(vFun, *(e2->maybeThunk(state, env)), v, pos);
<clever> so, when this runs __mul 5, it skips creating a thunk for `5`
<clever> it then has to inspect what type of lambda __mul is, and if its { a }:, try to match the attrs up...
<clever> increments nrFunctionCalls
<clever> fun.lambda.fun->body->eval(*this, env2, v);
<clever> and then evals it...
<clever> oh, but __mul is a primop, not nix
<clever> callPrimOp(fun, arg, v, pos);
<clever> so it generates a Value of type tPrimOpApp, and sticks the 1st ExprInt(5) into it
<clever> elvishjerricco: ah, primopapp's are linked lists, with each arg in one
<clever> so, `__mul 5 5` is a tPrimOpApp, that points to a tPrimOpApp, that points to a tPrimOp
<clever> and when it hits enough args, it will follow the list, to find the c++ function*, and all args
<clever> and if you force: `let f = x: x * 10; in f 5`, it will be calling the `ExprApp` (that runs __mul), with a scope setup to contain x=10;
<clever> hmmm, but back to the original thing i fixed, `let f = map (x: "--foo=${x}") [ "a" "b" ]; in f`
<clever> the let block must be to blame, since the `f` Value would hold things...
<clever> ExprLet contains an attrs pointer, and a body pointer...
<clever> `let key = "value"; in expr` passes a list of attrs to the attrs pointer, and `expr` as the body
<clever> ahh, its the same type as { key = "value"; }
<clever> elvishjerricco: i think i found the fault
<clever> every time you eval `let x = y in body`, and `y` is more complex then a var/string/int/path, it will create a new thunk for y
<clever> and if that let block is inside a function call, it creates a new thunk for each evaluation
<clever> changing `maybeThunk` to reuse the Value, might help
<{^_^}> nix#2665 (by cleverca22, 10 seconds ago, open): adjust `Expr::maybeThunk` to reuse `Value` objects
<clever> gchristensen: did i go too far into reading nix source? lol
endformationage has quit [Quit: WeeChat 2.3]
iqubic has quit [Ping timeout: 264 seconds]
jackdk_ has joined #nixos-chat
jackdk_ has quit [Remote host closed the connection]
flokli has quit [Quit: WeeChat 2.2]
flokli has joined #nixos-chat
<steveeJ> I'll just try my luck again today :) any chance someone is maintaining a php5x derivation?
<steveeJ> FYI, nixpkgs-channels rev e09c0adc63d10249dac8f90313f91e1050861d3c still contains php56
<steveeJ> probably better to use 846d8f8305192dcc3a63139102698b4ac6b9ef9f, which is the last to contain it
<MichaelRaskin> (looks at #nixos-unregistered) Hmmm, negotiating with {^_^} …
jasongrossman has quit [Ping timeout: 245 seconds]
<joepie91> hahaha
<joepie91> MichaelRaskin: that's a pretty generous definition of "negotiating" :)
<gchristensen> aww I missed it
<MichaelRaskin> It's a bit one-sided
jasongrossman has joined #nixos-chat
jasongrossman has quit [Ping timeout: 245 seconds]
<infinisil> Damn, if anybody needs a website to compare compression algorithms, this is the one you want: https://quixdb.github.io/squash-benchmark/
<ldlework> cool
<ldlework> i don't see pied piper on there though
<infinisil> ldlework: I can't find any info on that
<ldlework> infinisil: lol
<infinisil> Oh wait, I just found it's something from a TV series lol
<ldlework> :P
<infinisil> ldlework: But, http://www.piedpiper.com/#technology
<infinisil> They mention something about "Award-Winning Middle-Out Compression"
<infinisil> "After shattering the theoretical limit of compression at TechCrunch Disrupt, we’re now able to transport even the most complex data files across our network within seconds.‎"
<infinisil> Um okay
<ldlework> those data files are so complex!
<infinisil> Imagine they used /dev/urandom to test their compression algorithm, concluding that theirs is the best, because all others take longer than their own
<ldlework> infinisil: it's totally make believe
<lejonet> I really love the sales pitches :P
endformationage has joined #nixos-chat
drakonis has joined #nixos-chat
iqubic has joined #nixos-chat
<MichaelRaskin> Phew. Homebrew officially supports Windows, but only through WSL
aszlig has quit [Quit: Kerneling down for reboot NOW.]
aszlig has joined #nixos-chat
* infinisil also made a comment on the homebrew lobste.rs post, https://lobste.rs/s/woczuy/homebrew_2_0_is_released_with_official#c_jvyfwq
<gchristensen> infinisil: how about my reply there
<infinisil> Savage!
<infinisil> Feels a bit like cheating though, I think haskellPackages are included there as well, and some other big package sets
<gchristensen> it isn't cheating
<gchristensen> they're available and packaged, how is that cheating :P
<infinisil> haskellPackages at least don't all build, because it's just packaged automatically
<gchristensen> sure
<infinisil> But I guess packaged doesn't mean can't break :)
<gchristensen> I think the meme of no adoption is worth fighting
<infinisil> But then again, programming language packages is usually not what non-devs are interested in. Maybe a differentiation between dev packages and non-dev packages wouldn't be too bad
<infinisil> gchristensen: Yeah that definitely
<gchristensen> even still we build like 20k things for darwin / linux
<gchristensen> (each)
<manveru> damn, my psu got really loud with some ticking noise from the fan :(
<gchristensen> oops, its done
<infinisil> What is done?
<gchristensen> the PSU is going to die for sure
<gchristensen> it is done for
<joepie91> this is a recall of like, bath aroma fizzy things
<joepie91> that were made to look like cupcakes
<joepie91> because, in a twist that nobody expected, kids like to eat things that look like cupcakes
<gchristensen> n!
* joepie91 facedesks
<gchristensen> they look delicious!
<joepie91> yes and they are very toxic lol
<joepie91> idiots...
<joepie91> oh apparently this type isn't toxic, it can just cause choking
<joepie91> 'just'
<joepie91> I don't understand how at no point in the entire process from design to sales to packaging to shipping, nobody thought "hey uh maybe this isn't a great idea?"
<infinisil> joepie91: A combination of stupid people is always a joy..
<MichaelRaskin> I would assume that if bath aroma were also toxic, that would be a next level of facepalm
<joepie91> so, I just ran into https://www.mattsleeps.com/
<joepie91> which is super interesting
<joepie91> a composable mattress
drakonis has quit [Quit: WeeChat 2.3]
<joepie91> shipped as multiple separate layers, that you can rearrange and turn around to modify the properties and firmness of the mattress; for each side individually for the 2-person mattresses
<joepie91> never seen that before :P
<joepie91> can't find a lot of trustworthy reviews about it though, but the ones I can find are positive...
jasongrossman has joined #nixos-chat
<gchristensen> ok, anyone want to write a nix salespitch ? https://lobste.rs/s/woczuy/homebrew_2_0_is_released_with_official#c_yjxbf6 I don't think I could type that much tongiht.
<joepie91> gchristensen: this is technically a work-in-progress, but it may be useful for copypasting purposes: https://gist.github.com/joepie91/9fdaf8244b0a83afcce204e6da127c7d (there may still be errors or missing details though :P)
<joepie91> has a bunch of todo markers, too
<gchristensen> want an invite to lobsters so you can post it?: )
<joepie91> gchristensen: I have a lobsters account, but I'm currently mentally focused elsewhere and don't want to shift my attention :P
<joepie91> first motivated day in a long while
<gchristensen> I hear that
<infinisil> gchristensen: Yeah I just saw that too
<infinisil> Some points that should go in: pinning nixpkgs, using older versions without problems, reproducible, multiple different versions at the same time possible
jackdk has joined #nixos-chat
<infinisil> I guess I'll give it a go, I'll keep it short though
<gchristensen> yeah, over do-it and it sort of turns people off
<infinisil> I'm gonna try to find a really old nixpkgs version that can build emacs to tell how good it's reproducibility and support for older versions is is lol
<infinisil> s/is//
<gchristensen> make sure to note *why* it is valuable to be able to do that (ie: old projects, or across multiple computers)
<infinisil> As in "Great support for old versions, hell I can build an emacs from 200X!"
<jackdk> hope that works. emacs moves on geological timescales.
<infinisil> Turns out bisection over 170k commits takes quite a while..
<andi-> I always wonder why all the npm people accept npm -g while working on multiple projects.. That sort of thing is a prime example when working with different projects and having conflicting tooling
<joepie91> andi-: the common recommendation is actually to install every tool locally into the project, the problem is that a lot of people come from eg. Python and assume that global tool installations are therefore also the correct option in JS
<joepie91> (there are more issues like that, like the never-ending onslaught of people coming in asking for a "Rails-like framework in JS", sometimes *very* insistently)
<andi-> I come from Python and it was never right IMO.. That's why you start each project with pyvenv/pip/..
<joepie91> andi-: Python is just one of the examples :)
<joepie91> things are system-global in most languages
<joepie91> and people bring those assumptions with them...
<andi-> I guess arguing with people in here what the correct way is is kinda pointless, most of us probably agree :)
<clever> gchristensen: there was a recent convo about how the rpi foundation added drm to the camera modules
<gchristensen> yeah, that is insane!
<clever> they basically added an i2c crypto chip to the camera board, and if missing, the firmware wont allow you to use the camera
<clever> its very much like the SNES drm, just an extra chip, and no involvement in the actual data stream
<clever> so you could just steal it off official cameras
<clever> but the rpi foundation still gets their cut, because you have to buy donor official cams
<gchristensen> great, so you're not buying a camera, you're just buying this chip.
<clever> they do also sell the chip itself, i heard
<gchristensen> what has the rpif said about its motivation?
<colemickens> this drm prevents you from using non rpi-cameras with the rpi? hows that possible?
<clever> colemickens: the firmware will refuse to allow camera use if a special chip is missing
<clever> colemickens: and there are some dedicated pins on the camera connector, just for that chip
<MichaelRaskin> And of course the DRM is already broken
<clever> `but couldn't because the RPi Foundation wanted $25 per DRM chip to make their cameras work`
<gchristensen> guh
<clever> https://twitter.com/marcan42/status/1088474607718297600 has the secret key for the DRM
<clever> https://twitter.com/cybergibbons/status/916932973357862912 they are also agaisnt you changing the serial# of the board
<infinisil> gchristensen: Whatdya think of this https://paste.infinisil.com/NTYCQS9ShI.txt
<colemickens> I guess I didn't realize firmware worked like that, thanks for the link clever.
<clever> colemickens: the rpi is basically 198 cpu cores on a single chip, plus several modules like hardware h264
<gchristensen> ...198?
<clever> colemickens: you have 2 VC4 cores, 4 ARM cores, and 192 QPU cores
<clever> the start.elf and bootcode.bin blobs run on the VC4, and implement all of the "gpu firmware" and "bios" level stuff
<MichaelRaskin> And of course GPU cores are the ones that actually control things like booting and everything
<gchristensen> infinisil: "of gcc" -> "of gcc or clang"
<clever> the 192 QPU cores are for pixel/vertex shaders
<gchristensen> looks great, infinisil
<gchristensen> oh, those cores.
<clever> and yeah, when the rpi boots, it starts running a boot rom on the VC4 core
<infinisil> Unfortunately my quest for an old emacs already ended in 2010 on 3f287cfb1d3571e8149c47a0460d15282e47dc45: https://gist.github.com/Infinisil/6efe9d6f3b1b478c481258d4bde2bd6d
<clever> that searches for bootcode.bin, loads it into the L2 cache, and runs it
<clever> that powers up dram, searches for start.elf, and runs it
<clever> and then start.elf loads linux into ram, and powers on the ARM
<clever> so you must use the VC4 cores (and currently, the foundation blob) to even boot linux
<clever> and that blob is responsible for dealing with hdmi output, and various power management things
<infinisil> Wait, I bet that commit just isn't cached
<gchristensen> $ time cp /hydra-dump-2019-02-02.pg_dump /zfs/gzip-9/
<gchristensen> real26m26.732s
<colemickens> infinisil: this is tangential question sort of but are nixpkgs builds often called reproducible or repeatable? I thought only some packages are truly reproducible byte-for-byte, or maybe my understanding is wrong
<gchristensen> very much of nixos is byte-for-byte reproducible, but you're right
<infinisil> colemickens: I think they're still reproducible, just not byte-for-byte
<infinisil> I guess it's not "truly" reproducible
<clever> infinisil: there is also the nasty problem of upstream deleting source tarballs on us
<gchristensen> good thing we mirror almost all ofthem
<infinisil> Yeah
<colemickens> clever: I think I kind of knew there was a blob involved in boot, but I'm guessing here... that blob is also used (or another blob, etc) is then used to talk over this physical camera interface and in newer revisions, they've made the physical camera link, firmware, utilize a pair of extra crypto chips nefariously.
Synthetica has joined #nixos-chat
<MichaelRaskin> It still would be nice to be able to have fetchTarball and fetchVCS as alternative ways of obtaining the same source
<clever> colemickens: the blob has always been involved in the camera
<clever> colemickens: the GPU blob implements a fairly fancy AV pipeline, that lets you interconnect modules that do hardware encode/decode, muxing/demuxing, video/audio playback/capture, and more
<colemickens> infinisil: yeah, wasn't trying to nitpick, was more curious if there was a set of terminology that I should use to distinguish. I think "content addressable" is used sometimes?
<colemickens> clever: oh, doh, it seems obvious now... thanks
<clever> colemickens: so you can just use that pipeline to do: camera->hardware_h264->muxer, and you get a fully formed .mp4 stream out of the GPU
<clever> the linux "drivers" generally have zero clue how to actually control the hardware
<clever> they just talk to the blob over an inter-cpu channel
<clever> and since all cores share ram, it can trivially write to buffers the other core owns
<clever> the VC4 side also lacks an MMU, so it runs a realtime os called threadx
<clever> and funnily enough, the ARM side has 2 MMU's
<clever> there is an extra MMU between "arm physical address" and "real physical"
<clever> so you can entirely ban linux from reading certain regions of ram
<clever> which would be of use to protect hdmi encryption keys and gpu firmware, if this was on a more strict device
<gchristensen> ye ol reverse mmu
<clever> in the current firmware, you can just send an "execute code" message to the gpu, along with a physical address
<clever> and it will just run whatevr VC4 opcodes you throw at it
<clever> and you are free to try and patch the DRM code in the camera, at runtime :P
<clever> and the very next function, https://github.com/raspberrypi/firmware/wiki/Mailbox-property-interface#get-dispmanx-resource-mem-handle is an api call i personally requested!
<clever> i was using that to implement my own proper drivers for the V3D core (the QPU and stuff)
<infinisil> gchristensen: I'm also adding "- Customize builds easily. Useful for using forks of projects, using alternate dependencies, or changing configure flags."
<gchristensen> neat
<infinisil> Because homebrew apparently just removed all sorts of customization lol
<clever> infinisil: this guy managed to steal commit access :P
<infinisil> Should I also point out that nixpkgs doesn't have as good support for macOS as it has for Linux?
<infinisil> clever: Yeah saw that xD
<jasongrossman> Has someone added the point that Nix is cross-platform? That was a VERY big selling point for me. (I no longer use anything except NixOS, but I didn't know when I started with it that that would be the case.)
<jasongrossman> I've already made that point on the Hacker News thread on the same topic.
<infinisil> jasongrossman: Is that a selling point for homebrew users?
<clever> jasongrossman: i once wrote a nixos module to boot with an iscsi rootfs, on the rpi
<clever> jasongrossman: and i later just shoved that module into an x86 laptop, and it just worked
<clever> jasongrossman: at one point, i even had an SD card, that booted on both the rpi and x86-64, with a single rootfs, and a single configuration.nix
* colemickens has read some of those gists
<jasongrossman> infinisil: Sure. I was exclusively a Mac user at the time I was most worried about this.
<jasongrossman> clever: AWesome!
<jasongrossman> infinisil: It wouldn't be an issue for someone who's exclusively a Mac user and planning to always remain that way, but if you exclude the non-programmers who aren't using homebrew anyway then there will be plenty of Mac users who don't want to restrict themselves to Macs forever.
<infinisil> jasongrossman: Not sure what you're trying to say, I think I'll not mention the not-as-good mac support for now, it's not too bad anyways
<jasongrossman> I'm trying to say "Nix is cross-platform".
<infinisil> jasongrossman: Ahh that, right
MichaelRaskin has quit [Quit: MichaelRaskin]
<jasongrossman> infinisil++ gchristensen++
<{^_^}> gchristensen's karma got increased to 68, infinisil's karma got increased to 51
<infinisil> I didn't want to make it any longer, but i did add a "- ..." point to note that that's not all
<gchristensen> infinisil++
<{^_^}> infinisil's karma got increased to 52
<infinisil> We should have some presets, "Selling Nix to Homebrew users", "Selling Nix to grumpy C developers", "Selling NixOS to Arch users", ...
<gchristensen> yeah :)
<clever> infinisil: homebrew2nix when? :P
<lejonet> infinisil: "Selling Nix to grumpy person"? :P
<srhb> Grumpy X developer is a pleonasm anyway. :-)
<lejonet> srhb: fair enough :P
<jasongrossman> lol at (with) srhb
<lejonet> srhb: so, as you mentioned it, can you give me the elevator pitch for "Grumpy X developer"? ;) xD
* lejonet runs and hides
<gchristensen> every X developer must be grumpy
<lejonet> gchristensen: only eX developers are happy? ;)
<lejonet> But wayland is gonna happen any year now, from what I hear
<srhb> lejonet: I meant "foo" rather than X11 :-P
<srhb> Or.. X.org or whatever it's called nowadays.
<srhb> What was wrong with XFree86?!
* srhb grumps
<joepie91> "that X thing"
<infinisil> (unrelated, but I just read this and thought the examples were funny)
<jasongrossman> infinisil: Huh. I grew up fairly near Scunthorpe, but this had never occurred to me.
<jackdk> I first saw that called the clbuttic mistake
<lejonet> srhb: xD