worldofpeace_ changed the topic of #nixos-dev to: #nixos-dev NixOS Development (#nixos for questions) | NixOS 20.03 BETA Announced | | | 19.09 RMs: disasm, sphalerite; 20.03: worldofpeace, disasm |
drakonis has quit [Ping timeout: 240 seconds]
abathur has quit [Read error: Connection reset by peer]
abathur has joined #nixos-dev
<worldofpeace> gchristensen: I really love this PR, wish I could review it
<gchristensen> oh why can't you? :)
<gchristensen> just busy?
<worldofpeace> Well I did just look at the changes and they look good, but yeah, I don't have the time for the in depth worldofpeace review gchristensen 😸
<gchristensen> =)
<ryantm> cole-h: treewide PRs cause merge conflicts for one.
bhipple has joined #nixos-dev
<genesis> i get a bit crazy with some simple derivation, i think i missed something
<genesis> i try to package a script from a derivation , if i don't substituteAll for example @jq@/bin/jq , it don't find jq, ok i inherit jq , but after that, it doesn't find file in the package
<genesis> for example it can't find a plugin in {radare2}/share , and btw i wonder why buildinputs are not taken for a shell script, even propagatedbuildinput, so i think i make some very wrong stuff in the expression.
<clever> genesis: buildInputs get added to the PATH and gcc search paths
<clever> genesis: also, you probably want pkgs.substituteAll
<genesis> i'd like no substitute at all, i just want my dependencies available
<genesis> substitute makes script less easy to pass in a linter
<clever> genesis: pkgs.substituteAll { dir = "bin"; isExecutable = true; name = "appimage-exec"; src = ./; inherit jq radare2; }
<clever> genesis: try this and see what happens
<genesis> oki
<clever> if you want a linter to be happy, then you want wrapProgram to prepend to PATH, and then write your shell script assuming PATH has the tools already
<genesis> +1 clever
* genesis need to finish his nix-review before, unfinishing patchelf hook , this takes so long
<genesis> perharps i should have more than one nixpkgs to parallelise stuff .
<clever> genesis: look into git worktrees
<genesis> was sure someone would mention something like that :D
<clever> `git worktree add ../nixpkgs-foo -b foo master` will create a ../nixpkgs-foo, which is checked out to a new branch called foo, and based on the current master
<genesis> i look thanks
bhipple has quit [Ping timeout: 258 seconds]
drakonis has joined #nixos-dev
abathur has quit [Read error: Connection reset by peer]
abathur has joined #nixos-dev
<genesis> same issue clever
<genesis> => Cannot find 'elf_header' format.
<genesis> => unsquashfs: command not found
<genesis> works so great and easily in writeScript, too much magic.
<genesis> export PATH=/empty
<genesis> for i in @path@; do PATH=$PATH:$i/bin; done
<genesis> make it works better.
bhipple has joined #nixos-dev
<clever> > :p lib.makeBinPath [ radare2 libarchive jq squashfsTools coreutils file bash ]
<{^_^}> attribute 'attrPos' missing, at (string):88:9
<clever> genesis: there are functions to do that for you
<clever> though the bot isnt working right to show it
<clever> genesis: can you include the in the gist, so i can build it?
<genesis> with the tricks i found somewhere else in nixpkgs, i'm quite happy for now
<genesis> but yes nothing secret :)
<genesis> finally i merge appimage-run and appimageTools
<clever> genesis: gist supports multiple files in one url, which makes it much simpler to manage
<genesis> oki
<clever> both files in one gist, before my changes
<clever> export PATH=/nix/store/28yh3bkqzxwlih7ysjgv759jd34z351c-radare2-4.0.0/bin:/nix/store/g0kb16wrlxg541p56d6j9ky6znvm8iky-libarchive-3.4.0/bin:/nix/store/6hzm5w1v6qnr4yc2a1lc1xn0q8lpkb94-jq-1.6-bin/bin:/nix/store/bmjbrhznfsrkyjmd28n7px4j4knlc3zj-squashfs-4.4/bin:/nix/store/gnw6yrqy249n62r4q8vy12ispviv3dav-coreutils-8.31/bin:/nix/store/b5a98czixm1zhsbasmfi3b4l181cm3zq-file-5.37/bin:/nix/store/wd1jazzawjk4w1d31ism7fm7vdg4ma9l-bash-4.4-p23/bin
<clever> genesis: the gist i linked has been updated, it now generates the above
<clever> genesis: gist updated again
<clever> genesis: you can also see what i changed, by clicking on the revisions tab
<genesis> yes , that's good gist :) but i was not doing that since i used it as a pastebin .
<clever> updated one last time
<clever> though, i can also see other things to change...
<clever> genesis: updated again
<genesis> ho yes thanks
<genesis> i donno where i go with that tilpner seems a bit doubtful :D
<genesis> ho rev5 ;D
<genesis> thanks a lot.
<{^_^}> resolved: RootPartitionLowDiskSpace:
lovesegfault has quit [Quit: WeeChat 2.7.1]
lovesegfault has joined #nixos-dev
abathur has quit [Read error: Connection reset by peer]
abathur has joined #nixos-dev
<{^_^}> firing: HomepageUpdateStuck:
abathur has quit [Ping timeout: 258 seconds]
drakonis has quit [Quit: WeeChat 2.7.1]
evanjs- has joined #nixos-dev
evanjs has quit [Ping timeout: 258 seconds]
lovesegfault has quit [Quit: WeeChat 2.7.1]
lovesegfault has joined #nixos-dev
abathur has joined #nixos-dev
Emantor has quit [Quit: ZNC -]
Emantor has joined #nixos-dev
garbas has quit [Ping timeout: 255 seconds]
garbas has joined #nixos-dev
bhipple has quit [Ping timeout: 255 seconds]
bhipple has joined #nixos-dev
bhipple has quit [Remote host closed the connection]
orivej has joined #nixos-dev
abathur has quit [Ping timeout: 255 seconds]
abathur has joined #nixos-dev
nbp_ has joined #nixos-dev
nbp has quit [Ping timeout: 260 seconds]
<genesis> #81833 if you're interested, feel free to review, add needs etc.
<{^_^}> (by bignaux, 12 hours ago, open): [WIP] : unify appimagetool and appimage-run
<colemickens> I have libva in firefox' wrapper locally in anticipation of the stabilization of the vaapi support in Nightly.
<colemickens> How should I upstream this? Should I just always add it if on Linux, do I need to add another knob for the package, and if so, does it need to be exposed through nixpkgs (hopefully not)?
<lovesegfault> colemickens: talk to andi-
<colemickens> (and Nightly users (will) want this too, since the nightly overlay uses the nixpkgs wrapper as well. though, I haven't gotten it working e2e yet...)
abathur has quit [Ping timeout: 256 seconds]
orivej has quit [Ping timeout: 255 seconds]
<Profpatsch> I’m wondering how to reset the meta.position argument for cases where I override a derivation?
<Profpatsch> It’s magically called with mkDerivation, so if I just copy over the meta attrset, it will point to the old location
cole-h has quit [Ping timeout: 265 seconds]
<Profpatsch> there’s addMetaAttrs in lib/meta.nix, but that only does a dumb //
<Profpatsch> wow, this file is old. some lines are from 2009
Jackneill has joined #nixos-dev
lovesegfault has quit [Ping timeout: 272 seconds]
lovesegfault has joined #nixos-dev
ris has quit [Ping timeout: 268 seconds]
<{^_^}> firing: HomepageUpdateStuck:
<Profpatsch> hah, I think I have a function
<Profpatsch> And we can even use pname now it looks like
<Profpatsch> I added it to this PR if anybody wants to do a sanity check:
<Profpatsch> should probably add it to meta.nix? And maybe also update make-derivation to use pname instead of meta as position anchor? Iirc we enforce pname now
abathur has joined #nixos-dev
abathur has quit [Ping timeout: 260 seconds]
__monty__ has joined #nixos-dev
<garbas> s/tries/tried/
justanotheruser has quit [Ping timeout: 256 seconds]
<makefu> "Acts as immutable storage (in contrast to which acts as immutable storage)" <- this this on purpose or am i missing something?
<garbas> oh. right. that is clearly a copy paste typo :)
<garbas> makefu: tnx
<makefu> on the channels page it is equally confusing :D "Acts as immutable storage (in contrast to which act as immutable storage)"
<garbas> if fixed both pages
<garbas> channels is mutable (we are changing the redirects) and releases is immutable (we only upload there)
<makefu> okay, nice :)
janneke has quit [Remote host closed the connection]
pie_[bnc] has quit [Remote host closed the connection]
janneke_ has joined #nixos-dev
janneke_ is now known as janneke
m1cr0m4n has quit [Ping timeout: 265 seconds]
kraem has quit [Ping timeout: 265 seconds]
hl has quit [Ping timeout: 265 seconds]
cransom has quit [Ping timeout: 265 seconds]
pie_[bnc] has joined #nixos-dev
m1cr0m4n has joined #nixos-dev
kraem has joined #nixos-dev
hl has joined #nixos-dev
hl has joined #nixos-dev
hl has quit [Changing host]
cransom has joined #nixos-dev
<{^_^}> firing: HomepageUpdateStuck:
phreedom has quit [Ping timeout: 240 seconds]
phreedom has joined #nixos-dev
evils has quit [Read error: Connection reset by peer]
evils has joined #nixos-dev
evanjs- has quit [Quit: ZNC 1.7.5 -]
evanjs has joined #nixos-dev
<domenkozar[m]> someone that knows more about docker, am I right that docker layers are hashes of the filesystem? how does docker know when it can fetch a layer instead of building it?
<emily> probably of interest to a lot of people (changing nginx TLS config)
<{^_^}> #81891 (by emilazy, 1 minute ago, open): nginx: use Mozilla Intermediate TLS configuration
<domenkozar[m]> it seems to be mapping image ids to layers
<domenkozar[m]> but it's unclear to me how
<gchristensen> domenkozar[m]: it is a hash of the layer before it, and the hash of the instruction
<domenkozar[m]> so hash("RUN ...", layer - 1, layer)
<gchristensen> then those layers get mapped to a content-addressed hash of a tarball to represent the fs diff
<domenkozar[m]> so essentially it's a hash of inputs and outputs?
<gchristensen> no
<gchristensen> there are two different pieces here, but a layer's ID = hash(previousLayer, instruction)
<gchristensen> hash(previousLayer, "RUN ...")
<gchristensen> after building, the filesystem's diff is tar'd up and hashed and that hash is associated to that layer ID
<gchristensen> goes in to it some iirc
<emily> nixos is a merkle tree, docker is a blockchain 🤔
<gchristensen> no I don't
<gchristensen> domenkozar[m]: I can help explain more in like ~15 minutes :)
<domenkozar[m]> so it sensible to say that at the end of the day, image id maps to top layer id, which hash takes into account all inputs (instructions) and outputs (fs)
<domenkozar[m]> where instructions are how to fetch things, so no purity obviously
<gchristensen> wellllllllll that gets a bit hairy
<gchristensen> but srsly gotta step away for a few
<domenkozar[m]> k
<Profpatsch> emily: A blockchain is just a very simple merkle tree :)
abathur has joined #nixos-dev
<etu> Profpatsch: same with git :)
<Profpatsch> etu: everything’s converging (to turtles)
abathur has quit [Ping timeout: 255 seconds]
<emily> I was mostly just going for the cursed energy of describing the basic way rounds of hash functions work as a "blockchain" honestly
abathur has joined #nixos-dev
<gchristensen> domenkozar[m]: it Depends lol
<gchristensen> because images you pulled are treated a bit differently
<gchristensen> but generally sounds right-ish
evanjs has quit [Read error: Connection reset by peer]
<gchristensen> what is this for?
evanjs has joined #nixos-dev
<domenkozar[m]> I want to explain how is cachix different to docker hub
<domenkozar[m]> since I got the question
<gchristensen> cool
<domenkozar[m]> and it's really about explaning how caching in Nix and Docker works
<gchristensen> maybe your customer actually doesn't care about that detail and is looking for a different explanation?
<domenkozar[m]> and quickly realized I don't know to explain the difference
<domenkozar[m]> :D
<domenkozar[m]> I think as any developer they are looking for technical explanation, which should then follow with practical outcomes
<domenkozar[m]> I've found to be quite useful
<gchristensen> well then maybe this will be helpful -- because a lot of developers who use docker full time don't actually understand docker's model
<gchristensen> } elsif ($unit =~ /\.socket$/ || $unit =~ /\.path$/) {
<gchristensen> # FIXME: do something?
<domenkozar[m]> I know that code :D
<domenkozar[m]> it makes sockets not restart on nixos-rebuild switch
<domenkozar[m]> gchristensen: I think my missing piece is where the instructions fit in, still researching that
<gchristensen> domenkozar[m]: on an image you pulled, the instructions are basically not used at all
<gchristensen> they are just metadata in the image manifest
<domenkozar[m]> and locally, how does docker know what to take from cache and what to rebuild?
<domenkozar[m]> (in terms of layers)
<etu> gchristensen: Your blogpost about docker is something I've linked to people at work. And reffered to when I held a presentation for a tech crowd at work about nix. It's a great blogpost :)
<gchristensen> so starting "FROM foo", that establishes your initial "previousLayer" hash. then you add a "RUN foooo", so layerId is pretty much hash(previousLayer, "RUN foooo")
<gchristensen> this is why if you don't disrupt a command you don't rebuild it, but if you do update the 'foo' image in "FROM foo" , you do
<domenkozar[m]> so in docker the input (based on which hash we asses if it should be taken from cache or now) is based on previous outputs and instruction to run
<domenkozar[m]> or not*
<gchristensen> I suppose so, but maybe actually it hash(previousLayerId, "instruction") -- the difference being in this case the hashing being all about instructions once you get past the "FROQM"
<pie_[bnc]> could we get a flag like -K but one that keeps successful builds too
<domenkozar[m]> gchristensen: ok, so it's hash of the digest of previous layer + hash of instruction
<gchristensen> domenkozar[m]: possibly, but I can't say for sure! :)
<domenkozar[m]> hmm I'll write up a blog post, I think this is cool to understand where docker fails short
<gchristensen> I think you'd benefit from reading the source to make sure, especially if you're telling customers or writing a blog post
<domenkozar[m]> I'll try to find credible source
<domenkozar[m]> I can't possibly be the first one to wonder how it works
<gchristensen> definitely
<gchristensen> dive is really good
<domenkozar[m]> thanks gchristensen
<domenkozar[m]> I hope I'll manage to explain this well
<gchristensen> I would happily read a draft
<domenkozar[m]> thanks :) next week.
<niksnut> Ericson2314: RFC 49 meeting?
justanotheruser has joined #nixos-dev
<infinisil> niksnut: Why do floats in Nix only print ~6 digits? Would be great to use a more standard printing of floats
<infinisil> (or doubles rather)
<niksnut> dunno
<niksnut> didn't write float support :-)
<infinisil> I see
<domenkozar[m]> theuni did
<infinisil> nix#762
<{^_^}> (by ctheune, 4 years ago, merged): Implement floats
<infinisil> "As many digits are written as needed to represent the integral part, followed by the decimal-point character and six decimal digits."
<niksnut> the only worry about changing this is that it can change evaluation results in an incompatible way
<niksnut> probably not a big deal
<infinisil> Yeah I wouldn't think so too, and I won't have to wrestle with Profpatsch regarding with that change :)
<gchristensen> lopsided98: should we revert that merge?
<gchristensen> lopsided98: or was the only bad thing about that merge the version of the bootstrap tools?
drakonis has joined #nixos-dev
<infinisil> I'll try to do the float change
<Profpatsch> infinisil: let’s say if we fix the toString instance in nix, and newer nix versions are correct, I’d be okay to merge that display logic
<disasm> yikes! most recent hydra eval has almost 3x failures on release branch!
<infinisil> Profpatsch: But what do until that's nixpkgs minimum nix version?
<Profpatsch> infinisil: idk, display a warning?
<infinisil> People would get warnings every time they evaluate their config then
<infinisil> unsilencable
<infinisil> How about just rolling with the slightly worse precision until then?
<Profpatsch> I’m totally against silently coercing user inputs
<Profpatsch> Like 100% no-go
<infinisil> I mean I will have to use a float printing mechanism for that PR
<Profpatsch> You can leave out float support
<infinisil> No I can't really
<Profpatsch> It’s broken, so we don’t support it, done.
<infinisil> config formats support floats
<infinisil> If there's no float support they can't be represented fully
<Profpatsch> And when the fixed nix is merged into nixpkgs, we can add floats
<infinisil> Which is like the whole point of rfcs#42
<{^_^}> (by Infinisil, 49 weeks ago, open): [RFC 0042] NixOS settings options
<infinisil> Also builtins.fromJSON couldn't be used to import existing config files
<infinisil> (if they use any floats)
<infinisil> All this just because it only displays 6 digits?
<infinisil> I don't see how this could cause any serious problems
<Profpatsch> yes it can
<Profpatsch> > cat ~/tmp/float
<{^_^}> undefined variable 'cat' at (string):288:1
<Profpatsch> 2.453234233278
<infinisil> It's not like Nix is a neural network type of application
<Profpatsch> > builtins.fromJSON (builtins.readFile /home/philip/tmp/float) == 2.45323
<{^_^}> access to path '/home/philip/tmp/float' is forbidden in restricted mode
<Profpatsch> false
<Profpatsch> nix-repl> builtins.fromJSON (builtins.readFile /home/philip/tmp/float) == 2.453234233278
<Profpatsch> true
<LnL> gchristensen: for armv7 I'd wait with uploading the bootstrap tools until we have a native version built by hydra
<Profpatsch> It’s only the show instance that is broken.
<infinisil> And that causes serious problems how?
<infinisil> Tell me one place in NixOS where this is a problem
<Profpatsch> Every single. friggin. option. that. uses. floats
<Profpatsch> I don’t know how I can be more specific.
<infinisil> They all print the floats already
<Profpatsch> Then they are *all* *broken*
<Profpatsch> Individually. Which doesn’t mean it’s fine to make them all broken as a library feature
<Profpatsch> Because so far it was up to every single option writer to know the semantics of toString for floats
<infinisil> I haven't seen a single issue where somebody complained about that
<Profpatsch> After your change it’s *your fault* that you implemented a broken feature.
<Profpatsch> And every user that gets a different config than they expect will be *on you*.
<gchristensen> +1 to Profpatsch, adding float support there makes it much harder to fix floats later
<infinisil> Please just tell me a *concrete* problem
<Profpatsch> infinisil: Probably because they didn’t notice because software usually doesn’t tell you when you pass different values than you intend.
<infinisil> Like, did any of you ever run into having a broken config due to it only showing 6 decimals?
<gchristensen> today: a float is rendered at one thing and it works great, tomorrow nix floats render differently and causes a behavior difference and causes a problem
<Profpatsch> That’s a strawman.
<Profpatsch> infinisil: I have showed you a concrete proof of how the implementation is *not correct*, and a lack of a practical example isn’t going to change the proof
<infinisil> I don't want my RFC to be ruined or delayed for a year just because of that tiny issue..
<Profpatsch> infinisil: It’s not blocked. Just remove support for floats, or represent the type as a tuple of two decimals.
<infinisil> I CANT
<gchristensen> why not?
<infinisil> Every program that requires floats couldn't use the settings way
<gchristensen> if a program requires a float, is it likely that the program will be interested in a specific float?
<Profpatsch> Then users can fall back to the old way of passing options manually.
<infinisil> Maybe, maybe not, but all current modules seem to do fine with 6 digits of floats
<Profpatsch> Or they can construct a string from two decimals and pass that into the json
<Profpatsch> infinisil: I already told the reason.
<infinisil> strings in json arent' floats
<infinisil> If a program expects a float it can fail for strings
<gchristensen> I think at this point, it would be useful to set up a videocall to discuss this further
<Profpatsch> The option maintainers didn’t know the broken semantics
<infinisil> I'm already not that motivated anymore for my RFC.. I don't want this to be another hinderance
<infinisil> The impact of this is *minimal*
<infinisil> I really think you are overestimated how grave this is
<gchristensen> this sounds like it may be worth bringing up to the shepherd team
<Profpatsch> It’s already 95% okay, and not supporting floats is not a blocker, especially because we can fix it in the future.
<emily> i have a new question: why would any configuration option use floats ever
<emily> the only reals I have in my config are longitude and latitude
<emily> and iirc those are represented as strings
<infinisil> I've seen transformation matrices
<Profpatsch> infinisil: I just looked, and there’s exactly *two* options in man configuration.nix that use the float type.
<infinisil> parameters to compton I believe too
<Profpatsch> So we really don’t have a problem to leave out floats for now.
<infinisil> Profpatsch: No that's not what I mean
<emily> imo those should just be removed then
<gchristensen> emily: yikes
<emily> like, strings are better if it's just passing through the value with no processing
<infinisil> Profpatsch: Every program configuration file that could have a float setting
<emily> unless you're doing actual fp arithmetic/calculation i really don't see any reason to use a native float type
<Profpatsch> literally the only two options in nixos are `location.latitude` and `location.longitude`
<emily> it's more likely to cause problems than help anything
<infinisil> emily: Once the float printing problem is fixed there's no reason to not use floats either
<Profpatsch> I agree
<emily> yes there is: floats are a bad config interface
<emily> if you go down this route then you end up like Verilog, which has a "real" type that the implementations just treat as opaque strings
<emily> because you can't actually mangle them with rounding or anything
<emily> because that can break stuff
<LnL> there's a reason json doesn't have floats
<infinisil> I don't even really care about whether it's a good or bad interface, but JSON supports floats, TOML supports floats, etc. As such these types for the RFC need to support floats too
<Profpatsch> LnL: Oh, it doesn’t?
<emily> I'd start talking about whether Nix is ever going to support non-IEEE 754 platforms but that would get way too into cursed territory.
<infinisil> json does have floats though..
<clever> LnL: i have seen json files with ints over 64bits in length, which some json tools round to the nearest float
drakonis has quit [Ping timeout: 272 seconds]
<niksnut> I've had to deal with ints > 56 bits getting silently rounded to the nearest double
<LnL> oh hmm, not sure where I got that idea from
<emily> json ""has"" floats in the same way as it ""has"" integers
<simpson> emily: FWIW it's been a solved problem for several decades how to convert floats into strings in a way that doesn't lose precision. The problem is mostly one of effort and understanding. I am not against just having a single integral type; it's certainly simpler.
<niksnut> *52 bits
<simpson> Yep, JSON just has "numbers".
<emily> json: a syntax without a data model
<Profpatsch> I thought so much. JS only has numbers, some are ints incidentally
<emily> simpson: of course, floats can be printed correctly... but I still think we should be considering why we want them in the first place
<emily> like, does it actually make any sense to represent latitude/longitude configuration as a floating point value in our configurations? for all I know redshift doesn't store it as a float and can pass down arbitrary precision
<emily> hashes could also be represented as integers but we don't do that
<simpson> Sure, it's a valid design concern.
<emily> because we treat them as opaque identifiers
<garbas> i wonder why theuni implemented floats in the first place. they might use them at flyingcircus
<infinisil> lon/lat is very much a floating point value, it probably needs to do calculations with it
<gchristensen> the fundamental problem, though, emily, is that there is "discussing redshift" and there is "discussing infinisil's PR which implements a generic-to-everything config generation layer"
<infinisil> (redshift)
<Profpatsch> Okay, I propose the following way forward: 1) drop support for floats from the current RFC and implementation 2) add correct float printing to nix 3) add float support to the RFC once a non-broken nix is merged into nixpkgs that supports correct float printing.
<emily> gchristensen: right
<emily> gchristensen: but I think when you have two operations in the whole distribution that actually use the type it's a lot easier to argue for not supporting it and just changing those options instead /shrug
<emily> I'm all for richer types when they provide actual benefit, I just don't think they do in this case
<simpson> (Also yeah, what infinisil said. Even single-precision floats have enough precision for mapping stuff on a globe, and double-precision floats are how the maths is done in real-world mapping tools.)
<gchristensen> I agree with not supporting it for now. better to not support it and let people realize they need to do something special, than support it and silently do weird stuff
<infinisil> Um
<infinisil> Okay so
<infinisil> The promise of the RFC is to allow setting arbitrary config values
<infinisil> Implement settings once
<infinisil> Never worry about having too little flexibility anymore
<gchristensen> simpson: heck, Australia publishes GPS locations down to the precision of a chlorine atom, which is immediately wrong because Australia moves faster than their database updates.
<emily> it is really ok for something to error out in one tiny edge-case
<infinisil> Not supporting floats means that every time somebody wants to implement a settings, they need to go through *all* config values to make sure the program doesn't have anything that could use floats
<emily> gchristensen: nice
<gchristensen> infinisil: yeah, that is the apple you're biting out of :P
<gchristensen> it is a big and audacious goal
drakonis has joined #nixos-dev
<Profpatsch> infinisil: That’s why I proposed adding a warning that is only displayed when printing a float
<infinisil> It's not! Without this float issue nobody would have to do this!
<Profpatsch> That people can still use it if they know what they are doing.
* emily proposes rounding all floats to 3
<emily> it'll be accurate enough for pi
<simpson> gchristensen: Ah, good old precision-vs-accuracy
<Profpatsch> And we will be speedy and add a tiny patch to nix which is merged ASAP
<Profpatsch> And then we can remove the warning
<infinisil> Profpatsch: NixOS warnings can't be silenced though
<Profpatsch> infinisil: They are only displayed when floats are actually used
<Profpatsch> So only for users who wish to use them.
<Profpatsch> which, as we have determined, are not a lot of people.
<infinisil> Anyways that would be okay with me, floats are probably not used by most
<Profpatsch> Under the condition that we can add to a nix release asap.
<Profpatsch> Probably even have the nix patch merged before we merge the RCF
<Profpatsch> *RFC
<gchristensen> after `nixos-rebuild test`, does `nixos-rebuild rollback` roll back to the version immediately before `test`, or does it roll back to the version before version you were at before you tested? I'm suspicious it goes back to the one
<emily> note that yaml also supports, like, crazy custom-tagged types
<emily> so there's always going to be stuff you can't specify in nix
<clever> gchristensen: yeah, that is a tricky question.....
<emily> unless you want to add encodings for every horrible thing yaml has
<emily> (less relevant for JSON/TOML since they have less bloated data models)
<gchristensen> clever: it is, isn't it :P
<Profpatsch> emily: YAML is a lost cause anyway.
<Profpatsch> It will go away in a few years
<gchristensen> lol
<simpson> Right. If it weren't literally an IEEE standard with decades of culture and tooling and support in a serious fraction of typical CPUs (certainly more than support bigints!), I wouldn't even consider floats.
<emily> don't worry, TOML might add type tags too!
<{^_^}> toml-lang/toml#603 (by LongTengDao, 51 weeks ago, open): Will custom type syntax be good for TOML health?
<emily> Profpatsch: a-are you serious?
<emily> I don't think Kubernetes or whatever have any plans of switching off YAML
<simpson> emily: FWIW k8s is defined in JSON, and the YAML layer is sugar on top. Maybe YAML will be like XML, never going away but never being preferred.
<clever> emily: i just throw json in the face of anything wanting yaml :P
<emily> simpson: can JSON go away in a few years too, please?
<clever> emily: builtins.toFile "name.yaml" (builtins.toJSON things)
<simpson> Let us hope. Also, let us be the change we wish to see.
<emily> eh, that results in pretty bad formatting/error messages. you can use toJSON and ${pkgs.remarshal}/bin/json2yaml
<clever> emily: haskell also has a json2yaml
<{^_^}> firing: HomepageUpdateStuck:
<clever> 276 yaml2json = pkgs.haskell.lib.disableCabalFlag pkgs.haskellPackages.yaml "no-exe";
<clever> but it became optional to save space, and it defaults to off
<clever> so you have to rebuild the package
<emily> well that sounds worse than pkgs.remarshal then :p
<clever> emily: i have seen python based yaml parsers break
<emily> but it's json2yaml, you're only parsing JSON...?
<clever> emily: they dont handle the edge-cases iohk is using, but haskellPackages.yaml is using the exact same parser we already use
<clever> ah, for that direction, yeah, your less likely to have issues
<clever> emily: also, i'm reminded of a horror story from tesla
<clever> the CI system was parsing and testing a json file listing every supercharger, to ensure its valid
<clever> but, it was using a different JSON library from what the cars used
<clever> so one day, a "valid" json file got shipped out, that resulted in the main control app on the car, crashing
<clever> which leaves the car un-driveable, and unable to update its own software
<emily> given what i've heard of tesla's software engineering practices in general i can't say i'm terribly surprised
<clever> using the exact same library to both generate and parse the yaml, would avoid that type of issue
<Profpatsch> infinisil: sorry if I’m being a killjoy on this issue, but I really think correctness is not just bikeshedding, because I loathe every minute I have to debug stuff that could have been a trivial warning or even error message but instead silently coerced my inputs.
<gchristensen> +1
<Profpatsch> And I really don’t want to contribute to people experiencing stuff like that because of me
<clever> Profpatsch: you wouldnt like JS :P
<Profpatsch> clever: It might not look like it, but I’m not writing any JS if I can avoid it :P
<infinisil> Profpatsch: I would agree with you if I'd even have the suspicion of this ever causing a problem, but I don't
<gchristensen> it never does until it does
<infinisil> That's what premature optimization/abstraction is
<gchristensen> I don't think... so..?
<emily> wha?
<infinisil> Well no
<emily> "not silently doing the wrong thing is premature optimization"?
<infinisil> I mean
<infinisil> It's coding for a usecase you don't know you have yet
<emily> you can always make something more defined / less warny
<infinisil> Show me one instance of a floating point being trimmed to 6 digits is actually causing a problem and I'll shut up
<emily> the other way around is far harder, and the most pernicious problems are those that you dismiss ahead of time
<emily> (since the bugs that you can see coming, well, you see coming)
<infinisil> Oh and I'm talking a relevant example for Nix, I know how trimming to 6 digits would be a problem for math libraries
<emily> ok, show me one instance of a floating point in a config file you're converting then
<emily> explicitly not handling it is not coding for a usecase you don't know you have, it's explicitly deferring it until you have it...
drakonis has quit [Ping timeout: 272 seconds]
<emily> like I am saying "you should not silently bake in buggy behaviour" and you are saying "you aren't gonna need it" and that's just ???
<infinisil> compton has a bunch of floating point parameters
<emily> then supporting it with a warning, or just erroring out, until nix can be fixed, seems like the best thing to do
<emily> silently using the wrong function is never the right thing to do
myskran has joined #nixos-dev
<emily> your thing is meant to "convert data to config formats", not also "round off a bunch of the data lossily"
<infinisil> emily: Well yeah, but are those floating points being rounded to 6 significant digits a problem in compton?
<gchristensen> this is similar to my earlier question: if a program requires a float, is it likely that the program will be interested in a specific float?
<emily> you don't know that the user is using compton!
<emily> maybe they're a numeric analyst configuring their floating point benchmarks
<emily> in that case they will be really annoyed at you
<emily> especially once they learn you refused to even print a message telling them their nix is broken
<gchristensen> "uber self-driving car crashes; nix rounding to blame"
abathur has quit [Ping timeout: 256 seconds]
<emily> anyway you could try serializing it and reading it back in
<emily> and if it matches then print it out
<emily> otherwise raise an error
<emily> that way it's guaranteed to never cause any problems
<emily> and the error can be backcompatibly turned into working behaviour when nix is updated
<gchristensen> I think it is important to step away from the goal of getting this merged, and look more at the goal from the user's perspective
<emily> that way users aren't forced to deal with config warnings, no incorrect behaviour ever happens, and floats are supported
<infinisil> The worst thing I could imagine happening is number analyst paul who seems to be changing his configuration.nix to configure 0.000001 changes to the program config (always waiting for the rebuild for such tiny amounts), or maybe he's bisecting the best parameter with nixos-rebuilds? Who knows. But the worst thing I can imagine is that he changes it from 4.0347646 to 4.034645, and notices how
<infinisil> there's no change, the system rebuilds without building any new derivations. Yeah
<infinisil> Yes correctness is important, but we're talking about Nix here
<infinisil> We're not controlling rockets with Nix
<infinisil> Well I hope you're not at least..
<gchristensen> I don't know of any cases where Nix is near to rockets, only satelites
<clever> infinisil: now i want to rig it into a model rocket, lol
<gchristensen> but knowingly doing the wrong thing is pretty weird
<gchristensen> oh and space telescopes
<infinisil> I guess this summarizes my position nicely: The benefit of having basic float support without any warnings is far greater than the problems it will cause due to the rounding
<adisbladis> infinisil: Hey my entire data science pipeline is Nix.
<gchristensen> well, that is wrong, because you can't know how your user is using your software
<infinisil> gchristensen: It's an estimation
<infinisil> Coming from my knowledge with how NixOS modules are used
<infinisil> And configuration in genernal
<Profpatsch> emily: Oh! that’s great!
<Profpatsch> I like that proposal.
<gchristensen> and it is wrong, it is literally wrong, it is doing the wrong thing and the output is not what the user specified, how are we even discussing ignoring it being the right thing
<Profpatsch> I already showed it earlier without recognizing it
<Profpatsch> I thought we couldn’t verify.
<emily> infinisil: do you have any objection to my proposal?
<Profpatsch> So now there’s no case in which we need to display a warning!
<infinisil> emily: I don't think it can be done
<emily> it seems like you get practical float support for compton etc. and we get guaranteed correctness
<emily> hm? you can literally just eval (toString ...) or whatever
<infinisil> > toString 1.23475623652632
<{^_^}> "1.234756"
<infinisil> Oh
<infinisil> I think I get what you mean
<infinisil> Um, unfortunately there's no way in nixpkgs to convert a string to a float
<Profpatsch> Or use fromJson if you really care about json floats being handled correctly
<emily> yes, and if a user specifies that in their config they should be warned/errored!
<Profpatsch> infinisil: fromJSON
<emily> they can just manually truncate it to 1.234756 to fix it without silent loss of information, that's a good thing
<Profpatsch> errored
<infinisil> Ah
<Profpatsch> We can be correct now while still supporting the floats that are possible.
<emily> like, you can do
<emily> let approx n = fromJSON (toJSON n); canBePrecise n = approx n == n
<Profpatsch> And we can remove it when nixpkgs has a minimal nix that’s greater than the patch
<emily> needs better names, but
<gchristensen> > builtins.fromJSON (builtins.toJSON 1.213546) == 1.213546
<{^_^}> false
<gchristensen> > builtins.fromJSON (builtins.toJSON 1.21354) == 1.21354
<{^_^}> true
<infinisil> Yeah that sounds great emily
<gchristensen> oh emily already did it
<infinisil> I didn't think this was possible before
<Profpatsch> yeah, me too
<infinisil> But I still stand by what I said earlier
<infinisil> gchristensen: And regarding the point with correctness, I wouldn't count rounded floats as wrong
<gchristensen> you must be kidding, I cannot believe it.
<infinisil> Because 0.1 is already incorrect then
<infinisil> The thing about floating points
<emily> just checking, but have you read "What Every Computer Scientist Should Know About Floating-Point Arithmetic"? :p
<infinisil> And if it was a single-precision float (which it was originally in the Nix implementation!), then 6 digits would be right about the precision of it
<clever> infinisil: ive been dealing with fixed-point math on the rpi stuff lately
<infinisil> Where is the boundary from correct to incorrect for floating point precision
<clever> basically, just <<6 and >>6 at the right times, then treat it as a plain int
<infinisil> gchristensen: Do you get what I mean though?
<gchristensen> yes, I do
<gchristensen> thanks
<adisbladis> infinisil: It's not causing a problem until it is, and you may not even notice in time because of silent truncation.
orivej has joined #nixos-dev
<Profpatsch> If 64 bit floats are bad enough already, we shouldn’t make it worse while knowing about it
<adisbladis> Profpatsch++
<{^_^}> Profpatsch's karma got increased to 9
<gchristensen> Profpatsch++
<{^_^}> Profpatsch's karma got increased to 10
<infinisil> I just don't think it's a priority of Nix to have accurate floating points. We want a convenient user experience and things that Just Work
<gchristensen> I disagree on "convenient" being a priority like that, if I wanted convenient I'd be using debian
<infinisil> I spend a lot of time to make NixOS modules convenient for end users at least
<adisbladis> It conveniently drops a few decimal points
<gchristensen> yeah, things should be convenient and do the right thing, and in this case I think the convenient thing in an unhappy-path scenario is to refues to do the thing which will be decidedly inconvenient for the user
<infinisil> With the thing emily suggested sure
<emily> I would be sad if correctness is not a Nix(OS) priority :(
<gchristensen> emily: me too :)
<emily> mostly because I couldn't whine about the ways in which it fails to live up to that any more
<simpson> It's strange if one number type works normally and the other number type is unpredictable and eldritch.
* gchristensen vaguely motions to his TLA+ book / program, modeling a bug in `nixos-rebuild rollback`
<infinisil> I guess we'll just have to agree to disagree then
<gchristensen> scary
<gchristensen> but also: we've come to an agreement on the side of correctness: do the thing, unless it is incorrect
<adisbladis> That said: Arguably it's a bug that `toJSON` returns non-IEEE-754 floats
<gchristensen> let's add Unums to Nix
<infinisil> gchristensen: Scary because?
<gchristensen> (PMing)
<emily> gchristensen: get with the times, they're posits now
<emily> (and only slightly less cursed)
<gchristensen> oh
<emily> (it's just, now they're in the category of "ok, sometimes you might want this, if you really know what you're doing", rather than "literal non-starter nobody would ever have reason to use")
<gchristensen> I have to admit I know almost nothing about the field :x
* emily isn't a numerical analyst and doesn't even play one on TV ^^
<lovesegfault> IEEE-754 is a monstrosity
<emily> I'd like to think I've picked up at least a little from following Steve Canon on Twitter though
<emily> 754 has a lot of warts, but gets surprisingly closer to seeming like a crystalline jewel of near-perfect compromise the more I learn about it, honestly
<emily> NaN should definitely == NaN though.
<emily> they're tricky and unintuitive, but that's impossible to avoid in any real-approximation type, and none of the alternatives is "usually better in the general case" than binary floating point
lovesegfault has quit [Ping timeout: 258 seconds]
abathur has joined #nixos-dev
myskran has quit [Ping timeout: 260 seconds]
cole-h has joined #nixos-dev
ixxie has joined #nixos-dev
infinisil has quit [Quit: Configuring ZNC, sorry for the joins/quits!]
infinisil has joined #nixos-dev
<colemickens> I finally get my stuff together to attend Office Hours... :(
<colemickens> oh well, back to quarantine party
ris has joined #nixos-dev
orivej has quit [Ping timeout: 265 seconds]
orivej has joined #nixos-dev
claudiii has joined #nixos-dev
cole-h has quit [Quit: WeeChat 2.7.1]
<{^_^}> firing: HomepageUpdateStuck:
cole-h has joined #nixos-dev
janneke has quit [Quit: janneke quits Mes'sing]
<gchristensen> people might be interested in this:
<{^_^}> nixops#1245 (by grahamc, 3 minutes ago, open): Deploy Targets: Policy/Behavior-free Deployment Hooks (auto-rollbacks, drain events, etc.)
janneke has joined #nixos-dev
justanotheruser has quit [Ping timeout: 272 seconds]
pepesza has quit [Ping timeout: 260 seconds]
pepesza has joined #nixos-dev
abathur has quit [*.net *.split]
ajs124 has quit [*.net *.split]
octe has quit [*.net *.split]
multun has quit [*.net *.split]
MoreTea has quit [*.net *.split]
callahad8707 has quit [*.net *.split]
ivan has quit [*.net *.split]
johnny101 has quit [*.net *.split]
kgz has quit [*.net *.split]
callahad8707 has joined #nixos-dev
ivan has joined #nixos-dev
MoreTea has joined #nixos-dev
johnny101 has joined #nixos-dev
multun has joined #nixos-dev
octe has joined #nixos-dev
ajs124 has joined #nixos-dev
abathur has joined #nixos-dev
kgz has joined #nixos-dev
arianvp has quit [Ping timeout: 268 seconds]
arianvp has joined #nixos-dev
MichaelRaskin has joined #nixos-dev
orivej has quit [Ping timeout: 268 seconds]
drakonis has joined #nixos-dev
justanotheruser has joined #nixos-dev
__monty__ has quit [Quit: leaving]
drakonis has quit [Ping timeout: 240 seconds]
elvishjerricco has quit [Ping timeout: 240 seconds]
elvishjerricco has joined #nixos-dev
drakonis has joined #nixos-dev
drakonis has quit [Ping timeout: 256 seconds]
drakonis has joined #nixos-dev
justanotheruser has quit [Read error: Connection reset by peer]
abathur has quit [Ping timeout: 240 seconds]
<infinisil> Is this not a problem because by default it's turned off? (You need to enable addExtraHosts for it to do the extraHosts thing)
<infinisil> Come across this because there's #80113 which also uses IFD to add extra hosts
<{^_^}> (by pasqui23, 3 weeks ago, open): StevenBlack-hosts: init at 2.5.52
<infinisil> s/Come/Came
drakonis has quit [Ping timeout: 240 seconds]
<infinisil> Although, it's doing "builtins.readFile (pkgs.StevenBlack-hosts.override"
<infinisil> Which technically isn't IFD, but it has the same implications
<infinisil> I think