gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.03 release managers: fpletz and vcunat | https://logs.nix.samueldr.com/nixos-dev
pie___ has quit [Ping timeout: 264 seconds]
jtojnar has joined #nixos-dev
pie_ has joined #nixos-dev
pie__ has joined #nixos-dev
pie__ has quit [Client Quit]
catern has quit [Excess Flood]
catern has joined #nixos-dev
joepie91 has quit [Excess Flood]
joepie91 has joined #nixos-dev
joepie91 has quit [Changing host]
joepie91 has joined #nixos-dev
{`-`} has joined #nixos-dev
disasm has quit [Ping timeout: 240 seconds]
primeos_ has quit [Ping timeout: 268 seconds]
catern_ is now known as catern
primeos_ has joined #nixos-dev
lassulus has joined #nixos-dev
phreedom has quit [Ping timeout: 250 seconds]
lassulus_ has quit [Ping timeout: 256 seconds]
pie__ has quit [Ping timeout: 244 seconds]
phreedom has joined #nixos-dev
Lisanna has joined #nixos-dev
Lisanna has quit [Quit: Lisanna]
Jackneill has quit [Ping timeout: 256 seconds]
orivej has quit [Ping timeout: 268 seconds]
Lisanna has joined #nixos-dev
FRidh has joined #nixos-dev
<peti> domenkozar: Yay! It came out yesterday. And not one day too early. Lts-11 was getting old, really. It's a shame that ghc 8.6.1 is just around the corner, and then lts-12 will be outdated'ish again just a couple of weeks after its release.
ikwildrp1pper is now known as ikwildrpepper
jtojnar has quit [Remote host closed the connection]
jtojnar has joined #nixos-dev
<domenkozar> peti: yeah and most things weren't bumped
<domenkozar> lots of improvements to be made to the process :)
tilpner has quit [Ping timeout: 240 seconds]
<peti> The lts-12.x update breaks ~1200 builds: https://hydra.nixos.org/jobset/nixpkgs/haskell-updates
<domenkozar> should get better for 18.09
phreedom has quit [Ping timeout: 250 seconds]
phreedom_ has joined #nixos-dev
tilpner has joined #nixos-dev
FRidh has quit [Quit: Konversation terminated!]
ma27 has quit [Quit: WeeChat 2.0]
ma27 has joined #nixos-dev
orivej has joined #nixos-dev
orivej has quit [Ping timeout: 260 seconds]
disasm has joined #nixos-dev
<domenkozar> peti: would you be opposed to cabal2nix refactoring such that it would allow better use of it as a library?
<domenkozar> for example right now we have cabal2nixWithDb
<domenkozar> I'd like to refactor into cabal2nix :: Options -> ...
<domenkozar> and cabal2nix' :: Settings -> ...
<domenkozar> and have Options for cli parsing and then Settings be haskell configuration
<domenkozar> for example, hackage-db for options would be a path
<domenkozar> for Settings it would be Hackage type
<peti> domenkozar: I am hugely in favor or cleaning up that code. A lot of it is very ad hoc'ish. The only thing I'd suggest to do it incrementally in small steps and to coordinate the change before it's implemented because I have some ideas of my own and I'd like to make sure beforehand that those don't conflict.
<domenkozar> great
<domenkozar> I'll code up something that only introduces Settings and defaultSettings and let's go from there
<clever> angerman has also made Cabal2nix, which translates conditional statements in the cabal file into nix if statements
<domenkozar> yeah the adhockery has reached hocking levels
<peti> domenkozar: Personally, I want to throw out a lot of the functionality that's currently included in cabal2nix. IMHO, it was a mistake to introduce all those "fetchers" into it, for example. As far as I am concerned, cabal2nix 3.x will convert a local Cabal file to a Nix expression, but no more of that funky "download from github" stuff.
<peti> clever: Yes, I saw that.
<angerman> peti: could just take off from the cabal-to-nix in nix-tools ;-)
<domenkozar> yeah Cabal2nix is the way forward, but it's more like a PoC compared to current infrastructure
<domenkozar> it lacks all the tooling around it :)
<domenkozar> peti: I partially agree, I think fetchers are useful, but they should be implemented explicilty
<domenkozar> right now cabal2nix just tries them in order, but rather cabal2nix could accept --fetcher git
<domenkozar> etc
<clever> another thought i had, why not have stack2nix run super.callHackage "vector-sized" "1.0.3.0" {};
<clever> then you can just omit the data that nixpkgs already had
<domenkozar> it uses IFD
<clever> or would that result in a ton of IFD and worsen the performance?
<clever> ah
<domenkozar> would be interesting to measure evaluation performance, but I'd say it makes things wrose
<domenkozar> worse*.
* peti believes that going forward without IFD is just plain silly. That particular feature solves so many problems we currently have.
<peti> The fact that we commit megabytes of generated code into the repository makes no sense.
<domenkozar> I'm hoping hnix can have saner IFD implementation
<peti> Well, Nix's IFD implementation is okay, IMHO. The only problem is that it lacks a strategy for dealinig with IFD in restricted mode.
<clever> also, IFD interacts poorly with min-free and max-free based auto-gc
<clever> the IFD can trigger a gc, which eats things the current eval was about to grab a lock on
<domenkozar> IFD as-is has many dark corners
<domenkozar> suddenly cli args to nix stop working, etc
<domenkozar> but if we don't use it much, it won't improve :)
<domenkozar> angerman: does Cabal2nix succesfully output stackage snapshot and build it?
<peti> We already know a reasonable way forward, IMHO: in restricted more, import all derivations that exist and ignore those that don't exist. But there's no implementation. :-(
<peti> s/more/mode/
<angerman> domenkozar: well... I use it with cardano. It doesn't run the stackage solver but does an approximation.
<domenkozar> nice
<domenkozar> TH in a derivation
<domenkozar> magic
<domenkozar> angerman: does it cross-compile cardano?
<clever> domenkozar: yeah, we now have node and launcher running, but daedalus cant connect
<domenkozar> peti: yeah, I think Eelco will pick that up.
<domenkozar> clever: nice
<angerman> domenkozar: yes. clever was quite clever in helping set that up :-)
<clever> all TH runs under wine via iserv
<angerman> It's pretty close to how ghcjs does it by running the runner (iserv alike) in a nodejs process.
* peti can't wrap his head around Michael Snoyman's strategy for developing libraries. That guy writes some library A. Then he realizes that he'd prefer some different API, but instead of updating A he then releases a new library B. Then he maintains both A and B in parallel and uses both of those in his own projects, until eventually a new API emerges that's released as C. Now, once C is there, he
* peti deprecates both A and B. Meanwhile, he realizes that some other API would be cool and starts releasing D. The result of all that is that he's responsible for some 1e300 packages on Hackage, at least half of which are deprecated, and the other half is evenly divided between alternative Preludes, exception handling, and I/O functions.
<domenkozar> yeah I think conduit also has about 4 different interfaces
<domenkozar> most of which are deprecated
<manveru> :)
<gchristensen> my concern around the NUR has been around exactly this happening, which I understand is basically "sure, it could happen" and seeing this I'm thinking "duh, of course there is, it is inevitable"
<gchristensen> "OK but the tag line is asinine. As a regular user of a Linux distribution it is actually impossible for me to take the time to do a full analysis on every package I install to get work done.SOME level of trust has to be there or else the whole idea of a Linux distro can’t work."
<gchristensen> this comment specifically makes me concerned
<gchristensen> people _trust_ the AUR because it is supported by Arch, and we must be vigilent in not violating the trust we've accumulated
<gchristensen> https://screenshotscdn.firefoxusercontent.com/images/38d88abe-71f5-4057-bb69-9fa6d495df89.png I wonder how many users noticed, and out of that probably-small-number, heeded, the most important message in this block
<domenkozar> it's no much different to all language specific package managers
orivej has joined #nixos-dev
<gchristensen> I feel there is more trust in a linux distribution's packages to be safe, than in a language's package manager
<gchristensen> do you think that is not true?
<domenkozar> depends on the policy
<gchristensen> you're thinking like someone who sets policies
<domenkozar> debian has very strict rules, we don't read any of code before including packages
<gchristensen> true
<domenkozar> and one thing are backdoored post-back software
<domenkozar> catching faulty PRNG is another thing
<domenkozar> that I think is not in scope of a distro, although it may have momentum to do so
<domenkozar> gchristensen: but it's users fault, AUR and such clearly state it's not in core packages
<domenkozar> debian has the same thing that could happen
<Profpatsch> peti: Not to speak about Snoyman’s ideas about how to structure Haskell codebases and stuff like MonadBaseControl
<gchristensen> debian doesn't host arbitrary user packages
<domenkozar> well instead you have launchpad
<domenkozar> or whatever that ppa hosting site is
<gchristensen> a good point
<domenkozar> I mean, just changing the naming doesn't achieve much, I think
<domenkozar> it should be clearly communicated the difference between official packages, etc
<Profpatsch> gchristensen: tbh, I wouldn’t be suprised if there was a strange patch or two in nixpkgs (cf AUR)
<Profpatsch> So we shouldn’t show too much Schadenfreude :)
<gchristensen> I have no schadenfreude
<gchristensen> to me it isn't a if but when it happens to us
<gchristensen> and how do we appropriately demonstrate care and effort to prevent it
<manveru> how would you even go about preventing it in nixpkgs?
<manveru> right now nixpkgs is more like the AUR and official arch packages combined, or at least it feels that way to me :)
<gchristensen> you can't silently change something in nixpkgs without demonstrating good will for a bit
<gchristensen> is something
<domenkozar> I think it's more about how do we fail
<domenkozar> rather than how do we prevent fail
<manveru> exactly
<gchristensen> sure
<manveru> nobody can thoroughly review all the packages we have atm...
<domenkozar> usually thinking about that for a few hours spares a lot of panic when things happen
<domenkozar> writing down a draft of the process would be a good start
<Profpatsch> gchristensen: domenkozar manveru Let me point towards my suport matrix idea again :) https://nixos.wiki/wiki/User:Profpatsch#nixpkgs_support_matrix
<adisbladis[m]> Profpatsch: I was just thinking of that :)
<adisbladis[m]> But how do you communicate that efficiently to end users?
<adisbladis[m]> It's confusing to have a single package repo with different support levels inside
<Profpatsch> adisbladis[m]: meta attributes and warnings on evaluation.
<Profpatsch> And for unmaintained stuff you have something like NIXPKGS_ALLOW_UNMAINTAINED maybe.
<Profpatsch> Or even NIXPKGS_MIN_SUPPORT_LEVEL=support
<Profpatsch> and for mission-critical systems you’d set full
<Profpatsch> aszlig has some code to allow unfree packages on a per-package basis, maybe that can be reused to be a little less coarse in practice.
<gchristensen> or different repos
<Profpatsch> gchristensen: Not a fan of that, because it would introduce overhead in maintenance.
<aszlig> Profpatsch: well, that code is also ugly as hell :-D
<gchristensen> good overhead, I think
<Profpatsch> gchristensen: think even grepping for function usage will be one or two more actions.
<gchristensen> that would be bad overhead
<Profpatsch> gchristensen: But it’s what happens if you split into multiple repos.
ckauhaus has joined #nixos-dev
<Profpatsch> (never work with categories when you can work with tags)
<gchristensen> well I guess I won't learn monads then
<Profpatsch> :p
<adisbladis[m]> Different repos gives me exherbo flashbacks
<Profpatsch> adisbladis[m]: The “decentralised development done properly” part of their tagline?
FRidh has joined #nixos-dev
ckauhaus has quit [Ping timeout: 260 seconds]
<adisbladis[m]> Profpatsch: Yeah
<Profpatsch> adisbladis[m]: Is there a metarepository that has git submodules for all packages? :)
<adisbladis[m]> The current nixpkgs models makes for better testing imo
<Profpatsch> (don’t tell me packages can use whatever version control they wint)
<Profpatsch> *want
<adisbladis[m]> I remember that being a pain point of mine in exherbo because of the different repos
ckauhaus has joined #nixos-dev
<adisbladis[m]> Profpatsch: Not really iirc
<adisbladis[m]> No one thing exherbo did right is enforcing git
<Profpatsch> +2
<Profpatsch> *+1
ckauhaus has quit [Ping timeout: 240 seconds]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Remote host closed the connection]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Ping timeout: 244 seconds]
ckauhaus has joined #nixos-dev
ckauhaus has quit [Ping timeout: 264 seconds]
Jackneill has joined #nixos-dev
FRidh has quit [Quit: Konversation terminated!]
Enzime has joined #nixos-dev
<gchristensen> maybe we could do an experiment where we turn on CloudFront's APAC region and see how much it costs
<gchristensen> monitor the bill and turn it off whenever if it starts to look bad
<adisbladis[m]> gchristensen: Do we have any stats now so we can get a rough idea of what the distribution looks like?
<gchristensen> not sure what the stats would look like, but we have all the logs cloudfront provides
<gchristensen> access to those are tightly controlled (read: eelco can look and nobody else I think ;)) for privacy so I think to get some interesting data out we'd need to provide him a program to manipulate it in to giving the answer you want
<adisbladis[m]> Ok :) Sounds doable
<gchristensen> (note: I have no idea if this is realisticly something which would be done, but I think if it can be done, this would be how it would be done)
ekleog has quit [Quit: WeeChat 2.0]
ekleog has joined #nixos-dev
pie_ has joined #nixos-dev
Sonarpulse has joined #nixos-dev
disasm has quit [Ping timeout: 240 seconds]
Jackneill has quit [Read error: Connection reset by peer]
Jackneill has joined #nixos-dev
FRidh has joined #nixos-dev
disasm has joined #nixos-dev
Sonarpulse has quit [Ping timeout: 244 seconds]
Sonarpulse has joined #nixos-dev
pie_ has quit [Ping timeout: 240 seconds]
orivej has quit [Ping timeout: 268 seconds]
ckauhaus has joined #nixos-dev
ckauhaus has quit []
FRidh has quit [Quit: Konversation terminated!]
orivej has joined #nixos-dev
<thoughtpolice> gchristensen: Oh, it's certainly doable. I'm pretty sure you can just directly use AWS RedShift directly, since I think the CloudFront logs are just stored in S3 anyway.
<thoughtpolice> It would probably be more ideal to do something like have an AWS spot instance do some batch processing every day or something and submitting the calculated artifacts offsite. We really don't need much data out of the logs other than "Number of times this object was hit" and various rollups based on that.
<sphalerite> oh yeah, I've mentioned it a couple times in #nixos now but not here yet: I want to get rid of packageOverrides. Opinions welcome: https://github.com/NixOS/nixpkgs/issues/43266
<{^_^}> #43266 (by lheckemann, open): Deprecate packageOverrides?
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #nixos-dev
garbas has quit [Quit: WeeChat 1.9.1]
phreedom has joined #nixos-dev